Pluggable Client Transports / HTTP(S) proxy Support

The client side infrastructure has support to plug different transport mechanisms underneath (the API may still change in the future). A client side transport is represented by an instance of org.apache.pekko.http.scaladsl.ClientTransportorg.apache.pekko.http.javadsl.ClientTransport:

Scala
source@ApiMayChange
trait ClientTransport {
  def connectTo(host: String, port: Int, settings: ClientConnectionSettings)(
      implicit system: ActorSystem): Flow[ByteString, ByteString, Future[OutgoingConnection]]
}
Java
source@ApiMayChange
abstract class ClientTransport {
  def connectTo(host: String, port: Int, settings: ClientConnectionSettings, system: ActorSystem)
      : Flow[ByteString, ByteString, CompletionStage[OutgoingConnection]]
}

A transport implementation defines how the client infrastructure should communicate with a given host.

Note

In our model, SSL/TLS runs on top of the client transport, even if you could theoretically see it as part of the transport layer itself.

Configuring Client Transports

A ClientTransportClientTransport can be configured in the ClientConnectionSettingsClientConnectionSettings. Right now, this is not possible through config files but only by code. First, use ClientConnectionSettings.withTransport to configure a transport, then use ConnectionPoolSettings.withConnectionSettings. ClientConnectionSettingsClientConnectionSettings can be passed to all client-side entry points in HttpHttp.

Predefined Transports

TCP

The default transport is ClientTransport.TCP which simply opens a TCP connection to the target host.

HTTP(S) Proxy

A transport that connects to target servers via an HTTP(S) proxy. An HTTP(S) proxy uses the HTTP CONNECT method (as specified in RFC 7231 Section 4.3.6) to create tunnels to target servers. The proxy itself should transparently forward data to the target servers so that end-to-end encryption should still work (if TLS breaks, then the proxy might be fussing with your data).

This approach is commonly used to securely proxy requests to HTTPS endpoints. In theory it could also be used to proxy requests targeting HTTP endpoints, but we have not yet found a proxy that in fact allows this.

Instantiate the HTTP(S) proxy transport using ClientTransport.httpsProxy(proxyAddress).

The proxy transport can also be setup using ClientTransport.httpsProxy() or ClientTransport.httpsProxy(basicHttpCredentials) In order to define the transport as such, you will need to set the proxy host / port in your conf file like the following.

pekko.http.client.proxy {
 https {
   host = ""
   port = 443
 }
}

If host is left as "" and you attempt to setup an httpsProxy transport, an exception will be thrown.

Use HTTP(S) proxy with Http().singleRequestHttp.get(...).singleRequest

To make use of an HTTP proxy when using the singleRequest API you simply need to configure the proxy and pass the appropriate settings object when calling the single request method.

Scala
sourceimport org.apache.pekko
import pekko.actor.ActorSystem
import pekko.http.scaladsl.{ ClientTransport, Http }

import java.net.InetSocketAddress

implicit val system = ActorSystem()

val proxyHost = "localhost"
val proxyPort = 8888

val httpsProxyTransport = ClientTransport.httpsProxy(InetSocketAddress.createUnresolved(proxyHost, proxyPort))

val settings = ConnectionPoolSettings(system)
  .withConnectionSettings(ClientConnectionSettings(system)
    .withTransport(httpsProxyTransport))
Http().singleRequest(HttpRequest(uri = "https://google.com"), settings = settings)
Java
source
final ActorSystem system = ActorSystem.create();

ClientTransport proxy =
    ClientTransport.httpsProxy(InetSocketAddress.createUnresolved("192.168.2.5", 8080));
ConnectionPoolSettings poolSettingsWithHttpsProxy =
    ConnectionPoolSettings.create(system)
        .withConnectionSettings(ClientConnectionSettings.create(system).withTransport(proxy));

final CompletionStage<HttpResponse> responseFuture =
    Http.get(system)
        .singleRequest(
            HttpRequest.create("https://github.com"),
            Http.get(system).defaultClientHttpsContext(),
            poolSettingsWithHttpsProxy, // <- pass in the custom settings here
            system.log());

Use HTTP(S) proxy that requires authentication

In order to use an HTTP(S) proxy that requires authentication, you need to provide HttpCredentialsHttpCredentials that will be used when making the CONNECT request to the proxy:

Scala
sourceimport org.apache.pekko.http.scaladsl.model.headers

val proxyAddress = InetSocketAddress.createUnresolved(proxyHost, proxyPort)
val auth = headers.BasicHttpCredentials("proxy-user", "secret-proxy-pass-dont-tell-anyone")

val httpsProxyTransport = ClientTransport.httpsProxy(proxyAddress, auth)

val settings = ConnectionPoolSettings(system)
  .withConnectionSettings(ClientConnectionSettings(system)
    .withTransport(httpsProxyTransport))
Http().singleRequest(HttpRequest(uri = "http://pekko.apache.org"), settings = settings)
Java
sourceInetSocketAddress proxyAddress = InetSocketAddress.createUnresolved("192.168.2.5", 8080);
HttpCredentials credentials =
    HttpCredentials.createBasicHttpCredentials(
        "proxy-user", "secret-proxy-pass-dont-tell-anyone");

ClientTransport proxy =
    ClientTransport.httpsProxy(proxyAddress, credentials); // include credentials
ConnectionPoolSettings poolSettingsWithHttpsProxy =
    ConnectionPoolSettings.create(system)
        .withConnectionSettings(ClientConnectionSettings.create(system).withTransport(proxy));

final CompletionStage<HttpResponse> responseFuture =
    Http.get(system)
        .singleRequest(
            HttpRequest.create("https://github.com"),
            Http.get(system).defaultClientHttpsContext(),
            poolSettingsWithHttpsProxy, // <- pass in the custom settings here
            system.log());

Use HTTP(S) proxy with Http().singleWebSocketRequestHttp.get(…).singleWebSocketRequest

Making use of an HTTP proxy when using the singleWebSocketRequest is done like using singleRequest, except you set ClientConnectionSettings instead of ConnectionPoolSettings:

Scala
sourceimport java.net.InetSocketAddress

import org.apache.pekko
import pekko.actor.ActorSystem
import pekko.NotUsed
import pekko.http.scaladsl.{ ClientTransport, Http }
import pekko.http.scaladsl.settings.ClientConnectionSettings
import pekko.http.scaladsl.model.ws._
import pekko.stream.scaladsl._

implicit val system = ActorSystem()

val flow: Flow[Message, Message, NotUsed] =
  Flow.fromSinkAndSource(
    Sink.foreach(println),
    Source.single(TextMessage("hello world!")))

val proxyHost = "localhost"
val proxyPort = 8888

val httpsProxyTransport = ClientTransport.httpsProxy(InetSocketAddress.createUnresolved(proxyHost, proxyPort))

val settings = ClientConnectionSettings(system).withTransport(httpsProxyTransport)
Http().singleWebSocketRequest(WebSocketRequest(uri = "wss://example.com:8080/some/path"), clientFlow = flow,
  settings = settings)
Java
source
final ActorSystem system = ActorSystem.create();
final Materializer materializer = Materializer.createMaterializer(system);

final Flow<Message, Message, NotUsed> flow =
    Flow.fromSinkAndSource(
        Sink.foreach(System.out::println), Source.single(TextMessage.create("hello world")));

ClientTransport proxy =
    ClientTransport.httpsProxy(InetSocketAddress.createUnresolved("192.168.2.5", 8080));
ClientConnectionSettings clientSettingsWithHttpsProxy =
    ClientConnectionSettings.create(system).withTransport(proxy);

Http.get(system)
    .singleWebSocketRequest(
        WebSocketRequest.create("wss://example.com:8080/some/path"),
        flow,
        Http.get(system).defaultClientHttpsContext(),
        null,
        clientSettingsWithHttpsProxy, // <- pass in the custom settings here
        system.log(),
        materializer);

Use HTTP(S) proxy that requires authentication for Web Sockets

Here is an example for Web Socket:

Scala
sourceimport org.apache.pekko.http.scaladsl.model.headers

val proxyAddress = InetSocketAddress.createUnresolved(proxyHost, proxyPort)
val auth = headers.BasicHttpCredentials("proxy-user", "secret-proxy-pass-dont-tell-anyone")

val httpsProxyTransport = ClientTransport.httpsProxy(proxyAddress, auth)

val settings = ClientConnectionSettings(system).withTransport(httpsProxyTransport)
Http().singleWebSocketRequest(WebSocketRequest(uri = "wss://example.com:8080/some/path"), clientFlow = flow,
  settings = settings)
Java
sourceInetSocketAddress proxyAddress = InetSocketAddress.createUnresolved("192.168.2.5", 8080);
HttpCredentials credentials =
    HttpCredentials.createBasicHttpCredentials(
        "proxy-user", "secret-proxy-pass-dont-tell-anyone");

ClientTransport proxy =
    ClientTransport.httpsProxy(proxyAddress, credentials); // include credentials
ClientConnectionSettings clientSettingsWithHttpsProxy =
    ClientConnectionSettings.create(system).withTransport(proxy);

Http.get(system)
    .singleWebSocketRequest(
        WebSocketRequest.create("wss://example.com:8080/some/path"),
        flow,
        Http.get(system).defaultClientHttpsContext(),
        null,
        clientSettingsWithHttpsProxy, // <- pass in the custom settings here
        system.log(),
        materializer);

Custom Host Name Resolution Transport

You can use ClientTransport.withCustomResolverClientTransport.withCustomResolver to customize host name resolution. The given resolution function will be called for every connection attempt to resolve a hostname / port combination (potentially asynchronously) to an InetSocketAddress.

As a backend to implement the resolution function you can use Apache Pekko’s Async DNS Resolution.

Potential use cases:

  • in a managed setting this can be used to query for SRV DNS records that contain both address and port for a service.
  • if the DNS server returns multiple addresses, you can implement a load balancing algorithm to select a different target address for each connection

Implementing Custom Transports

Implement ClientTransport.connectTo to implement a custom client transport.

Here are some ideas for custom (or future predefined) transports:

  • SSH tunnel transport: connects to the target host through an SSH tunnel
  • Per-host configurable transport: allows choosing transports per target host
1.1.0