Kubernetes

An Apache Pekko gRPC service can be deployed to a Kubernetes cluster just like any other HTTP-based application, but if you want to make sure HTTP/2 is used across the board there are some things to consider.

There are 3 ways to use HTTP/2:

  • HTTP/2 over TLS, using ALPN to detect support for HTTP/2 in the TLS handshake.
  • HTTP/2 over plaintext TCP (‘h2c’), starting with HTTP/1 and using the Upgrade mechanism to negotiate moving to HTTP/2.
  • HTTP/2 over plaintext TCP (‘h2c’), without negotiation. This assumes the client has prior knowledge that the server supports HTTP/2.

A straightforward way to expose a TCP endpoint outside the cluster is to create a Kubernetes Service of type LoadBalancer. Beyond that, Kubernetes supports many types of Ingress Controllers. GCE and NGINX are the most widely known.

LoadBalancer Service

By creating a service of type LoadBalancer you can expose your services using a direct TCP loadbalancer. This means all features supported by the underlying implementation are available.

Using a LoadBalancer as the Ingress has some limitations, for example:

  • you are restricted to 1 service per IP
  • more advanced features such as TLS termination and path-based routing are not available

NGINX Ingress

Of the three ways to use HTTP/2 presented at the beginning of this page, NGINX by default only supports h2c with negotiation via the Upgrade mechanism. Because the Upgrade mechanism is more complicated than the other 2 approaches (“TLS”, and h2c with “prior knowledge”), gRPC client support for h2c with the Upgrade mechanism is not ubiquitous. Also, the loadbalancer will use HTTP/1 rather than HTTP/2 for the connection from the loadbalancer to the service. While this may happen to work it is not optimal.

GCE Ingress

GCE only supports HTTP/2 over TLS. This means you must configure GCE to use HTTPS both on the ‘outside’ and on the ‘inside’, and you will have to make sure your service is exposed over HTTPS internally as well.

To make sure the GCE loadbalancer uses HTTP/2 to connect to your service you must use the cloud.google.com/app-protocols annotation the service:

apiVersion: v1
kind: Service
metadata:
  name: my-service-3
  annotations:
    cloud.google.com/app-protocols: '{"my-grpc-port":"HTTP2"}'
spec:
  type: NodePort
  selector:
    app: metrics
    department: sales
  ports:
  - name: my-grpc-port
    port: 443
    targetPort: 8443

By default the loadbalancer will accept any certificate, so your service can be configured with a self-signed certificate.

The GCE load balancer only routes to pods that pass the health check. The default health check is a request to ‘/’ that expects a 200 OK return value, so it might be helpful to add this to your route.

Google Cloud Endpoints

The Google cloud has a Cloud Endpoints feature that allows exposing a gRPC API in a more ‘controlled’ way: you can configure API key management, authentication, monitoring quote/rate limiting, generate a ‘developer portal’ for your API’s users and much more. You need to provide your .proto definitions when creating the endpoint, and still need to configure your own loadbalancer to expose this proxy to the outside.

1.1.1