Migrate to Extensible Service Proxy V2 Beta

The Extensible Service Proxy V2 Beta (ESPv2 Beta) is an Envoy-based proxy that enables Cloud Endpoints to provide API management features. ESPv2 Beta replaces the NGINX-based Extensible Service Proxy (ESP).

This document describes how to migrate an existing Endpoints API deployment from ESP to ESPv2 Beta.

Before you begin

Before you begin the migration, consider the unsupported use cases and breaking API changes described below.

ESPv2 Beta unsupported use cases

  • App Engine flexible environment is not supported

    App Engine flexible environment has built-in Endpoints support that is enabled by setting endpoints_api_service in the application's app.yaml file. This built-in Endpoints implementation only supports ESP; it can not be migrated to ESPv2 Beta.

    If you want to use ESPv2 Beta with App Engine flexible environment, disable endpoints_api_service in app.yaml. You can deploy ESPv2 Beta as a separate Cloud Run service used to manage your application in App Engine flexible environment. The deployment works in the same way that ESPv2 Beta is used to support App Engine standard environment.

  • Custom NGINX Configuration is not supported

    ESPv2 Beta is an Envoy-based proxy. It cannot support custom NGINX proxy configuration. If your ESP configuration uses the -n or --nginx_config flags, your implementation may rely on a custom NGINX configuration that can't easily be migrated to ESPv2 Beta.

Breaking changes

  • The X-Endpoint-API-UserInfo header data format is changed. If your application uses this header, it must be changed to use the new format. See Handle JWTs in the backend service for more details.
  • If an API key is required for a request, ESP sends the X-Endpoint-API-Project-ID header with the consumer project ID to the backend application. ESPv2 Beta uses two different headers,X-Endpoint-API-Consumer-Type and X-Endpoint-API-Consumer-Number, to send required details. See the Service Infrastructure reference documentation for more details about the Consumer-Type and Consumer-Number sent with these headers.

  • The HTTP error response body format is changed. When ESPv2 Beta rejects an HTTP request, it generates an error response body in a new format. If your implementation uses client code to process the HTTP error JSON response body, the client code must be updated. See HTTP Error JSON response body for more details.

  • New startup flags are available and some ESP flags have been deprecated or replaced in ESPv2 Beta. See Startup flag changes between ESP and ESPv2 Beta.

Migrating your Endpoints APIs to use ESPv2 Beta

The migration steps required to use ESPv2 Beta with serverless platforms (Cloud Run, Cloud Functions, App Engine) differ from the steps required for non-serverless platforms (Google Kubernetes Engine, Compute Engine, and Kubernetes).

The migration steps required for each platform type are described below:

Non-serverless platforms: GKE, Compute Engine, Kubternetes

ESPv2 Beta is a drop-in replacement for ESP. For most configurations, you only need to update to the docker image tag.

However, you may need to adjust your startup flags if you configured ESP with:

  • More than one port via the --http_port, http2_port, and/or --ssl_port flags.
  • The SSL, DNS, Client IP, or another rarely-used flag.

New startup flags are available for ESPv2 Beta and some ESP flags have been deprecated or replaced. See Startup flag changes between ESP and ESPv2 Beta for more details.

GKE and Kubernetes

To migrate Endpoints configurations for GKE and Kubernetes, change the ESP image tag from :1 to :2 in the deployment yaml file. For example:

- name: esp
  image: gcr.io/endpoints-release/endpoints-runtime:2
  args: [
    "--http_port=8081",
    "--backend=127.0.0.1:8080",
    "--service=SERVICE_NAME",
    "--rollout_strategy=managed",
  ]

Compute Engine

Both ESP and ESPv2 Beta are deployed to the docker container using the docker run command. To migrate Endpoints for Compute Engine toESPv2 Beta, update the docker image tag from :1 to :2 in the command. For example:

sudo docker run \
    --detach \
    DOCKER_ARGUMENTS \
    gcr.io/endpoints-release/endpoints-runtime:2 \
    --service=SERVICE_NAME \
    --rollout_strategy=managed \
    --backend=YOUR_API_CONTAINER_NAME:8080

Serverless platforms (Cloud Run, Cloud Functions, App Engine)

For serverless platforms, ESPv2 Beta is deployed as a Cloud Run service to manage your application running on Cloud Run, Cloud Function or App Engine. To migrate Endpoints to ESPv2 Beta, you must build your existing Endpoints service configuration into a new ESPv2 Beta docker image, then deploy the image to your ESPv2 Beta Cloud Run service.

The deployment steps for ESP and ESPv2 Beta are identical, except for the following details:

  • The image tag should be changed from :1 to :2 in the ESPv2 Beta when you deploy ESPv2 Beta to Cloud Run. For example:

    gcloud run deploy CLOUD_RUN_SERVICE_NAME \
    --image="gcr.io/endpoints-release/endpoints-runtime-serverless:2" \
    --allow-unauthenticated \
    --platform managed \
    --project=ESP_PROJECT_ID  
    
  • The gcloud_build_image script is downloaded from a different location. It uses gcr.io/endpoints-release/endpoints-runtime-serverless:2 as the base image.

  • An environment variable is used to specify startup flags. The variable name for ESP is ESP_ARGS. The name for ESPv2 Beta is ESPv2_ARGS. See Extensible Service Proxy V2 Beta startup options for more information on setting ESPv2_ARGS and the available startup flags.

Startup flag changes between ESP and ESPv2 Beta

As with Extensible Service Proxy, you can specify configuration flags when deploying ESPv2 Beta services. With the change from the NGINX-based ESP to the Envoy-based ESPv2 Beta, some flags have been deprecated or replaced and new flags have been added. This section uses three tables to describe the changes:

  • Table 1 describes new flags that replace deprecated flags.
  • Table 2 describes new flags.
  • Table 3 describes deprecated flags.

Replaced Flags

New flags Replaced flags Description
--listener_port --http_port, --http2_port, --ssl_port A single Envoy listener port supports http, http2 and ssl in ESPv2 Beta. There is no need to specify separate ports.
--ssl_server_cert_path --ssl_port When --ssl_server_cert_path is used, ESPv2 Beta uses certs from server.key and server.crt files. With ESPv2 Beta you can specify server cert paths other than /etc/nginx/ssl. This flag replaces --ssl_port in ESP, which uses certs from the /etc/nginx/ssl/nginx.key and /etc/nginx/ssl/nginx.crt file paths.
--ssl_backend_client_cert_path --tls_mutual_auth, --enable_grpc_backend_ssl, --grpc_backend_ssl_private_key_file, --grpc_backend_ssl_cert_chain_file When --ssl_backend_client_cert_path is used, ESPv2 Beta uses certs from client.key and client.crt files. With ESPv2 Beta, you can specify client cert paths other than /etc/nginx/ssl. This flag replaces --tls_mutual_auth in ESP, which uses certs from the /etc/nginx/ssl/backend.key and /etc/nginx/ssl/backend.crt file paths.
--ssl_backend_client_root_certs_file --grpc_backend_ssl_root_certs_file With ESPv2 Beta, --ssl_backend_client_root_certs_file works for all backends. This flag replaces --grpc_backend_ssl_root_certs_file in ESP, which only works for gRPC backends.
--ssl_minimum_protocol,--ssl_maximum_protocol --ssl_protocols When using --ssl_protocols in ESP, you must list all desired ssl protocols. In ESPv2 Beta, you can specify a min and max protocol.
--envoy_use_remote_address,--envoy_xff_num_trusted_hops --xff_trusted_proxy_list,--client_ip_header,--client_ip_position Envoy requires use_remote_address and xff_num_trusted_hops to configure client ip extraction.
--dns_resolver_addresses --dns The replacement flag has the same behavior, but a different default value. ESP uses 8.8.8.8 as a DNS resolver. ESPv2 Beta uses the DNS resolver configured in /etc/resolv.conf.

New Flags

New flags Description
--http_request_timeout_s Sets the timeout for all http/https remote calls, except for backend calls and Google Service Control calls, in seconds.
--service_control_check_timeout_ms Sets the timeout for Google Service Control Check calls, in milliseconds.
--service_control_report_timeout_ms Sets the timeout for Google Service Control Report calls.
--service_control_quota_timeout_ms Sets the timeout for Google Service Control Quota calls.
--service_control_check_retries Specifies the retry number for Google Service Control Check calls.
--service_control_report_retries Specifies the retry number for Google Service Control Report calls.
--service_control_quota_retries Specifies the retry number for Google Service Control Quota calls.
--backend_dns_lookup_family Envoy specific config used to define the DNS lookup family for all backends.
--disable_tracing A overall flag used to disable all traces.
--tracing_project_id Used to set the id of the project that owns the trace data.
--tracing_incoming_context used to specify the incoming trace context.
--tracing_outgoing_context Used to specify the outgoingtrace context.

Deprecated Flags

Deprecated flags Description
--enable_websocket Websocket is enabled by default in Envoy.
--experimental_proxy_backend_host_header Not supported.
--allow_invalid_headers Not supported. This is a NGINX config: ignore_invalid_headers. If a HTTP request has invalid header names, it will be rejected by ESPv2 Beta. Valid header names are composed of English letters, digits, hyphens, and possibly underscores. In ESPv2 Beta, the flag --underscores_in_headers determines whether underscores are allowed in headers.
--client_max_body_size NGINX config, not supported.
--client_body_buffer_size NGINX config, not supported.
--large_client_header_buffers NGINX config, not supported.
--keepalive_timeout NGINX config, not supported.
--client_body_timeout NGINX config, not supported.
--rewrite Not supported.
--experimental_enable_multiple_api_configs Not supported.
--enable_backend_routing Not needed. Backend routing is automatically enabled for serverless platforms.
--rollout_fetch_throttle_window_in_s Not needed.
--nginx_config Not supported.

Please see Extensible Service Proxy V2 Beta startup options for more detail regarding ESPv2 Beta startup flags. Additional generic examples and help text for flags can be found in the GitHub repository.

Handle JWTs in the backend service

When using JWTs to perform authentication, both proxies send the authentication result in the X-Endpoint-API-UserInfo header to the backend API. We recommend using this header instead of the original Authorization header, as the original Authorization header may be modified in serverless platforms. However, the format of the header has been changed for ESPv2 Beta.

In ESP, the X-Endpoint-API-UserInfo header contains a Base64Url encoded JSON object that contains the JWT payload and a few specific fields added by ESP.

If available in the JWT, ESP adds the values of the id, issuer, email, and audiences properties to the encoded JSON object. It also adds the claims property that includes the original payload of the JWT. The JSON object has the form:

{
  "id": "from-sub",
  "issuer": "from-iss",
  "email": "from-email",
  "audiences": ["from-aud"],
  "claims": {
     original-jwt-payload
   }
}

In ESPv2 Beta, the X-Endpoint-API-UserInfo header only contains the Base64Url encoded payload of the original JWT, without modification.

If your backend service expects the X-Endpoint-API-UserInfo header to use the ESP JSON format, you must update your service to handle the new, simplified format.

See Using a custom method to authenticate users and Authentication between services for more on using JWTs with authentication.

Error JSON response body format

If an HTTP request is rejected by ESP or ESPv2 Beta, the response body contains a status code and an error message in JSON format. The response body format has changed in ESPv2 Beta, as shown in the examples below:

The error response body from ESP

{
 "code": 5,
 "message": "Method does not exist.",
 "details": [
  {
   "@type": "type.googleapis.com/google.rpc.DebugInfo",
   "stackEntries": [],
   "detail": "service_control"
  }
 ]
}

The error response body from ESPv2 Beta

{
 "code": 400,
 "message": "Method does not exist.",
}

There are two primary differences:

  • In ESPv2 Beta, the code field contains an http status code, rather than RPC status code found in ESP.
  • The error response body in ESPv2 Beta does not contain a details field.

What's next

Learn about: