The product described by this documentation, Anthos Clusters on AWS (previous generation), is now in maintenance mode. All new installs must use the current generation product, Anthos clusters on AWS.

Using a proxy with GKE on AWS

This topic shows you how to route traffic from the GKE on AWS management service and Connect through an HTTP/HTTPS proxy.

GKE on AWS requires internet access for the following reasons:

The management service registers user clusters with Connect.
User clusters run the Connect Agent.
All management service and user cluster nodes download images from Container Registry.

You can route this traffic through an HTTP or HTTPS proxy. As of GKE on AWS 1.8, you can configure individual proxy settings for your management service and each of your clusters. For more information on how to change proxy settings, see Changing cluster proxy settings.

You declare proxy settings in a JSON file that is referenced in anthos-gke.yaml.

Prerequisites

Before you configure GKE on AWS to use a proxy, you must enable AWS VPC endpoints, define a security group for the proxy, and allow outbound connections to certain Google Cloud services.

To use an HTTP proxy, your user clusters must be running Kubernetes 1.17 or higher.

Enable VPC endpoints

Before you configure a proxy, you must create VPC endpoints for your GKE on AWS installation. VPC endpoints let resources in private subnets access AWS services without public internet access.

The following table lists the AWS services that GKE on AWS requires VPC endpoints for, along with the type of endpoint and the Security Groups for each GKE on AWS component that requires access to the endpoint.

Service	Endpoint type	Security groups
Auto Scaling	Interface	Management, control plane, node pools
EC2	Interface	Management, control plane, node pools
Load Balancing	Interface	Management, control plane, node pools
Key Management Service	Interface	Management, control plane, node pools
S3	Gateway	Management, control plane, node pools
Security Token Service (STS)	Interface	Management, control plane, node pools

You can create endpoints from the AWS VPC Console. The options you set when creating VPC endpoints depend on your VPC configuration.

Define a security group

Your GKE on AWS components must be able to connect to the proxy server. Create or locate an AWS security group that allows outbound connections to your proxy server. The security group should allow outbound access from your Management, Control plane, and Node pool security groups to the proxy address and port. Save the ID of this security group (for example, sg-12345678).

Type	Protocol	From port	To port	Address
Egress	TCP	Proxy port	Proxy port	Proxy security group

Allow outbound connections to Google Cloud services

For GKE on AWS to connect to Google Cloud services, the proxy server must allow traffic to the following domains:

gkeconnect.googleapis.com
gkehub.googleapis.com
oauth2.googleapis.com
storage.googleapis.com
www.googleapis.com
gcr.io
k8s.gcr.io
EC2-REGION.ec2.archive.ubuntu.com

Replace EC2-REGION with the AWS EC2 region where your GKE on AWS installation runs. For example, us-west-1.ec2.archive.ubuntu.com/.

If you are using Cloud Service Mesh with Prometheus and Kiali, allow outbound access from the following domains:

docker.io
quay.io

Creating the proxy configuration JSON file

The proxy configuration JSON file contains one object that contains three key/value pairs, with key names httpProxy, httpsProxy, and noProxy. Usernames and passwords are envelope-encrypted with your AWS KMS key before applying them to the server.

{
  "httpProxy": "HTTP_PROXY",
  "httpsProxy": "HTTPS_PROXY",
  "noProxy": "NO_PROXY"
}

Replace the following:

HTTP_PROXY with the proxy server address to route HTTP requests through.
HTTPS_PROXY with the proxy server to route HTTPS requests through.
NO_PROXY with an optional list of IPs, CIDR ranges, and domains where proxy is not used.

See Configuration reference for more information. For example, the following configuration uses a proxy at 10.0.0.254 and excludes a number of sites.

{
  "httpProxy": "http://user:password@10.0.0.254:80",
  "httpsProxy": "http://user:password@10.0.0.254:443",
  "noProxy": "localhost,127.0.0.1,169.254.169.254,10.0.0.0/16,example.com"
}

Adding the proxy to anthos-gke.yaml

Configure your AWSManagementService by performing the following steps:

Change to the directory with your GKE on AWS configuration. You created this directory when Installing the management service.
```
cd anthos-aws
```
Open anthos-gke.yaml in a text editor. Add a value for spec.proxy as in the following example.
```
apiVersion: multicloud.cluster.gke.io/v1
kind: AWSManagementService
metadata:
  name: management
spec:
  version: aws-1.14.1-gke.0
  proxy: PROXY_JSON_FILE
  securityGroupIDs:
  - SECURITY_GROUP_ID
  ...
```
Replace the following:
- PROXY_JSON_FILE with the relative path of your proxy's JSON configuration.
- SECURITY_GROUP_ID with the ID of a security group with access to the proxy server.
Save the file.
Run anthos-gke aws management init to generate an anthos-gke.status.yaml file with additional configuration. The init command also validates the AWSManagementService object in your anthos-gke.yaml file.
```
anthos-gke aws management init
```
Run anthos-gke aws management apply to update your management service on AWS.
```
anthos-gke aws management apply
```
Next, Create a user cluster. Include the proxy's security group in the AWSCluster and AWSNodePool's spec.controlPlane.securityGroupIDs and spec.securityGroupIDs, respectively. The cluster then has access to the proxy.

Configuration reference

Field	Description	Example	Required
`httpProxy`	A proxy server URL. The value should include a hostname/IP address and optionally a port, username, and password.	`"http://user:password@10.184.37.42:80"`	Yes
`httpsProxy`	A proxy URL for encrypted, HTTPS traffic. The `httpProxy` URL will be used if `httpsProxy` isn't given.	`"http://user:password@10.101.16.31:80"`	No
`noProxy`	A comma-separated list of URLs to exclude from proxying. Each value may be an IP address, a CIDR range, a domain name, or the asterix character (``). Domains specified with a leading dot (e.g. ".google.com") indicate that a subdomain is required. A single asterix `` indicates that no proxying should be done.	`"1.2.3.4,10.0.0.0/16,example.com,.site.com"`	No

What's next

For more information about completing a GKE on AWS installation, see the following:

If you need to create a dedicated AWS VPC, see Installing a management service.
If you have an existing AWS VPC, see Integrating with existing infrastructure.

For information about changing your cluster's proxy settings, see:

Changing cluster proxy settings