Updates coming for Authorized Networks and Cloud Run/Functions on GKE

Update September 30, 2022: The steps described in the original post were completed on September 2, 2022. Most clusters (99%) now have the new firewall rules; they are not reachable from Cloud Run, Cloud Functions and App Engine unless you’ve configured Serverless VPC Access. The remaining 1% of clusters that were using this access will be migrated as follows:

• Affected Anthos Service Mesh clusters have been notified by email and will be migrated automatically without any customer action.
• Cloud Run, Cloud Functions, and App Engine customers that were using this mode of access have been provided with migration steps to enable VPC access via email.

We recently received helpful information through the Vulnerability Rewards Program for Authorized Networks and Cloud Run/Functions on Google Kubernetes Engine (GKE). Based on that information, we updated our product documentation and prioritized a plan to make engineering changes to GKE to restrict access to only GKE-related services. Those changes will roll out automatically to over 99% of our GKE customers by late August, and we will proactively reach out to the remaining customers to work on migration issues together.

Our existing firewall rules allow the Kubernetes API server’s IP address to be reachable from the Cloud Run and Cloud Functions services. However, even with this access, calls to the API still need to be authenticated and authorized using either Google Identity and Access Management or GKE role-based access control. To further improve security, we will soon limit that access to GKE-related services and block access from Cloud Run and Cloud Functions.

We plan to take the following steps: 

• Migrate core GKE services that communicate with the API server onto a dedicated set of IP addresses. 
• Notify customers that currently rely on being able to communicate from other cloud services to the Kubernetes API server that the access will be removed (approximately 1% of clusters). We will provide instructions to migrate to a new solution and allowlist existing customer usage to give them time to migrate.
• Remove the existing firewall rule and introduce a targeted rule allowing only the dedicated set of IP addresses belonging to the core GKE services.

Once these steps are complete, 99% of private clusters won’t be accessible from Cloud Run or Cloud Functions, with no action required from those customers. The remaining 1% will migrate on their own timeline as those customers need time to move their access to new solutions. Public clusters (where nodes have public IPs) will continue to be accessible from Google Cloud IPs as this is necessary for those nodes to communicate with the API server.

To access the GKE API server from serverless environments such as Cloud Run and Cloud Functions, customers can use Serverless VPC Access and connect through its private IP address. For customers who already access their GKE API server using this method, no further action is required.*

We look forward to continuing to work with all our partners and customers, and the research community, to advance security for everyone.

^{*This blog was edited on June 17, 2022 to provide additional customer guidance on their use of Serverless VPC Access}