Google's primary support objective is to resolve production incidents as quickly as possible. We do this by understanding your configuration, analyzing logs and metrics, and collaborating with partners to solve incidents quickly.
Google Cloud offers a variety of support packages to accommodate your support needs. All Google Cloud Support packages include support for GKE Enterprise and GKE On-Prem. If you have an existing Google Cloud Support package, you already have support for GKE Enterprise and GKE On-Prem.
For more information, see the Google Cloud Support documentation.
Recommendations for getting support
You can get the best troubleshooting for business-critical incidents by performing the following recommended steps:
- Check that the environment is current with the published end-of-support timeframes. See Version support policy below.
Before you create your clusters, enable Cloud Logging and Cloud Monitoring for system components by configuring the
stackdriverfield of your GKE On-Prem configuration file. For more details, see the Support tools section).Create a configuration snapshot by running
gkectl diagnose snapshotagainst your clusters. Provide this snapshot in your support case.
Support tools
To troubleshoot a GKE On-Prem incident, Google Cloud Support relies on three pieces of information:
- Your environment's configuration
- Logs from your admin and user clusters
- Metrics from your admin and user clusters
Environment configuration
When you open a support case, you are asked to run the
gkectl diagnose snapshot --seed-config command and attach the resulting
tarball to the support case. gkectl diagnose snapshot --seed-config captures
information about Kubernetes and your nodes.
The tool is highly configurable and includes several predefined scenarios. You can also pass a YAML file with a customized set of information to gather. To learn more, refer to Diagnosing Clusters.
You can add an excludeWords field to your configuration file to omit
sensitive or confidential information. Be sure to carefully review the
information captured by the tool. Highly confidential or sensitive information
should not be attached to your support case.
Cluster logs
When you create a new GKE On-Prem cluster, Cloud Logging agents are enabled by default and scoped only to system-level components. This replicates system-level logs into the Google Cloud project associated with the cluster. System-level logs are from Kubernetes pods running in one of five namespaces:
- kube-system
- gke-system
- gke-connect
- istio-system
- config-management-system
Logs can be queried from the Cloud Logging console.
For more details, see Logging and Monitoring.
Cluster metrics
In addition to logs, metrics are also captured by the Cloud Monitoring agent. This replicates system-level metrics into the Google Cloud project associated with the cluster. System-level metrics are from Kubernetes pods running in the same namespaces listed in Logs.
For more details, see Logging and Monitoring.
How we troubleshoot your environment
Here is an example of a typical support incident:
- Someone—for example, the cluster administrator—opens a support case via
Google Cloud console or the Google Cloud Support Center, and selects
__GKE Enterprise and GKE On-Prem as Category and
Component, respectively. They enter the information required and attach the
output of
gkectl diagnose snapshotto the case. - The support case is routed to a Technical Support Engineer specializing in GKE On-Prem.
- The support engineer examines the contents of the snapshot to gain context of the environment.
- The support engineer examines the logs and metrics in the Google Cloud project, entering the support case ID as the business justification, which is logged internally.
- The support engineer responds to the case with an assessment and recommendation. The support engineer and the user continue troubleshooting until they come to a resolution.
Collaborative Support Partners
Google maintains collaborative support relationships with select partners to deliver a more seamless support experience for GKE On-Prem. With these relationships, Google is able to collaborate closely with that partner on behalf of our shared customers.
To benefit from Collaborative Support, you must maintain support agreements with both Google and the partner in question.
Google currently has a collaborative support relationship in place with the partners specified on the Collaborative Support Partners page.
Data about support issues may be shared with Collaborative Support Partners, as described in Google's Technical Support Services Guidelines.
What does Google support?
Generally, the Google Cloud Support team supports all software components shipped as part of GKE On-Prem plus open source Istio. The table below details this further:
| GCP Support | Collaborative Support | Not Supported |
|---|---|---|
| Kubernetes and the container runtime |
VMware vSphere (vCenter Server and ESXi) |
VMware products beyond vSphere |
| Canonical Ubuntu for guest/node OS |
F5 BIG-IP load balancers |
Customer code (see Developer Support below) |
| vCenter controller |
Hardware and hyper-converged infrastructure solutions as listed in the Collaborative Support Partners page |
Customer choice of host OS |
| F5 controller |
Physical server, storage, and network |
|
| Calico and related network policies |
External DNS, DHCP, and identity systems |
|
| Ingress controller |
Calico Enterprise Edition |
|
| Prometheus and Grafana |
||
| Stackdriver Monitoring, Stackdriver Logging, and Stackdriver agents |
||
| Identity federation with OIDC compliant providers |
||
| Hub, Connect, and the Connect Agent |
||
| Open source Istio |
Version support policy
Support for Google Distributed Cloud follows the Anthos Version Support Policy. Google supports the current version and the previous two (n-2) minor versions of GKE On-Prem.
The following table shows the supported and unsupported versions of this product.
| Release version | Release date | End of life date* |
|---|---|---|
| 1.29 (latest) | April 29, 2024 | April 29, 2025 or 1.32 release date |
| 1.28 | December 14, 2023 | December 14, 2024 or 1.31 release date |
| 1.16 | August 23, 2023 | August 23, 2024 or 1.30 release date |
| 1.15 (unsupported) | May 2, 2023 | May 2, 2024 or 1.29 release date |
*The EOL will be the later of these two dates.
Shared responsibility model
Running a business-critical production application on GKE On-Prem requires multiple parties to carry different responsponsibilities. While not an exhaustive list, the sections below list the roles and responsibilities.
Google responsibilities
- Maintenance and distribution of the GKE On-Prem software package
including Kubernetes, vCenter and F5 controllers, ingress controller, Connect
and Stackdriver agents, and
gkectlcommand line tool. - Maintenance and distribution of the Ubuntu admin workstation and node machine images including regular patching and security fixes.
- Notifying users of available upgrades for GKE On-Prem, and producing upgrade scripts for the previous version; GKE On-Prem supports sequential upgrades only (1.2 → 1.3 → 1.4 only and not 1.2 → 1.4).
- Operating the Connect, and Stackdriver services.
- Troubleshooting, providing workarounds, and correcting the root cause of any issues related to Google-provided components
User responsibilities
- Overall system administration for on-premises clusters.
- Maintaining any application workload deployed on the cluster.
- Running, maintaining, and patching the data center infrastructure, including networking, servers, storage, and connectivity to Google Cloud.
- Running, maintaining, and patching vSphere and network load balancers.
- Maintaining support contracts with VMware and F5 (if deployed).
- Upgrading GKE On-Prem versions on a regular basis.
- Testing and deployment updated node machine images with Ubuntu patches.
- Monitoring of the cluster and applications, and responding to any incidents.
- Ensuring Cloud Logging and Stackdriver agents are deployed to clusters.
- Providing Google with environmental details for troubleshooting purposes.
Developer Support
Google does not provide support for application workloads running on GKE On-Prem. However, Google does provide best-effort developer support to ensure your developers can easily run applications on GKE On-Prem. Google believes that engaging earlier during development can prevent critical incidents later in the deployment.
Developer support is available to customers with a paid support package, and is treated as a P3 for an issue blocking a launch, or a P4 for general consultation.