This page shows you how to resolve issues with privileged workloads that you deploy in Google Kubernetes Engine (GKE) Autopilot clusters.
Allowlist synchronization issues
When you deploy an AllowlistSynchronizer
, GKE attempts to
install and synchronize the allowlist files that you specify. If this
synchronization fails, the status
field of the AllowlistSynchronizer
reports the error.
Get the status of the AllowlistSynchronizer
object:
kubectl get allowlistsynchronizer ALLOWLIST_SYNCHRONIZER_NAME -o yaml
The output is similar to the following:
...
status:
conditions:
- type: Ready
status: "False"
reason: "SyncError"
message: "some allowlists failed to sync: example-allowlist-1.yaml"
lastTransitionTime: "2024-10-12T10:00:00Z"
observedGeneration: 2
managedAllowlistStatus:
- filePath: "gs://path/to/allowlist1.yaml"
generation: 1
phase: Installed
lastSuccessfulSync: "2024-10-10T10:00:00Z"
- filePath: "gs://path/to/allowlist2.yaml"
phase: Failed
lastError: "Initial install failed: invalid contents"
lastSuccessfulSync: "2024-10-08T10:00:00Z"
The conditions.message
field and the managedAllowlistStatus.lastError
field
provide detailed information about the error. Use this information to resolve
the issue.
Privileged workload deployment issues
After successfully installing an allowlist, you deploy the corresponding privileged workload in your cluster. In some cases, GKE might reject the workload.
Try the following resolution options:
- Ensure that the GKE version of your cluster meets the version requirement of the workload.
- Ensure that the workload that you're deploying is the workload to which the allowlist file applies.
To see why a privileged workload was rejected, request detailed information from GKE about allowlist violations:
Get a list of the installed allowlists in the cluster:
kubectl get workloadallowlist
Find the name of the allowlist that should apply to the privileged workload.
Open the YAML manifest of the privileged workload in a text editor. If you can't access the YAML manifests, for example if the workload deployment process uses other tooling, contact the workload provider to open an issue. Skip the remaining steps.
Add the following label to the
spec.metadata.labels
section of the privileged workload Pod specification:labels: cloud.google.com/matching-allowlist: ALLOWLIST_NAME
Replace
ALLOWLIST_NAME
with the name of the allowlist that you obtained in the previous step. Use the name from the output of thekubectl get workloadallowlist
command, not the path to the allowlist file.Save the manifest and apply the workload to the cluster:
kubectl apply -f WORKLOAD_MANIFEST_FILE
Replace
WORKLOAD_MANIFEST_FILE
with the path to the manifest file.The output provides detailed information about which fields in the workload didn't match the specified allowlist, like in the following example:
Error from server (GKE Warden constraints violations): error when creating "STDIN": admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: =========================================================================== Workload Mismatches Found for Allowlist (example-allowlist-1): =========================================================================== HostNetwork Mismatch: Workload=true, Allowlist=false HostPID Mismatch: Workload=true, Allowlist=false Volume[0]: data - data not found in allowlist. Verify volume with matching name exists in allowlist. Container[0]: - Envs Mismatch: - env[0]: 'ENV_VAR1' has no matching string or regex pattern in allowlist. - env[1]: 'ENV_VAR2' has no matching string or regex pattern in allowlist. - Image Mismatch: Workload=k8s.gcr.io/diff/image, Allowlist=k8s.gcr.io/pause2. Verify that image string or regex match. - SecurityContext: - Capabilities.Add Mismatch: the following added capabilities are not permitted by the allowlist: [SYS_ADMIN SYS_PTRACE] - VolumeMount[0]: data - data not found in allowlist. Verify volumeMount with matching name exists in allowlist.
In this example, the following violations occur:
- The workload specifies
hostNetwork: true
, but the allowlist doesn't specifyhostNetwork: true
. - The workload specifies
hostPID: true
, but the allowlist doesn't specifyhostPID: true
. - The workload specifies a volume named
data
, but the allowlist doesn't specify a volume nameddata
. - The container specifies environment variables named
ENV_VAR1
andENV_VAR2
, but the allowlist doesn't specify these environment variables. - The container specifies the image
k8s.gcr.io/diff/image
, but the allowlist specifiesk8s.gcr.io/pause2
. - The container adds the
SYS_ADMIN
andSYS_PTRACE
capabilities, but the allowlist doesn't allow adding these capabilities. - The container specifies a volume mount named
data
, but the allowlist doesn't specify a volume mount nameddata
.
- The workload specifies
If you're deploying a workload that's provided by a third-party provider, open an issue with that provider to resolve the violations. Provide the output from the previous step in the issue.
Bugs and feature requests for privileged workloads and allowlists
Partners are responsible for creating, developing, and maintaining their privileged workloads and allowlists. If you encounter a bug or have a feature request for a privileged workload or allowlist, contact the corresponding partner.