Troubleshoot deployment issues in App Engine

Several factors can cause deployment errors in App Engine, including missing permissions, changes to organization policies, and issues in your app configuration.

This page describes the following common deployment errors in App Engine and methods to troubleshoot them:

Permission errors

This section describes errors that might occur when you deploy your app due to missing account permissions or changes to organization policies.

To identify the active account you use to access Google Cloud CLI and other tools in the Google Cloud Platform, do one of the following:

  • If you used the Google Cloud CLI to deploy, run the gcloud auth list command.

  • If you deployed from an IDE, view the settings for the Cloud Tools plugin.

To learn why assigning only the App Engine Deployer (roles/appengine.deployer) role might not be sufficient in some cases, see App Engine roles.

For more information about granting roles, see Manage access to projects, folders, and organizations.

Deployment fails for new projects

You might see the following error when you deploy your app for the first time in a new project:

ERROR: (gcloud.app.deploy) Error Response: [13] Failed to create cloud build: com.google.net.rpc3.client.RpcClientException:..........invalid bucket "staging.PROJECT_ID.appspot.com"; service account PROJECT_ID@appspot.gserviceaccount.com does not have access to the bucket

To resolve this issue, grant the Storage Admin (roles/storage.admin) role to the default service account. For more information, see Store build logs in a user-created bucket.

If you have already granted the Storage Admin role, along with the other required roles based on different permission errors you encounter during deployment, and are still unable to deploy your app, it might be due to the following changes to organization policies:

  • As of May 2024, Google Cloud enforces secure-by-default organization policies for all organization resources. This policy prevents App Engine from granting the Editor role to the App Engine default service accounts.

  • In June 2024, Cloud Build changed the default behavior for how Cloud Build uses service accounts in new projects. This is detailed in Cloud Build service account change. As a result of this change, new projects deploying versions for the first time might be using the default App Engine service account with insufficient permissions for deploying versions.

To resolve this issue, do the following:

The caller doesn't have permission to access project

The following error occurs if the service account doesn't have permission to deploy apps in the current project:

User EMAIL_ADDRESS does not have permission to access project PROJECT_ID (or it may not exist).

To resolve this issue, grant the App Engine Deployer (roles/appengine.deployer) role to the service account.

Failed to fetch metadata from the registry

The following error occurs if you use the gcloud app deploy command from a service account that doesn't have the Storage Admin (roles/storage.admin) role:

Failed to fetch metadata from the registry, with reason: generic::permission_denied

To resolve this issue, grant the Storage Admin role to the service account.

Service accounts must have permissions on the image

The following error occurs when you deploy your app:

The App Engine appspot and App Engine flexible environment service accounts must
have permissions on the image IMAGE_NAME

This error occurs due to one of the following reasons:

  • The default App Engine service account doesn't have the Storage Object Viewer (roles/storage.objectViewer) role.

    To resolve this issue, grant the Storage Object Viewer role to the service account.

  • Your project has a VPC Service Controls service perimeter that limits access to the Cloud Storage API using access levels.

    To resolve this issue, add the service account you use to deploy your app to the corresponding VPC Service Controls service perimeter accessPolicies.

  • After May 15, 2024, Artifact Registry hosts images for the gcr.io domain in Google Cloud projects without previous Container Registry usage. If you deploy an existing application in a new project created after this date, the service account might not have the required permissions to deploy the app. To grant the required permissions, see Deploying to App Engine.

Failed to create Cloud Build

The following error occurs if you use the gcloud app deploy command from a service account that doesn't have the Cloud Build Editor (roles/cloudbuild.builds.editor) role.

Failed to create cloud build: Permission denied

To resolve this issue, grant the Cloud Build Editor role to the service account.

Error fetching application

The following error occurs when the service account that you used to deploy your app doesn't have the App Engine Deployer role.

Permissions error fetching application apps/app_name. Please make sure you are using the correct project ID and that you have permission to view applications on the project.

If you are running Google Cloud CLI version 328 or later, the following error occurs
when you deploy your app:

make sure that you have permission to view applications on the project and that
SERVICE_ACCOUNT has the App Engine Deployer (roles/appengine.deployer) role.

To resolve this issue, grant the App Engine Deployer role to the service account that you used to deploy your app.

Error when deploying a service with a Serverless VPC Access connector

The following error occurs when the user or service account that is trying to deploy the app with a Serverless VPC Access connector doesn't have the required permissions:

Please ensure you have [compute.globalOperations.get] on the service project

To resolve this issue, ensure that the user or service account used for deployment has the Serverless VPC Access User and Compute Viewer IAM roles.

Timed out waiting for the app infrastructure to become healthy

The following error occurs when you deploy your app:

Timed out waiting for the app infrastructure to become flex_await_healthy

Various factors can cause this error, such as missing permissions, code errors, insufficient CPU or memory, or failed health checks.

To resolve this issue, rule out the following potential causes:

  1. Check whether the organization policy for your project restricts access to external IP addresses. For more information, see App Engine flexible environment known issues.

  2. Verify that you have granted the following roles to the service account that you use to run your application (usually the default service account, app-id@appspot.gserviceaccount.com):

  3. Grant the remaining roles based on your deployment errors, if the service account doesn't have them.

  4. If you deploy in a Shared VPC setup and configure an instance_tag in your app.yaml file, see Invalid value error when deploying in a Shared VPC setup to fix the issue.

Error when restarting instances under running versions

The following error occurs when you deploy your app:

error when restarting the instance under the running versions

As of May 2024, Google Cloud enforces the secure-by-default organization policies for all new organizations. This policy requires all VM instances created in new projects to enable VM Manager. For new and existing projects, this constraint prevents metadata updates that disable VM Manager at the project or instance level.

To resolve this issue, you must disable the organization policy constraint Require OS Config (constraints/compute.requireOsConfig).

If this issue continues to persist, you must also disable the following organization policies that might have been enabled at the project or the organization level:

Required compute.firewalls.list permission

The following error occurs when you deploy your app on a Shared VPC network:

Request to https://compute.googleapis.com/compute/v1/projects/projects/PROJECT_ID/global/firewalls?key failed, details: Required 'compute.firewalls.list' permission for 'projects/PROJECT_ID'

This error occurs if the following service accounts for the host project don't have the Compute Network User (roles/compute.networkUser) role:

To resolve this issue, grant the Compute Network User role to the Google APIs Service Agent and App Engine flexible environment Service Agent service accounts for the host project.

Deployment fails due to an organization policy constraint

The following error occurs when you deploy an app:

ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred while processing task /app-engine-flex/....: Request to https://compute.googleapis.com/compute/VERSION/projects/PROJECT_ID/... failed, details: Constraint constraints/compute.disableGuestAttributesAccess violated for project PROJECT_ID.

This might be due to the enforcement of the constraints/compute.disableGuestAttributesAccess constraint when deploying your app. All apps in the App Engine flexible environment enforce this organization policy by default.

To resolve this issue, you must disable the constraints/compute.disableGuestAttributesAccess constraint.

Common deployment errors

This section describes troubleshooting strategies for configuration errors in your app or project.

Invalid value error when deploying in a Shared VPC setup

The following error shows in Cloud Logging for flexible VM instances when you deploy your app:

Invalid value for field 'resource.tags.items[1]': 'aef-instance'. Duplicate
tags are not allowed: aef-instance on compute.instances.insert

This is a known issue where setting the instance_tag in your app.yaml file results in errors when creating instances.

To resolve the issue, remove the instance_tag field from your app.yaml file and redeploy.

Errors when deploying apps with three or less maximum instances

Apps deployed with max_instances set to three or less might experience unexpected errors or downtime. To resolve the issue, specify at least four maximum instances in your app.yaml file and redeploy.

Exceed the limit on maximum instances

The following error occurs when you deploy your app:

You may not have more than 'xx' total max instances in your project.

There is a limit for the maximum number of instances you can create per project. Requests to create additional instances fail if you exceed this limit.

To resolve this issue, set the value of max_instances in your app.yaml file to a value less than this limit or delete some services or versions to bring the sum of max_instances within the limit.

Build during deployment fails without errors in logs

The following error occurs when you deploy your app:

ERROR: (gcloud.app.deploy) Cloud build failed. Check logs at https://console.cloud.google.com/cloud-build/builds/BUILD_ID?project=PROJECT_NUMBER Failure status: UNKNOWN: Error Response: [2] Build failed; check build logs for details

If you click the link in the error message and find that all build steps were successful, but the app still failed to build, it might be due to either of the following reasons:

To resolve this issue, change the following settings for your bucket:

Errors when deploying to an existing App Engine version

The following error might occur when deploying to an existing version in the App Engine flexible environment:

ERROR: (gcloud.app.deploy) Error Response: [9] An internal error occurred while
processing task /app-engine-flex/flex_await_healthy/flex_await_healthy

This error indicates that updating an unhealthy deployment with a working Docker image doesn't always result in a healthy deployment. The outcome depends on the state of instances from the unhealthy deployment. Despite the error, if you provide a good Docker image, the deployment might eventually become healthy. Updating an existing version with a new Docker image, though allowed, is not a good practice. There is no rollback in case of version failure.