Jump Start Solution: AI/ML image processing on Cloud Functions

Last reviewed 2023-08-29 UTC

This guide helps you understand, deploy, and use the AI/ML image processing on Cloud Functions Jump Start Solution. This solution uses pre-trained machine learning models to analyze images provided by users and generate image annotations.

Deploying this solution creates an image processing service that can help you do the following, and more:

Handle unsafe or harmful user-generated content.
Digitize text from physical documents.
Detect and classify objects in images.

This document is intended for developers who have some familiarity with backend service development, the capabilities of AI/ML, and basic cloud computing concepts. Though not required, Terraform experience is helpful.

Objectives

Learn how a serverless architecture is used to create a scalable image processing service.
Understand how the image processing service uses pre-trained machine learning models for image analysis.
Deploy the image processing service and invoke it through REST API calls or in response to image upload events.
Review configuration and security settings to understand how to adapt the image processing service to different needs.

Products used

The solution uses the following Google Cloud products:

Cloud Vision API: An API offering powerful pre-trained machine learning models for image annotation. The solution uses the Cloud Vision API to analyze images and obtain image annotation data.
Cloud Storage: An enterprise-ready service that provides low-cost, no-limit object storage for diverse data types. Data is accessible from within and outside of Google Cloud and is replicated geo-redundantly. The solution uses Cloud Storage to store input images and resulting image annotation data.
Cloud Run functions: A lightweight serverless compute service that lets you create single-purpose, standalone functions that can respond to Google Cloud events without the need to manage a server or runtime environment. The solution uses Cloud Run functions to host the image processing service's endpoints.

For information about how these products are configured and how they interact, see the next section.

Architecture

The solution consists of an example image processing service that analyzes input images and generates annotations for the images using pre-trained machine learning models. The following diagram shows the architecture of the Google Cloud resources used in the solution.

Architecture of the infrastructure required for the AI/ML image processing on Cloud Functions solution.

The service can be invoked in two ways: directly through REST API calls or indirectly in response to image uploads.

Request flow

The request processing flow of the image processing service depends on how users invoke the service. The following steps are numbered as shown in the preceding architecture diagram.

When the user invokes the image processing service directly through a REST API call:

The user makes a request to the image processing service's REST API endpoint, deployed as a Cloud Run function. The request specifies an image as a URI or a base64 encoded stream.
The Cloud Run function makes a call to the Cloud Vision API to generate annotations for the specified image. The image annotation data is returned in JSON format in the function's response to the user.

When the user invokes the image processing service indirectly in response to image uploads:

The user uploads images to a Cloud Storage bucket for input.
Each image upload generates a Cloud Storage event that triggers a Cloud Run function to process the uploaded image.
The Cloud Run function makes a call to the Cloud Vision API to generate annotations for the specified image.
The Cloud Run function writes the image annotation data as a JSON file in another Cloud Storage bucket for output.

Cost

For an estimate of the cost of the Google Cloud resources that the AI/ML image processing on Cloud Functions solution uses, see the precalculated estimate in the Google Cloud Pricing Calculator.

Use the estimate as a starting point to calculate the cost of your deployment. You can modify the estimate to reflect any configuration changes that you plan to make for the resources that are used in the solution.

The precalculated estimate is based on assumptions for certain factors, including the following:

The Google Cloud locations where the resources are deployed.
The amount of time that the resources are used.
The amount of data stored in Cloud Storage.
The number of times the image processing service is invoked.

Before you begin

To deploy this solution, you first need a Google Cloud project and some IAM permissions.

Create or choose a Google Cloud project

When you deploy the solution, you choose the Google Cloud project where the resources are deployed. You can either create a new project or use an existing project for the deployment.

If you want to create a new project, do so before you begin the deployment. Using a new project can help avoid conflicts with previously provisioned resources, such as resources that are used for production workloads.

To create a project, complete the following steps:

In the Google Cloud console, go to the project selector page.

Go to project selector
Click Create project.
Name your project. Make a note of your generated project ID.
Edit the other fields as needed.
Click Create.

Get the required IAM permissions

To start the deployment process, you need the Identity and Access Management (IAM) permissions that are listed in the following table.

If you created a new project for this solution, then you have the roles/owner basic role in that project and have all the necessary permissions. If you don't have the roles/owner role, then ask your administrator to grant these permissions (or the roles that include these permissions) to you.

IAM permission required	Predefined role that includes the required permissions
`serviceusage.services.enable`	Service Usage Admin (`roles/serviceusage.serviceUsageAdmin`)
`iam.serviceAccounts.create`	Service Account Admin (`roles/iam.serviceAccountAdmin`)
`resourcemanager.projects.setIamPolicy`	Project IAM Admin (`roles/resourcemanager.projectIamAdmin`)
`config.deployments.create` `config.deployments.list`	Cloud Infrastructure Manager Admin (`roles/config.admin`)
`iam.serviceAccount.actAs`	Service Account User (`roles/iam.serviceAccountUser`)

About temporary service account permissions

If you start the deployment process through the console, Google creates a service account to deploy the solution on your behalf (and to delete the deployment later if you choose). This service account is assigned certain IAM permissions temporarily; that is, the permissions are revoked automatically after the solution deployment and deletion operations are completed. Google recommends that after you delete the deployment, you delete the service account, as described later in this guide.

View the roles assigned to the service account

These roles are listed here in case an administrator of your Google Cloud project or organization needs this information.

roles/serviceusage.serviceUsageAdmin
roles/iam.serviceAccountAdmin
roles/resourcemanager.projectIamAdmin
roles/cloudfunctions.admin
roles/run.admin
roles/storage.admin
roles/pubsublite.admin
roles/iam.securityAdmin
roles/logging.admin
roles/artifactregistry.reader
roles/cloudbuild.builds.editor
roles/compute.admin
roles/iam.serviceAccountUser

Deploy the solution

This section guides you through the process of deploying the solution.

To help you deploy this solution with minimal effort, a Terraform configuration is provided in GitHub. The Terraform configuration defines all the Google Cloud resources that are required for the solution.

You can deploy the solution by using one of the following methods:

Through the console: Use this method if you want to try the solution with the default configuration and see how it works. Cloud Build deploys all the resources that are required for the solution. When you no longer need the deployed solution, you can delete it through the console. Any resources that you create after you deploy the solution might need to be deleted separately.

To use this deployment method, follow the instructions in Deploy through the console.
Using the Terraform CLI: Use this method if you want to customize the solution or if you want to automate the provisioning and management of the resources by using the infrastructure as code (IaC) approach. Download the Terraform configuration from GitHub, optionally customize the code as necessary, and then deploy the solution by using the Terraform CLI. After you deploy the solution, you can continue to use Terraform to manage the solution.

To use this deployment method, follow the instructions in Deploy using the Terraform CLI.

Deploy through the console

Complete the following steps to deploy the preconfigured solution.

In the Google Cloud Jump Start Solutions catalog, go to the AI/ML image processing on Cloud Functions solution.

Go to the AI/ML image processing on Cloud Functions solution
Review the information that's provided on the page, such as the estimated cost of the solution and the estimated deployment time.
When you're ready to start deploying the solution, click Deploy.

A step-by-step configuration pane is displayed.
Complete the steps in the configuration pane.

Note the name that you enter for the deployment. This name is required later when you delete the deployment.

When you click Deploy, the Solution deployments page is displayed. The Status field on this page shows Deploying.
Wait for the solution to be deployed.

If the deployment fails, the Status field shows Failed. You can use the Cloud Build log to diagnose the errors. For more information, see Errors when deploying through the console.

After the deployment is completed, the Status field changes to Deployed.
To view the Google Cloud resources that are deployed and their configuration, take an interactive tour.

Start the tour

Next, to try the solution out yourself, see Explore the solution.

When you no longer need the solution, you can delete the deployment to avoid continued billing for the Google Cloud resources. For more information, see Delete the deployment.

Deploy using the Terraform CLI

This section describes how you can customize the solution or automate the provisioning and management of the solution by using the Terraform CLI. Solutions that you deploy by using the Terraform CLI are not displayed in the Solution deployments page in the Google Cloud console.

Set up the Terraform client

You can run Terraform either in Cloud Shell or on your local host. This guide describes how to run Terraform in Cloud Shell, which has Terraform preinstalled and configured to authenticate with Google Cloud.

The Terraform code for this solution is available in a GitHub repository.

Clone the GitHub repository to Cloud Shell.

A prompt is displayed to confirm downloading the GitHub repository to Cloud Shell.
Click Confirm.

Cloud Shell is launched in a separate browser tab, and the Terraform code is downloaded to the $HOME/cloudshell_open directory of your Cloud Shell environment.
In Cloud Shell, check whether the current working directory is $HOME/cloudshell_open/terraform-ml-image-annotation-gcf/infra. This is the directory that contains the Terraform configuration files for the solution. If you need to change to that directory, run the following command:
```
cd $HOME/cloudshell_open/terraform-ml-image-annotation-gcf/infra
```
Initialize Terraform by running the following command:
```
terraform init
```
Wait until you see the following message:
```
Terraform has been successfully initialized!
```

Configure the Terraform variables

The Terraform code that you downloaded includes variables that you can use to customize the deployment based on your requirements. For example, you can specify the Google Cloud project and the region where you want the solution to be deployed.

Make sure that the current working directory is $HOME/cloudshell_open/terraform-ml-image-annotation-gcf/infra. If it isn't, go to that directory.
In the same directory, create a text file named terraform.tfvars.
In the terraform.tfvars file, copy the following code snippet, and set values for the required variables.
- Follow the instructions that are provided as comments in the code snippet.
- This code snippet includes only the variables for which you must set values. The Terraform configuration includes other variables that have default values. To review all the variables and the default values, see the variables.tf file that's available in the $HOME/cloudshell_open/terraform-ml-image-annotation-gcf/infra directory.
- Make sure that each value that you set in the terraform.tfvars file matches the variable type as declared in the variables.tf file. For example, if the type that’s defined for a variable in the variables.tf file is bool, then you must specify true or false as the value of that variable in the terraform.tfvars file.
```
# This is an example of the terraform.tfvars file.
# The values in this file must match the variable types declared in variables.tf.
# The values in this file override any defaults in variables.tf.

# ID of the project in which you want to deploy the solution
project_id = "PROJECT_ID"
```

Validate and review the Terraform configuration

Make sure that the current working directory is $HOME/cloudshell_open/terraform-ml-image-annotation-gcf/infra. If it isn't, go to that directory.
Verify that the Terraform configuration has no errors:
```
terraform validate
```
If the command returns any errors, make the required corrections in the configuration and then run the terraform validate command again. Repeat this step until the command returns the following message:
```
Success! The configuration is valid.
```
Review the resources that are defined in the configuration:
```
terraform plan
```
If you didn't create the terraform.tfvars file as described earlier, Terraform prompts you to enter values for the variables that don't have default values. Enter the required values.

The output of the terraform plan command is a list of the resources that Terraform provisions when you apply the configuration.

If you want to make any changes, edit the configuration and then run the terraform validate and terraform plan commands again.

Provision the resources

When no further changes are necessary in the Terraform configuration, deploy the resources.

Make sure that the current working directory is $HOME/cloudshell_open/terraform-ml-image-annotation-gcf/infra. If it isn't, go to that directory.
Apply the Terraform configuration:
```
terraform apply
```
If you didn't create the terraform.tfvars file as described earlier, Terraform prompts you to enter values for the variables that don't have default values. Enter the required values.

Terraform displays a list of the resources that will be created.
When you're prompted to perform the actions, enter yes.

Terraform displays messages showing the progress of the deployment.

If the deployment can't be completed, Terraform displays the errors that caused the failure. Review the error messages and update the configuration to fix the errors. Then run the terraform apply command again. For help with troubleshooting Terraform errors, see Errors when deploying the solution using the Terraform CLI.

After all the resources are created, Terraform displays the following message:
```
Apply complete!
```
The Terraform output also includes the image processing service's entry point URL, the name of the input Cloud Storage bucket for uploading images, and the name of the output Cloud Storage bucket that contains image annotation data, as shown in the following example output:
```
vision_annotations_gcs = "gs://vision-annotations-1234567890"
vision_input_gcs = "gs://vision-input-1234567890"
vision_prediction_url = [
  "https://annotate-http-abcde1wxyz-wn.a.run.app",
  "ingressIndex:0",
  "ingressValue:ALLOW_ALL",
  "isAuthenticated:false",
]
```
To view the Google Cloud resources that are deployed and their configuration, take an interactive tour.

Start the tour

Next, you can explore the solution and see how it works.

When you no longer need the solution, you can delete the deployment to avoid continued billing for the Google Cloud resources. For more information, see Delete the deployment.

Explore the solution

In this section, you can try using the solution to see it in action. The image processing service can be invoked in two ways: by calling its REST API directly or by uploading images to the input Cloud Storage bucket.

Invoke the service through the REST API

In scenarios where you want to process images synchronously in a request-response flow, use the image processing service's REST API.

The annotate-http function deployed by the solution is the entry point to the image processing service's REST API. You can find the URL of this function in the console, or if you deployed by using the Terraform CLI, in the output variable vision_prediction_url. This entry point URL exposes an endpoint named /annotate for making image processing requests. The /annotate endpoint supports GET and POST requests with the following parameters:

Parameter	Description
`image`	(`POST` requests only) Image content, uploaded in binary format or specified as base64-encoded image data.
`image_uri`	A URI pointing to an image.
`features`	(Optional) A comma-separated list of Vision API annotation features to request. Possible feature values are: `CROP_HINTS` `DOCUMENT_TEXT_DETECTION` `FACE_DETECTION` `IMAGE_PROPERTIES` `LABEL_DETECTION` `LANDMARK_DETECTION` `LOGO_DETECTION` `OBJECT_LOCALIZATION` `PRODUCT_SEARCH` `SAFE_SEARCH_DETECTION` `TEXT_DETECTION` `WEB_DETECTION`

To specify the image to be analyzed, only include one of the image or image_uri parameters. If you specify both, image_uri is used.

For example, to perform object detection on an image with an internet URI, you can send a GET request such as the following using curl:

curl "YOUR_ENTRYPOINT_URL/annotate?features=OBJECT_LOCALIZATION&image_uri=YOUR_IMAGE_URI"

Alternatively, to specify image content directly using a local image file, you can use a POST request such as the following:

curl -X POST -F image=@YOUR_IMAGE_FILENAME -F features=OBJECT_LOCALIZATION "YOUR_ENTRYPOINT_URL/annotate"

The response contains the image annotations from the Vision API in JSON format.

Invoke the service by uploading images to Cloud Storage

In scenarios where you want to process images asynchronously or by batch upload, use the image processing service's Cloud Storage trigger, which automatically invokes the service in response to image uploads.

Follow the steps to analyze images using the Cloud Storage trigger:

In the console, go to the Cloud Storage Buckets page.

Go to Cloud Storage
Click the name of your input bucket (vision-input-ID) to go to its Bucket details page.
In the Objects tab, click Upload files.
Select the image file or files you want to analyze.
After the upload is complete, go back to the Cloud Storage Buckets page.

Go to Cloud Storage
Click the name of your annotation output bucket (vision-annotations-ID) to go to its Bucket details page.
The Objects tab contains a separate JSON file for each image you uploaded. The JSON files contain the annotation data for each image.

Note: If a corresponding JSON file doesn't appear in the annotation output bucket, wait a moment for image processing to complete and refresh the page.

Customize the solution

This section provides information that Terraform developers can use to modify the AI/ML image processing on Cloud Functions solution in order to meet their own technical and business requirements. The guidance in this section is relevant only if you deploy the solution by using the Terraform CLI.

The Terraform configuration for this solution provides the following variables you can use to customize the image processing service:

Variable	Description	Default value
`region`	The Google Cloud region in which to deploy the Cloud Run functions and other solution resources. See Cloud Run functions Locations for more information.	`us-west4`
`gcf_max_instance_count`	The maximum number of Cloud Run functions instances for the service. This helps control the service's scaling behavior. See Using maximum instances for more information.	10
`gcf_timeout_seconds`	The timeout for requests to the service, in seconds. This controls how long the service can take to respond. See Function timeout for more information.	120
`gcf_http_ingress_type_index`	Controls whether the service can be invoked by resources outside of your Google Cloud project. See Ingress settings for more information. Possible values are: 0 (Allow all) 1 (Allow internal only) 2 (Allow internal and Cloud Load Balancing)	0 (Allow all)
`gcf_require_http_authentication`	Controls whether authentication is required to make a request to the service. See Authenticating for invocation for more information.	`false`
`gcf_annotation_features`	A comma-separated list of Vision API annotation features for the service to include by default. This can be overridden for individual requests. Possible feature values are: `CROP_HINTS` `DOCUMENT_TEXT_DETECTION` `FACE_DETECTION` `IMAGE_PROPERTIES` `LABEL_DETECTION` `LANDMARK_DETECTION` `LOGO_DETECTION` `OBJECT_LOCALIZATION` `PRODUCT_SEARCH` `SAFE_SEARCH_DETECTION` `TEXT_DETECTION` `WEB_DETECTION`	`FACE_DETECTION,PRODUCT_SEARCH,SAFE_SEARCH_DETECTION`

To customize the solution, complete the following steps in Cloud Shell:

Make sure that the current working directory is $HOME/cloudshell_open/terraform-ml-image-annotation-gcf/infra. If it isn't, go to that directory.
Open your terraform.tfvars file and make the required changes, specifying appropriate values for the variables listed in the previous table.

Note: For guidance about the effects of such customization on reliability, security, performance, cost, and operations, see Design recommendations.
Validate and review the Terraform configuration.
Provision the resources.

Design recommendations

As you make changes to the solution either by changing the values of the provided Terraform variables or modifying the Terraform configuration itself, refer to the resources in this section to help you develop an architecture that meets your requirements for security, reliability, cost, and performance.

Note the following:

Before you make any design changes, assess the cost impact and consider potential trade-offs with other features. You can assess the cost impact of design changes by using the Google Cloud Pricing Calculator.
To implement design changes in the solution, you need expertise in Terraform coding and advanced knowledge of the Google Cloud services that are used in the solution.
If you modify the Google-provided Terraform configuration and if you then experience errors, create issues in GitHub. GitHub issues are reviewed on a best-effort basis and are not intended for general usage questions.
For more information about designing and setting up production-grade environments in Google Cloud, see Landing zone design in Google Cloud and Google Cloud setup checklist.

Security

By default, the image processing service allows requests from the internet and does not require authentication for requests. In a production environment, you might want to restrict access to the service.

You can control where requests to your service are allowed to originate by modifying the gcf_http_ingress_type_index Terraform variable. Take caution against unintentionally making the solution's service endpoints publicly accessible on the internet. See Configuring network settings in the Cloud Run functions documentation for more information.

You can require authentication for requests to the image processing service's REST API by modifying the gcf_require_http_authentication Terraform variable. This helps to control individual access to the service. If you require authentication, then callers of the service must provide credentials to make a request. See Authenticating for invocation in the Cloud Run functions documentation for more information.

For more security recommendations, see the Google Cloud Architecture Framework guidelines for Security, privacy, and compliance.

Reliability

When users upload images to the input Cloud Storage bucket, they might experience varying levels of latency in the resulting annotation output. By default, users must poll the output bucket to determine when the annotations are available. To make your application reliably take action as soon as image processing is complete, you can subscribe to Cloud Storage events in the output bucket. For example, you might deploy another Cloud Run function to process the annotation data - see Cloud Storage triggers in the Cloud Run functions documentation for more information.

For more recommendations, refer to the following guides to help optimize the reliability of the products used in this solution:

Performance

The throughput of the image processing service is directly affected by the Cloud Run functions scaling ability. Cloud Run functions scales automatically by creating function instances to handle the incoming traffic load, up to a configurable instance limit. You can control the scaling of the functions, and in turn the image processing service's throughput, by changing the maximum instance limit or removing the limit altogether. Use the gcf_max_instance_count Terraform variable to change the limit. See Using maximum instances and Auto-scaling behavior in the Cloud Run functions documentation for more information.

Additionally, you can help optimize performance by adhering to the following best practices:

Cost

Use the recommendations in the following guide to help optimize the cost of your solution: Google Cloud Architecture Framework: Cost optimization

Delete the deployment

When you no longer need the solution deployment, to avoid continued billing for the resources that you created, delete the deployment.

Delete through the console

Use this procedure if you deployed the solution through the console.

In the Google Cloud console, go to the Solution deployments page.

Go to Solution deployments
Select the project that contains the deployment that you want to delete.
Locate the deployment that you want to delete.
In the row for the deployment, click Actions and then select Delete.

You might need to scroll to see Actions in the row.
Enter the name of the deployment and then click Confirm.

The Status field shows Deleting.

If the deletion fails, see the troubleshooting guidance in Error when deleting a deployment.

When you no longer need the Google Cloud project that you used for the solution, you can delete the project. For more information, see Optional: Delete the project.

Delete using the Terraform CLI

Use this procedure if you deployed the solution by using the Terraform CLI.

In Cloud Shell, make sure that the current working directory is $HOME/cloudshell_open/terraform-ml-image-annotation-gcf/infra. If it isn't, go to that directory.
Remove the resources that were provisioned by Terraform:
```
terraform destroy
```
Terraform displays a list of the resources that will be destroyed.
When you're prompted to perform the actions, enter yes.

Terraform displays messages showing the progress. After all the resources are deleted, Terraform displays the following message:
```
Destroy complete!
```
If the deletion fails, see the troubleshooting guidance in Error when deleting a deployment.

When you no longer need the Google Cloud project that you used for the solution, you can delete the project. For more information, see Optional: Delete the project.

Optional: Delete the project

If you deployed the solution in a new Google Cloud project, and if you no longer need the project, then delete it by completing the following steps:

In the Google Cloud console, go to the Manage resources page.
Go to Manage resources
In the project list, select the project that you want to delete, and then click Delete.
At the prompt, type the project ID, and then click Shut down.

If you decide to retain the project, then delete the service account that was created for this solution, as described in the next section.

Optional: Delete the service account

If you deleted the project that you used for the solution, then skip this section.

As mentioned earlier in this guide, when you deployed the solution, a service account was created on your behalf. The service account was assigned certain IAM permissions temporarily; that is, the permissions were revoked automatically after the solution deployment and deletion operations were completed, but the service account isn't deleted. Google recommends that you delete this service account.

If you deployed the solution through the Google Cloud console, go to the Solution deployments page. (If you're already on that page, refresh the browser.) A process is triggered in the background to delete the service account. No further action is necessary.
If you deployed the solution by using the Terraform CLI, complete the following steps:
1. In the Google Cloud console, go to the Service accounts page.
  
  Go to Service accounts
2. Select the project that you used for the solution.
3. Select the service account that you want to delete.
  
  The email ID of the service account that was created for the solution is in the following format:
```
goog-sc-DEPLOYMENT_NAME-NNN@PROJECT_ID.iam.gserviceaccount.com
```
  The email ID contains the following values:
  - DEPLOYMENT_NAME: the name of the deployment.
  - NNN: a random 3-digit number.
  - PROJECT_ID: the ID of the project in which you deployed the solution.
4. Click Delete.

Troubleshoot errors

The actions that you can take to diagnose and resolve errors depend on the deployment method and the complexity of the error.

Errors when deploying through the console

If the deployment fails when you use the console, do the following:

Go to the Solution deployments page.

If the deployment failed, the Status field shows Failed.
View the details of the errors that caused the failure:
1. In the row for the deployment, click Actions.
  
  You might need to scroll to see Actions in the row.
2. Select View Cloud Build logs.
Review the Cloud Build log and take appropriate action to resolve the issue that caused the failure.

Errors when deploying using the Terraform CLI

If the deployment fails when you use Terraform, the output of the terraform apply command includes error messages that you can review to diagnose the problem.

The examples in the following sections show deployment errors that you might encounter when you use Terraform.

API not enabled

If you create a project and then immediately attempt to deploy the solution in the new project, the deployment might fail with an error like the following:

Error: Error creating Network: googleapi: Error 403: Compute Engine API has not
been used in project PROJECT_ID before or it is disabled. Enable it by visiting
https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project=PROJECT_ID
then retry. If you enabled this API recently, wait a few minutes for the action
to propagate to our systems and retry.

If this error occurs, wait a few minutes and then run the terraform apply command again.

If an API not enabled error persists, follow the link in the error message to enable the API. Wait a few moments for the API to become enabled and then run the terraform apply command again.

Cannot assign requested address

When you run the terraform apply command, a cannot assign requested address error might occur, with a message like the following:

Error: Error creating service account:
 Post "https://iam.googleapis.com/v1/projects/PROJECT_ID/serviceAccounts:
 dial tcp [2001:db8:ffff:ffff::5f]:443:
 connect: cannot assign requested address

If this error occurs, run the terraform apply command again.

Error when deleting a deployment

In certain cases, attempts to delete a deployment might fail:

After deploying a solution through the console, if you change any resource that was provisioned by the solution, and if you then try to delete the deployment, the deletion might fail. The Status field on the Solution deployments page shows Failed, and the Cloud Build log shows the cause of the error.
After deploying a solution by using the Terraform CLI, if you change any resource by using a non-Terraform interface (for example, the console), and if you then try to delete the deployment, the deletion might fail. The messages in the output of the terraform destroy command show the cause of the error.

Review the error logs and messages, identify and delete the resources that caused the error, and then try deleting the deployment again.

If a console-based deployment doesn't get deleted and if you can't diagnose the error by using the Cloud Build log, then you can delete the deployment by using the Terraform CLI, as described in the next section.

Delete a console-based deployment by using the Terraform CLI

This section describes how to delete a console-based deployment if errors occur when you try to delete it through the console. In this approach, you download the Terraform configuration for the deployment that you want to delete and then use the Terraform CLI to delete the deployment.

Identify the region where the deployment's Terraform code, logs, and other data are stored. This region might be different from the region that you selected while deploying the solution.
1. In the Google Cloud console, go to the Solution deployments page.
  
  Go to Solution deployments
2. Select the project that contains the deployment that you want to delete.
3. In the list of deployments, identify the row for the deployment that you want to delete.
4. Click View all row content.
5. In the Location column, note the second location, as highlighted in the following example:
In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
Create environment variables for the project ID, region, and name of the deployment that you want to delete:
```
export REGION="REGION"
export PROJECT_ID="PROJECT_ID"
export DEPLOYMENT_NAME="DEPLOYMENT_NAME"
```
In these commands, replace the following:
- REGION: the location that you noted earlier in this procedure.
- PROJECT_ID: the ID of the project where you deployed the solution.
- DEPLOYMENT_NAME: the name of the deployment that you want to delete.

Get the ID of the latest revision of the deployment that you want to delete:

export REVISION_ID=$(curl \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    "https://config.googleapis.com/v1alpha2/projects/${PROJECT_ID}/locations/${REGION}/deployments/${DEPLOYMENT_NAME}" \
    | jq .latestRevision -r)
    echo $REVISION_ID

The output is similar to the following:

projects/PROJECT_ID/locations/REGION/deployments/DEPLOYMENT_NAME/revisions/r-0

Get the Cloud Storage location of the Terraform configuration for the deployment:

export CONTENT_PATH=$(curl \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    "https://config.googleapis.com/v1alpha2/${REVISION_ID}" \
    | jq .applyResults.content -r)
    echo $CONTENT_PATH

The following is an example of the output of this command:

gs://PROJECT_ID-REGION-blueprint-config/DEPLOYMENT_NAME/r-0/apply_results/content

Download the Terraform configuration from Cloud Storage to Cloud Shell:
```
gcloud storage cp $CONTENT_PATH $HOME --recursive
cd $HOME/content/infra
```
Wait until the Operation completed message is displayed, as shown in the following example:
```
Operation completed over 45 objects/268.5 KiB
```

Initialize Terraform:

terraform init

Wait until you see the following message:

Terraform has been successfully initialized!

Remove the deployed resources:
```
terraform destroy
```
Terraform displays a list of the resources that will be destroyed.

If any warnings about undeclared variables are displayed, ignore the warnings.
When you're prompted to perform the actions, enter yes.

Terraform displays messages showing the progress. After all the resources are deleted, Terraform displays the following message:
```
Destroy complete!
```

Delete the deployment artifact:

curl -X DELETE \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    "https://config.googleapis.com/v1alpha2/projects/${PROJECT_ID}/locations/${REGION}/deployments/${DEPLOYMENT_NAME}?force=true&delete_policy=abandon"

Wait a few seconds and then verify that the deployment artifact was deleted:

curl -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    "https://config.googleapis.com/v1alpha2/projects/${PROJECT_ID}/locations/${REGION}/deployments/${DEPLOYMENT_NAME}" \
    | jq .error.message

If the output shows null, wait a few seconds and then run the command again.

After the deployment artifact is deleted, a message as shown in the following example is displayed:

Resource 'projects/PROJECT_ID/locations/REGION/deployments/DEPLOYMENT_NAME' was not found

Submit feedback

Jump Start Solutions are for informational purposes only and are not officially supported products. Google may change or remove solutions without notice.

To troubleshoot errors, review the Cloud Build logs and the Terraform output.

To submit feedback, do the following:

For documentation, in-console tutorials, or the solution, use the Send Feedback button on the page.

For unmodified Terraform code, create issues in the GitHub repository. GitHub issues are reviewed on a best-effort basis and are not intended for general usage questions.

For issues with the products that are used in the solution, contact Cloud Customer Care.

What's next

Learn more about serverless computing on Google Cloud.
Learn more about machine learning for image analysis on Google Cloud.
Learn more about event-driven architectures.
Understand the capabilities and limits of products used in this solution: