Creating a CI/CD pipeline with Azure Pipelines and Google Kubernetes Engine

This tutorial shows how to use Azure Pipelines, Google Kubernetes Engine (GKE), and Container Registry to create a continuous integration/continuous deployment (CI/CD) pipeline. The tutorial uses the ASP.NET MusicStore web app, which is based on ASP.NET Core.

The CI/CD pipeline uses two separate GKE clusters, one for development and one for production, as the following diagram shows.

At the beginning of the pipeline, developers commit changes to the example codebase. This action triggers the pipeline to create a release and to deploy it to the development cluster. A release manager can then promote the release so that it's deployed into the production cluster.

This tutorial is intended for developers and DevOps engineers. It assumes that you have basic knowledge of .NET Core, Azure Pipelines, and GKE. The tutorial also requires you to have administrative access to an Azure DevOps account.

Objectives

• Connect Container Registry to Azure Pipelines for publishing Docker images.
• Prepare a .NET Core sample app for deployment into GKE.
• Authenticate securely against GKE without having to use legacy authentication.
• Use Azure Pipelines release management to orchestrate GKE deployments.

Costs

This tutorial uses the following billable components of Google Cloud:

• GKE
• Cloud Load Balancing
• Cloud Storage (for Container Registry)

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Cleaning up.

Check the Azure DevOps pricing page for any fees that might apply to using Azure DevOps.

Before you begin

It's usually advisable to use separate projects for development and production workloads so that Identity and Access Management (IAM) roles and permissions can be granted individually. For the sake of simplicity, this tutorial uses a single project for both GKE clusters, one for development and one for production.

1. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to the project selector page

2. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

3. Make sure you have an Azure DevOps account and have administrator access to it. If you don't yet have an Azure DevOps account, you can sign up on the Azure DevOps home page.

Note: You can either use a command-line Git client or Visual Studio to follow this tutorial. If you use Visual Studio, make sure that it's connected to your Azure DevOps account.

Creating an Azure DevOps project

You use Azure DevOps to manage the source code, run builds and tests, and orchestrate the deployment to GKE. To begin, you create a project in your Azure DevOps account.

1. Go to the Azure DevOps home page (https://dev.azure.com/YOUR_AZURE_DEVOPS_ACCOUNT_NAME).
2. Click New Project.
3. Enter a project name, such as Music Store.
4. Set Visibility to Private, and then click Create.
5. After you create the project, in the menu on the left, click Repos.
6. Click Import to fork the MusicStore repository from GitHub, and then set the following values:
   • Source type: Git
   • Clone URL: https://github.com/aspnet/MusicStore.git

7. Click Import.

   When the import process is done, you see the source code of the MusicStore app.

Building continuously

You can now use Azure Pipelines to set up continuous integration. For each commit that's pushed to the Git repository, Azure Pipelines builds the code and packages the build artifacts into a Docker container. The container is then published to Container Registry.

Setting the default branch

This tutorial was built and tested with version 2.1 of the MusicStore app. To ensure that you are using the same version, configure Azure Pipelines to use the release/2.1 branch as the default branch:

1. In the Azure DevOps menu, select Project settings.
2. Select Repos > Repositories.
3. In the list of repositories, select the Git repository you imported previously. It should have the same name as your Azure DevOps project.
4. Expand the list of branches by clicking the arrow next to Branches.
5. Select the release/2.1 branch. A ... button appears next to the name of the branch.
6. Click ... and select Set as default branch.

Build the code

After you create the branch, you can begin to automate the build. Because MusicStore is an ASP.NET Core app, building the code includes four steps:

• Downloading and installing dependencies.
• Compiling the code.
• Running unit tests.
• Publishing the build results.

Later you'll add additional steps to deploy to GKE. Because GKE is a Linux-based environment, you'll set up the entire build process to run on Linux-based build agents.

1. Using Visual Studio or a command-line git client, clone your new Git repository.
2. In the root of the repository, create a file named azure-pipelines.yml.

3. Copy the following code into the file:

   resources:
   - repo: self
     fetchDepth: 1
   queue:
     name: Hosted Ubuntu 1604
   trigger:
   - release/2.1
   variables:
     TargetFramework: 'netcoreapp2.1'
     RestoreBuildProjects: 'samples/**/*.csproj'
     TestProjects: 'test/MusicStore.Test/*.csproj'
     BuildConfiguration: 'Release'
     DockerImageName: 'PROJECT_ID/musicstore'
   steps:
   - task: DotNetCoreCLI@2
     displayName: Restore
     inputs:
       command: restore
       projects: '$(RestoreBuildProjects)'
       feedsToUse: config
       nugetConfigPath: NuGet.config
   - task: DotNetCoreCLI@2
     displayName: Build
     inputs:
       projects: '$(RestoreBuildProjects)'
       arguments: '--configuration $(BuildConfiguration) --framework=$(TargetFramework)'
   - task: DotNetCoreCLI@2
     displayName: Test
     inputs:
       command: test
       projects: '$(TestProjects)'
       arguments: '--configuration $(BuildConfiguration) --framework=$(TargetFramework)'
   - task: DotNetCoreCLI@2
     displayName: Publish
     inputs:
       command: publish
       publishWebProjects: True
       arguments: '--configuration $(BuildConfiguration) --framework=$(TargetFramework)'
       zipAfterPublish: false
       modifyOutputPath: false
   

   Replace PROJECT_ID with the name of your project, and then save the file.

4. Commit your changes and push them to Azure Pipelines.

   Visual Studio

   1. Open Team Explorer and click the Home icon.
   2. Click Changes.
   3. Enter a commit message like Add build definition.
   4. Click Commit All and Push.

   Command line

   1. Stage all modified files:

      git add -A
      

   2. Commit the changes to the local repository:

      git commit -m "Add build definition"
      

   3. Push the changes to Azure DevOps:

      git push
      

5. In Azure, select Pipelines.

   A build definition is created based on the YAML file that you committed to the Git repository.

Publish Docker images

To deploy the MusicStore app to GKE, you must package the app as a Docker container image and make it available in Container Registry. You use the Container Registry in your production project for this task.

Set up a service account for publishing images

Before you can configure Azure Pipelines to build and publish a container image, you need to ensure that Azure Pipelines can authenticate with Container Registry. For this purpose, create a Google Cloud service account in your production project:

1. Switch to your project in the Cloud Console.

2. In the Cloud Console, activate Cloud Shell.

   Activate Cloud Shell

3. To save time typing your project ID and Compute Engine zone options, set default configuration values by running the following commands:

   gcloud config set project PROJECT_ID
   gcloud config set compute/zone ZONE

   Replace PROJECT_ID with the ID of your GCP project and replace ZONE with the name of the zone that you're going to use for creating resources. If you're unsure about which zone to pick, use us-central1-a.

   Example:

   gcloud config set project azure-pipelines-test-project-12345
   gcloud config set compute/zone us-central1-a

4. Enable the Container Registry API for your project:

   gcloud services enable containerregistry.googleapis.com

5. Create a service account for Azure Pipelines to publish Docker images:

   gcloud iam service-accounts create azure-pipelines-publisher \
       --display-name "Azure Pipelines Publisher"

6. Assign the Storage Admin IAM role to the service account:

   AZURE_PIPELINES_PUBLISHER=azure-pipelines-publisher@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com
   
   gcloud projects add-iam-policy-binding $GOOGLE_CLOUD_PROJECT \
       --member serviceAccount:$AZURE_PIPELINES_PUBLISHER \
       --role roles/storage.admin
   

7. Generate a service account key:

   gcloud iam service-accounts keys create azure-pipelines-publisher.json \
       --iam-account $AZURE_PIPELINES_PUBLISHER
   
   tr -d '\n' < azure-pipelines-publisher.json > azure-pipelines-publisher-oneline.json

8. In the Cloud Shell menu, click Open Editor.

9. Open the file named azure-pipelines-publisher-oneline.json. You'll need the content of this file in one of the following steps.

Connect Azure Pipelines to Container Registry

After you create the service account, you can connect Azure Pipelines to Container Registry.

1. In the Azure DevOps menu, select Project settings, and then select Pipelines > Service connections.
2. Click New service connection.
3. From the list, select Docker Registry, and then click Next.
4. In the dialog, enter values for the following fields:
   • Registry type: Others
   • Docker Registry: https://gcr.io/PROJECT_ID, replacing PROJECT_ID with the name of your project (for example, https://gcr.io/azure-pipelines-test-project-12345).
   • Docker ID: _json_key
   • Password: Paste the content of azure-pipelines-publisher-oneline.json.
   • Service connection name: gcr-tutorial
5. Click Save to create the connection.

Create a Dockerfile

You can now create the Dockerfile that Azure Pipelines uses to build the container image.

1. In the root of the repository, create a file named Dockerfile.

2. Copy the following code into the file, and then save the file:

   FROM mcr.microsoft.com/dotnet/core/aspnet:2.1
   WORKDIR /app
   COPY samples/MusicStore/bin/Release/netcoreapp2.1/publish /app/
   COPY samples/MusicStore/config.json /app/
   
   ENV ASPNETCORE_URLS=http://0.0.0.0:8080
   
   ENTRYPOINT ["dotnet", "MusicStore.dll"]
   

3. Create a another file named deployment.yaml in the root of your Git workspace. Leave the file empty for now.

4. Commit your changes:

   Visual Studio

   1. Open Team Explorer and click the Home icon at upper left to switch to the Home view.
   2. Click Changes.
   3. Enter a commit message like Add Dockerfile and placeholder for the Kubernetes manifest.
   4. Click Commit All.

   Command line

   1. Stage all modified files:

      git add -A
      

   2. Commit the changes to the local repository:

      git commit -m "Add Dockerfile and placeholder for the Kubernetes manifest"
      

Extend the build definition to build a Docker image

With all of the necessary files checked in, you can now extend the build definition.

1. Open the file azure-pipelines.yml.

2. Extend the build definition by appending the following piece of code to the file:

   - task: CmdLine@1
     displayName: 'Lock image version in deployment.yaml'
     inputs:
       filename: /bin/bash
       arguments: '-c "awk ''{gsub(\"MUSICSTORE_IMAGE\", \"gcr.io/$(DockerImageName):$(Build.BuildId)\", $0); print}'' deployment.yaml > $(build.artifactstagingdirectory)/deployment.yaml"'
   - task: PublishBuildArtifacts@1
     displayName: 'Publish Artifact'
     inputs:
       PathtoPublish: '$(build.artifactstagingdirectory)'
   - task: Docker@0
     displayName: 'Build image'
     inputs:
       containerregistrytype: 'Container Registry'
       dockerRegistryConnection: 'gcr-tutorial'
       imageName: '$(DockerImageName):$(Build.BuildId)'
   - task: Docker@0
     displayName: 'Publish image'
     inputs:
       containerregistrytype: 'Container Registry'
       dockerRegistryConnection: 'gcr-tutorial'
       action: 'Push an image'
       imageName: '$(DockerImageName):$(Build.BuildId)'
   

3. Commit your changes and push them to Azure Pipelines:

   Visual Studio

   1. Open Team Explorer and click the Home icon at upper left to switch to the Home view.
   2. Click Changes.
   3. Enter a commit message like Extend build definition to build Docker image.
   4. Click Commit All and Push.

   Command line

   1. Stage all modified files:

      git add -A
      

   2. Commit the changes to the local repository:

      git commit -m "Extend build definition to build Docker image"
      

   3. Push the changes to Azure DevOps:

      git push
      

4. In the Azure DevOps menu, select Pipelines.

   A new build is automatically triggered. It might take about two minutes for the build to complete. It might take around 2 minutes for the build to complete.

5. To verify that the image has been published to Container Registry, switch to the Cloud Console, select Container Registry > Images, and then click musicstore.

   A single image and the tag of this image are displayed. The tag corresponds to the numeric ID of the build that was run in Azure Pipelines.

Deploying continuously

With Azure Pipelines automatically building your code and publishing Docker images for each commit, you can now turn your attention to deployment.

Unlike some other continuous integration systems, Azure Pipelines distinguishes between building and deploying, and it provides a specialized set of tools labeled Release Management for all deployment-related tasks.

Azure Pipelines Release Management is built around these concepts:

• A release refers to a set of artifacts that make up a specific version of your app and that are usually the result of a build process.
• Deployment refers to the process of taking a release and deploying it into a specific environment.
• A deployment performs a set of tasks, which can be grouped in jobs.
• Stages let you segment your pipeline and can be used to orchestrate deployments to multiple environments—for example, development and testing environments.

The primary artifact that the MusicStore build process produces is the Docker image. However, because the Docker image is published to Container Registry, the image is outside the scope of Azure Pipelines. The image therefore doesn't serve well as the definition of a release.

To deploy to Kubernetes, you also need a manifest, which resembles a bill of materials. The manifest not only defines the resources that Kubernetes is supposed to create and manage, but also specifies the exact version of the Docker image to use. The Kubernetes manifest is well suited to serve as the artifact that defines the release in Azure Pipelines Release Management.

Configure the Kubernetes deployment

To run the MusicStore in Kubernetes, you need the following resources:

• A Deployment that defines a single pod that runs the Docker image produced by the build.
• A NodePort service that makes the pod accessible to a load balancer.
• An Ingress that exposes the application to the public internet by using a Cloud HTTP(S) load balancer.

With the MusicStore application, you can use either SQL Server or an embedded, locally stored database. For the sake of simplicity, use the default configuration that relies on the embedded database, although it comes with two restrictions:

• Only a single copy of the pod can run at a time. Otherwise users might see different data depending on which pod serves them.
• Any data changes are lost whenever the pod is restarted, unless you change the deployment to use persistent volumes. (We do not cover this scenario in the tutorial.)

To define these Kubernetes resources, you perform the following steps:

1. Open deployment.yaml and paste in the following code, and then save the file:

   apiVersion: v1
   kind: Service
   metadata:
     name: musicstore
   spec:
     ports:
     - port: 80
       targetPort: 8080
       protocol: TCP
       name: http
     selector:
       app: musicstore
     type: NodePort
   
   ---
   apiVersion: extensions/v1beta1
   kind: Ingress
   metadata:
     name: musicstore
   spec:
     backend:
       serviceName: musicstore
       servicePort: 80
   
   ---
   apiVersion: extensions/v1beta1
   kind: Deployment
   metadata:
     name: musicstore
   spec:
     replicas: 1
     template:
       metadata:
         labels:
           app: musicstore
       spec:
         containers:
         - name: musicstore
           image: MUSICSTORE_IMAGE
           ports:
             - containerPort: 8080
           livenessProbe:      # Used by deployment controller
             httpGet:
               path: /
               port: 8080
             initialDelaySeconds: 5
             periodSeconds: 5
           readinessProbe:     # Used by Ingress/GCLB
             httpGet:
               path: /
               port: 8080
             initialDelaySeconds: 3
             periodSeconds: 5
           resources:
             limits:
               memory: 1024Mi
             requests:
               memory: 768Mi
   

2. Commit your changes and push them to Azure Pipelines:

   Visual Studio

   1. Open Team Explorer and click the Home icon at upper left to switch to the Home view.
   2. Click Changes.
   3. Enter a commit message like Add Kubernetes manifest.
   4. Click Commit All and Push.

   Command line

   1. Stage all modified files:

      git add -A
      

   2. Commit the changes to the local repository:

      git commit -m "Add Kubernetes manifest"
      

   3. Push the changes to Azure DevOps:

      git push
      

Set up the development and production environments

Before returning to Azure Pipelines Release Management, you need to create the GKE clusters.

Create GKE clusters

1. Return to your Cloud Shell instance.

2. Enable the GKE API for your project:

   gcloud services enable container.googleapis.com

3. Create the development cluster by using the following command. Note that it might take a few minutes to complete:

   gcloud container clusters create azure-pipelines-cicd-dev

4. Create the production cluster by using the following command. Note that it might take a few minutes to complete:

   gcloud container clusters create azure-pipelines-cicd-prod

Connect Azure Pipelines to the development cluster

Just as you can use Azure Pipelines to connect to an external Docker registry like Container Registry, Azure Pipelines supports integrating external Kubernetes clusters.

It is possible to authenticate to Container Registry using a Google Cloud service account, but using Google Cloud service accounts is not supported by Azure Pipelines for authenticating with GKE. Instead, you have to use a Kubernetes service account.

To connect Azure Pipelines to your development cluster, you therefore have to create a Kubernetes service account first.

1. In Cloud Shell, connect to the development cluster:

   gcloud container clusters get-credentials azure-pipelines-cicd-dev

2. Create a Kubernetes service account for Azure Pipelines:

   kubectl create serviceaccount azure-pipelines-deploy

3. Assign the cluster-admin role to the service account by creating a cluster role binding:

   kubectl create clusterrolebinding azure-pipelines-deploy --clusterrole=cluster-admin --serviceaccount=default:azure-pipelines-deploy

4. Determine the IP address of the cluster:

   gcloud container clusters describe azure-pipelines-cicd-dev --format=value\(endpoint\)
   

   You will need this address in a moment.

5. In the Azure DevOps menu, select Project settings and then select Pipelines > Service connections.

6. Click New service connection.

7. Select Kubernetes and click Next.

8. Configure the following settings.

   Note: To get some of these values, you must run kubectl commands in Cloud Shell and then copy the values into the Azure page.
   • Authentication method: Service account.
   • Server URL: https://[PRIMARY_IP]/. Replace [PRIMARY_IP] with the IP address that you determined earlier.
   • Secret: Run the following command in Cloud Shell and copy the output:

     kubectl get secret $(kubectl get serviceaccounts azure-pipelines-deploy -o custom-columns=":secrets[0].name") -o yaml

   • Service connection name: azure-pipelines-cicd-dev.

9. Click Save.

Connect Azure Pipelines to the production cluster

To connect Azure Pipelines to your production cluster, you can follow the same approach.

1. In Cloud Shell, connect to the production cluster:

   gcloud container clusters get-credentials azure-pipelines-cicd-prod

2. Create a Kubernetes service account for Azure Pipelines:

   kubectl create serviceaccount azure-pipelines-deploy

3. Assign the cluster-admin role to the service account by creating a cluster role binding:

   kubectl create clusterrolebinding azure-pipelines-deploy --clusterrole=cluster-admin --serviceaccount=default:azure-pipelines-deploy

4. Determine the IP address of the cluster:

   gcloud container clusters describe azure-pipelines-cicd-prod --format=value\(endpoint\)
   

   You will need this address in a moment.

5. In the Azure DevOps menu, select Project settings and then select Pipelines > Service connections.

6. Click New service connection.

7. Select Kubernetes and click Next.

8. Configure the following settings:

   Note: To get some of these values, you must run kubectl commands in Cloud Shell and then copy the values into the Azure page.
   • Authentication method: Service account.
   • Server URL: https://[PRIMARY_IP]/. Replace [PRIMARY_IP] with the IP address that you determined earlier.
   • Secret: Run the following command in Cloud Shell and copy the output:

     kubectl get secret $(kubectl get serviceaccounts azure-pipelines-deploy -o custom-columns=":secrets[0].name") -o yaml

   • Service connection name: azure-pipelines-cicd-prod.

9. Click Save.

Configure the release pipeline

After you set up the GKE infrastructure, you return to Azure Pipelines to automate the deployment, which includes the following:

• Deploying to the development environment.
• Requesting manual approval before initiating a deployment to the production environment.
• Deploying to the production environment.

Create a release definition

As a first step, create a new release definition.

1. In the Azure DevOps menu, select Pipelines > Releases.
2. Click New pipeline.
3. From the list of templates, select Empty job.
4. When you're prompted for a name for the stage, enter Dev.
5. At the top of the screen, name the release MusicStore-KubernetesEngine.
6. In the pipeline diagram, next to Artifacts, click Add.

7. Select Build and add the following settings:

   • Source type: Build
   • Source (build pipeline): Select the build definition (there should be only one option)
   • Default version: Latest
   • Source Alias: manifest

8. Click Add.

9. On the Artifact box, click Continuous deployment trigger (the lightning bolt icon) to add a deployment trigger.

10. Under Continuous deployment trigger, set the switch to Enabled.

11. Click Save.

12. Enter a comment if you want, and then confirm by clicking Save.

    The pipeline displays similar to the following.

    

Deploy to the development cluster

With the release definition created, you can now configure the deployment to the GKE development cluster.

1. In the menu, switch to the Tasks tab.
2. Click Agent job.
3. Set Agent specification to ubuntu-18.04.
4. Next to Agent job, click Add a task to agent job to add a step to the phase.
5. Select the Deploy to Kubernetes task and click Add.

6. Click the newly added task and configure the following settings:

   • Display name: Deploy
   • Action: deploy
   • Kubernetes service connection: azure-pipelines-cicd-dev
   • Namespace: default
   • Strategy: None
   • Manifests: manifest/drop/deployment.yaml

7. Click Save.

8. Enter a comment if you want, and then confirm by clicking OK.

Deploy to the production cluster

Finally, you configure the deployment to the GKE production cluster.

1. In the menu, switch to the Pipeline tab.
2. In the Stages box, select Add > New stage.
3. From the list of templates, select Empty job.
4. When you're prompted for a name for the stage, enter Production.
5. Click the lightning bolt icon of the newly created stage.

6. Configure the following settings:

   • Select trigger: After stage
   • Stages: Dev
   • Pre-deployment approvals: (enabled)
   • Approvers: Select your own user name.

   The pipeline now looks like this:

   

7. Switch to the Tasks tab.

8. Hold the mouse over the Tasks tab and select Tasks > Production.

9. Click Agent job.

10. Set Agent specification to ubuntu-18.04.

11. Click Add a task to agent job to add a step to the phase.

12. Select the Deploy to Kubernetes task and click Add.

13. Click the newly added task and configure the following settings:

    • Display name: Deploy
    • Action: deploy
    • Kubernetes service connection: azure-pipelines-cicd-prod
    • Namespace: default
    • Strategy: None
    • Manifests: manifest/drop/deployment.yaml

14. Click Save.

15. Enter a comment if you want, and then confirm by clicking OK.

Run the pipeline

Now that you've configured the entire pipeline, you can test it by performing a source code change:

1. On your local computer, open the file samples\MusicStore\config.json from the Git repository you cloned earlier.
2. In line 3, change the SiteTitle setting to ASP.NET MVC Music Store on Kubernetes Engine.

3. Commit your changes, and then push them to Azure Pipelines.

   Visual Studio

   1. Open Team Explorer and click the Home icon.
   2. Click Changes.
   3. Enter a commit message like Change site title.
   4. Click Commit All and Push.

   Command line

   1. Stage all modified files:

      git add -A
      

   2. Commit the changes to the local repository:

      git commit -m "Change site title"
      

   3. Push the changes to Azure Pipelines:

      git push
      

4. In the Azure DevOps menu, select Pipelines. A build is triggered.

5. After the build is finished, select Pipelines > Releases. A release process is initiated.

6. Click Release-1 to open the details page, and wait for the status of the Development stage to switch to Succeeded.

7. In the Cloud Console, select Kubernetes Engine > Services & Ingress.

8. Locate the Ingress service for the azure-pipelines-cicd-dev cluster, and wait for its status to switch to Ok. This might take several minutes.

9. Open the link in the Endpoints column of the same row. You might see an error at first because the load balancer takes a few minutes to become available. When it's ready, observe that the Music Store has been deployed and is using the custom title:

   

10. In Azure Pipelines, click the Approve button located under the Prod stage to promote the deployment to the production environment:

    

    If you don't see the button, you might need to first approve or reject a previous release.

11. Enter a comment if you want, and then confirm by clicking Approve.

12. Wait for the status of the Prod environment to switch to Succeeded. You might need to manually refresh the page in your browser.

13. In the Cloud Console, refresh the Services page.

14. Locate the Ingress service for the azure-pipelines-cicd-prod cluster and wait for its status to switch to Ok. This might take several minutes.

15. Open the link in the Endpoints column of the same row. Again, you might see an error at first because the load balancer take a few minutes to become available. When it's ready, you see the MusicStore app with the custom title again, this time running in the production cluster.

Cleaning up

To avoid incurring further costs after you complete this tutorial, delete the entities that you created.

Delete the Azure Pipelines project

To delete the Azure Pipelines project, see the Azure DevOps Services documentation. Deleting the Azure Pipelines project causes all source code changes to be lost.

Delete the Google Cloud development and production projects

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.

1. In the Cloud Console, go to the Manage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next

• Configure fine-grained access control for Container Registry.
• Learn how to deploy a highly available SQL Server group on Compute Engine.
• Read about .NET on Google Cloud Platform.
• Install Cloud Tools for Visual Studio.
• Try out other Google Cloud features for yourself. Have a look at our tutorials.