Use Zonal DNS for your internal DNS type

Zonal DNS mitigates the risk of cross-regional outages and improves the overall reliability of your projects on Compute Engine. Using zonal DNS reduces the impact of single-point failures which can occur when using global DNS.

Before you begin

  • If you haven't already, then set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

    Select the tab for how you plan to use the samples on this page:

    Console

    When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

    gcloud

    1. Install the Google Cloud CLI, then initialize it by running the following command: 

      gcloud init
    2. Set a default region and zone.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      Install the Google Cloud CLI, then initialize it by running the following command: 

      gcloud init

    For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Required roles

To get the permissions that you need to migrate to zonal DNS, ask your administrator to grant you the following IAM roles:

  • Create or update an organization policy: Organization Policy Administrator (roles/orgpolicy.policyAdmin) on the folder or organization
  • Determine if a folder is ready to migrate to zonal DNS: Browser (roles/browser) on the folder or organization
  • Migrate a project to use zonal DNS: Project Editor (roles/resourcemanager.projectEditor) on the project
  • Migrate VMs to zonal DNS within a project: Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1) on the project
  • If your VM uses a service account: Service Account User (roles/iam.serviceAccountUser) on the service account or project

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to migrate to zonal DNS. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to migrate to zonal DNS:

  • Set an organization policy constraint: orgpolicy.*
  • Determine if a folder is ready to migrate to zonal DNS:
    • resourcemanager.folders.get
    • resourcemanager.folders.list
    • resourcemanager.organizations.get
    • resourcemanager.projects.get
    • resourcemanager.projects.list
  • Check for global DNS names and VM metadata: compute.projects.get
  • Set metadata on a VM: compute.instances.setMetadata
  • Set project-wide metadata: compute.projects.setCommonInstanceMetadata
  • If your VMs use service accounts: iam.serviceAccounts.actAs

You might also be able to get these permissions with custom roles or other predefined roles.

About DNS names

Zonal DNS names include the name of your VM, the zone where your VM is located, and the project that owns the VM. Global DNS names don't include the zone where the VM is located.

When you make a call to a VM using a global DNS name:

  • The name is resolved at a global level.
  • Each VM must have a unique DNS name.
  • When creating a new VM, the DNS name for the VM must be checked against all other global DNS names registered in the same project to avoid a DNS name collision.

When you make a call to a VM using a zonal DNS name:

  • The name is resolved within a zone.
  • Zonal DNS names must be unique within a zone. For example, my-vm.zone1.google.com must be unique for zone1. But, unlike global DNS names my-vm.zone2.google.com is also a valid DNS name when using zonal DNS.

Zonal DNS is the default internal DNS resolution method for Compute Engine for organizations created after September 6, 2018. Zonal DNS names in one zone function independently of other zones, letting you build more fault-tolerant multiregional applications with better availability characteristics. There is no charge for using zonal DNS. Zonal DNS is separate from Cloud DNS.

Projects created before September 6, 2018 were configured to use global DNS by default. These projects can continue to use global DNS, but Google highly recommends that organizations migrate these projects to zonal DNS to prevent service disruptions in another region from affecting local regional resources. Using zonal DNS provides higher reliability compared to global DNS by isolating failures in the DNS registration to individual zones. This reduces the impact of single-point failures. If an outage in Google Cloud occurs, it's isolated to a single zone, and resources and costs are not significantly impacted.

Migrate from global DNS to zonal DNS

Zonal DNS has replaced global DNS as the default internal DNS type for all new organizations onboarded to Google Cloud after September 6, 2018. If your existing projects are still using global DNS, Google strongly recommends that you switch these projects to use zonal DNS names. By not migrating to zonal DNS, you risk encountering the following issues:

  • Unable to create new VMs in any region experiencing control plane failures, where you have or previously had a project.
  • Inability to use some Compute Engine services critical to your workloads, such as autoscaling or autohealing for managed-instance groups (MIGs).

An alternative approach to improving the reliability of your workloads that use global DNS is to migrate to Cloud DNS private zones. Using Cloud DNS private zones removes the name uniqueness check required by global DNS and isolates failures to allow for DNS name resolution. For more information about this option, see the Cloud DNS documentation, or contact Cloud Customer Care or your account team representative. For information about how Cloud DNS handles zonal and regional outages, see Architecting disaster recovery for cloud infrastructure outages.

Overview of the migration process

The zonal DNS migration process has two levels:

  • Organization or folder level: Determine folder or organization readiness to migrate to zonal DNS. Prevent new projects from using global DNS. Performed by the organization administrator.
  • Project level: Migrate a single project from global DNS to zonal DNS. Usually performed by the project owner.

The image shows a flow chart of the steps involved in migrating to zonal DNS

Generally, the process involves the following steps:

  1. Check folder or organization readiness for migration to zonal DNS.
  2. If the folder or organization is ready to migrate to zonal DNS:
    1. Organization administrator sets an organizational policy for the folder or organization to prevent future use of global DNS.
    2. Project owners migrate their projects to use zonal DNS.
  3. If the folder or organization is not ready to migrate to zonal DNS:
    1. Project owners migrate ready projects to zonal DNS.
    2. Project owners investigate global DNS usage in projects that aren't ready.
    3. Project owners apply search path improvements or other changes to make the project ready to migrate to zonal DNS.
    4. Organization administrator rechecks the status of folder or organization readiness for zonal DNS migration.

Limitations

  • Zonal DNS is not a full replacement of global DNS. There are some query types (cross-zonal) that might not be resolved by zonal DNS with automatic search path lookup. Check the migration readiness of your organization's folders and projects to make sure they are compatible with zonal DNS prior to migration.

  • The global DNS to zonal DNS migration process introduces a new domain (ZONE.c.PROJECT_ID.internal) in the search path. If a VM running Linux or Unix already has 6 domains in the search path, make sure the VM is running with glibc version 2.26 or greater. DHCP clients with glibc 2.25 and earlier only support up to 6 search domains, so there might be a risk of discarding an existing search domain. This limit does not apply to VMs using:

    • Windows images
    • Container-Optimized OS images
    • Debian 10 or later images
    • Fedora CoreOS (version 27 or later)
    • RHEL 8 or later images
    • Ubuntu release 18.04 or later images
    • Other images with glibc version 2.26 or higher

  • Enabling search path improvement adds a few more search domains, such as NEIGHBOR_ZONE.c.PROJECT_ID.internal. As mentioned in the previous limitation, existing domains in the search path might be removed if the search path domain limit is exceeded when using glibc version 2.25 or earlier.

  • To use search path improvements with Google Kubernetes Engine, you must use Google Kubernetes Engine version 1.26 or later.

Check the version of glibc

To check the glibc version used by your VM:

  1. Connect to your Linux VM.
  2. Run ldd --version to get the glibc version.

Check the number of search domains if using glibc 2.25 or earlier

  1. Connect to your Linux VM.

  2. Check if your Linux VM already has 6 domains in the search path. You can run cat /etc/resolv.conf to view this information.

Organization administrator steps

As the organization administrator, you perform the following tasks:

Check if your organization uses global DNS by default

The default type of internal DNS name for your organization is determined by the date the organization was created and whether the organization policy constraint constraints/compute.setNewProjectDefaultToZonalDNSOnly is enforced.

Console

  1. Go to the IAM & Admin>Identity & Organization page in the console.

    Go to Identity & Organization

  2. Check the organization signup date.

    A screenshot of the Identity & Organization console page showing the Signup completed date

    • If your organization was created after September 6, 2018, your organization uses zonal DNS by default. In this case, no action is required.
    • If your organization was created before September 6, 2018, your organization uses global DNS by default, and should be migrated to zonal DNS.

  3. If your organization uses global DNS by default, check if an organization policy constraint sets the default DNS type for all newly created projects to zonal DNS.

    1. Go to the IAM & Admin>Organization Policies page in the Google Cloud console.
    2. In the Filter field, enter constraints/compute.setNewProjectDefaultToZonalDNSOnly.
    3. If the constraint is configured, click the name Sets the internal DNS setting for new projects to Zonal DNS Only.
    4. On the Policy details page, check the Status.
      • If the status is Enforced, then the default internal DNS type is zonal DNS for all new projects created in the organization.
      • Otherwise, the default DNS type for the project is still determined by the organization creation time.
    5. If the constraint wasn't configured for the organization, the default DNS type for the project is determined by the organization creation time, as described in the first step.

gcloud

Use the organizations describe command and the resource-manager org-policies list command to determine the default DNS type for an organization.

  1. Check the organization creationTime metadata value.

    gcloud organizations describe ORGANIZATION_ID

    Replace ORGANIZATION_ID with either the organization ID number or the organization domain name.

    • If your organization was created after September 6, 2018, your organization uses zonal DNS by default. In this case, your organization is already using zonal DNS and no further action is required.
    • If your organization was created before September 6, 2018, your organization uses global DNS by default, and should be migrated to zonal DNS.

  2. If your organization uses global DNS by default, determine if an organization policy constraint is configured to set the default DNS type for all newly created projects to zonal DNS.

    gcloud resource-manager org-policies list --organization=ORGANIZATION_ID \
   --filter="constraints/compute"

    In the output, look for constraints/compute.setNewProjectDefaultToZonalDNSOnly.

    1. If the constraint is configured, and the Status is Enforced, the default internal DNS type is zonal DNS for all new projects created in the organization.
    2. If the constraint wasn't configured for the organization or isn't enforced, the default internal DNS type is determined by the organization creation time, as described in the first step.

Determine which projects in a folder or organization use global DNS

To determine how many projects are using global DNS, we recommend using BigQuery to create a table that lists the relative projects for your organization and their metadata. You can then use this table to run a query that exposes the count of total projects that use global DNS.

  1. Create a BigQuery dataset.

  2. Export the asset metadata for your organization to a BigQuery table.

    1. Make sure the Cloud Asset Inventory API is enabled.
    2. Configure the permissions that are required to use the Cloud Asset Inventory API.

    3. Use the following gcloud CLI command to export the compute.googleapis.com/Project asset:

      gcloud asset export \
   --content-type resource \
   --organization 'ORGANIZATION_ID' \
   --bigquery-table 'projects/PROJECT_ID/datasets/DATASET_ID/tables/TABLE_NAME' \
   --asset-types='compute.googleapis.com/Project' \
   --output-bigquery-force

      Replace the following:

      • ORGANIZATION_ID: the organization ID number
      • PROJECT_ID: the project ID
      • DATASET_ID: the name of the BigQuery dataset
      • TABLE_NAME: the table you're exporting your metadata to. If the table doesn't exist, it's created.

  3. Go to the BigQuery page in the Google Cloud console.

  4. Select Compose a new query.

  5. In the query editor text area, enter the following GoogleSQL query and then click  Run.

    SELECT
  JSON_VALUE(SAFE.PARSE_JSON(resource.data).vmDnsSetting) AS vmDnsSetting,
  count(*) as project_count
FROM PROJECT_ID.DATASET_ID.TABLE_NAME
GROUP BY 1

    Replace the following:

    • PROJECT_ID: the project ID
    • DATASET_ID: the name of the BigQuery dataset
    • TABLE_NAME: the table that contains the exported metadata, from Step 2.

    Projects with the value ZONAL_ONLY for vmDnsSetting have zonal DNS configured. Otherwise, the projects are set to use global DNS.

  6. Optional: For a detailed view of the vmDnsSetting for each project, enter the following GoogleSQL query and then click  Run.

    SELECT
  SUBSTR(name,35) as project_id,
  JSON_VALUE(SAFE.PARSE_JSON(resource.data).vmDnsSetting) AS vmDnsSetting
FROM PROJECT_ID.DATASET_ID.TABLE_NAME

Determine if a folder is ready to migrate to zonal DNS

This step uses a bash script and the BigQuery table created in the previous section to determine the migration readiness of the folder.

  • The folder is ready if all projects haven't made any queries that are incompatible with zonal DNS in the last 30 days.
  • If a folder is not ready for migration, the script responds with the project IDs in the folder that are causing the folder to not be ready for migration. The projects in this result list are not yet compatible with zonal DNS and require additional action.

Complete the following steps:

  1. Get the folder ID. If you don't know the folder ID:
    1. In the Google Cloud console, go to the Managed resources page.
    2. Apply the filter Name:FOLDER_NAME to get the folder ID.

  2. Query the BigQuery table with the exported compute.Project assets data.

    See Determine which projects in a folder or organization use global DNS for the instructions on how to create the BigQuery table.

    Enter the following GoogleSQL query and then click  Run:

    SELECT
  SUBSTR(name,35) AS project_id,
FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE CONTAINS_SUBSTR(ancestors, 'FOLDER_NUMBER')

    Replace the following:

    • PROJECT_ID: the project ID
    • DATASET_ID: the name of the BigQuery dataset
    • TABLE_NAME: the table that contains the exported metadata
    • FOLDER_NUMBER: the folder ID number

  3. Copy the project ID list and save it to a file.

  4. Run the following bash script. The script iterates through the project IDs in the saved file to determine whether a folder is ready for migration.

#!/bin/bash
inaccessible_projects=()
unready_projects=()

for project in $(cat ~/FILENAME | tr '\n' ' '); do
  echo -e "Checking project $project..."
  ERROR=`curl -s --request POST "https://monitoring.googleapis.com/v3/projects/$project/timeSeries:query"   -H "Authorization: Bearer $(gcloud auth print-access-token)"   -H "Accept: application/json"   -H "Content-Type: application/json"   --data '{"query":"fetch compute.googleapis.com/Location | metric '"'"'compute.googleapis.com/global_dns/request_count'"'"' | filter metric.zonal_dns_readiness = '"'"'zonal_dns_risky'"'"' | every 30d | within 30d"}'   --compressed | jq --raw-output '.error'`
  if ! [[ "$ERROR" -eq "null" ]]; then
    inaccessible_projects+=($project)
    continue
  fi
  QUERY_COUNT=`curl -s --request POST "https://monitoring.googleapis.com/v3/projects/$project/timeSeries:query"   -H "Authorization: Bearer $(gcloud auth print-access-token)"   -H "Accept: application/json"   -H "Content-Type: application/json"   --data '{"query":"fetch compute.googleapis.com/Location | metric '"'"'compute.googleapis.com/global_dns/request_count'"'"' | filter metric.zonal_dns_readiness = '"'"'zonal_dns_risky'"'"' | every 30d | within 30d"}'   --compressed | jq --raw-output '.timeSeriesData[0].pointData[0].values[0].int64Value'`
  if [[ "$QUERY_COUNT" -ne "null" ]] && [[ "$QUERY_COUNT" -ne "0" ]]; then
    unready_projects+=($project)
  fi
done

error_len=${#inaccessible_projects[@]}
unready_len=${#unready_projects[@]}

echo -e "$error_len projects were inaccessible"
echo -e "$unready_len projects were not ready for migration"

if [ $error_len -ne 0 ]; then
  echo "Unable to access the following projects:"
  for project in "${inaccessible_projects[@]}"; do
    echo "$project"
  done
fi
if [ $unready_len -ne 0 ]; then
  echo "The following projects are not ready for migration:"
  for project in "${unready_projects[@]}"; do
    echo "$project"
  done
fi

if (( $error_len + $unready_len > 0 )); then
  echo "This folder is NOT ready for gDNS -> zDNS migration."
else
  echo "This folder is ready for gDNS -> zDNS migration."
fi

Replace FILENAME with the name of the file that you saved the project ID list in.

Convey the results of the migration readiness analysis with the project owners:

Enforce zonal DNS by default for new projects

If a new project is created under an organization that was created before September 6, 2018, by default the internal DNS type used by the project is global DNS. To isolate any failures in DNS registration to individual zones, you can enforce the organization policy constraints/compute.setNewProjectDefaultToZonalDNSOnly at an organization or folder level.

When you set an organization policy to override the default internal DNS type, newly created projects use zonal DNS by default. The organization policy doesn't impact existing projects where the Compute Engine API is already enabled. To switch existing projects to use zonal DNS, see switching existing projects to zonal DNS.

To enforce this policy change, you must have folder or organization level access with the IAM role roles/orgpolicy.policyAdmin.

Use the following steps to set the organization policy for a folder or organization.

  1. Sign in to the Google Cloud console as a Google Workspace or Cloud Identity super administrator.

  2. In the console, go to the Organization policies page.

    Go to Organization policies

  3. Select the folder or organization for which you want to view organization policies. The Google Cloud console displays a list of organization policy constraints that are available. The list might span multiple pages.

  4. To find the policy to enforce zonal DNS, click Filter and select Name, then set the filter name to Sets the internal DNS setting for new projects to Zonal DNS Only.

  5. Click the policy name to see its details.

    The policy details page provides information about the constraint and how the constraint is applied.

    By default, the enforcement is undefined for a folder or organization. However, if a parent folder has a defined enforcement, then the enforcement is inherited from the closest parent folder that has a defined enforcement. For more information, see Understanding hierarchy evaluation.

  6. To customize the organization policy, click Edit.

  7. In the edit page, select Customize.

  8. Under Enforcement, select On.

    This sets the default internal DNS type for all new projects in the organization to zonal DNS.

  9. Click Save.

To validate the organization policy change, you can create a new project under the folder or organization, then create and start a VM instance, and check whether your VM is enabled for zonal DNS.

If global DNS is needed to resolve a DNS name query built into your workload, you can rollback this change at the organization or folder level by disabling the enforcement.

Exempt folders not ready to migrate to zonal DNS

If there are only a few folders within an organization that aren't ready to migrate to zonal DNS, we recommend setting an organization policy at the organization level, but granting an exemption for folders that are not yet ready to migrate.

The following example shows an organization hierarchy where only one folder is not ready to migrate to zonal DNS.

organization/Example.com

  • folders/101
    • projects/301 (migration-ready)
    • folders/201
      • projects/303 (NOT ready)
      • projects/304 (migration-ready)
  • folders/102
    • projects/302 (migration-ready)

To exempt a folder from the organization policy, complete the following steps to set the enforcement option for the policy at the folder level to Off.

  1. Sign in to the Google Cloud console as a Google Workspace or Cloud Identity super administrator.

  2. In the console, go to the Organization policies page.

    Go to Organization policies

  3. Click Select, and then select the folders that you want to exempt from the organization policy.

    The Google Cloud console displays a list of organization policy constraints for that folder on one or more pages.

  4. To find the organization policy constraint that enforces zonal DNS:

    1. Click Filter.
    2. Select Name.
    3. Set the filter name to Sets the internal DNS setting for new projects to Zonal DNS Only.

  5. Click the organization policy constraint name to open the Policy details page.

  6. Click Edit.

  7. On the Edit page, select Customize.

  8. Under Enforcement, select Off to disable enforcement of the constraint. This means that the default internal DNS type for all projects in the folder are determined by the organization creation time.

  9. Click Save.

For more information about customizing organization policy constraints, see Customizing policies for boolean constraints in the Resource Manager documentation.

Project owner steps

As the project owner, you perform the following tasks during the migration from global DNS to zonal DNS:

Project owners can also complete these optional tasks:

Check if your project uses global DNS by default

Check your projects to see if they need to migrate from using global DNS to zonal DNS. You only need to migrate projects that are configured to use global DNS as the default for any internal DNS names created within the project.

Console

  1. In the Google Cloud console, go to the Metadata page.

    Go to Metadata

  2. On the Metadata tab, view the vmdnssetting setting. The value indicates whether the project uses global DNS by default.

    • GlobalDefault: the project has global DNS enabled.
    • ZonalOnly: the project has zonal DNS enabled. This project doesn't need to be migrated.

gcloud

    In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  1. Run the following gcloud CLI command to check the value of vmDnsSetting.

      gcloud compute project-info describe --project=PROJECT_ID --flatten="vmDnsSetting"

    Replace PROJECT_ID with the name of the project.

    The returned value indicates whether the project uses global DNS by default.

    • GLOBAL_DEFAULT: the project has global DNS enabled.
    • ZONAL_ONLY: the project has zonal DNS enabled. This project doesn't need to be migrated.

REST

Check the value of vmDnsSetting by using the projects.get method. This example uses a fields query parameter to only include the fields you want to view.

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID?fields=id,name,vmDnsSetting

Replace PROJECT_ID with the project ID.

The value of vmDnsSetting indicates whether the project uses global DNS by default.

  • GLOBAL_DEFAULT: the project has global DNS enabled.
  • ZONAL_ONLY: the project has zonal DNS enabled. This project doesn't need to be migrated.

Determine if your project is ready to migrate

In the Google Cloud console, on the VM instances page, if your project needs to migrate to zonal DNS, then you should see a notification banner about zonal DNS migration.

When you view the VM instances page for your project, if your project is ready for migration (compatible with zonal DNS), then the banner includes a recommendation to Use Zonal DNS. This recommendation is based on internal DNS usage in the project, but is limited to the last 100 days.

A screenshot of the ready to migrate to zonal DNS banner in the console

If your project is not ready to migrate to zonal DNS, a different banner appears.

A screenshot of the not ready for migration to zonal DNS banner in the console

To view your global DNS usage, click the View Global DNS Usage button. This takes you to the Logs Explorer page in the Google Cloud console. On this page you can view migration blocking query logs from the past 30 days. When you click the link in the banner, the Logs Explorer page is configured to use the logName filter to pull global DNS queries and relative log fields.

To view this information without using the button in the banner, do the following:

  1. In the Google Cloud console, go to the Logs Explorer page.

    Go to Logs Explorer

  2. Select the project.

  3. Apply the resource and log name filters:

    1. Click Resource.
    2. In the Select resource dialog, select VM Instance, and then click Apply.
    3. Click Log name.

    4. In the Select log names dialog, select gdnsusage, and then click Apply.

Alternatively, you can enter the following into the query field:

   resource.type="gce_instance"
   log_name="projects/PROJECT_ID/logs/compute.googleapis.com%2Fgdnsusage"

Track global DNS usage within the project

There are two metrics created for tracking project readiness to migrate to zonal DNS:

  • zonal_dns_ready: The aggregated number of queries completed over the specified time interval that can be resolved by using zonal DNS instead of global DNS.
  • zonal_dns_risky: The aggregated number of queries completed over the specified time interval that can't be resolved by using zonal DNS. For these queries, Compute Engine was unable to determine the relative internal IP address from the current hostname. If this metric has a non-zero value, then the project is not ready for migration.

The time interval used by these metrics is a 100 day period.

To view these metrics, use the Metrics Explorer in the Google Cloud console.

  1. In the Google Cloud console, go to the  Metrics explorer page:

    Go to Metrics explorer

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. On the right side of the toolbar that contains the Select a metric field, click Code editor, MQL, or PromQL.

  3. If the query input field isn't titled MQL Query, then in the lower right corner of the query input field, for Language, select MQL.

  4. In the query input field, enter the following text exactly as it appears:

    fetch compute.googleapis.com/Location
| metric 'compute.googleapis.com/global_dns/request_count'
| every 1d
| within 7d

  5. Click the Run query button.

    The Google Cloud console displays a chart of the two metrics (zonal_dns_ready and zonal_dns_risky) and the corresponding number of queries made over the time period for each metric.

    Screenshot of the chart for global DNS usage metrics

  6. Check the value for the zonal_dns_risky metric.

    • If the value is 0, then the project is ready for migration to zonal DNS. You can migrate the project, as described in Migrate projects that are ready to zonal DNS.
    • If the value is a non-zero number, such as 0.02k as shown in the previous screenshot, then there are some queries that might not work after you migrate to zonal DNS. The project is not ready for migration. Continue with the steps in Modify projects that are not ready to migrate.

Migrate projects that are ready to zonal DNS

In general, you can use the following migration process:

  1. Check your applications and update them to resolve compatibility issues with zonal-only settings:
    • If you have any application that uses hard-coded global DNS names, update them to use zonal DNS names.
    • If any application uses VM names to access VMs in another zone, make sure that the destination zone name is added in the query, for example: DESTINATION_VM_NAME.DESTINATION_ZONE_NAME.
    • To resolve DNS names of VMs in service projects that use a Shared VPC network, you must use the zonal fully qualified domain name (FQDN) of the VMs.

  2. Click the Use Zonal DNS button in the banner on the VM instances page of the Google Cloud console. This changes the project metadata to use zonal DNS.

    Alternatively, you can manually modify your projects to use zonal DNS by default, as described in Manually update projects and VMs to use zonal DNS and Prevent new projects from using global DNS by default.

Manually update projects and VMs to use zonal DNS

After you have determined that your project is ready to migrate to zonal DNS, you can configure the project and VMs to use only zonal DNS names by updating their metadata. Enable zonal DNS for your VMs by setting the vmDnsSetting metadata entry for either the project or the VM. If you set the vmDnsSetting metadata entry for a specific VM, it overrides any vmDnsSetting setting inherited from the project metadata for that VM.

The vmDnsSetting variable supports the following settings:

  • Recommended: Set vmDnsSetting=ZonalOnly in the project metadata. This means your VMs can be accessed only by their zonal DNS names (VM_NAME.ZONE.c.PROJECT_ID.internal) when using search paths. The VMs still retain both the zonal and global search paths, but their global DNS names, which don't include the ZONE as part of the internal DNS name, no longer function. Only VMs in the same zone and same project can access each other using the global name when this setting is in place. Other VMs can access VMs with vmDnsSetting set to ZonalOnly using only their zonal DNS names and can't access these VMs using their global DNS names or search paths. This is the preferred and more reliable option as long as your applications can support it.

    The setting vmDnsSetting=ZonalOnly is the default setting for VMs in standalone projects and projects created in an organization that enabled the Compute Engine API after September 6, 2018.

  • Set vmDnsSetting=GlobalDefault so that VMs register both global and zonal DNS names, but use only global DNS names as default domain names and search path entries. This is the default setting for VMs in standalone projects and projects created in an organization that enabled the Compute Engine API before September 6, 2018.

For information about how to set project metadata or VM metadata values, see Setting custom metadata.

After you configure the vmDnsSetting metadata entry for a VM, refresh the DHCP lease on the VM. You can refresh the lease by restarting the VM, or by waiting for the lease to expire, or by running one of the following commands:

  • Linux VMs: sudo dhclient -v -r
  • Windows Server VMs: ipconfig /renew

After refreshing the DHCP lease, check whether your VM is enabled for zonal DNS.

Modify projects that are not ready to migrate

A project that is not ready to migrate means that at least one risky DNS query was made within a certain amount of time, such as the last 30 days or the last 100 days. A risky query is a call to a VM using a global DNS name (VM_NAME.c.PROJECT_ID.internal) that can't be seamlessly resolved using a zonal DNS name (VM_NAME.ZONE.c.PROJECT_ID.internal) instead. Risky queries might have the following attributes:

  • Make a call to a VM in a different project
  • Make a call to a VM in a different zone

For projects with risky queries, usually some extra work is needed to eliminate all the risky DNS lookups ahead of the migration.

For those projects that aren't ready for migration, complete the following steps:

  1. Enable search path improvement to resolve cross-zonal DNS name lookups. This might make your projects ready for migration without affecting your resources.
  2. If the search path adjustment does not transition all of your cross-zonal queries, you can manually update the queries to use zonal DNS names.

About the search path improvement feature

By default, most Linux distributions store DHCP information in resolv.conf. The following is a minimal global resolv.conf file:

domain c.PROJECT_ID.internal
search c.PROJECT_ID.internal. google.internal.
nameserver 169.254.169.254

The search configuration option is used to list the host names to use when performing DNS resolution. The DNS server attempts to resolve the query using each of the host names in the search path in turn until a DNS record match is found. For example, if a VM calls ping my-vm, the domains in the search path are appended to the original query to get the fully-qualified domain names (FQDN), for example, using my-vm.c.PROJECT_ID.internal. If there is a match, the DNS resolver returns an internal IP address in the answer. Otherwise it attempts to resolve the DNS name using the next domain in the search path.

Adding extra zonal domains in the VM search path can make some projects ready for migration. For example, assume your VM is located in the us-west4-c zone. This VM makes a call to another VM named myapp-vm which is located in the us-west4-b zone.

  • When you make a call to the VM as myapp-vm, Compute Engine attempts to resolve the DNS name by using a FQDN that appends one of the search paths to the VM name, such as myapp-vm.c.PROJECT_ID.internal.
  • If the target VM uses zonal DNS, the FQDN for the target VM is registered as myapp-vm.us-west4-b.c.PROJECT_ID.internal. As a result, the DNS name can't be resolved.
  • If you add us-west4-b.c.PROJECT_ID.internal to the search list, the DNS name myapp-vm can then be resolved when us-west4-b.c.PROJECT_ID.internal is appended to the VM name.

Google provides a search path improvement feature that automatically searches for the zonal DNS name for a VM in all zones in the same region as the VM. As a result, cross-zonal queries can be resolved without requiring an update to the resolv.conf file or your DHCP policy. Search path improvements can work to resolve in-region cross-zonal queries up to 80% of the time. This feature should help the majority of projects with risky queries become ready to migrate to zonal DNS.

Enable search path improvement to resolve cross-zonal DNS name lookups

Use the following steps to enable the search path improvement feature.

  1. Run the project-info add-metadata command as follows.

    gcloud compute project-info add-metadata  \
    --metadata=enable-search-path-improvement=true

  2. Allow the project to use this setting for a few days and then check the monitoring metric to see if there are still any risky or cross-zonal global DNS queries.

    • If the value for the project is 0, then the project is now ready to migrate.
    • If the project returns a non-zero value, change all global DNS query names to use the zonal FQDN, as described in the next section.

Manually update the queries to use zonal DNS names

If using the search path improvement feature to add extra zonal domains in the DNS name search path does not transition all of your cross-zonal queries, you can use Logs Explorer to view your global DNS usage within the project.

To identify the global DNS queries that need to be manually changed to use zonal fully-qualified domain names (FQDN) instead, complete the following steps:

  1. Follow the steps in Determine if your project is ready to migrate to view your global DNS usage for a project. Use the Logs Explorer to access and query the global DNS usage for VMs in your project.

  2. In the Query results pane, each query has a jsonPayload field. Each jsonPayload field contains the following information:

    • The source VM name, its project ID, and zone name.
    • The destination VM name, its project ID, and zone name.

    • A debug message that provides information about how to update the global DNS query that can't be resolved by using zonal DNS names. These are considered migration blocking queries that you should debug and fix.

      "To use Zonal DNS, update the Global DNS query sent from the source VM
VM_NAME.c.PROJECT_ID.internal to the following zonal
FQDN: VM_NAME.ZONE.c.PROJECT_ID.internal"

    • A query count that shows how many migration blocking queries the source VM sends to the destination VM for that day.

    The following screenshot shows the jsonPayload field information in the Logs Explorer console page.

    Screenshot of the jsonPayload in the gdnsusage log query results

  3. Use the information in the jsonPayload to determine what FQDN to use to manually update your global DNS queries to use zonal DNS, and make the project ready for migration. The most common use cases for where to update the FQDN and resolve compatibility are:

    • Internal DNS names from the metadata server: No action is required because the returned DNS name will change to a zonal FQDN immediately after the migration to zonal DNS. If the DNS name is cached, you just need to make one more call to update the cache value.
    • Internal DNS names used to access VMs in another region: If you have an application that uses the internal DNS names for VMs in different regions, you can modify the DHCP policy or configuration file to include the zone in the other region.
    • Hard-coded global FQDN: If you have an application that uses hard-coded global FQDN names for VMs, you can update the call within the application to use the internal DNS name or zonal FQDN instead. You can make this change through a code change or configuration change in Terraform.
    • VMs in service projects that use a Shared VPC network: To resolve DNS names of VMs in service projects that use a Shared VPC network, you must use the zonal FQDNs of the VMs.

After you have updated the global DNS queries to use zonal DNS:

  1. Use the Logs Explorer page to query the global DNS usage again. After you fix all blocking global DNS queries, there should be no debug logs displayed in the query results.
  2. Recheck the monitoring metric to see if all risky DNS queries have been removed.

Verify the project migration to zonal DNS is complete

  1. Repeat the steps in Check if your project uses global DNS by default.

  2. To test that the project metadata has been updated, you can run the following command:

    gcloud compute project-info describe --flatten="vmDnsSetting"

    The command should return ZONAL_ONLY.

  3. Check that the internal DNS name for a VM uses a zonal DNS name.

  4. To verify that the organization policy constraints/compute.setNewProjectDefaultToZonalDNSOnly is being enforced, you can:

    1. Create a new project under the folder or organization.
    2. Create and start a VM instance.
    3. Check whether the VM uses a zonal DNS name.

Revert to using global DNS

You can undo the migration to zonal DNS by changing the default internal DNS type back to global DNS. You can do this at the organization, project, VM, or container level.

Revert to using global DNS for an organization or folder

To revert an organization or folder to using global DNS, complete the following steps.

  1. Disable the organization policy constraints/compute.setNewProjectDefaultToZonalDNSOnly at the organization or folder level. For instructions on how to modify this policy, see Enforce zonal DNS by default for new projects.

    Set the enforcement of Sets the internal DNS setting for new projects to Zonal DNS Only to Off.

  2. If you want to revert to using global DNS for the entire organization, verify that none of the folders in the organization are enforcing the organization policy constraints/compute.setNewProjectDefaultToZonalDNSOnly.

  3. Use the following sections to check that global DNS is configured for projects, VMs, and containers.

Revert to using global DNS for a project

To revert a project to using global DNS, complete the following steps.

  1. Add the following to the project's metadata: vmDnsSetting=GlobalDefault.

    For information about how to set project metadata or VM metadata values, see Setting custom metadata.

  2. Verify that none of the VMs in the project have the vmDnsSetting metadata value set to ZonalOnly.

  3. Refresh the DHCP lease on each VM. You can refresh the lease by restarting the VM, waiting for the lease to expire, or by running one of the following commands:

    • Linux VMs: sudo dhclient -v -r
    • Windows Server VMs: ipconfig /renew

Revert to using global DNS for a VM

To revert a specific VM to using global DNS, complete the following steps.

  1. Add the following to the VM's metadata: vmDnsSetting=GlobalDefault.

    For information about how to set VM metadata values, see Setting custom metadata.

  2. To force the DNS configuration change, restart the network of the VM using one of the following commands:

  • For Container-Optimized OS or Ubuntu:

    sudo systemctl restart systemd-networkd

  • For CentOS, RedHat EL, Fedora CoreOS, or Rocky Linux:

    sudo systemctl restart network

    or

    sudo systemctl restart NetworkManager.service

  • For Debian:

    sudo systemctl restart networking

  • For Linux systems with nmcli:

    sudo nmcli networking off
sudo nmcli networking on

  • For Windows:

    ipconfig /renew

Revert to using global DNS for a container

If you run your application or workload in containers, on Google Kubernetes Engine, or on App Engine flexible environment, the DNS configuration in your container settings might not automatically update until you restart the containers. To disable zonal DNS on these container apps, complete the following steps.

  1. Set the project metadata setting vmDnsSetting to GlobalDefault on the projects that own the containers and VMs.

  2. Restart the containers so that their DNS settings revert to the original state.

Troubleshoot the global DNS to zonal DNS migration process

If you have issues with the migration process, refer to the troubleshooting guide.

Hide the zonal DNS migration banner in the Google Cloud console

You can click the Dismiss button in the zonal DNS migration notification banner that appears on the VM instances page of the Google Cloud console. This prevents the banner from appearing for the project indefinitely.

If you dismissed the banner, but want it to appear again, contact Cloud Customer Care for assistance.

What's next