Unable to resolve internal DNS names

Problem

Your Google Compute Engine instance is unable to resolve internal DNS after altering the name server configuration or configuring an outbound server policy. Existing Cloud DNS zones also seem to not be respected.

However, querying directly against the metadata server (169.254.169.254) works.

dig example.com @169.254.169.254

Environment

  • Google Compute Engine instance using an alternative name server

Solution

  1. Configure the alternative name server to forward *.internal DNS to the metadata server.
  2. To restore existing Cloud DNS zones, forward configured domains to the metadata server.

Cause

By default, Google Compute Engine instances use the metadata server to resolve records in Compute Engine internal DNS (the internal zone). Modifying the default DNS server will break internal DNS resolution and existing Cloud DNS zones unless your replacement DNS server forwards these queries to the metadata server. Per the VPC name resolution, when an outbound server policy exists within a VPC, VMs within that VPC will solely rely on the alternative server for all DNS queries.