Overview

Service Directory is a Service covered by Google's obligations set forth in the Data Processing and Security Terms.

Service Directory is a single place to publish, discover, and connect to services in a consistent and reliable way, regardless of their environment. Service Directory supports services in Google Cloud, multi-cloud, and on-premises environments and can scale up to thousands of services and endpoints for a single project.

Service Directory has the following features:

  • A Registration and Lookup API for creating and resolving namespaces, services, and endpoints
  • Integration with Cloud DNS. Service Directory zones allow services to be made available on Virtual Private Cloud (VPC).
  • IAM Integration to assign and control service visibility and permissions
  • Built-in gcloud command-line interface and Cloud Console support for interacting with Service Directory
  • Cloud Monitoring and Cloud Logging integration for monitoring, auditing and debugging Service Directory operations

Why Service Directory

As applications adopt services, being able to resolve the location of a service becomes more difficult as the endpoints of those services change. Services deployed across hybrid environments present additional obstacles as neither may share the same naming system, making resolving and connecting services challenging. To illustrate the problem, consider the following.

Imagine that you are building a simple API and that your code needs to call some other application. When endpoint information remains static, you can hard-code these locations into your code or store them in a small configuration file. However, with microservices and multi-cloud, this problem becomes much harder to solve as instances, services, and environments can all change.

Service Directory without a load balancer (click to enlarge)
Different changing services (click to enlarge)

With Service Directory, you can register all of your services in a single place and resolve them via HTTP, gRPC, and DNS.

Let's revisit the previous diagram, but this time adding Service Directory. Below each service instance is registered with Service Directory. These registrations are immediately reflected in DNS and can be queried via HTTP/gRPC regardless of their implementation and environment.

Service Directory with a load balancer (click to enlarge)
Service Directory with a load balancer (click to enlarge)

You can create a universal service name that works across Google Cloud products, like App Engine and GKE. You can make these services available over DNS. You can apply access controls to services based on on network, project, and IAM roles of serviceaccounts.

Service Directory solves the following problems:

  1. Interoperability: Service Directory is a universal naming service that works across Google Cloud, multi-cloud, and on-premises. You can migrate services between these environments and still use the same service name to register and resolve endpoints.
  2. Service management: Service Directory is a managed service. Your organization doesn't have to worry about the high availability, redundancy, scaling, or maintenance concerns of maintaining your own service registry.
  3. Access Control: With Service Directory, you can control who can register and resolve your services using IAM. Assign Service Directory roles to teams, service accounts, and organizations.
  4. Limitations of pure DNS: DNS resolvers can be unreliable in terms of respecting TTLs and caching, cannot handle larger record sizes, and do not offer an easy way to serve metadata to users. In addition to DNS support, Service Directory offers HTTP and gRPC APIs to query and resolve services.

Key Concepts

Endpoint

An endpoint is an individual IP/port pair fronting a service that can handle requests. Any given service can have zero or more endpoints. The endpoint can be a VM, container, load balancer, or other entity capable of handling the requests. An example would be a cluster of user-managed Redis servers.

Endpoints can have optional metadata, in the form of key:value pairs, that can be used by clients. For example, a Redis service might have metadata like replica:server. Metadata can be used to store URLs. You can use a tool such as jq to pull the new URL out of the response.

For Service Directory, an endpoint can only exist within a service and is identified with an endpoint name that is a string identifier that is unique to that service.

Service

A service is a collection of endpoints (IP/ports) that provide a set of behaviors. Clients look up a service by its name and then connect to the endpoints that provide that service. Services can also have optional metadata (key-value pairs) associated with them (for example, use_https:true).

A service must belong to a namespace. Each service name must be unique within that namespace.

Namespace

A namespace holds a set of service names along with their endpoint information. A namespace is a means of grouping services together for easier management.

Namespaces are regional resources and belong to a project. The name of a namespace must be unique for that project and region.

Although a namespace is a regional resource, the services that belong to that namespace can exist anywhere and be queried from anywhere. We recommend customers choose a region that maps to where their endpoints are running.

Service Directory zone

A Service Directory zone is a specific type of Cloud DNS private zone whose complete and authoritative source of information is a Service Directory namespace. It allows users to look up Service Directory data using DNS requests.

If a namespace is attached to a Service Directory zone, then all services in that namespace are visible to all networks that are allowed to query the zone.

Using Cloud DNS with Service Directory

Cloud DNS is a fast, scalable, and reliable Domain Name System (DNS) service running on Google's infrastructure.

In addition to public DNS zones, Cloud DNS also provides a managed internal DNS solution for private networks on Google Cloud. Private DNS zones enable you to internally name your virtual machine (VM) instances, load balancers, or other resources. DNS queries for those private DNS zones are restricted to your private networks.

The diagram below illustrates how you can use Service Directory zones to make service names available via DNS lookups.

Using Cloud DNS with Service Directory (click to enlarge)
Using Cloud DNS with Service Directory (click to enlarge)

Overview of the individual components:.

  1. The endpoints are registered directly with Service Directory via the Service Directory API. This can be done for both GCP and non-GCP services.
  2. Both external and internal clients can look up those services at: https://servicedirectory.googleapis.com
  3. To enable DNS requests, create a Service Directory zone in Cloud DNS that is associated with a Service Directory namespace.
  4. Internal clients can resolve this service via DNS, HTTP, and gRPC. External clients (clients not on the private network) must use HTTP or gRPC to resolve service names.

Example configuration

How to Expose a Service over DNS

The below diagram illustrates how a microservice architecture is modeled in Service Directory and made available via DNS. Notice that the services and endpoints are maintained entirely by Service Directory, but the private zone is in Cloud DNS.

Exposing a service over DNS (click to enlarge)
Exposing a service over DNS (click to enlarge)

In the above diagram (left side), the payments service is registered to a namespace with the name backend-namespace, the region us-east1, and the project gcp-project. The namespace is linked to the private zone example.com.

To do a DNS lookup, the client requests the SRV records for the domain name _payments._tcp.payments.example.com, which resolves to the port numbers and address records for the payment service's endpoints.

What's next