Security

Google Distributed Cloud Hosted (GDCH) is a turnkey integrated hardware and software solution designed to operate in air-gapped environments, disconnected from the public Internet. It has been architected from the ground up to handle the most secure applications in the world, including workloads that need to answer strict data and operational sovereignty requirements. This document highlights the key security attributes of GDCH in the following categories:

GDCH takes a security-first approach with multiple layers of security to deliver maximum control and maintain compliance with statutory regulations and safeguard confidential data. It is designed to run on dedicated and secured hardware in a local data center to provide strict tenant isolation. The layers of security provided by GDCH includes hardware security, identity and access management, Hardware Security Module key management, support for external identity providers based on SAML or OIDC for federation of user accounts, and a secure software supply chain.

Google, or your operating partner, is responsible for security other layers of the infrastructure. The customer is responsible for securing the project setup and application layer, including application containers, base images, and dependencies.

The GDCH shared responsibility model is as follows:

  • Customer responsibility: Responsible for application containers such as base image, dependencies, and application code.
  • Google+ Infrastructure Operator responsibility: Responsible for Kubernetes system containers that include logging and Domain Name System (DNS), Kubernetes on the node and control plane, container runtime, operating system (OS) userspace, Linux kernel, firmware and bootloader, virtualization, hardware, networking, and physical security.

From a security perspective, there are three separate roles for GDCH:

  • The Infrastructure Operator (IO) who manages the day-to-day operations of the system infrastructure, but has no access to customer data. This role is the highest level administrator of the system and could be Google, a contracted third party or the customer, depending on the nature of sovereignty restrictions.

  • The Platform Administrator (PA) is a customer role that manages resources and permissions for projects. This is the highest level of administrative privilege granted to the customer and is the only administrative level that can grant access to customer data.

  • The Application Operator (AO) who develops applications to deploy and execute, and configures granular resources and permissions on GDCH. As such, this user holds the most accountability for ensuring that security compliance controls and requirements for which the Customer is responsible (i.e., the CRM) are met.

Facility security

GDCH is deployed in customer-designated, air-gapped data centers so location-specific physical security varies based on individual customer requirements. For data centers that Google is responsible for managing, physical security controls match those of Google's public cloud data centers and include controls such as security cameras, access cards, and physical incident management process that meet ISO 27001 and FedRAMP certification.

Hardware security

All GDCH hardware is purchased and assembled from hardware partners who are certified under a customer's specific sovereignty requirements. Hardware assembly is done in certified centers that have workers who are staff meeting customers requirements for supply chain security. These partnerships also include specialized resources for warranty and repair as well as specifications for secure decommission and data destruction. Google partners with similarly credentialed hardware vendors in other countries where required.

Node security

Secure boot and host operating system

GDCH uses a secure network boot mechanism with a custom-packaged image of Linux that is FIPS 140-2 certified, passes DISA STIG evaluation, and meets security and compliance requirements with respect to IL5 and IL6. SSH access for administration is managed with a custom certificate authority (CA) in the root admin cluster, unique host keys for each host and key rotation.

Antivirus and malware detection

GDCH provides separate anti-virus and anti-malware detection systems for the OS running on the bare metal nodes and a solution to scan the storage systems. Google packages updates for these scanners and they are included in regular system updates that are applied by the IO using the GDCH upgrade process. Detection alerts are sent to the IO.

Network security

Physical network

While GDCH is designed to be used in disconnected environments, in practicality, GDCH systems are likely to be connected to a wider protected environment that is connected across a customer's internal private network, such as the Defense Information Systems Network, while still remaining disconnected from the public Internet.

All network traffic between the customer's private network and GDCH systems passes through a hardware Firewall and Intrusion Detection & Prevention systems (IDS/IPS). This firewall provides the first line of defense to control access to the system. The firewall controls inbound and outbound traffic based on configurable security policies and uses machine learning to automatically analyze traffic to block unauthorized entry. It can also integrate with the customer's security information and event management (SIEM) systems for additional security analysis and forensics. For data exfiltration prevention, network egress traffic is blocked by default, although this behavior can be changed by specifying alternate egress rules.

Inside GDCH, there are two physically separate networks to isolate the administration and control functions from customer workload data. The control plane network is solely used for administrative functions such as logging and configuration changes of the network devices, storage appliances, and other hardware components and is accessible only by the Infrastructure Operator. Network access control lists (ACL) are configured so that only those servers that are in the root admin cluster controlled by the IO are granted access to this network. The data plane network connects the workload servers together and is only accessible by the Platform Administrator and Application Operators.

Denial of service protection

All traffic into GDCH passes through the IDS/IPS firewall and a set of load balancers. The firewalls detect and drop unauthorized traffic and the load balancers can throttle traffic appropriately to avoid loss of service. Because any external connectivity is handled through the customer's private network, it is up to the customer to provide an extra layer of denial of service protection for distributed attacks.

Encryption

At rest

All GDCH data that is stored on persistent storage is encrypted at rest using multiple levels of encryption. All block storage is encrypted with tenant isolated keys that are protected by FIPS 140-2 level 3 hardware security modules (HSM). Drives are also self-encrypted with keys protected by the HSMs. Object storage also supports S3 SSE and SSE-C object encryption methods. For object storage used in multi-tenant environments, it is strongly recommended to use client-side encryption. See Encryption at rest for details.

In transit

All GDCH traffic is encrypted by default using FIPS 140-2 certified encryption and industry standard protocols such as TLS 1.2+, HTTPS, or IPSec tunnels. Because securing the application layer is the customer's responsibility in the shared responsibility model, it is up to the customer to ensure that encryption requirements are met for all applications deployed on top of GDCH.

Secure software development life cycle

To protect against software supply chain attacks, all GDCH software is developed in accordance with the best practices recommended by Supply chain Levels for Software Artifacts (SLSA) supply chain security framework developed by Google in partnership with organizations including the CNCF and the Linux Foundation.

SLSA is a checklist of standards and controls to prevent tampering, improve integrity, and secure packages within the software supply chain. It is designed to prevent exploits that attack source code repositories, build systems and packages of third-party dependencies. All GDCH source code and dependencies are stored in a Google version control system that is isolated and encrypted in secure Google data centers. All code has a verified history and requires a second person to review code changes before those changes are accepted into the version control system.

GDCH Secure Software Development Life Cycle Builds are done using a Google release orchestration system that leverages Google Cloud Build and Google automated test tools for a fully automated build, test and release of container images and binaries. No individual has unilateral access in the software delivery process. All images are scanned for vulnerabilities using multiple application security scanners and are signed using a combination of public and private keys and SHA256 checksums. Each release contains a Software Bill of Materials (SBOM) that includes details about the software components, dependencies and libraries, and data delivered.

Secure operations

Identity and access

Access to systems in GDCH is based on the rules of least privilege. No operators have unilateral access to customer systems. The Infrastructure Operator has access to the layers they need to administer the systems but do not have access to customer data. The Platform Administrator and the Application Operators have access to appropriate customer data, but not to system layers. GDCH uses Anthos Identity Service (AIS) as the core identity management infrastructure. AIS supports both SAML and OIDC for simplified centralization of identity. Role Based Access Control (RBAC) is used to authorize all control plane and data plane API access and OPA Gatekeeper is used to enforce IO and PA defined custom policies.

Logging and monitoring

GDCH ships with an observability platform that provides monitoring, logging, tracing and reporting functionalities for the GDCH platform, services, and workloads. The logging subsystem ingests and aggregates logs from all core GDCH platforms and services as well as customer workloads into a time series database and makes that data available via an interactive UI and APIs. Critical audit log events include:

  • User authentication and authorization requests
  • Generation and revocation of authentication tokens
  • All Administrative access and changes
  • All control plane API actions
  • All security events (IDS, IPS, Firewall, Anti-virus)

Emergency access

An IO does not have access to customer data. However, in emergency situations, there might be cases where such access is needed. In those cases, GDCH has a set of emergency access procedures that allow a PA to explicitly grant access to an IO for troubleshooting. For example, in a case where the identity and access management configuration has been accidentally corrupted and is preventing user access, an IO uses an SSH certificate-based authentication through secured work stations and requires multi-party approval to establish access. All taken actions are tracked and auditable. Additionally, because all customer data is encrypted using keys managed by a customer, the customer's data is secure even during emergency access scenarios. After an emergency access incident, the SSH keys are changed.

Disaster recovery

GDCH ships with a backup system that lets PAs create and manage backup policies for clusters, namespaces, or virtual machines. The backup system contains typical systems for scheduling, running backups, restoring data, and reporting.

Security incident response

In the event that vulnerabilities are discovered in code that has been deployed to GDCH or there is an incident that has occurred within the system, there is a set of incident response processes that are executed by an IO as well as the GDCH security team to mitigate, create, and apply security patches, and communicate to affected parties. This incident management process parallels Google's standard incident management process with modifications for the disconnected nature of GDCH. For each CVE or incident that is discovered, an incident commander (IC) is assigned to run a response process. The IC is responsible for managing the process, which includes verifying and validating the incident, assigning the vulnerability a CVSS score and managing the process of getting a patch created and ready for delivery and creating appropriate communications.

Upgrade and update system

Since GDCH is deployed in an air-gapped environment, an Infrastructure Operator processes all updates to customer systems. Google posts binary updates in a secure location for the IO to download. The IO validates the binaries and then moves them into the GDCH for distribution and deployment. This system includes all software and firmware updates, including updates to the GDCH software, the node-level OS, anti-virus and malware definitions and signatures, storage and network appliance and device updates, and any other binary updates.

Compliance certifications

GDCH is currently going through the certification processes for FedRAMP, ISO 27001/27017, NIST 800-53, IL5, and IL6.