ContainerAttachedCluster

Property Value
Google Cloud Service Name Anthos Multi-Cloud
Google Cloud Service Documentation /anthos/clusters/docs/multi-cloud/attached
Google Cloud REST Resource Name v1.projects.locations.attachedClusters
Google Cloud REST Resource Documentation /anthos/clusters/docs/multi-cloud/reference/rest/v1/projects.locations.attachedClusters
Config Connector Resource Short Names gcpcontainerattachedcluster
gcpcontainerattachedclusters
containerattachedcluster
Config Connector Service Name gkemulticloud.googleapis.com
Config Connector Resource Fully Qualified Name containerattachedclusters.containerattached.cnrm.cloud.google.com
Can Be Referenced by IAMPolicy/IAMPolicyMember No
Config Connector Default Average Reconcile Interval In Seconds 600

Prerequisites

Before you can use this resource, you must prepare the target cluster so that the multi-cloud service can connect to the target cluster. To prepare the cluster, follow the steps to deploy an install-agent into the target cluster:

  1. Get the manifest yaml file for the install-agent:

gcloud container attached clusters generate-install-manifest my-cluster --location=GOOGLE_CLOUD_REGION --platform-version=PLATFORM_VERSION --output-file=manifest.yaml

Example command:

gcloud container attached clusters generate-install-manifest kcc-attached-cluster --location=us-west1 --platform-version=1.25.0-gke.5 --output-file=manifest.yaml

  1. Check out the target cluster and get the kubeconfig context:

Amazon Elastic Kubernetes Service cluster: aws eks update-kubeconfig --region $AWS_REGION --name $CLUSTER

Azure Kubernetes Service cluster: az aks get-credentials -n $CLUSTER -g $AZURE_RESOURCE_GROUP

export KUBECONFIG_CONTEXT=$(kubectl config current-context)

  1. Apply the manifest.yaml file to the target cluster:

(Optional if you used the previous command to switch context) kubectl use-context $KUBECONFIG_CONTEXT

kubectl apply -f manifest.yaml

You should see the following logs:

 namespace/gke-install created
 serviceaccount/gke-install-agent created
 clusterrolebinding.rbac.authorization.k8s.io/multicloud-install-agent-admin created
 deployment.apps/gke-multicloud-agent created
  1. Switch back to the Google Kubernetes Engine(GKE) cluster with Config Connector installed:

Run kubectl config get-contexts to see all configured contexts. You should see at least two contexts: one associated with the target cluster, and one associated with the GKE cluster with Config Connector installed.

Run kubectl config use-context GKE_CONTEXT to switch back to the GKE context.

Custom Resource Definition Properties

Annotations

Fields
cnrm.cloud.google.com/state-into-spec

Spec

Schema

annotations:
  string: string
authorization:
  adminUsers:
  - string
binaryAuthorization:
  evaluationMode: string
deletionPolicy: string
description: string
distribution: string
fleet:
  membership: string
  projectRef:
    external: string
    name: string
    namespace: string
location: string
loggingConfig:
  componentConfig:
    enableComponents:
    - string
monitoringConfig:
  managedPrometheusConfig:
    enabled: boolean
oidcConfig:
  issuerUrl: string
  jwks: string
platformVersion: string
projectRef:
  external: string
  name: string
  namespace: string
resourceID: string
Fields

annotations

Optional

map (key: string, value: string)

Optional. Annotations on the cluster. This field has the same restrictions as Kubernetes annotations. The total size of all keys and values combined is limited to 256k. Key can have 2 segments: prefix (optional) and name (required), separated by a slash (/). Prefix must be a DNS subdomain. Name must be 63 characters or less, begin and end with alphanumerics, with dashes (-), underscores (_), dots (.), and alphanumerics between.

authorization

Optional

object

Configuration related to the cluster RBAC settings.

authorization.adminUsers

Optional

list (string)

Users that can perform operations as a cluster admin. A managed ClusterRoleBinding will be created to grant the 'cluster-admin' ClusterRole to the users. Up to ten admin users can be provided. For more info on RBAC, see https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles.

authorization.adminUsers[]

Optional

string

binaryAuthorization

Optional

object

Binary Authorization configuration.

binaryAuthorization.evaluationMode

Optional

string

Configure Binary Authorization evaluation mode. Possible values: ["DISABLED", "PROJECT_SINGLETON_POLICY_ENFORCE"].

deletionPolicy

Optional

string

Policy to determine what flags to send on delete.

description

Optional

string

A human readable description of this attached cluster. Cannot be longer than 255 UTF-8 encoded bytes.

distribution

Required

string

Immutable. The Kubernetes distribution of the underlying attached cluster. Supported values: "eks", "aks".

fleet

Required

object

Fleet configuration.

fleet.membership

Optional

string

The name of the managed Hub Membership resource associated to this cluster. Membership names are formatted as projects//locations/global/membership/.

fleet.projectRef

Required

object

The number of the Fleet host project where this cluster will be registered.

fleet.projectRef.external

Optional

string

Allowed value: string of the format `projects/{{value}}`, where {{value}} is the `number` field of a `Project` resource.

fleet.projectRef.name

Optional

string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

fleet.projectRef.namespace

Optional

string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

location

Required

string

Immutable. The location for the resource.

loggingConfig

Optional

object

Logging configuration.

loggingConfig.componentConfig

Optional

object

The configuration of the logging components.

loggingConfig.componentConfig.enableComponents

Optional

list (string)

The components to be enabled. Possible values: ["SYSTEM_COMPONENTS", "WORKLOADS"].

loggingConfig.componentConfig.enableComponents[]

Optional

string

monitoringConfig

Optional

object

Monitoring configuration.

monitoringConfig.managedPrometheusConfig

Optional

object

Enable Google Cloud Managed Service for Prometheus in the cluster.

monitoringConfig.managedPrometheusConfig.enabled

Optional

boolean

Enable Managed Collection.

oidcConfig

Required

object

OIDC discovery information of the target cluster. Kubernetes Service Account (KSA) tokens are JWT tokens signed by the cluster API server. This fields indicates how GCP services validate KSA tokens in order to allow system workloads (such as GKE Connect and telemetry agents) to authenticate back to GCP. Both clusters with public and private issuer URLs are supported. Clusters with public issuers only need to specify the 'issuer_url' field while clusters with private issuers need to provide both 'issuer_url' and 'jwks'.

oidcConfig.issuerUrl

Required

string

Immutable. A JSON Web Token (JWT) issuer URI. 'issuer' must start with 'https://'.

oidcConfig.jwks

Optional

string

Immutable. OIDC verification keys in JWKS format (RFC 7517).

platformVersion

Required

string

The platform version for the cluster (e.g. '1.23.0-gke.1').

projectRef

Required

object

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

projectRef.external

Optional

string

Allowed value: The `name` field of a `Project` resource.

projectRef.name

Optional

string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

projectRef.namespace

Optional

string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

resourceID

Optional

string

Immutable. Optional. The name of the resource. Used for creation and acquisition. When unset, the value of `metadata.name` is used as the default.

Status

Schema

clusterRegion: string
conditions:
- lastTransitionTime: string
  message: string
  reason: string
  status: string
  type: string
createTime: string
errors:
- message: string
kubernetesVersion: string
observedGeneration: integer
reconciling: boolean
state: string
uid: string
updateTime: string
workloadIdentityConfig:
- identityProvider: string
  issuerUri: string
  workloadPool: string
Fields
clusterRegion

string

Output only. The region where this cluster runs. For EKS clusters, this is an AWS region. For AKS clusters, this is an Azure region.
conditions

list (object)

Conditions represent the latest available observation of the resource's current state.
conditions[]

object
conditions[].lastTransitionTime

string

Last time the condition transitioned from one status to another.
conditions[].message

string

Human-readable message indicating details about last transition.
conditions[].reason

string

Unique, one-word, CamelCase reason for the condition's last transition.
conditions[].status

string

Status is the status of the condition. Can be True, False, Unknown.
conditions[].type

string

Type is the type of the condition.
createTime

string

Output only. The time at which this cluster was created.
errors

list (object)

A set of errors found in the cluster.
errors[]

object
errors[].message

string

Human-friendly description of the error.
kubernetesVersion

string

The Kubernetes version of the cluster.
observedGeneration

integer

ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource.
reconciling

boolean

If set, there are currently changes in flight to the cluster.
state

string

The current state of the cluster. Possible values: STATE_UNSPECIFIED, PROVISIONING, RUNNING, RECONCILING, STOPPING, ERROR, DEGRADED.
uid

string

A globally unique identifier for the cluster.
updateTime

string

The time at which this cluster was last updated.
workloadIdentityConfig

list (object)

Workload Identity settings.
workloadIdentityConfig[]

object
workloadIdentityConfig[].identityProvider

string

The ID of the OIDC Identity Provider (IdP) associated to the Workload Identity Pool.
workloadIdentityConfig[].issuerUri

string

The OIDC issuer URL for this cluster.
workloadIdentityConfig[].workloadPool

string

The Workload Identity Pool associated to the cluster.

Sample YAML(s)

Container Attached Cluster Basic

# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: containerattached.cnrm.cloud.google.com/v1beta1
kind: ContainerAttachedCluster
metadata:
  name: containerattachedcluster-sample-basic
spec:
  # Replace ${ATTACHED_CLUSTER_NAME?} with the name of the underlying attached cluster
  resourceID: ${ATTACHED_CLUSTER_NAME?}
  location: us-west1
  projectRef:
    # Replace ${PROJECT_ID?} with your Google Cloud project id
    external: ${PROJECT_ID?}
  description: "Test attached cluster basic sample"
  # Replace ${DISTRIBUTION?} with the Kubernetes distribution of the underlying attached cluster
  # Supported values: "eks", "aks".
  distribution: ${DISTRIBUTION?}
  oidcConfig:
    # Replace ${ISSUER_URL?} with the OIDC issuer URL of the underlying attached cluster
    issuerUrl: ${ISSUER_URL?}
  # Replace ${PLATFORM_VERSION?} with the platform version of the underlying attached cluster
  platformVersion: ${PLATFORM_VERSION?}
  fleet:
    projectRef:
      name: containerattachedcluster-dep-basic
---
apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
kind: Project
metadata:
  name: containerattachedcluster-dep-basic
  annotations:
    cnrm.cloud.google.com/deletion-policy: abandon
spec:
  # Replace ${PROJECT_ID?} with your Google Cloud project id
  resourceID: ${PROJECT_ID?}
  organizationRef:
    # Replace ${ORG_ID?} with your Google Cloud ord id your project associates to
    external: "${ORG_ID?}"
  # Replace ${PROJECT_ID?} with your Google Cloud project id
  name: ${PROJECT_ID?}

Container Attached Cluster Full

# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: containerattached.cnrm.cloud.google.com/v1beta1
kind: ContainerAttachedCluster
metadata:
  name: containerattachedcluster-sample-full
spec:
  # Replace ${ATTACHED_CLUSTER_NAME?} with the name of the underlying attached cluster
  resourceID: ${ATTACHED_CLUSTER_NAME?}
  location: us-west1
  projectRef:
    # Replace ${PROJECT_ID?} with your Google Cloud project id
    external: ${PROJECT_ID?}
  description: "Test attached cluster full sample"
  # Replace ${DISTRIBUTION?} with the Kubernetes distribution of the underlying attached cluster
  # Supported values: "eks", "aks".
  distribution: ${DISTRIBUTION?}
  annotations:
    label-one: "value-one"
  authorization:
    admin_users: [ "user1@example.com", "user2@example.com"]
  oidcConfig:
    # Replace ${ISSUER_URL?} with the OIDC issuer URL of the underlying attached cluster
    issuerUrl: ${ISSUER_URL?}
  # Replace ${PLATFORM_VERSION?} with the platform version of the underlying attached cluster
  platformVersion: ${PLATFORM_VERSION?}
  fleet:
    projectRef:
      name: containerattachedcluster-dep-full
  loggingConfig:
    componentConfig:
      enableComponents: ["SYSTEM_COMPONENTS", "WORKLOADS"]
  monitoringConfig:
    managedPrometheusConfig:
      enabled: true
---
apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
kind: Project
metadata:
  name: containerattachedcluster-dep-full
  annotations:
    cnrm.cloud.google.com/deletion-policy: abandon
spec:
  # Replace ${PROJECT_ID?} with your Google Cloud project id
  resourceID: ${PROJECT_ID?}
  organizationRef:
    # Replace ${ORG_ID?} with your Google Cloud ord id your project associates to
    external: "${ORG_ID?}"
  # Replace ${PROJECT_ID?} with your Google Cloud project id
  name: ${PROJECT_ID?}

Container Attached Cluster Ignore Errors

# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: containerattached.cnrm.cloud.google.com/v1beta1
kind: ContainerAttachedCluster
metadata:
  name: containerattachedcluster-sample-ignore-errors
spec:
  # Replace ${ATTACHED_CLUSTER_NAME?} with the name of the underlying attached cluster
  resourceID: ${ATTACHED_CLUSTER_NAME?}
  location: us-west1
  projectRef:
    # Replace ${PROJECT_ID?} with your Google Cloud project id
    external: ${PROJECT_ID?}
  description: "Test attached cluster ignore errors sample"
  # Replace ${DISTRIBUTION?} with the Kubernetes distribution of the underlying attached cluster
  # Supported values: "eks", "aks".
  distribution: ${DISTRIBUTION?}
  oidcConfig:
    # Replace ${ISSUER_URL?} with the OIDC issuer URL of the underlying attached cluster
    issuerUrl: ${ISSUER_URL?}
  # Replace ${PLATFORM_VERSION?} with the platform version of the underlying attached cluster
  platformVersion: ${PLATFORM_VERSION?}
  fleet:
    projectRef:
      name: containerattachedcluster-dep-ignore-errors
  deletionPolicy: "DELETE_IGNORE_ERRORS"
---
apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
kind: Project
metadata:
  name: containerattachedcluster-dep-ignore-errors
  annotations:
    cnrm.cloud.google.com/deletion-policy: abandon
spec:
  # Replace ${PROJECT_ID?} with your Google Cloud project id
  resourceID: ${PROJECT_ID?}
  organizationRef:
    # Replace ${ORG_ID?} with your Google Cloud ord id your project associates to
    external: "${ORG_ID?}"
  # Replace ${PROJECT_ID?} with your Google Cloud project id
  name: ${PROJECT_ID?}