Declarative multi-tenancy with project namespaces
This tutorial shows how to use the Project Namespace blueprint with Config Controller to create a dedicated namespace to manage Google Cloud resources in a specific project. Follow along if you are an infrastructure admin and want to allow your internal tenants to declaratively manage their project configuration.
Config Controller is a hosted service to provision and orchestrate Anthos and Google Cloud resources. It offers an API endpoint that can provision, actuate, and orchestrate Google Cloud resources as part of Anthos Config Management.
KRM blueprints are a way to package resources that are commonly used together while codifying best practices that you can roll out across your organization.
The Project Namespace blueprint is a KRM blueprint that includes all the resources you need to provision a namespace in Config Controller where you or a tenant can manage Google Cloud project resources. You can instantiate the blueprint multiple times to set up multiple project namespaces.
The Project Namespace blueprint makes it easy to manage one or more project namespaces,
but if you want to see the manual steps using Google Cloud CLI
, check out
Configure Config Connector to manage resources in your namespaces.
Objectives
- Configure a project namespace in Config Controller.
- Apply the configuration using
kpt live apply
.
Costs
This tutorial uses the following billable components of Google Cloud:
For a full list of resources included in the Project Namespace blueprint, see the Resources section of the Project Namespace blueprint package.
To generate a cost estimate based on your projected usage, use the pricing calculator.
When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Cleaning up.
Requirements
You must have a Config Controller instance.
You must have a Google Cloud project. This is the tenant project your project namespace will manage.
You must have billing enabled for your tenant project.
Learn how to confirm that billing is enabled for your project.
Before you begin
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
You run all commands in this tutorial from Cloud Shell.
Set up the environment
In Cloud Shell, run the following commands:
Install
kubectl
, the primary command-line interface for Kubernetes:gcloud components install kubectl
Install
kpt
, the primary command-line interface for KRM blueprints:gcloud components install kpt
Configure
kubectl
andkpt
to connect with Config Controller:gcloud alpha anthos config controller get-credentials INSTANCE_NAME \ --location COMPUTE_REGION \ --project ADMIN_PROJECT_ID
Replace the following:
INSTANCE_NAME
: the name of your Config Controller instance.COMPUTE_REGION
: the region of your Config Controller instance (for example,us-central1
).ADMIN_PROJECT_ID
: the project ID of your Config Controller instance.
Enable the Resource Manager API:
The Resource Manager API is used by Config Connector to manage enablement of other service APIs.
gcloud services enable cloudresourcemanager.googleapis.com \ --project TENANT_PROJECT_ID
Replace
TENANT_PROJECT_ID
with the ID of your tenant project.Install the ResourceGroup CRD, if not already installed:
kpt live install-resource-group
Verify that Config Connector is configured and healthy in the
config-control
namespace:kubectl get ConfigConnectorContext -n config-control \ -o "custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HEALTHY:.status.healthy"
Expected output:
NAMESPACE NAME HEALTHY config-control configconnectorcontext.core.cnrm.cloud.google.com true
Give Config Controller permission to manage Google Cloud resources in the tenant project:
export TENANT_PROJECT_ID=TENANT_PROJECT_ID export SA_EMAIL="$(kubectl get ConfigConnectorContext -n config-control \ -o jsonpath='{.items[0].spec.googleServiceAccount}' 2> /dev/null)" gcloud projects add-iam-policy-binding "${TENANT_PROJECT_ID}" \ --member "serviceAccount:${SA_EMAIL}" \ --role "roles/owner" \ --project "${TENANT_PROJECT_ID}"
Configure a project namespace
To create a project namespace in which to manage resources for another project, you run the following commands.
Fetch the Project Namespace blueprint with
kpt
, from within the desired working directory:kpt pkg get \ https://github.com/GoogleCloudPlatform/blueprints.git/catalog/project/kcc-namespace@main \ TENANT_PROJECT_ID
Replace
TENANT_PROJECT_ID
with the ID of your tenant project.In this blueprint, the project ID is also used as the name of the project namespace.
Move into the package directory:
cd ./TENANT_PROJECT_ID/
Configure the package by modifying the
setters.yaml
file:cat > setters.yaml << EOF apiVersion: v1 kind: ConfigMap metadata: name: setters annotations: config.kubernetes.io/local-config: "true" data: project-id: TENANT_PROJECT_ID management-project-id: ADMIN_PROJECT_ID management-namespace: ADMIN_NAMESPACE projects-namespace: PROJECTS_NAMESPACE networking-namespace: NETWORKING_NAMESPACE EOF
Replace the following:
ADMIN_PROJECT_ID
: the ID of the project that contains your Config Controller cluster.The admin project will be used to bootstrap Config Connector workload identity.
ADMIN_NAMESPACE
: the namespace to use to manage project namespaces (for example,config-control
).The admin namespace will be used to bootstrap Config Connector workload identity in the admin project.
PROJECTS_NAMESPACE
: the namespace to use to manage project permissions (for example,config-control
).The project's namespace will be used to manage things like IAM policy, so that tenants with access to the tenant namespace cannot escalate their permissions.
If you used the Landing Zone blueprint, you should already have a
projects
namespace for managing projects. If not, set this toconfig-control
.NETWORKING_NAMESPACE
: the namespace used to manage Shared VPCs (for example,config-control
).This will allow you to use cross-namespace resource references when selecting which network or subnet to use.
If you used the Landing Zone blueprint, you should already have a
networking
namespace for managing Shared VPCs. If not, set this toconfig-control
.
Render the setter values into the templated resources:
kpt fn render
Example output:
Package "example-1234": [RUNNING] "gcr.io/kpt-fn/apply-setters:v0.1" [PASS] "gcr.io/kpt-fn/apply-setters:v0.1" Results: [INFO] set field value to "cnrm-network-viewer-example-1234" in file "kcc-namespace-viewer.yaml" in field "metadata.name" [INFO] set field value to "config-control" in file "kcc-namespace-viewer.yaml" in field "metadata.namespace" [INFO] set field value to "cnrm-controller-manager-example-1234" in file "kcc-namespace-viewer.yaml" in field "subjects[0].name" [INFO] set field value to "cnrm-project-viewer-example-1234" in file "kcc-namespace-viewer.yaml" in field "metadata.name" ...(20 line(s) truncated, use '--truncate-output=false' to disable) Successfully executed 1 function(s) in 1 package(s).
Configure tenant permissions
To allow a tenant to provision Google Cloud project resources in their tenant project, you grant them permission to use the tenant project namespace with the following commands.
Create a
project-admin.yaml
file with the followingRoleBinding
:cat > project-admin.yaml << EOF apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: project-admin namespace: TENANT_PROJECT_ID roleRef: kind: ClusterRole name: cnrm-admin apiGroup: rbac.authorization.k8s.io subjects: - kind: User name: TENANT_EMAIL apiGroup: rbac.authorization.k8s.io EOF
Replace
TENANT_EMAIL
with the email of the tenant Google Cloud user account (for example,janedoe@example.com
).The
cnrm-admin
role will allow the tenant to create Config Connector resources in the project namespace.For details about how to authorize a group or service account, see Assigning Roles using RoleBindings or ClusterRoleBindings.
Apply config changes
Local changes in the previous steps do not affect the cloud until applied.
To apply your configuration changes, you run the following commands.
Initialize the working directory with kpt, which creates a resource to track changes:
kpt live init --namespace ADMIN_NAMESPACE
Replace
ADMIN_NAMESPACE
with the namespace used for managing project namespaces (for example,config-control
).Preview the resources that will be created:
kpt live apply --dry-run
All resources should say "created (dry-run)".
Example output:
namespace/example-1234 created (dry-run) rolebinding.rbac.authorization.k8s.io/cnrm-network-viewer-example-1234 created (dry-run) rolebinding.rbac.authorization.k8s.io/cnrm-project-viewer-example-1234 created (dry-run) rolebinding.rbac.authorization.k8s.io/project-admin created (dry-run) configconnectorcontext.core.cnrm.cloud.google.com/configconnectorcontext.core.cnrm.cloud.google.com created (dry-run) iampartialpolicy.iam.cnrm.cloud.google.com/example-1234-sa-workload-identity-binding created (dry-run) iampartialpolicy.iam.cnrm.cloud.google.com/kcc-example-1234-owners-permissions created (dry-run) iamserviceaccount.iam.cnrm.cloud.google.com/kcc-example-1234 created (dry-run) 8 resource(s) applied. 8 created, 0 unchanged, 0 configured, 0 failed (dry-run) 0 resource(s) pruned, 0 skipped, 0 failed (dry-run)
Apply the resources with
kpt
:kpt live apply
All resources should say "created".
Example output:
namespace/example-1234 created rolebinding.rbac.authorization.k8s.io/cnrm-network-viewer-example-1234 created rolebinding.rbac.authorization.k8s.io/cnrm-project-viewer-example-1234 created rolebinding.rbac.authorization.k8s.io/project-admin created configconnectorcontext.core.cnrm.cloud.google.com/configconnectorcontext.core.cnrm.cloud.google.com created iampartialpolicy.iam.cnrm.cloud.google.com/example-1234-sa-workload-identity-binding created iampartialpolicy.iam.cnrm.cloud.google.com/kcc-example-1234-owners-permissions created iamserviceaccount.iam.cnrm.cloud.google.com/kcc-example-1234 created 8 resource(s) applied. 8 created, 0 unchanged, 0 configured, 0 failed 0 resource(s) pruned, 0 skipped, 0 failed
Verify success
To verify that your changes are applied and the resources they specify are provisioned, you run the following commands.
Wait until the resources are ready:
kpt live status --output table --poll-until current
This command will poll until all the resources have a status of
Current
and a condition ofReady
or<None>
(for resources that do not support the Ready condition). TheReady
will be green if Ready=true and red if Ready=false.Use
ctrl-c
to interrupt, if needed.Example output:
NAMESPACE RESOURCE STATUS CONDITIONS AGE MESSAGE Namespace/example-1234 Current <None> 13s Resource is current config-con IAMPartialPolicy/example-1234-sa-workloa Current Ready 11s Resource is Ready config-con IAMPartialPolicy/kcc-example-1234-owners Current Ready 11s Resource is Ready config-con IAMServiceAccount/kcc-example-1234 Current Ready 11s Resource is Ready config-con RoleBinding/cnrm-network-viewer-example- Current <None> 13s Resource is current config-con RoleBinding/cnrm-project-viewer-example- Current <None> 12s Resource is current example-12 ConfigConnectorContext/configconnectorco Current <None> 12s Resource is current example-12 RoleBinding/project-admin Current <None> 12s Resource is current
In the case of error, use the default event output to see full error messages:
kpt live status
Cleaning up
If you decide to stop using Config Controller, you should first clean up all resources created with Config Controller and then delete Config Controller itself.
Delete the resources with
kpt
, from within the working directory:kpt live destroy
Wait until all resources are deleted:
until [ -z "$(kubectl get -R -f . --ignore-not-found | tee /dev/fd/2)" ]; \ do sleep 1; done
This command will poll until all the resources have a status of
Deleted
.Use
ctrl-c
to interrupt, if needed.
What's next
- Browse the KRM blueprint catalog.