Declarative multi-tenancy with project namespaces

This tutorial shows how to use the Project Namespace blueprint with Config Controller to create a dedicated namespace to manage Google Cloud resources in a specific project. Follow along if you are an infrastructure admin and want to allow your internal tenants to declaratively manage their project configuration.

Config Controller is a hosted service to provision and orchestrate Anthos and Google Cloud resources. It offers an API endpoint that can provision, actuate, and orchestrate Google Cloud resources as part of Anthos Config Management.

KRM blueprints are a way to package resources that are commonly used together while codifying best practices that you can roll out across your organization.

The Project Namespace blueprint is a KRM blueprint that includes all the resources you need to provision a namespace in Config Controller where you or a tenant can manage Google Cloud project resources. You can instantiate the blueprint multiple times to set up multiple project namespaces.

The Project Namespace blueprint makes it easy to manage one or more project namespaces, but if you want to see the manual steps using Google Cloud CLI, check out Configure Config Connector to manage resources in your namespaces.

Objectives

Configure a project namespace in Config Controller.
Apply the configuration using kpt live apply.

Costs

This tutorial uses the following billable components of Google Cloud:

For a full list of resources included in the Project Namespace blueprint, see the Resources section of the Project Namespace blueprint package.

To generate a cost estimate based on your projected usage, use the pricing calculator.

When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Cleaning up.

Requirements

You must have a Config Controller instance.

Learn how to set up Config Controller.
You must have a Google Cloud project. This is the tenant project your project namespace will manage.

Learn how to create and manage projects.
You must have billing enabled for your tenant project.

Learn how to confirm that billing is enabled for your project.

Before you begin

In the Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Set up the environment

In Cloud Shell, run the following commands:

Install kubectl, the primary command-line interface for Kubernetes:
```
gcloud components install kubectl
```
Install kpt, the primary command-line interface for KRM blueprints:
```
gcloud components install kpt
```
Note: This tutorial requires kpt v1+.
Configure kubectl and kpt to connect with Config Controller:
```
gcloud alpha anthos config controller get-credentials INSTANCE_NAME \
    --location COMPUTE_REGION \
    --project ADMIN_PROJECT_ID
```
Replace the following:
- INSTANCE_NAME: the name of your Config Controller instance.
- COMPUTE_REGION: the region of your Config Controller instance (for example, us-central1).
- ADMIN_PROJECT_ID: the project ID of your Config Controller instance.
Note: While in preview, Config Controller is only available in the us-central1 region.
Enable the Resource Manager API:

The Resource Manager API is used by Config Connector to manage enablement of other service APIs.
```
gcloud services enable cloudresourcemanager.googleapis.com \
    --project TENANT_PROJECT_ID
```
Replace TENANT_PROJECT_ID with the ID of your tenant project.
Install the ResourceGroup CRD, if not already installed:
```
kpt live install-resource-group
```

Verify that Config Connector is configured and healthy in the config-control namespace:

kubectl get ConfigConnectorContext -n config-control \
    -o "custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HEALTHY:.status.healthy"

Expected output:

NAMESPACE        NAME                                                HEALTHY
config-control   configconnectorcontext.core.cnrm.cloud.google.com   true

Give Config Controller permission to manage Google Cloud resources in the tenant project:

export TENANT_PROJECT_ID=TENANT_PROJECT_ID
export SA_EMAIL="$(kubectl get ConfigConnectorContext -n config-control \
    -o jsonpath='{.items[0].spec.googleServiceAccount}' 2> /dev/null)"
gcloud projects add-iam-policy-binding "${TENANT_PROJECT_ID}" \
    --member "serviceAccount:${SA_EMAIL}" \
    --role "roles/owner" \
    --project "${TENANT_PROJECT_ID}"

Configure a project namespace

To create a project namespace in which to manage resources for another project, you run the following commands.

Fetch the Project Namespace blueprint with kpt, from within the desired working directory:
```
kpt pkg get \
    https://github.com/GoogleCloudPlatform/blueprints.git/catalog/project/kcc-namespace@main \
    TENANT_PROJECT_ID
```
Replace TENANT_PROJECT_ID with the ID of your tenant project.

In this blueprint, the project ID is also used as the name of the project namespace.
Move into the package directory:
```
cd ./TENANT_PROJECT_ID/
```
Configure the package by modifying the setters.yaml file:
```
cat > setters.yaml << EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: setters
  annotations:
    config.kubernetes.io/local-config: "true"
data:
  project-id: TENANT_PROJECT_ID
  management-project-id: ADMIN_PROJECT_ID
  management-namespace: ADMIN_NAMESPACE
  projects-namespace: PROJECTS_NAMESPACE
  networking-namespace: NETWORKING_NAMESPACE
EOF
```
Replace the following:
- ADMIN_PROJECT_ID: the ID of the project that contains your Config Controller cluster.
  
  The admin project will be used to bootstrap Config Connector workload identity.
- ADMIN_NAMESPACE: the namespace to use to manage project namespaces (for example, config-control).
  
  The admin namespace will be used to bootstrap Config Connector workload identity in the admin project.
- PROJECTS_NAMESPACE: the namespace to use to manage project permissions (for example, config-control).
  
  The project's namespace will be used to manage things like IAM policy, so that tenants with access to the tenant namespace cannot escalate their permissions.
  
  If you used the Landing Zone blueprint, you should already have a projects namespace for managing projects. If not, set this to config-control.
- NETWORKING_NAMESPACE: the namespace used to manage Shared VPCs (for example, config-control).
  
  This will allow you to use cross-namespace resource references when selecting which network or subnet to use.
  
  If you used the Landing Zone blueprint, you should already have a networking namespace for managing Shared VPCs. If not, set this to config-control.

Render the setter values into the templated resources:

kpt fn render

Example output:

Package "example-1234":
[RUNNING] "gcr.io/kpt-fn/apply-setters:v0.1"
[PASS] "gcr.io/kpt-fn/apply-setters:v0.1"
  Results:
    [INFO] set field value to "cnrm-network-viewer-example-1234" in file "kcc-namespace-viewer.yaml" in field "metadata.name"
    [INFO] set field value to "config-control" in file "kcc-namespace-viewer.yaml" in field "metadata.namespace"
    [INFO] set field value to "cnrm-controller-manager-example-1234" in file "kcc-namespace-viewer.yaml" in field "subjects[0].name"
    [INFO] set field value to "cnrm-project-viewer-example-1234" in file "kcc-namespace-viewer.yaml" in field "metadata.name"
    ...(20 line(s) truncated, use '--truncate-output=false' to disable)

Successfully executed 1 function(s) in 1 package(s).

Configure tenant permissions

To allow a tenant to provision Google Cloud project resources in their tenant project, you grant them permission to use the tenant project namespace with the following commands.

Create a project-admin.yaml file with the following RoleBinding:
```
cat > project-admin.yaml << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: project-admin
  namespace: TENANT_PROJECT_ID
roleRef:
  kind: ClusterRole
  name: cnrm-admin
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: User
  name: TENANT_EMAIL
  apiGroup: rbac.authorization.k8s.io
EOF
```
Replace TENANT_EMAIL with the email of the tenant Google Cloud user account (for example, janedoe@example.com).

The cnrm-admin role will allow the tenant to create Config Connector resources in the project namespace.

For details about how to authorize a group or service account, see Assigning Roles using RoleBindings or ClusterRoleBindings.

Apply config changes

Local changes in the previous steps do not affect the cloud until applied.

To apply your configuration changes, you run the following commands.

Initialize the working directory with kpt, which creates a resource to track changes:
```
kpt live init --namespace ADMIN_NAMESPACE
```
Replace ADMIN_NAMESPACE with the namespace used for managing project namespaces (for example, config-control).

Preview the resources that will be created:

kpt live apply --dry-run

All resources should say "created (dry-run)".

Example output:

namespace/example-1234 created (dry-run)
rolebinding.rbac.authorization.k8s.io/cnrm-network-viewer-example-1234 created (dry-run)
rolebinding.rbac.authorization.k8s.io/cnrm-project-viewer-example-1234 created (dry-run)
rolebinding.rbac.authorization.k8s.io/project-admin created (dry-run)
configconnectorcontext.core.cnrm.cloud.google.com/configconnectorcontext.core.cnrm.cloud.google.com created (dry-run)
iampartialpolicy.iam.cnrm.cloud.google.com/example-1234-sa-workload-identity-binding created (dry-run)
iampartialpolicy.iam.cnrm.cloud.google.com/kcc-example-1234-owners-permissions created (dry-run)
iamserviceaccount.iam.cnrm.cloud.google.com/kcc-example-1234 created (dry-run)
8 resource(s) applied. 8 created, 0 unchanged, 0 configured, 0 failed (dry-run)
0 resource(s) pruned, 0 skipped, 0 failed (dry-run)

Apply the resources with kpt:

kpt live apply

All resources should say "created".

Example output:

namespace/example-1234 created
rolebinding.rbac.authorization.k8s.io/cnrm-network-viewer-example-1234 created
rolebinding.rbac.authorization.k8s.io/cnrm-project-viewer-example-1234 created
rolebinding.rbac.authorization.k8s.io/project-admin created
configconnectorcontext.core.cnrm.cloud.google.com/configconnectorcontext.core.cnrm.cloud.google.com created
iampartialpolicy.iam.cnrm.cloud.google.com/example-1234-sa-workload-identity-binding created
iampartialpolicy.iam.cnrm.cloud.google.com/kcc-example-1234-owners-permissions created
iamserviceaccount.iam.cnrm.cloud.google.com/kcc-example-1234 created
8 resource(s) applied. 8 created, 0 unchanged, 0 configured, 0 failed
0 resource(s) pruned, 0 skipped, 0 failed

Verify success

To verify that your changes are applied and the resources they specify are provisioned, you run the following commands.

Wait until the resources are ready:

kpt live status --output table --poll-until current

This command will poll until all the resources have a status of Current and a condition of Ready or <None> (for resources that do not support the Ready condition). The Ready will be green if Ready=true and red if Ready=false.

Use ctrl-c to interrupt, if needed.

Example output:

NAMESPACE   RESOURCE                                  STATUS      CONDITIONS                                AGE     MESSAGE
            Namespace/example-1234                    Current     <None>                                    13s     Resource is current
config-con  IAMPartialPolicy/example-1234-sa-workloa  Current     Ready                                     11s     Resource is Ready
config-con  IAMPartialPolicy/kcc-example-1234-owners  Current     Ready                                     11s     Resource is Ready
config-con  IAMServiceAccount/kcc-example-1234        Current     Ready                                     11s     Resource is Ready
config-con  RoleBinding/cnrm-network-viewer-example-  Current     <None>                                    13s     Resource is current
config-con  RoleBinding/cnrm-project-viewer-example-  Current     <None>                                    12s     Resource is current
example-12  ConfigConnectorContext/configconnectorco  Current     <None>                                    12s     Resource is current
example-12  RoleBinding/project-admin                 Current     <None>                                    12s     Resource is current

In the case of error, use the default event output to see full error messages:
```
kpt live status
```

Cleaning up

If you decide to stop using Config Controller, you should first clean up all resources created with Config Controller and then delete Config Controller itself.

Delete the resources with kpt, from within the working directory:
```
kpt live destroy
```
Wait until all resources are deleted:
```
until [ -z "$(kubectl get -R -f . --ignore-not-found | tee /dev/fd/2)" ]; \
do sleep 1; done
```
This command will poll until all the resources have a status of Deleted.

Use ctrl-c to interrupt, if needed.

What's next

Browse the KRM blueprint catalog.