Why use Service Catalog?
Service Catalog allows for applications running on a cluster to easily discover and connect to external services without having to manually import information such as credentials or endpoints. External service dependencies are modeled as Kubernetes resources, which can be easily integrated into your existing deployment process.
In this tutorial, we will use the following features of the Service Catalog:
- Discovering Google Cloud services in Service Broker via Service Catalog.
- Provisioning a service instance to create the Cloud resource.
- Binding the provisioned service instance in the Kubernetes application to inject credentials into the cluster.
- Using the service binding in the application to access the service instance, which points to the Cloud resource.
For more information on Service Catalog, refer to the Kubernetes Service Catalog documentation.
Objectives
This tutorial covers the following steps:
- Create a namespace
- Create a Cloud Identity and Access Management service account
- Create a Pub/Sub topic
- Create a Pub/Sub subscription to the topic
- Test the Pub/Sub service with a demo app
- Clean up
Before you begin
Take the following steps to enable the Kubernetes Engine API:- Visit the Kubernetes Engine page in the Google Cloud Console.
- Create or select a project.
- Wait for the API and related services to be enabled. This can take several minutes.
-
Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.
Install the following command-line tools used in this tutorial:
-
gcloud
is used to create and delete Kubernetes Engine clusters.gcloud
is included in the Google Cloud SDK. -
kubectl
is used to manage Kubernetes, the cluster orchestration system used by Kubernetes Engine. You can installkubectl
usinggcloud
:gcloud components install kubectl
Set defaults for the gcloud
command-line tool
To save time typing your project ID
and Compute Engine zone options in the gcloud
command-line tool, you can set the defaults:
gcloud config set project [PROJECT_ID] gcloud config set compute/zone [COMPUTE_ENGINE_ZONE]
Set up Service Catalog
- Install Service Catalog and Service Broker in your GKE cluster.
Verify that the Service Catalog is running by using
kubectl
in the command-line to list all the Service Catalog processes:kubectl get deployment -n service-catalog
All the deployments should report as
AVAILABLE
.(Optional) Install the
svcat
CLI tool, which improves the user experience around working with the Service Catalog resources.
Step 1: Create a Namespace
A Namespace is optional, but it is recommended to make cleanup of this tutorial easier.
Alias the Namespace name as an environment variable:
NAMESPACE=pubsub-app
Create the Namespace for the application using Pub/Sub. Run the following command:
kubectl create namespace [NAMESPACE]
It should output the following message:
namespace "pubsub-app" created
Step 2: Create a Cloud IAM service account
You need a Cloud Identity and Access Management service account in order to use
Google Cloud services in your application. For this tutorial, we will
create one with the ID pubsub-sa
.
Otherwise, you can create a Cloud IAM service account using Service Catalog. This can be done by
provisioning and binding to a service instance of the
cloud-iam-service-account
service class.
Provision a Cloud IAM service account instance
Use the following command to store the service account name in an environment variable:
SERVICE_ACCOUNT_ID=pubsub-sa
svcat
Use the svcat
command to provision a service account instance:
svcat provision gcp-iam \ --class cloud-iam-service-account \ --plan beta \ --namespace [NAMESPACE] \ --param accountId=$SERVICE_ACCOUNT_ID
The command should output the following message:
Name: gcp-iam
Namespace: pubsub-app
Status:
Class: cloud-iam-service-account
Plan: beta
Parameters:
accountId: pubsub-sa
Check the status of the service instance with the following command:
svcat get instance gcp-iam --namespace [NAMESPACE]
The output should appear similar to the following:
NAME NAMESPACE CLASS PLAN STATUS
+---------+------------+---------------------------+------+--------+
gcp-iam pubsub-app cloud-iam-service-account beta Ready
The service instance is successfully provisioned when its STATUS is Ready
.
kubectl
Create a ServiceInstance YAML file called iam_instance.yaml
with the
following spec. Make sure the value for serviceAccount
field is the same
as echo $SERVICE_ACCOUNT_ID
:
Use the kubectl
command to provision the service account instance using
the YAML file:
kubectl apply -f iam_instance.yaml
The command should output the following message:
serviceinstance "gcp-iam" created
Check the status of the service instance with the following command:
kubectl get serviceinstances gcp-iam -n [NAMESPACE] -o 'custom-columns=INSTANCE-NAME:.metadata.name,SERVICE:.spec.clusterServiceClassExternalName,PLAN:.spec.clusterServicePlanExternalName,STATUS:.status.conditions[0].reason'
The output should appear similar to the following:
INSTANCE-NAME SERVICE PLAN STATUS
gcp-iam cloud-iam-service-account beta ProvisionedSuccessfully
Once the service instance has been provisioned successfully, you can also check
the corresponding Cloud IAM service account
pubsub-sa@[PROJECT_ID].iam.gserviceaccount.com
in the Cloud
Console.
Bind to the Cloud IAM service account instance
Binding to the service account instance will create a secret in the same namespace. The secret will contain the service account name and a private key, which can be easily referenced by your application.
Use the following command to store the secret name in an environment variable:
SA_SECRET_NAME=pubsub-credentials
svcat
Use the svcat
command to bind to the service account instance:
svcat bind gcp-iam --namespace [NAMESPACE] --secret-name $SA_SECRET_NAME
The command should output the following message:
Name: gcp-iam
Namespace: pubsub-app
Status:
Instance: gcp-iam
Parameters:
{}
Check the status of the service binding with the following command:
svcat get binding gcp-iam --namespace [NAMESPACE]
The output should appear similar to the following:
NAME NAMESPACE INSTANCE STATUS
+---------+------------+----------+--------+
gcp-iam pubsub-app gcp-iam Ready
Wait until the service binding's status is Ready
before continuing. This
could take a minute.
kubectl
Create a ServiceBinding YAML file called iam_binding.yaml
with the
following spec:
Run the following command to create the service binding:
kubectl apply -f iam_binding.yaml
Check the status of the service binding by running the following command:
kubectl get servicebindings gcp-iam -n [NAMESPACE] -o 'custom-columns=BINDING-NAME:.metadata.name,SERVICE-INSTANCE:.spec.instanceRef.name,STATUS:.status.conditions[0].reason,OUTPUT-SECRET:.spec.secretName'
The output should be similar to the following:
BINDING-NAME SERVICE-INSTANCE STATUS OUTPUT-SECRET
gcp-iam gcp-iam InjectedBindResult gcp-iam
Once the service binding has been successfully created, confirm the secret
exists with the following kubectl
command:
kubectl get secrets $SA_SECRET_NAME --namespace [NAMESPACE]
The output should appear similar to the following:
NAME TYPE DATA AGE
pubsub-credentials Opaque 2 1m
You can also view the secret containing the service account credentials:
kubectl get secrets $SA_SECRET_NAME --namespace [NAMESPACE] -o yaml
Notice that the privateKeyData
value is Base64 encoded and contains the
service account private key.
Step 3: Create a Pub/Sub topic
Provision a Pub/Sub service instance
Now we will use Service Catalog to provision a Pub/Sub service instance. Doing so will create a Pub/Sub topic for our application to publish to and read from.
svcat
Use the svcat
command to provision a Pub/Sub service instance:
svcat provision gcp-pubsub \ --class cloud-pubsub \ --plan beta \ --namespace [NAMESPACE] \ --param topicId=pubsub-app
The command should output the following message:
Name: gcp-pubsub
Namespace: pubsub-app
Status:
Class: cloud-pubsub
Plan: beta
Parameters:
topicId: pubsub-app
Check the status of the service instance with the following command:
svcat get instance gcp-pubsub --namespace [NAMESPACE]
The output should appear similar to the following:
NAME NAMESPACE CLASS PLAN STATUS
+------------+------------+--------------+------+--------+
gcp-pubsub pubsub-app cloud-pubsub beta Ready
The service instance is successfully provisioned when its STATUS is Ready
.
kubectl
Create a ServiceInstance YAML file called pubsub_instance.yaml
with the
following spec:
Provision a Pub/Sub service instance with the following command:
kubectl create -f pubsub_instance.yaml
It should output the following message:
serviceinstance "gcp-pubsub" created
Check the Pub/Sub service instance status with the following command:
kubectl get serviceinstances gcp-pubsub -n [NAMESPACE] -o 'custom-columns=INSTANCE-NAME:.metadata.name,SERVICE:.spec.clusterServiceClassExternalName,PLAN:.spec.clusterServicePlanExternalName,STATUS:.status.conditions[0].reason'
It should output a status message similar to the following:
INSTANCE-NAME SERVICE PLAN STATUS
gcp-pubsub cloud-pubsub beta ProvisionedSuccessfully
Once the Pub/Sub service instance has been provisioned successfully, you can
also check the corresponding Pub/Sub topic
projects/[PROJECT_ID]/topics/pubsub-app
in the Cloud
Console.
Bind to the Pub/Sub service instance
Binding to a Pub/Sub service instance does the following:
- Grant the specified Cloud IAM service account appropriate permissions to access the Pub/Sub service.
- Make the information necessary to access the service instance available to the Kubernetes application in the form of a Kubernetes secret.
- Optionally, create a subscription to the Pub/Sub topic.
This is required for the demo application, because the app will need to be permitted to access the service in order to execute as the assumed identity of the service account to publish messages to a Pub/Sub topic.
The Pub/Sub service class has multiple types of service bindings. First, we will
create a service binding of type publisher
, which contains two roles:
roles/pubsub.publisher
and roles/pubsub.viewer
. The binding will allow our
service account to publish to this topic and view the messages.
Tip: For more information on Pub/Sub's Cloud IAM roles and permissions, please refer to Pub/Sub's [access control documentation](/pubsub/docs/access-control#tbl_roles).
svcat
Use the svcat
command to bind to the Pub/Sub topic as a publisher
:
svcat bind gcp-pubsub --namespace [NAMESPACE] --params-json '{ "serviceAccount": "'${SERVICE_ACCOUNT_ID}'", "roles": [ "roles/pubsub.publisher", "roles/pubsub.viewer" ] }'
The command should output the following message (serviceAccount
value may
be the name of the service account you created before):
Name: gcp-pubsub
Namespace: pubsub-app
Status:
Instance: gcp-pubsub
Parameters:
roles:
- roles/pubsub.publisher
- roles/pubsub.viewer
serviceAccount: pubsub-sa
Check the status of the service binding with the following command:
svcat get binding gcp-pubsub --namespace [NAMESPACE]
The output should appear similar to the following:
NAME NAMESPACE INSTANCE STATUS
+------------+------------+------------+--------+
gcp-pubsub pubsub-app gcp-pubsub Ready
Wait until the service binding's status is Ready
before continuing. This
could take a minute.
kubectl
Create a ServiceBinding YAML file called pubsub_binding.yaml
with the
following spec to bind to the Pub/Sub topic as a publisher
. Make sure the
value for serviceAccount
field is the same as echo $SERVICE_ACCOUNT_ID
:
Bind to the Pub/Sub service instance:
kubectl create -f pubsub_binding.yaml
Check the status of the service binding by running the following command:
kubectl get servicebindings gcp-pubsub -n [NAMESPACE] -o 'custom-columns=BINDING-NAME:.metadata.name,SERVICE-INSTANCE:.spec.instanceRef.name,STATUS:.status.conditions[0].reason,OUTPUT-SECRET:.spec.secretName'
It should display a InjectedBindResult
status.
The gcp-pubsub
secret created as a result of binding contains the project (projectId
),
service account (serviceAccount
), and topic ID (topicId
). You can check the
secret by running the following command:
kubectl get secrets gcp-pubsub --namespace [NAMESPACE] -o yaml
Step 4: Create a Pub/Sub subscription to the topic
Like binding to the Pub/Sub service instance for publishing to the topic, the demo application also needs a service binding in order to subscribe to and read messages from the topic.
Now we will create another service binding to the Pub/Sub service instance, this
time of type subscriber
, which contains a subscription and two roles:
roles/pubsub.subscriber
and roles/pubsub.viewer
. This will create a Pub/Sub
subscription to this topic and allow our service account to pull and view
messages from this subscription.
svcat
Use the svcat
command to bind to the Pub/Sub topic as a subscriber
:
svcat bind gcp-pubsub \ --name gcp-pubsub-subscription \ --namespace [NAMESPACE] \ --params-json '{ "serviceAccount": "'${SERVICE_ACCOUNT_ID}'", "roles": [ "roles/pubsub.subscriber", "roles/pubsub.viewer" ], "subscription": { "subscriptionId": "pubsub-app" } }'
The command should output the following message (serviceAccount
value may
be the name of the service account you created before):
Name: gcp-pubsub-subscription
Namespace: pubsub-app
Status:
Instance: gcp-pubsub
Parameters:
roles:
- roles/pubsub.subscriber
- roles/pubsub.viewer
serviceAccount: pubsub-sa
subscription:
subscriptionId: pubsub-app
Check the status of the service binding with the following command:
svcat get binding gcp-pubsub-subscription --namespace [NAMESPACE]
The output should appear similar to the following:
NAME NAMESPACE INSTANCE STATUS
+-------------------------+------------+------------+--------+
gcp-pubsub-subscription pubsub-app gcp-pubsub Ready
Wait until the service binding's status is Ready
before continuing. This
could take a minute.
kubectl
Create a ServiceBinding YAML file called pubsub_subscription_binding.yaml
that binds to the Pub/Sub topic as a subscriber
. Make sure the value for
serviceAccount
field is the same as echo $SERVICE_ACCOUNT_ID
:
Bind to the Pub/Sub service instance:
kubectl create -f pubsub_subscription_binding.yaml
Check the status of the service binding by running the following command:
kubectl get servicebindings gcp-pubsub-subscription -n [NAMESPACE] -o 'custom-columns=BINDING-NAME:.metadata.name,SERVICE-INSTANCE:.spec.instanceRef.name,STATUS:.status.conditions[0].reason,OUTPUT-SECRET:.spec.secretName'
It should display a InjectedBindResult
status.
The new subscription projects/[PROJECT_ID]/subscriptions/pubsub-app
can be
found in the Cloud
Console. The
gcp-pubsub-subscription
secret created as a result of binding contains the
project (projectId
), service account(serviceAccount
), topic ID (topicId
),
and subscription ID (subscriptionId
). You can check the secret by running the
following command:
kubectl get secrets gcp-pubsub-subscription --namespace [NAMESPACE] -o yaml
Step 5: Test the Pub/Sub service with a demo app
Run the demo app in your GKE cluster to test the connection with the Pub/Sub service.
Create a Kubernetes job file called hello_pubsub_world.yaml
with the following
spec (replace the value of secretName
to [SA_SECRET_NAME]
if you manually
created the secret to store the service account private key):
Note how it uses values from the [SA_SECRET_NAME]
(should be
pubsub-credentials
if you created a new Cloud IAM service
account), gcp-pubsub
, and
gcp-pubsub-subscription
secrets, which were created via bindings to the Cloud IAM
service account and Pub/Sub service instances.
Create the job:
kubectl apply -f hello_pubsub_world.yaml
The command should output the following message:
job "hello-pubsub-world" created
View the list of pods:
kubectl get pods --namespace [NAMESPACE] --show-all
It should display the details of the pod running in that namespace, similar to the following:
NAME READY STATUS RESTARTS AGE
hello-pubsub-world-zh8hm 0/1 Completed 0 1m
Wait for the pod to have a status of Completed
, and then check the logs of
that pod with the following command:
kubectl -n [NAMESPACE] logs [POD_NAME]
You should see lines that contain entries similar to:
Publishing 10 messages.
Publishing done.
Subscribing 10 messages.
Got message: "hello world #9"
Got message: "hello world #0"
Got message: "hello world #1"
Got message: "hello world #2"
Got message: "hello world #3"
Got message: "hello world #4"
Got message: "hello world #5"
Got message: "hello world #6"
Got message: "hello world #7"
Got message: "hello world #8"
Subscription done.
Cleaning up
To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:
Delete the demo namespace, which will delete the app, service bindings, and service instances:
kubectl delete namespace [NAMESPACE]
For more information about cleaning up Service Catalog, please refer to this guide
What's next
- Try our end-to-end samples on GitHub.
- Explore other Kubernetes Engine tutorials.
- Try out other Google Cloud Platform features for yourself. Have a look at our tutorials.