This document shows you how to commit a schema revision for Pub/Sub topics.
Before you begin
- Understand how Pub/Sub schemas work.
- Create a schema.
Required roles and permissions
To get the permissions that you need to commit a schema revision and manage schemas,
ask your administrator to grant you the
Pub/Sub Editor (roles/pubsub.editor
) IAM role on your project.
For more information about granting roles, see Manage access to projects, folders, and organizations.
This predefined role contains the permissions required to commit a schema revision and manage schemas. To see the exact permissions that are required, expand the Required permissions section:
Required permissions
The following permissions are required to commit a schema revision and manage schemas:
-
Create schema:
pubsub.schemas.create
-
Attach schema to topic:
pubsub.schemas.attach
-
Commit a schema revision:
pubsub.schemas.commit
-
Delete a schema or a schema revision:
pubsub.schemas.delete
-
Get a schema or schema revisions:
pubsub.schemas.get
-
List schemas:
pubsub.schemas.list
-
List schema revisions:
pubsub.schemas.listRevisions
-
Rollback a schema:
pubsub.schemas.rollback
-
Validate a message:
pubsub.schemas.validate
-
Get the IAM policy for a schema:
pubsub.schemas.getIamPolicy
-
Configure the IAM policy for a schema:
pubsub.schemas.setIamPolicy
You might also be able to get these permissions with custom roles or other predefined roles.
You can grant roles and permissions to principals such as users, groups, domains, or service accounts. You can create a schema in one project and attach it to a topic located in a different project. Ensure that you have the required permissions for each project.
Revise a schema
You can commit a schema revision using the Google Cloud console, the gcloud CLI CLI, the Pub/Sub API, or the Cloud Client Libraries.
The following are some guidelines to commit a schema revision:
You can revise a schema within specific constraints:
For Protocol Buffer schemas, you can add or remove optional fields. You cannot add or delete other fields. You also cannot edit any existing field.
For Avro schemas, refer to the Avro documentation for rules about schema resolution. A new revision must follow the rules as though it is both the reader schema and the writer schema.
A schema can have a maximum of 20 revisions at one time. If you exceed the limit, delete a schema revision before creating another.
Each revision has a unique revision ID associated with it. The revision ID is an auto-generated eight-character UUID.
When you update the revision range or the revision of a schema used for topic validation, it might take a few minutes for the changes to take effect.
Console
To create a schema revision, follow these steps:
In the Google Cloud console, go to the Pub/Sub schemas page.
Click the Schema ID of an existing schema.
The Schema details page for the schema opens.
Click Create revision.
The Create schema revision page opens.
Make changes as required.
For example, for the sample schema in Avro that you created in Create a schema, you can add an additional optional field called
Price
as follows:{ "type": "record", "name": "Avro", "fields": [ { "name": "ProductName", "type": "string", "default": "" }, { "name": "SKU", "type": "int", "default": 0 }, { "name": "InStock", "type": "boolean", "default": false }, { "name": "Price", "type": "double", "default": "0.0" } ] }
Click Validate definition to check if the schema definition is correct.
You can also validate the messages for the schema.
Click Test message to test a sample message.
In the Test message window, select a type of Message encoding.
In the Message body, enter a test message.
For example, here's a sample message for the test schema. In this example, select the Message encoding as
JSON
.{"ProductName":"GreenOnions", "SKU":34543, "Price":12, "InStock":true}
Click Test.
Click Commit to save the schema.
gcloud
gcloud pubsub schemas commit SCHEMA_ID \ --type=SCHEMA_TYPE \ --definition=SCHEMA_DEFINITION
Where:
- SCHEMA_TYPE is either
avro
orprotocol-buffer
. - SCHEMA_DEFINITION is a
string
containing the definition of the schema, formatted according to the chosen schema type.
You can also specify the schema definition in file:
gcloud pubsub schemas commit SCHEMA_ID \ --type=SCHEMA_TYPE \ --definition-file=SCHEMA_DEFINITION_FILE
Where:
- SCHEMA_TYPE is either
avro
orprotocol-buffer
. - SCHEMA_DEFINITION_FILE is a
string
containing the path to the file with the definition of the schema, formatted according to the chosen schema type.
REST
To commit a schema revision, send a POST request like the following:
POST https://pubsub.googleapis.com/v1/projects/PROJECT_ID/schemas/SCHEMA_ID:commit Authorization: Bearer $(gcloud auth application-default print-access-token) Content-Type: application/json --data @response-body.json
Specify the following fields in the request body:
{ "definition": SCHEMA_DEFINITION "type": SCHEMA_TYPE "name": SCHEMA_NAME }
Where:
- SCHEMA_TYPE is either
AVRO
orPROTOCOL_BUFFER
. - SCHEMA_DEFINITION is a string containing the definition of the schema, formatted according to the chosen schema type.
- SCHEMA_NAME is the name of an existing schema.
The response body should contain a JSON representation of a schema resource. For example:
{ "name": SCHEMA_NAME, "type": SCHEMA_TYPE, "definition": SCHEMA_DEFINITION "revisionId": REVISION_ID "revisionCreateTime": REVISION_CREATE_TIME }
Where:
- REVISION_ID is the server-generated ID for the revision.
- REVISION_CREATE_TIME is the ISO 8601 timestamp at which the revision was created.
Go
Before trying this sample, follow the Go setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Go API reference documentation.
Avro
Proto
C++
Before trying this sample, follow the C++ setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub C++ API reference documentation.
Avro
Proto
Java
Before trying this sample, follow the Java setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Java API reference documentation.
Avro
Proto
Python
Before trying this sample, follow the Python setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Python API reference documentation.
Avro
Proto
Node.js
Before trying this sample, follow the Node.js setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Node.js API reference documentation.
Avro
Proto
Node.js
Before trying this sample, follow the Node.js setup instructions in Quickstart: Using Client Libraries. For more information, see the Pub/Sub Node.js API reference documentation.
Avro
Proto
After you commit a schema revision, you can see the details of the new revision in the Schemas page.