Filtering messages

This page explains how to create filters for subscriptions.

After you create a subscription with a filter, the subscription only delivers the messages that match the filter. Pub/Sub automatically acknowledges the messages that don't match the filter. You can filter messages by their attributes.

When you receive messages from a subscription with a filter, you don't incur egress fees for the messages that Pub/Sub automatically acknowledges. For a limited time after the beta launch stage starts, you don't incur message delivery fees for these messages. All other fees apply.

Creating subscriptions with filters

You can create a subscription with a filter using the gcloud command-line tool or the Pub/Sub API.

gcloud

To create a subscription with a filter, use the gcloud beta pubsub subscriptions create command and --message-filter flag:

gcloud beta pubsub subscriptions create SUBSCRIPTION_ID \
  --topic=TOPIC_ID \
  --message-filter='FILTER'

Replace the following:

  • SUBSCRIPTION_ID: the ID of the subscription to create
  • TOPIC_ID: the ID of the topic to attach to the subscription
  • FILTER: an expression in the filtering syntax

Protocol

To create a subscription with a filter, use the projects.subscriptions.create method.

PUT https://pubsub.googleapis.com/v1/projects/PROJECT_ID/subscriptions/SUBSCRIPTION_ID
Authorization: Bearer $(gcloud auth print-access-token)

Replace the following:

  • PROJECT_ID: the project ID for the project to create the subscription in
  • SUBSCRIPTION_ID: the ID of the subscription to create

Specify the filter in the request body:

{
  "topic": TOPIC_ID,
  "filter": FILTER
}

Replace the following:

  • TOPIC_ID: the ID of the topic to attach to the subscription
  • FILTER: an expression in the filtering syntax

The maximum length of a filter is 256 bytes. After creating the subscription, you can't modify the filter. The backlog metrics might include messages that don't match the filter.

Filtering syntax

To filter messages, write an expression that operates on attributes. You can write an expression that matches the key or value of the attributes. The attributes identifier selects the attributes in the message.

For example, the filters in the following table select the domain attribute:

Filter Description
attributes:domain Messages with the domain attribute
NOT attributes:domain Messages without the domain attribute
attributes.domain = "com" Messages with the domain attribute and the value of com
attributes.domain != "com" Messages without the domain attribute and the value of com
hasPrefix(attributes.domain, "co") Messages with the domain attribute and a value that starts with co
NOT hasPrefix(attributes.domain, "co") Messages without the domain attribute and a value that starts with co

Comparison operators

You can filter attributes with the following comparison operators:

  • :
  • =
  • !=

The : operator matches a key in a list of attributes.

attributes:KEY

The equality operators match keys and values. The value must be a string literal.

attributes.KEY = "VALUE"

An expression with an equality operator must begin with a key, and the equality operator must compare a key and a value.

  • Valid: Filter compares a key and a value

    attributes.domain = "com"
    
  • Invalid: The left-hand side of the filter is a value

    "com" = attributes.domain
    
  • Invalid: Filter compares two keys

    attributes.domain = attributes.website
    

The key and value are case-sensitive and must match the attribute exactly. If a key contains characters other than hyphens, underscores, or alphanumeric characters, use a string literal.

attributes."iana.org/language_tag" = "en"

To use backslashes, quotation marks, and non-printing characters in a filter, escape the characters within a string literal. You can also use Unicode, hexadecimal, and octal escape sequences within a string literal.

  • Valid: Filter escapes characters within a string literal

    attributes:"\u307F\u3093\u306A"
    
  • Invalid: Filter escapes characters without a string literal

    attributes:\u307F\u3093\u306A
    

Boolean operators

You can use boolean operators in a filter. For example, the following filter is for messages with the iana.org/language_tag attribute, but without the domain attribute and the com value.

attributes:"iana.org/language_tag" AND NOT attributes.domain = "com"

The NOT operator has the highest precedence. To combine the AND and OR operators, use parentheses and complete expressions.

  • Valid: AND and OR operators with parentheses

    attributes:"iana.org/language_tag" AND (attributes.domain = "net" OR attributes.domain = "org")
    
  • Invalid: AND and OR operators without parentheses

    attributes:"iana.org/language_tag" AND attributes.domain = "net" OR attributes.domain = "org"
    
  • Invalid: AND and OR operators combine incomplete expressions

    attributes.domain = "com" AND ("net" OR "org")
    

You can also use the unary minus operator instead of the NOT operator.

attributes.domain = "com" AND -attributes:"iana.org/language_tag"

Functions

The hasPrefix function filters for attributes with values that start with a substring.

hasPrefix(attributes.KEY, "SUBSTRING")

Replace the following:

  • KEY: the name of the attribute
  • SUBSTRING: a substring of the value