List your Google Cloud Managed Service for Apache Kafka consumer groups

To list your consumer groups in a cluster, you can use the Google Cloud console, the Google Cloud CLI, the client library, the Managed Kafka API, or the open source Apache Kafka APIs.

Required roles and permissions to list your consumer groups

To get the permissions that you need to list your consumer groups, ask your administrator to grant you the Managed Kafka Viewer (roles/managedkafka.viewer) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to list your consumer groups. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to list your consumer groups:

List consumer groups: managedkafka.consumerGroups.list
Get consumer group details: managedkafka.consumerGroups.get

You might also be able to get these permissions with custom roles or other predefined roles.

For more information about the Managed Kafka Viewer role, see Managed Service for Apache Kafka predefined roles.

List your consumer groups

To list all your consumer groups for a specific cluster, follow these steps:

Console

In the Google Cloud console, go to the Cluster page.

Go to Clusters
Click the cluster for which you want to see the consumer groups.
The cluster details page is displayed. In the cluster details page, under the Resources tab, the consumer groups are listed.

gcloud

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
Run the gcloud beta managed-service-for-apache-kafka consumer-groups list command:
```
gcloud beta managed-kafka consumer-groups list CLUSTER_ID \
    --location=LOCATION \
    --limit=LIMIT
```
This command lists all Managed Service for Apache Kafka consumer groups within a specified cluster and location.

Replace the following:
- CLUSTER_ID: The ID or name of the cluster.
- LOCATION: The location of the cluster.
- LIMIT: The maximum number of consumer groups to list.

Go

import (
	"context"
	"fmt"
	"io"

	"cloud.google.com/go/managedkafka/apiv1/managedkafkapb"
	"google.golang.org/api/iterator"
	"google.golang.org/api/option"

	managedkafka "cloud.google.com/go/managedkafka/apiv1"
)

func listConsumerGroups(w io.Writer, projectID, region, clusterID string, opts ...option.ClientOption) error {
	// projectID := "my-project-id"
	// region := "us-central1"
	// clusterID := "my-cluster"
	ctx := context.Background()
	client, err := managedkafka.NewClient(ctx, opts...)
	if err != nil {
		return fmt.Errorf("managedkafka.NewClient got err: %w", err)
	}
	defer client.Close()

	clusterPath := fmt.Sprintf("projects/%s/locations/%s/clusters/%s", projectID, region, clusterID)
	req := &managedkafkapb.ListConsumerGroupsRequest{
		Parent: clusterPath,
	}
	consumerGroupIter := client.ListConsumerGroups(ctx, req)
	for {
		res, err := consumerGroupIter.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			return fmt.Errorf("consumerGroupIter.Next() got err: %w", err)
		}
		fmt.Fprintf(w, "Got consumer group: %v", res)
	}
	return nil
}

Java

import com.google.api.gax.rpc.ApiException;
import com.google.cloud.managedkafka.v1.ClusterName;
import com.google.cloud.managedkafka.v1.ConsumerGroup;
import com.google.cloud.managedkafka.v1.ManagedKafkaClient;
import java.io.IOException;

public class ListConsumerGroups {

  public static void main(String[] args) throws Exception {
    // TODO(developer): Replace these variables before running the example.
    String projectId = "my-project-id";
    String region = "my-region"; // e.g. us-east1
    String clusterId = "my-cluster";
    listConsumerGroups(projectId, region, clusterId);
  }

  public static void listConsumerGroups(String projectId, String region, String clusterId)
      throws Exception {
    try (ManagedKafkaClient managedKafkaClient = ManagedKafkaClient.create()) {
      ClusterName clusterName = ClusterName.of(projectId, region, clusterId);
      // This operation is being handled synchronously.
      for (ConsumerGroup consumerGroup :
          managedKafkaClient.listConsumerGroups(clusterName).iterateAll()) {
        System.out.println(consumerGroup.getAllFields());
      }
    } catch (IOException | ApiException e) {
      System.err.printf("managedKafkaClient.listConsumerGroups got err: %s", e.getMessage());
    }
  }
}

Python

from typing import List

from google.cloud import managedkafka_v1


def list_consumer_groups(
    project_id: str,
    region: str,
    cluster_id: str,
) -> List[str]:
    """
    List Kafka consumer groups in a cluster.

    Args:
        project_id: Google Cloud project ID.
        region: Cloud region.
        cluster_id: ID of the Kafka cluster.
    """

    client = managedkafka_v1.ManagedKafkaClient()

    request = managedkafka_v1.ListConsumerGroupsRequest(
        parent=client.cluster_path(project_id, region, cluster_id),
    )

    response = client.list_consumer_groups(request=request)
    for consumer_group in response:
        print("Got consumer group:", consumer_group)

    return [consumer_group.name for consumer_group in response]

What's next?

Apache Kafka® is a registered trademark of The Apache Software Foundation or its affiliates in the United States and/or other countries.