Delete a Connect cluster

Deleting a Connect cluster erases all associated data, including connector configurations stored in its primary Kafka cluster. This action is irreversible.

To delete a Connect cluster, you can use the Google Cloud console, the gcloud CLI, the client library, or the Managed Kafka API. You can't use the open source Apache Kafka API to delete a Connect cluster.

Required roles and permissions to delete a Connect cluster

To get the permissions that you need to delete a Connect cluster, ask your administrator to grant you the Managed Kafka Connect Cluster Editor (roles/managedkafka.connectClusterEditor) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to delete a Connect cluster. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to delete a Connect cluster:

  • Grant the delete a Connect cluster permission on the Connect cluster: managedkafka.connectClusters.delete
  • Grant the list Connect clusters permission on the specified location. This permission is only required for deleting a Connect cluster using the Google Cloud console: managedkafka.connectClusters.list

You might also be able to get these permissions with custom roles or other predefined roles.

For more information about the Managed Kafka Connect Cluster Editor role, see Managed Service for Apache Kafka predefined roles.

Delete a Connect cluster

  • Understand data loss implications: Deleting a Connect cluster erases all data stored within the Connect cluster itself. This includes the following:

    • Connectors and their configurations

    • Any other data directly managed by the Connect cluster

    Deleting a Connect cluster does not delete data in your source or target Kafka clusters. If you're using a source connector to move data to a Kafka topic, deleting the Connect cluster does not delete the data that is already published to that Kafka topic. Similarly, deleting a Connect cluster does not delete the Kafka clusters that the Connect cluster is associated with.

  • Plan for service disruption: Any applications or services depending on data read or written by the Connect cluster might experience disruptions. Plan for this service disruption before deleting the cluster.

  • Review billing implications: You stop incurring charges for the cluster after you delete it. You might still be billed for resources used until deletion.

  • Expect asynchronous operation: Cluster deletion is asynchronous by default. The command returns immediately, and you can track the deletion progress separately.

Console

  1. In the Google Cloud console, go to the Connect Clusters page.

    Go to Connect Clusters

  2. Select the Connect cluster that you want to delete. You can select more than one.

  3. Click Delete.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Use the gcloud alpha managed-kafka connect-clusters delete command to delete Connect clusters:

    gcloud alpha managed-kafka connect-clusters delete CONNECT_CLUSTER \
    --location=LOCATION [--async]

    Replace the following:

    • CONNECT_CLUSTER: The ID of the Connect cluster you want to delete.
    • LOCATION: The location of the Connect cluster.

    The following flag is optional:

    • --async: Return immediately, without waiting for the operation in progress to complete.

Go

Before trying this sample, follow the Go setup instructions in Install the client libraries. For more information, see the Managed Service for Apache Kafka Go API reference documentation.

To authenticate to Managed Service for Apache Kafka, set up Application Default Credentials(ADC). For more information, see Set up ADC for a local development environment.

import (
	"context"
	"fmt"
	"io"

	"cloud.google.com/go/managedkafka/apiv1/managedkafkapb"
	"google.golang.org/api/option"

	managedkafka "cloud.google.com/go/managedkafka/apiv1"
)

func deleteConnectCluster(w io.Writer, projectID, region, clusterID string, opts ...option.ClientOption) error {
	// projectID := "my-project-id"
	// region := "us-central1"
	// clusterID := "my-connect-cluster"
	ctx := context.Background()
	client, err := managedkafka.NewManagedKafkaConnectClient(ctx, opts...)
	if err != nil {
		return fmt.Errorf("managedkafka.NewManagedKafkaConnectClient got err: %w", err)
	}
	defer client.Close()

	clusterPath := fmt.Sprintf("projects/%s/locations/%s/connectClusters/%s", projectID, region, clusterID)
	req := &managedkafkapb.DeleteConnectClusterRequest{
		Name: clusterPath,
	}
	op, err := client.DeleteConnectCluster(ctx, req)
	if err != nil {
		return fmt.Errorf("client.DeleteConnectCluster got err: %w", err)
	}
	err = op.Wait(ctx)
	if err != nil {
		return fmt.Errorf("op.Wait got err: %w", err)
	}
	fmt.Fprint(w, "Deleted connect cluster\n")
	return nil
}

Java

Before trying this sample, follow the Java setup instructions in Install the client libraries. For more information, see the Managed Service for Apache Kafka Java API reference documentation.

To authenticate to Managed Service for Apache Kafka, set up Application Default Credentials. For more information, see Set up ADC for a local development environment.


import com.google.api.gax.longrunning.OperationFuture;
import com.google.api.gax.longrunning.OperationSnapshot;
import com.google.api.gax.longrunning.OperationTimedPollAlgorithm;
import com.google.api.gax.retrying.RetrySettings;
import com.google.api.gax.retrying.TimedRetryAlgorithm;
import com.google.api.gax.rpc.ApiException;
import com.google.cloud.managedkafka.v1.ConnectClusterName;
import com.google.cloud.managedkafka.v1.DeleteConnectClusterRequest;
import com.google.cloud.managedkafka.v1.ManagedKafkaConnectClient;
import com.google.cloud.managedkafka.v1.ManagedKafkaConnectSettings;
import com.google.cloud.managedkafka.v1.OperationMetadata;
import com.google.protobuf.Empty;
import java.io.IOException;
import java.time.Duration;

public class DeleteConnectCluster {

  public static void main(String[] args) throws Exception {
    // TODO(developer): Replace these variables before running the example.
    String projectId = "my-project-id";
    String region = "my-region"; // e.g. us-east1
    String clusterId = "my-connect-cluster";
    deleteConnectCluster(projectId, region, clusterId);
  }

  public static void deleteConnectCluster(String projectId, String region, String clusterId)
      throws Exception {

    // Create the settings to configure the timeout for polling operations
    ManagedKafkaConnectSettings.Builder settingsBuilder = ManagedKafkaConnectSettings.newBuilder();
    TimedRetryAlgorithm timedRetryAlgorithm = OperationTimedPollAlgorithm.create(
        RetrySettings.newBuilder()
            .setTotalTimeoutDuration(Duration.ofHours(1L))
            .build());
    settingsBuilder.deleteConnectClusterOperationSettings()
        .setPollingAlgorithm(timedRetryAlgorithm);

    try (ManagedKafkaConnectClient managedKafkaConnectClient = ManagedKafkaConnectClient.create(
        settingsBuilder.build())) {
      DeleteConnectClusterRequest request = DeleteConnectClusterRequest.newBuilder()
          .setName(ConnectClusterName.of(projectId, region, clusterId).toString())
          .build();
      OperationFuture<Empty, OperationMetadata> future = managedKafkaConnectClient
          .deleteConnectClusterOperationCallable().futureCall(request);

      // Get the initial LRO and print details. CreateConnectCluster contains sample
      // code for polling logs.
      OperationSnapshot operation = future.getInitialFuture().get();
      System.out.printf(
          "Connect cluster deletion started. Operation name: %s\nDone: %s\nMetadata: %s\n",
          operation.getName(),
          operation.isDone(),
          future.getMetadata().get().toString());

      future.get();
      System.out.println("Deleted connect cluster");
    } catch (IOException | ApiException e) {
      System.err.printf("managedKafkaConnectClient.deleteConnectCluster got err: %s\n", 
          e.getMessage());
    }
  }
}

Python

Before trying this sample, follow the Python setup instructions in Install the client libraries. For more information, see the Managed Service for Apache Kafka Python API reference documentation.

To authenticate to Managed Service for Apache Kafka, set up Application Default Credentials. For more information, see Set up ADC for a local development environment.

from google.api_core.exceptions import GoogleAPICallError
from google.cloud.managedkafka_v1.services.managed_kafka_connect import (
    ManagedKafkaConnectClient,
)
from google.cloud import managedkafka_v1

# TODO(developer)
# project_id = "my-project-id"
# region = "us-central1"
# connect_cluster_id = "my-connect-cluster"

connect_client = ManagedKafkaConnectClient()

request = managedkafka_v1.DeleteConnectClusterRequest(
    name=connect_client.connect_cluster_path(project_id, region, connect_cluster_id),
)

try:
    operation = connect_client.delete_connect_cluster(request=request)
    print(f"Waiting for operation {operation.operation.name} to complete...")
    operation.result()
    print("Deleted Connect cluster")
except GoogleAPICallError as e:
    print(f"The operation failed with error: {e}")

What's next?

Apache Kafka® is a registered trademark of The Apache Software Foundation or its affiliates in the United States and/or other countries.