Troubleshooting an Amazon Kinesis Data Streams import topic

This document provides some common troubleshooting tips for Amazon Kinesis Data Streams import topics.

After you configure the import topic and start ingesting messages, you can check the relevant Cloud Monitoring metrics to see if data is ingested. Perform the following steps:

  1. In the console, go to the Topics page.

    Go to Topics

  2. Click the Amazon Kinesis Data Streams import topic that you want to troubleshoot.

  3. In the topic details page, click the Metrics tab.

  4. Check the chart for the metric Ingestion byte count.

  5. If no data is being ingested, in the topic details page, check to see if there is an error for the Topic state field.

  6. You can also check the metric Ingestion data source state. To do so, in the topic details page, click the Metrics tab.

  7. Here is a list of errors that you might encounter:

    Error Code Description Fix
    KINESIS_PERMISSION_DENIED Error consuming Kinesis data due to permission issues. Verify the AWS role ARN for accuracy and check that the AWS role has the required Kinesis read permissions. For more information, see Create a policy in AWS.
    Ensure that a service account exists and that it is correctly configured as described in the procedure Create a role in AWS using a custom trust policy. For more information about how to create a service account, see Create a service account in Google Cloud.
    Verify that the Pub/Sub service account has the iam.serviceAccounts.getOpenIdToken permission. For more information, see Add the Service Account Token Creator role to the Pub/Sub service account.
    Verify that you have added the service account user role to the service account. For more information, see Add the service account user role to the service account.
    PUBLISH_PERMISSION_DENIED Error publishing to the topic due to permission issues. Grant the Pub/Sub service account the necessary publish permissions. For more information, see Add the Pub/Sub publisher role to the Pub/Sub service account.
    STREAM_NOT_FOUND The specified Kinesis stream cannot be found. Verify the stream ARN for accuracy.
    CONSUMER_NOT_FOUND The Kinesis consumer cannot be found. Verify the consumer ARN for accuracy. If necessary, re-create the consumer.

  8. If there are no errors till this point, check the best practices in Maintain a healthy publisher.

You can enable or disable Platform logs for Amazon Kinesis Data Streams import topics. However, Kinesis stream data is always compatible with Pub/Sub's message requirements and can be published. Therefore, no logs are written.