Google Cloud IoT Core is being retired on August 16, 2023. Contact your Google Cloud account team for more information.

Viewing device logs

Cloud IoT Core can optionally send device activity logs to Cloud Logging. Device activity logs include information such as device connections and errors. These are called events.

Authentication failures are not logged. A device must be authenticated to Cloud IoT Core to generate event logs.

Device events lifecycle

All Cloud IoT Core devices follow a similar lifecycle with the following events. Some of these events and their underlying details are captured in device logs.

  1. Connecting to Cloud IoT Core.
  2. Sending and receiving telemetry and or/state events.
  3. Disconnecting from or dropping the connection to Cloud IoT Core.

How these events occur and are logged is determined by whether the devices use the MQTT bridge or the HTTP bridge.

Logging device activity

Cloud IoT Core uses log levels to determine which device events are sent to Cloud Logging. You can set device log levels for a registry and all of its devices or for individual devices. The log level set for a registry applies to all devices within the registry. The log level set for an individual device overrides its registry's setting.

The following table describes the available device log levels:

Log level Description
None No device event logs are collected.
ERROR Captures all ERROR events, such as failed publishes. See the list of logged device events for a full list of ERROR events.
INFO (MQTT only) Captures all INFO events, such as connections and disconnections over MQTT. Also captures all ERROR events. See the list of logged device events for a full list of INFO events.
DEBUG Captures all DEBUG events, such as publishes, subscribes, and heartbeats. Useful for pinpointing problems with specific devices. Also captures all ERROR and INFO events. See the list of logged device events for a full list of DEBUG events. See Device debug logging for more information.

List of logged device events

The following tables show:

  • Which device events are logged.
  • The eventType of the event as logged in Cloud Logging.
  • Whether successes, failures, or both are logged for each event.
  • The log level that must be set on a registry or device to log the event.

MQTT bridge

Device event eventType Success Failure
Authenticate to server N/A Not logged* Not logged
Connect to server CONNECT INFO ERROR
Disconnect from server DISCONNECT INFO Not logged
Attach a device to a gateway ATTACH_TO_GATEWAY INFO ERROR
Detach a device from a gateway DETACH_FROM_GATEWAY INFO ERROR
Publish a telemetry event or state to server PUBLISH (on /devices/{device-id}/events or /devices/{device-id}/state MQTT topic) DEBUG ERROR
Receive a configuration update PUBLISH (on /devices/{device-id}/config MQTT topic) DEBUG ERROR
Subscribe to Pub/Sub configuration topic SUBSCRIBE DEBUG ERROR
Unsubscribe from Pub/Sub configuration topic UNSUBSCRIBE DEBUG ERROR
PUBACK received from server PUBACK DEBUG ERROR
Keep-alive ping sent to server PINGREQ DEBUG ERROR
Command sent to device from server PUBLISH DEBUG ERROR
Command PUBACK sent to server PUBACK DEBUG ERROR

* If MQTT authentication succeeds, then the device connects to Cloud IoT Core and a CONNECT event is logged rather than an "Authenticate to server" event.

HTTP bridge

Device event methodName Success Failure
Authenticate to server N/A Not logged** Not logged
Publish a telemetry event google.cloud.iot.v1.PublishEvent DEBUG ERROR
Set device state google.cloud.iot.v1.SetDeviceState DEBUG ERROR
Receive a configuration update google.cloud.iot.v1.GetDeviceConfig DEBUG ERROR

**If HTTP authentication succeeds, then the device connects to Cloud IoT Core and the relevant request message (GET if a configuration is received, or POST if a message is published) is logged.

Enabling, changing, and disabling device logging

You can choose what types of logs are reported to Cloud Logging when you create or edit a registry or device in Google Cloud console or with the gcloud tool.

Setting log levels for a registry

You can enable, change, or disable device logs for a registry using Google Cloud console or gcloud. Registry log settings automatically apply to all devices within the registry.

Console

To create or edit a registry and set its log level:

  1. Go to the Registries page in Google Cloud console.

    Go to the Registries page

  2. At the top of the page, click Create Registry.

    To edit an existing registry, click its ID on the Registries page, and then click Edit registry at the top of the page.

  3. Under Stackdriver Logging, select a log level.

  4. Click Create (if creating a new registry) or Update (if editing an existing registry).

gcloud

To create a registry and set its log level, run the gcloud iot registries create command with the --log-level flag:

gcloud iot registries create REGISTRY_ID \
    --project=PROJECT_ID \
    --region=REGION \
    [--event-notification-config=topic=TOPIC,[subfolder=SUBFOLDER] [--event-notification-config=...]]
    [--state-pubsub-topic=STATE_PUBSUB_TOPIC] \
    --log-level={NONE|INFO|ERROR|DEBUG}

To update a device's log level, run the gcloud iot registries update command with the --log-level flag:

gcloud iot registries update REGISTRY_ID \
    --project=PROJECT_ID \
    --region=REGION \
    --log-level={NONE|INFO|ERROR|DEBUG}

Setting log levels for a device

You can enable, change, or disable device logs for a device using Google Cloud console or gcloud. A device's log level overrides its registry's log level.

Console

To set the log level for a new or existing device:

  1. Go to the Registries page in Google Cloud console.

    Go to the Registries page

  2. Click the ID of the registry for the device.

  3. In the registry menu on the left, click Devices.

  4. To create a new device, click Create a device.

    To edit an existing device, click its ID on the Devices page, and then click Edit device at the top of the page.

  5. Under Stackdriver Logging, select a log level.

  6. Click Create (if creating a new device) or Update (if editing an existing device).

gcloud

To create a device and set a log level, run the gcloud iot devices create command with the --log-level flag:

gcloud iot devices create DEVICE_ID \
    --project=PROJECT_ID \
    --region=REGION \
    --registry=REGISTRY_ID \
    --public-key path=PUBLIC_KEY,type=TYPE \
    --log-level={NONE|INFO|ERROR|DEBUG}

To update a device's log level, run the gcloud iot devices update command with the --log-level flag:

gcloud iot devices update DEVICE_ID \
    --project=PROJECT_ID \
    --region=REGION \
    --registry=REGISTRY_ID \
    --log-level={NONE|INFO|ERROR|DEBUG}

Viewing logs

You can view device activity logs for your project in the Logs Explorer on Google Cloud console. Select device_activity in the log name menu.

For more details, see Using the Logs Explorer.

API

To read your log entries through the Logging API, see entries.list.

gcloud

To read your log entries using gcloud, see Reading log entries.

Exporting device logs

You can export device logs in the same way you export other kinds of logs. For details about how to export your logs, see Exporting Logs.

The following examples describe why you might want to export device logs:

  • To keep device logs for a longer period of time or to use more powerful search capabilities, you can export copies of your device logs to Cloud Storage, BigQuery, or Pub/Sub. Using Pub/Sub, you can export to other applications, other repositories, and to third parties.

  • To manage your device logs across an entire organization, you can create aggregated export sinks that can export logs from any or all projects in the organization.

Device logging limits

Device log quotas and limits apply at the project level. They are measured separately from and do not count toward other Cloud Logging quotas and limits. If device log quotas are exhausted, Cloud Logging quotas for other services are unaffected. The reverse is also true.

Device log quotas are based on two factors:

  • The number of device events logged per second. Events include publishes, connections, disconnections, and so forth.
  • The total size (in bytes) of device events logged per minute.

If either of the device log quotas is exceeded, device logs stop temporarily.

For a list of device log quotas and limits, see Quotas & Limits.

Best practices

Device debug logging

As shown in the list of logged device events, setting the DEBUG log level for a registry or an individual device can generate large amounts of logging information. As a result, if you turn on debug logging for a registry with a large fleet of devices, logging records may be dropped due to the high velocity and quantity of logs.

As an example, suppose that you have a registry with 100,000 devices, and the debug log level is set for the registry. If each device publishes one telemetry event every second, then only 1,000 out of 100,000 telemetry events will be logged every second. This is because, as shown in Quotas & Limits, the maximum number of logged entries is 1,000 per second.

For best results, enable debug logging either for only a short period of time or for a small number of devices.

Troubleshooting errors when writing logs

When you first enable the Google Cloud IoT Core API for a project, a new service account for the project is automatically assigned a role (cloudiot.serviceAgent) that enables writing logs to Cloud Logging. If you later remove this default role from the relevant project service account, you might encounter errors. If you are unable to write device activity to Cloud Logging, complete the following steps:

  1. On the IAM page in Google Cloud console, verify that the role Cloud IoT Core Service Agent appears in the Members list for the relevant project service account. (Look for the project service account that ends in @gcp-sa-cloudiot.iam.gserviceaccount.com.)

  2. If the Cloud IoT Core Service Agent role does not appear in the Members list, use gcloud to add the cloudiot.serviceAgent role to the relevant project service account. This role includes permission to write logs to Cloud Logging.

    Run the following command to add the cloudiot.serviceAgent role to your project. To find the PROJECT_ID and PROJECT_NUMBER, refer to Identifying projects.

    gcloud projects add-iam-policy-binding PROJECT_ID \
  --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-cloudiot.iam.gserviceaccount.com \
  --role=roles/cloudiot.serviceAgent