Viewing Device Logs

Cloud IoT Core can optionally send device activity logs to Stackdriver Logging. Device activity logs include information such as device connections and errors. These are called events.

Authentication failures are not logged. A device must be authenticated to Cloud IoT Core to generate event logs.

Device events lifecycle

All Cloud IoT Core devices follow a similar lifecycle with the following events. Some of these events and their underlying details are captured in device logs.

  1. Connecting to Cloud IoT Core.
  2. Sending and receiving telemetry and or/state events.
  3. Disconnecting from or dropping the connection to Cloud IoT Core.

How these events occur and are logged is determined by whether the devices use the MQTT bridge or the HTTP bridge.

Logging device activity

Cloud IoT Core uses log levels to determine which device events are sent to Stackdriver Logging. You can set device log levels for a registry and all of its devices or for individual devices. The log level set for a registry applies to all devices within the registry. The log level set for an individual device overrides its registry's setting.

The following table describes the available device log levels:

Log level Description
None No device event logs are collected.
ERROR Captures all ERROR events, such as failed publishes. See the list of logged device events for a full list of ERROR events.
INFO (MQTT only) Captures all INFO events, such as connections and disconnections over MQTT. Also captures all ERROR events. See the list of logged device events for a full list of INFO events.
DEBUG Captures all DEBUG events, such as publishes, subscribes, and heartbeats. Useful for pinpointing problems with specific devices. Also captures all ERROR and INFO events. See the list of logged device events for a full list of DEBUG events. See Device debug logging for more information.

List of logged device events

The following tables show:

  • Which device events are logged.
  • The eventType of the event as logged in Stackdriver Logging.
  • Whether successes, failures, or both are logged for each event.
  • The log level that must be set on a registry or device to log the event.

MQTT bridge

Device event eventType Success Failure
Authenticate to server N/A Not logged* Not logged
Connect to server CONNECT INFO ERROR
Disconnect from server DISCONNECT INFO Not logged
Publish a telemetry event or state to server PUBLISH (on /devices/{device-id}/events or /devices/{device-id}/state MQTT topic) DEBUG ERROR
Receive a configuration update PUBLISH (on /devices/{device-id}/config MQTT topic) DEBUG ERROR
Subscribe to Cloud Pub/Sub configuration topic SUBSCRIBE DEBUG ERROR
Unsubscribe from Cloud Pub/Sub configuration topic UNSUBSCRIBE DEBUG ERROR
PUBACK received from server PUBACK DEBUG ERROR
Keep-alive ping sent to server PINGREQ DEBUG ERROR
Command sent to device from server PUBLISH DEBUG ERROR
Command PUBACK sent to server PUBACK DEBUG ERROR

* If MQTT authentication succeeds, then the device connects to Cloud IoT Core and a CONNECT event is logged rather than an "Authenticate to server" event.

HTTP bridge

Device event methodName Success Failure
Authenticate to server N/A Not logged** Not logged
Publish a telemetry event google.cloud.iot.v1.PublishEvent DEBUG ERROR
Set device state google.cloud.iot.v1.SetDeviceState DEBUG ERROR
Receive a configuration update google.cloud.iot.v1.GetDeviceConfig DEBUG ERROR

**If HTTP authentication succeeds, then the device connects to Cloud IoT Core and the relevant request message (GET if a configuration is received, or POST if a message is published) is logged.

Enabling, changing, and disabling device logging

You can choose what types of logs are reported to Stackdriver Logging when you create or edit a registry or device in GCP Console or with the gcloud tool.

Setting log levels for a registry

You can enable, change, or disable device logs for a registry using GCP Console or gcloud. Registry log settings automatically apply to all devices within the registry.

Console

To create or edit a registry and set its log level:

  1. Go to the Registries page in GCP Console.

    Go to the Registries page

  2. At the top of the page, click Create Registry.

    To edit an existing registry, click its ID on the Registries page, and then click Edit registry at the top of the page.

  3. Under Stackdriver Logging, select a log level.

  4. Click Create (if creating a new registry) or Update (if editing an existing registry).

gcloud

To create a registry and set its log level, run the gcloud iot registries create command with the --log-level flag:

gcloud iot registries create REGISTRY_ID \
    --project=PROJECT_ID \
    --region=REGION \
    [--event-notification-config=topic=TOPIC,[subfolder=SUBFOLDER] [--event-notification-config=...]]
    [--state-pubsub-topic=STATE_PUBSUB_TOPIC] \
    --log-level={NONE|INFO|ERROR|DEBUG}

To update a device's log level, run the gcloud iot registries update command with the --log-level flag:

gcloud iot registries update REGISTRY_ID \
    --project=PROJECT_ID \
    --region=REGION \
    --log-level={NONE|INFO|ERROR|DEBUG}

Setting log levels for a device

You can enable, change, or disable device logs for a device using GCP Console or gcloud. A device's log level overrides its registry's log level.

Console

To set the log level for a new or existing device:

  1. Go to the Registries page in GCP Console.

    Go to the Registries page

  2. Click the ID of the registry for the device.

  3. On the Registry details page, click Create device to create a new device.

    To edit an existing device, click its ID on the Registry details page, and then click Edit device at the top of the page.

  4. Under Stackdriver Logging, select a log level.

  5. Click Create (if creating a new device) or Update (if editing an existing device).

gcloud

To create a device and set a log level, run the gcloud iot devices create command with the --log-level flag:

gcloud iot devices create DEVICE_ID \
    --project=PROJECT_ID \
    --region=REGION \
    --registry=REGISTRY_ID \
    --public-key path=PUBLIC_KEY,type=TYPE \
    --log-level={NONE|INFO|ERROR|DEBUG}

To update a device's log level, run the gcloud iot devices update command with the --log-level flag:

gcloud iot devices update DEVICE_ID \
    --project=PROJECT_ID \
    --region=REGION \
    --registry=REGISTRY_ID \
    --log-level={NONE|INFO|ERROR|DEBUG}

Viewing logs

You can view device activity logs for your project in the Logs Viewer on Google Cloud Platform Console. Select device_activity in the log name menu:

Screenshot of the log name menu

For more details, see the following options and Logs Viewer filter interfaces.

Basic Viewer

You can use the Logs Viewer basic interface to retrieve your device log entries by doing the following:

  1. In the first menu, select the device resource type.
  2. In the second menu, select the device_activity log name. If you do not see the device_activity option, then there are no device logs available.

Advanced Viewer

  1. Switch to the advanced filter interface in the Logs Viewer.
  2. Create a filter that specifies the resource types and log names you want. For more information, see Monitored resource types.

API

To read your log entries through the Logging API, see entries.list.

gcloud

To read your log entries using gcloud, see Reading log entries.

Exporting device logs

You can export device logs in the same way you export other kinds of logs. For details about how to export your logs, see Exporting Logs.

The following examples describe why you might want to export device logs:

  • To keep device logs for a longer period of time or to use more powerful search capabilities, you can export copies of your device logs to Cloud Storage, BigQuery, or Cloud Pub/Sub. Using Cloud Pub/Sub, you can export to other applications, other repositories, and to third parties.

  • To manage your device logs across an entire organization, you can create aggregated export sinks that can export logs from any or all projects in the organization.

Device logging limits

Device log quotas and limits apply at the project level. They are measured separately from and do not count toward other Stackdriver Logging quotas and limits. If device log quotas are exhausted, Stackdriver Logging quotas for other services are unaffected. The reverse is also true.

Device log quotas are based on two factors:

  • The number of device events logged per second. Events include publishes, connections, disconnections, and so forth.
  • The total size (in bytes) of device events logged per minute.

If either of the device log quotas is exceeded, device logs stop temporarily.

For a list of device log quotas and limits, see Quotas & Limits.

Best practices

Device debug logging

As shown in the list of logged device events, setting the DEBUG log level for a registry or an individual device can generate large amounts of logging information. As a result, if you turn on debug logging for a registry with a large fleet of devices, logging records may be dropped due to the high velocity and quantity of logs.

As an example, suppose that you have a registry with 100,000 devices, and the debug log level is set for the registry. If each device publishes one telemetry event every second, then only 1,000 out of 100,000 telemetry events will be logged every second. This is because, as shown in Quotas & Limits, the maximum number of logged entries is 1,000 per second.

For best results, enable debug logging either for only a short period of time or for a small number of devices.

Troubleshooting errors when writing logs

When you first enable the Google Cloud IoT Core API for a project, a new service account for the project is automatically assigned a role (cloudiot.serviceAgent) that enables writing logs to Stackdriver Logging. If you later remove this default role from the relevant project service account, you might encounter errors. If you are unable to write device activity to Stackdriver Logging, complete the following steps:

  1. On the IAM page in Google Cloud Platform Console, verify that the role Cloud IoT Core Service Agent appears in the Members list for the relevant project service account. (Look for the project service account that ends in @gcp-sa-cloudiot.iam.gserviceaccount.com.)

  2. If the Cloud IoT Core Service Agent role does not appear in the Members list, use gcloud to add the cloudiot.serviceAgent role to the relevant project service account. This role includes permission to write logs to Stackdriver Logging.

    Run the following command to add the cloudiot.serviceAgent role to your project. To find the PROJECT_ID and PROJECT_NUMBER, refer to Identifying projects.

    gcloud projects add-iam-policy-binding PROJECT_ID \
      --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-cloudiot.iam.gserviceaccount.com \
      --role=roles/cloudiot.serviceAgent
    

Was this page helpful? Let us know how we did:

Send feedback about...

Google Cloud Internet of Things Core