Migrate environments from IoT Core

Last reviewed 2023-02-01 UTC

Google has announced the deprecation of IoT Core. This document is intended to help you design and implement a migration plan from an IoT Core-based environment to a new environment that doesn't depend on IoT Core for any of the following:

  • Edge device authentication
  • Edge device management
  • Communication between your edge devices and Google Cloud

The document also provides guidance for if you're evaluating the opportunity to migrate following the deprecation of IoT Core and want to explore what a migration might look like.

This document is part of a series of documents that provide information about IoT architectures on Google Cloud and about migrating from IoT Core. The other documents in this series include the following:

The workload to migrate

This document assumes that the workloads that you want to migrate away from IoT Core have the following parts:

  • A part that runs on edge devices (deployed at the edges of your environment, and next to the data that you want to process)
  • A backend that runs on Google Cloud

The following diagram shows the architecture of a typical workload that uses IoT Core. In this architecture, Cloud IoT integrates edge devices that have a backend that runs on Google Cloud.

Flow of events from edge devices to Cloud IoT (summarized in following text).

The preceding diagram can be summarized as follows:

To effectively plan your migration, we recommend that you make an assessment to gain a full understanding of your source environment architecture. In this case, source environment refers to your current, IoT Core-based environment.

This document assumes that you can update the configuration of and the software components running on your edge devices for your migration. In some cases, this approach might be infeasible. For example, your edge devices or your deployment processes might not support this use case. In this case, we recommend that you plan to retire the edge devices that don't support updates. For more information about designing and implementing automated provisioning and configuration processes for edge devices, see Best practices for automatically provisioning and configuring edge and bare metal systems and servers.

Design the migration

To migrate your IoT Core-based environment, we recommend that you follow the migration framework described in Migration to Google Cloud.

The following diagram describes the path of your migration journey:

Migration path with four phases.

As shown in the preceding diagram, this journey has four phases:

  1. Assess: In this phase, you assess your source environment, assess the workloads and the edge devices that you want to migrate away from Cloud IoT Core.
  2. Plan: In this phase, you create the basic infrastructure on Google Cloud, such as provisioning the resource hierarchy and setting up network access.
  3. Deploy: In this phase, you deploy the new solution to use instead of IoT Core, and migrate your edge devices to the new solution.
  4. Optimize: In this phase, you optimize your target environment. In this case, target environment refers to the environment you're migrating to that is not based on IoT Core.

Assess the source environment and workloads

In the assessment phase, you gather information about your source environment, the edge devices, and the use of IoT Core in your organization. This information helps you to design a migration plan and to ensure that you have the resources that you need for the migration and for your target environment.

In the assessment phase, you do the following:

  1. Build an inventory of the edge devices that you registered to Cloud IoT Core.
  2. Build an inventory of the backend workloads that integrate with Cloud IoT Core.
  3. Categorize your edge devices and backend workloads.
  4. Experiment and design proofs of concept.
  5. Calculate total cost of ownership.
  6. Design the architecture of the target environment.
  7. Choose the edge devices and the backend workloads to migrate first.

At the end of the assessment phase, you have two inventories: an inventory for your edge devices and an inventory for your backend workloads.

To avoid inconsistencies, we recommend that you pause the deployment of new edge devices and backend workloads before you build these inventories. We also recommend that you don't deploy new edge devices and background workloads after you create the inventories.

Build an inventory of your edge devices

To scope your migration and to design your migration plan, you need to know how many edge devices exist in your source environment. You must also understand how the devices interact with IoT Core, and if you can categorize them by common characteristics, by behavior, by purpose, or by dependencies.

Each edge device that you register to IoT Core belongs to an IoT Core registry. The first step for building the inventory of your edge devices is to list the IoT Core registries you created. You then gather information about the edge devices registered in each registry.

To build the inventory of your edge devices, consider the following information for each edge device, and how the device integrates with IoT Core:

  • Identifiers: Gather the following information about the IoT Core identifiers of the edge device:

    • The user-defined identifier
    • The server-defined, non-editable ID that IoT Core automatically generates when you register an edge device to IoT Core
    • The resource name that uniquely identifies the edge device using the identifier of the IoT Core registry where you registered the edge device

  • Deployment state: Assess the current deployment state of the edge device. For example, the edge device might be in one of the following states:

    • Not yet manufactured, or currently being manufactured
    • Ready to be deployed, but not yet registered to IoT Core
    • Already deployed in its destination site, and registered to IoT Core

  • IoT Core device type: Assess the IoT Core device type. Each edge device that you register to IoT Core can act in one of two ways. It can be a client that connects directly to IoT Core. Or, it can be a gateway for the clients that you cannot or don't want to connect to IoT Core directly.

  • Communication protocol: IoT Core supports two protocols to communicate with edge devices: HTTP and MQTT. Assess which protocol your edge devices use to communicate with IoT Core. For the MQTT protocol, you must also determine the quality of service that your edge devices and backend workloads rely on.

  • Credentials: IoT Core authenticates edge devices using a key pair and short-lived tokens generated using that key pair. It can optionally verify the signature of the public part of the key pair using a certificate-based verification method. Assess how the authentication of the edge device is configured. Check if you're using the certificate-based verification mechanism for the IoT Core registry that the device belongs to.

  • Device metadata: In IoT Core, you can define metadata for each edge device, in the form of key-value pairs. For example, you can define a hardware thumbprint, a serial number, manufacturer information, or any other attribute that is relevant for an edge device. You can define metadata when you add an edge device to an IoT Core registry, or edit an edge device that is already in a registry. Metadata is never sent to or from an edge device through IoT Core. Gather information about the metadata you defined for the edge device.

  • Device state: In IoT Core, each edge device can report information about its state as arbitrary structured or unstructured data. For example, an edge device might report the version of the firmware that it's running. Or, it might report information about its health, according to specific metrics. IoT Core publishes the received information about the device state as messages in a Pub/Sub topic that you configure. Assess how your edge device reports information about its state, and to which Pub/Sub topics IoT Core publishes those messages to. Determine which components of your architecture rely on information about the edge device state.

  • Telemetry events: Each edge device that you add to an IoT Core registry can send telemetry events as arbitrary structured or unstructured data to IoT Core. IoT Core publishes the received telemetry events as messages in a Pub/Sub topic that you configure. Assess how your edge device reports telemetry events, and to which Pub/Sub topics IoT Core publishes those messages to. Determine which components of your architecture rely on telemetry events reported by the edge device.

  • Device configuration: In IoT Core, you can define the configuration of an edge device as arbitrary structured or unstructured data. IoT Core also lets you define updates to the configuration of a device as new versions of such configuration that will then push to the edge device. Assess if the edge device receives configuration from Cloud IoT Core, and gather information about all the configuration versions you defined.

  • Commands: In IoT Core, edge devices can receive commands from IoT Core and then react accordingly to those commands. Assess if your edge devices support reacting to commands coming from IoT Core.

  • Software and configuration updates: During the migration, you might need to update the software components running on the edge device, or the configuration of those components. Assess the update mechanisms that your edge device supports. Determine if the device also supports a rollback mechanism to get it back to a known working state if there are issues and problems during such updates.

  • Downtime: During the migration, backend workloads or other parts of the source environment might be unavailable. Assess if your edge device supports downtimes, its fallback mechanisms, and how it recovers after the downtime.

Build an inventory of your backend workloads that integrate with IoT Core

After building the inventory of your edge devices, you gather information about the backend workloads in your source environment that integrate with Cloud IoT Core. A backend workload can integrate with IoT Core in the following ways:

  • By sending commands to edge devices, and updating the configuration of your edge devices using IoT Core.
  • By subscribing to Pub/Sub topics to which IoT Core publishes messages about edge device telemetry events and device state.
  • By integrating with IoT Core APIs either directly, or using an infrastructure provisioning tool. For example, you might be using Terraform to provision IoT Core registries and devices.

To build the inventory of the backend workloads that integrate with Cloud IoT Core, consider the following for each backend workload:

  • Commands and device configuration: Assess if the backend workload sends commands to edge devices, and if it updates the device configuration. Both actions require an integration with IoT Core APIs.
  • Telemetry events and device state: Assess if the backend workload subscribes to the Pub/Sub topics where IoT Core publishes messages about telemetry events and device state.

  • Integration with other IoT Core APIs: Assess if the backend workload interacts with any other IoT Core APIs, besides the ones to send commands and update device configurations. For example, your backend workload might rely on IoT Core APIs to do the following:

    • Create IoT Core registries and update their configuration.
    • Create IoT Core devices and update their configuration.
    • Gather information about IoT Core registries and devices.
    • Use IoT Core logging metrics and device activity logs.

Categorize your edge devices and backend workloads

After you create the inventories of your edge devices and your backend workload, you categorize the items in each inventory based on their characteristics. This categorization helps you to draft your migration plan and choose which edge devices and backend workloads to migrate first.

To categorize your edge devices, we recommend a categorization based on the kinds of interactions that can happen between edge devices and backend workloads. Consider the following kinds of interactions:

  • When an edge device sends data to backend workloads using telemetry events or information about device state.
  • When backend workloads send directives to edge devices using commands or device configuration updates.

For each of the preceding kinds of interaction, the message types exchanged during interactions of that kind are different. However, the messages have similar purposes. Some devices send data from an edge device to backend workloads, such as telemetry events and information about device state. Some devices send directives from backend workloads to edge devices, such as commands and device configuration updates.

Based on the proposed kinds of interactions, we recommend the following categories for your edge devices:

  • Transmit-only: Edge devices that send telemetry events or information about device state, but don't receive commands or device configuration updates from backend workloads.
  • Receive-only: Edge devices that don't send telemetry events or information about device state, but receive commands or device configuration updates from backend workloads.
  • Receive and transmit: Edge devices that send telemetry events and information about device state, and receive commands or device configuration updates from backend workloads.

To categorize your backend workloads, you can follow an approach that is similar to the one you followed to categorize edge devices. Based on the proposed kinds of interactions, we recommend the following categories for your backend workloads:

  • Receive-only: Backend workloads that receive telemetry events or information about device state from edge devices, but don't send commands or device configuration updates.
  • Send-only: Backend workloads that don't receive telemetry events or information about device state, but send commands or device configuration updates.
  • Send and receive: Backend workloads that receive telemetry events or information about device state from edge devices, and send commands or device configuration updates.

Complete the assessment

After building the inventories, you must complete the following parts of the assessment phase:

After completing these activities, continue reading this document.

Design the architecture of the target environment

After completing the previous assessment activities, you design the architecture for the target environment.

This document focuses on migrating the source environment to the target environment. However, migrating your environment from IoT Core is also an opportunity for you to plan for new features and updates. When you design the architecture of the target environment, think about any limitations that you might have experienced in the source environment. Consider how you'd like to configure the target environment to avoid those limitations. You can also consider any potential new features that you might require in the target environment.

Based on how you categorized your edge devices and backend workloads, you might see the following complementary IoT Core use cases emerging from the assessment of the source environment:

  • Ingestion of data coming from edge devices into a backend running on Google Cloud.
  • Running edge device management backend workloads on Google Cloud.

To reduce the complexity of your migration, we recommend that you focus on the use cases that emerged from the assessment of your source environment. For example, if you're ingesting data coming from edge devices but you're not using any of the IoT Core device management features, we recommend that you focus on designing the target environment. This approach lets you support the data ingestion use case only, with no need to consider the device management use case.

The design of the target environment can vary depending on which of these Cloud IoT Core use cases you implemented in the source environment, and how you want to implement them in the target environment. You must consider the following factors:

  • If you implemented both use cases in the source environment, you can design the target environment to implement both use cases with a single solution. You can also implement the two use cases separately using distinct solutions.
  • If you implement only one of the two use cases in the source environment, you can design the target environment to implement that use case with a single solution.

The following diagram shows a series of example questions to consider when you decide how to design the architecture of the target environment.

Example questions, stated in following text.

The preceding diagram can be summarized as follows:

  • Do you need to both ingest data from edge devices, and manage edge devices?

    • If yes, continue to the next question.
    • If no, continue to the edge device management use case questions.

  • Do you need a single solution to implement both data ingestion and edge device management use cases?

    • If yes, deploy a solution for both data ingestion and edge device management on Google Cloud.
    • If no, deploy an edge device management solution on Google Cloud, and then continue to the data ingestion use case evaluation questions.

  • Do you need to manage edge devices?

    • If yes, deploy an edge device management solution on Google Cloud, and then continue to the data ingestion use case evaluation questions.
    • If no, continue to the data ingestion use case evaluation questions.

  • Do you need to ingest data coming from edge devices?

    • If yes, go to the next question.
    • If no, either you completed the migration, or you don't need to migrate your source environment. In both cases, you can decommission the source environment.

  • What is the preferred communication protocol?

    • If MQTT, deploy an MQTT broker on Google Cloud.
    • If HTTP or gRPC, ingest data coming from edge devices using Pub/Sub.
    • Otherwise, evaluate data ingestion solutions that are compatible with your preferred communication protocols.

When designing the architecture of the target environment, consider the following:

  • Managing any component of the architecture requires knowledge and effort. We recommend that you evaluate how many additional resources you need to account for the target environment.
  • Provisioning many edge devices poses security, scalability, and operational challenges. For more information about provisioning edge devices, refer to Best practices for automatically provisioning and configuring edge and bare metal systems and servers.
  • Using Pub/Sub to ingest data from your edge devices frees you from managing and scaling a distributed messaging platform. If you use Pub/Sub to ingest data coming from your edge devices, consider both Pub/Sub quotas and limits, especially if you need to ingest data from many devices.

  • To authenticate your edge devices against the target environment and manage their identities, we recommend that you assess which authentication methods and credential stores the target environment supports. Consider how they compare to the ones you're using with IoT Core in the source environment.

    After you gather this information, we recommend that you follow the guidance in the IoT backend security guide to design and implement an authentication and identity management mechanism for your edge devices.

Choose which edge devices and the backend workloads to migrate first

After designing the architecture of the target environment, you define the following:

  1. The categories of edge devices and backend workloads to migrate first.
  2. The migration batches (the groups of items to migrate from the source environment to the target environment).

Define the categories of edge devices to migrate first

The categories of edge devices and backend workloads might offer different challenges and difficulties to migrate. One example would be the migration of transmit-only edge devices, which might be easier than migrating receive and transmit edge devices.

To develop a deeper understanding of how to choose which categories of edge devices and backend workloads to migrate first, see Choosing the apps to migrate first.

This section summarizes the considerations that you must make for each edge device category when you decide which to migrate first.

Transmit-only edge devices

These edge devices send telemetry events and information about device state using either MQTT or HTTP.

If the devices use MQTT, you might only need to update their configuration to connect to and authenticate against the MQTT broker in your target environment. You can keep publishing telemetry events and information about device state through the MQTT broker in the target environment. In some cases, you might not have an MQTT broker in your target environment, and need to migrate to a different kind of target environment, such as a third-party solution. In this case, you need to assess the capabilities and the integration interfaces that the solution provides. You can then design and implement a suitable migration plan.

If the devices use HTTP, you might need to update their configuration to connect to and authenticate against the target environment. You might also need to refactor the semantics of how the devices communicate to account for the differences in your target environment when compared to using IoT Core APIs. For example, if you're using Pub/Sub in the target environment, you might migrate from using IoT Core APIs to publish messages to Pub/Sub topics to using Pub/Sub APIs for the same purpose. In some cases, you might not use Pub/Sub in your target environment, and therefore need to migrate to a different kind of target environment, such as a third-party solution. In this case, you must assess the capabilities and the integration interfaces that the third-party solution provides to design and implement a suitable migration plan.

Receive-only edge devices

These edge devices receive commands using MQTT and configuration updates using either MQTT or HTTP. IoT Core doesn't support the use of HTTP to send commands.

If the devices receive commands and configuration updates using MQTT, similar considerations as the previous edge device category apply. To migrate this category of edge devices, you update the configuration of the edge devices to connect to and authenticate against the MQTT broker in your target environment. You keep subscribing to MQTT topics where IoT Core publishes commands and device configuration updates. In some cases, you might not have an MQTT broker in your target environment, and need to migrate to a different kind of target environment, such as a third-party solution. In this case, you need to assess the capabilities and the integration interfaces that the solution provides to design and implement a suitable migration plan.

If the devices receive configuration updates using HTTP, similar considerations as the previous edge device category apply. To migrate this category of edge devices, you might need to update their configuration to connect to and authenticate against the target environment. To receive configuration updates, you might also need to refactor the semantics of the communication to account for the differences in your target environment, compared to using IoT Core APIs. For example, if you're migrating to a different kind of target environment, such as a third-party solution, you need to assess the capabilities and the integration interfaces that the solution provides to design and implement a suitable migration plan.

Receive and transmit edge devices

These edge devices might be the hardest to migrate because they are both consumers of data coming from backend workloads, and producers of data that edge devices receive. The considerations to migrate the categories of edge devices discussed above both apply in this case, so you need particular care to handle the migration of this backend workload category.

After choosing which categories of edge devices to migrate first, you choose which categories of backend workloads to migrate first.

Receive-only backend workloads

These backend workloads are decoupled from edge devices that produce telemetry events or information about device state, so, for the following reasons, can be relatively simple to migrate:

  • Backend workloads subscribe to Pub/Sub topics. For this reason, the devices don't need to know about the producers of those pieces of information to consume it. You might not need to update the configuration of or the software running on your edge devices.
  • Backend workloads don't send commands or device configuration updates to edge devices. Therefore, you don't need to take this use case into account during the migration of these backend workloads.
  • You might keep the existing Pub/Sub topics to publish or consume messages. In such a case, your backend workloads can stay subscribed to the existing Pub/Sub topics if your target environment continues forwarding telemetry events and information about device state to those Pub/Sub topics.

Send-only backend workloads

These backend workloads require a comprehensive assessment to understand how they interact with edge devices when sending commands and device configuration updates, and how to migrate them to the target environment. For example, if you're migrating to a target environment with an MQTT broker, those backend workloads might migrate from using IoT Core APIs to send commands or device configuration updates to publish messages through MQTT. You might not need to perform a software or configuration update on your edge devices in some cases. For example, if backend workloads publish commands and configuration updates in the same format and to the same MQTT topics where IoT Core publishes messages about commands and device configuration updates in your source environment. If you're migrating to a different kind of target environment, such as a third-party solution, you need to assess the capabilities and the integration interfaces that the solution provides to design and implement a suitable migration plan.

Send and receive backend workloads

These backend workloads might be the hardest to migrate because they are both consumers of data coming from edge devices, and producers of data that edge devices receive. The considerations to migrate the categories of backend workloads discussed above both apply in this case, so you need particular care to handle the migration of this backend workload category.

Define migration batches

To reduce the risks and the complexity of migrating a high number of items in a single batch, you divide the items to migrate in each category in migration batches. To plan migration batches, you do the following:

  1. Design migration batches by grouping homogeneous items: To group the items to migrate in batches, we recommend that you choose a set of criteria so that the items in a migration batch share common characteristics. For example, you might group edge devices in batches according to the following:

    • The deployment region
    • The IoT Core registry that the devices are registered to
    • If there is a meaningful IoT Core device metadata set
    • The deployment state of the devices

  2. Decide the size of each migration batch: For each category of items to migrate, we recommend that you plan the first migration batches in that category to be relatively small. You increase the size of the batches as you gain experience and momentum during the migration.

  3. Assess if your migration batches need ad-hoc strategies: Depending on how you grouped the items to migrate in migration batches, the migration strategy to apply for a given migration batch might depend on the characteristics of the items in that batch. For example, to migrate edge devices grouped by deployment state, you must make the following considerations:

    • If your devices are not yet manufactured, or currently being manufactured, you can instruct the manufacturer to update their configuration and the software to migrate them to the target environment.
    • If your devices are ready to be deployed, but not yet registered to IoT Core, you can instruct the deployer to recall those edge devices. You can then update their configuration and the software to migrate them to the target environment.
    • If your devices are already deployed in their destination site, and registered to IoT Core, you can update their configuration and the software to migrate them to the target environment, either remotely or on-site.

Plan and build your foundation

In the planning and building phase, you provision and configure the cloud infrastructure and services that support your workloads on Google Cloud as follows:

  1. Build a resource hierarchy.
  2. Configure Identity and Access Management.
  3. Set up billing.
  4. Set up network connectivity.
  5. Harden your security.
  6. Set up monitoring and alerting.

For guidance about how to build the cloud infrastructure and services that support your workloads and their dependencies, see Migration to Google Cloud: Building your foundation. Follow these guidelines to build a foundation for your environments. You then continue with the activities described in the next sections of this document.

Migrate the edge devices and backend workloads

After you build your foundation for your target environment, you must do the following to migrate your edge devices and backend workloads to the target environment.

  1. Provision and configure resources to implement the architecture of the target environment: As the first step of the migration process, you create and set up the infrastructure of the new platform.
  2. Migrate edge devices and backend workloads to the target environment: After you verify that the target environment is ready, you then migrate the backend workloads and edge devices to the target environment. Depending on the architecture of your target environment, and your use cases, there might be different approaches to migrate. This document discusses a two-step migration strategy which allows your source and target environment to co-exist for a period of time. This approach means that if there are failures during the migration, you can roll back to the source environment.
  3. Decommission the source environment: After you have verified that the target environment is fully operational, you decommission the source environment.

Provision and configure resources for the architecture of the target environment

In this phase, you provision and configure the target environment. As described in Design the architecture of the target environment, the architecture of the target environment can be summarized as follows:

  • An MQTT broker running on Google Cloud: You run an MQTT broker on Google Cloud and forward telemetry events and information about device state from the MQTT broker to backend workloads. Your backend workloads publish commands and controls to the MQTT topic.
  • Pub/Sub: Your edge devices publish telemetry events and information about device state to Pub/Sub and receive commands from Pub/Sub.
  • A third-party data ingestion and device management platform: You set up a third-party solution for telemetry events and information about device state ingestion and device management.

For more information about each architecture, see connected devices architectures on Google Cloud.

Migrate edge devices and backend workloads to the target environment

After you provision and configure the resources in your target environment, you migrate edge devices and backend workloads to the target environment. In this section, you migrate edge devices and backend workloads from the source environment to the target environment. The source and target environment co-exist until you decommission the source environment.

To reduce downtimes, the migration process has the following phases:

  1. Monitoring the source and the target environments.
  2. Migrating the edge device metadata information from the source environment to the target environment. This includes device credentials, device configuration and device state.
  3. Updating edge devices to connect to both the source and the target environment.
  4. Migrating backend workloads from the source environment to the target environment.
  5. Updating edge devices to connect to only the source environment.

We recommend that you monitor both the source and the target environment during each migration phase, and that you verify the results of each phase before moving to the next phase.

In addition to monitoring the environment, you might want to introduce black-box testing to verify if the environment works as expected. An example of such a test would be a use case where your backend workload sends an email notification to operators when it detects a specific event, such as a temperature greater than 50 degrees celsius. You might create a test case which has telemetry data for a temperature greater than 50 degrees celsius, and test if the backend workload sends an email to operators.

Monitor the source and the target environments

To monitor the source and the target environments, we recommend that you consider the following metrics:

  • Active device count: The number of devices that have recently sent data to IoT Core.
  • Device communication error count: The number of errors that backend workloads encountered when communicating with edge devices, grouped by error type, in a given period. This metric is useful to understand if backend workloads are having issues communicating with edge devices.
  • Device operation count: The number of operations performed by edge devices, such as connection or disconnection requests, message publishing, grouped by operation type, in a given period. This metric helps you understand if edge devices are running as expected. For example, if both Device error count and Device operation count values are increasing, the environment might have issues sending messages to the edge devices.
  • Received bytes count: The number of bytes received from edge devices in a given period. This metric helps you reason about network ingress traffic statistics.
  • Sent bytes count: Delta count of the number of bytes sent to edge devices. This metric helps you reason about network egress traffic statistics.
  • Message throughput: The number of messages that backend workloads processed in a given period. This metric helps you understand if the environment scales according to the volume of traffic between edge devices and backend workloads. For example, if both active device count and device operation count go up, but message throughput does not change too much, you might want to check if backend workloads have sufficient resources to handle increasing messages.
  • Message delivery latency: The amount of time elapsed after an edge device publishes a message and before a backend workload receives it for processing. For example, if the latency value goes up, you might want to check if there are issues that slow the message delivery.
  • Undeliverable messages: The number of messages that can't be delivered to edge devices and backend workloads. Failure to deliver messages to consumers might mean that the edge devices or backend workloads might not be responding.
  • Cloud resource quota usage: Monitor cloud resource quota usage to ensure that the environment has sufficient resources to scale.
Monitor the source environment

Cloud Monitoring automatically collects metrics from IoT Core and Pub/Sub. For example, IoT Core exposes the device/active_devices, device/error_count, and device/operation_count metrics. This data helps you to understand how many edge devices connected to the source environment, and how many edge devices are facing errors communicating with IoT Core. The device/received_bytes_count metrics and device/sent_bytes_count metrics help you to monitor network bandwidth consumption.

To monitor message delivery status and health, you use Monitoring Query Language to measure the delivery latency health score for a subscription, message throughput, and undeliverable messages.

Monitor the target environment

You monitor the target environment to understand if the migration is successful. Depending on the architecture of the target environment, the MQTT broker or the third-party IoT platform might provide the following metrics:

  • MQTT broker: If the target environment is based on an MQTT broker, the broker might provide metrics about edge devices and message delivery. To monitor bytes sent and received, refer to the metrics provided by Cloud Load Balancing. If the MQTT broker is running on a GKE cluster, you can configure Cloud Monitoring to define which metrics are sent to Monitoring. If the MQTT broker is running on a Compute Engine instance, you can use the default dashboard or install Ops Agent to collect detailed telemetry from the Cloud Monitoring instance.

  • Pub/Sub: If the target environment is based on Pub/Sub, you use Pub/Sub topics and subscriptions. For example, you can use the Monitoring Query Language to conduct a query to fetch the delivery latency health score for a subscription, message throughput and undeliverable messages.

  • IoT platform: If the target environment is based on an IoT platform, the platform might provide information about edge devices and message delivery. If the third-party IoT platform is running on a GKE cluster, you can configure Logging and Monitoring to configure which metrics are sent to Cloud Monitoring. If the third-party IoT platform is running on a Cloud Monitoring instance, you can use the default dashboard or install Ops Agent to collect detailed telemetry from the Cloud Monitoring instance.

Migrate the edge device metadata information from the source environment to the target environment

To migrate to the new IoT platform, you migrate edge devices metadata information to the target environment. To migrate the edge device metadata, consider the following metadata categories:

  • Device credentials: IoT Core authenticates edge devices using a key pair and short-lived tokens. Follow the required steps of the target environment to register devices to the target environment and create device credentials in the target environment according to its authentication mechanism.

  • Device configuration: It might be that your target environment is a third party IoT platform that provides device configuration service and your use case requires edge devices to be configured with the latest desired state. In this case, you need to migrate device configuration to the target environment. During the migration, ensure that device configuration is in sync between the source environment and the target environment. If your target environment is based on an MQTT broker or Pub/Sub and it doesn't provide a way to manage device configuration, you might want to store device configurations in Cloud Storage buckets as a long term archive.

  • Information about device state: Ensure that edge devices update their state when they connect to the target environment for the first time, so that the target environment has the most up to date information about device state.

After you complete this step, verify if the required device information and credentials are correctly configured and that edge devices can connect to and authenticate against the target environment.

Update edge devices to connect to both the source and the target environment

When you reach this stage, your target environment is ready to accept connections from edge devices. You can update edge devices to connect them to both the source and the target environment to send telemetry events and information about device state. When updating edge devices, the approach that you must take depends on the edge device category.

For edge devices that do send telemetry events or information about device state, but don't receive commands or device configuration updates from backend workloads, you do the following:

  1. Update the edge devices to have them send telemetry events and information about device state to both the source and the target environment.
  2. Verify that the target environment correctly receives telemetry events and information about device state.

Conversely, for edge devices that don't send telemetry events or information about device state, but do receive commands or configuration updates from backend workloads, you do the following:

  1. Update edge devices to receive commands and configuration updates from the target environment.
  2. Ensure that your edge devices report the result of command execution or configuration updates to the target environment.
  3. Send a command and a configuration update from the target environment to the edge devices.
  4. Verify that the execution of the command and of the configuration update are successful.

For edge devices that both send telemetry events and information about device state and also receive commands or configuration updates from backend workloads, you do the following:

  1. Update the edge devices to have them send telemetry events and information about device state to both the source and the target environment.
  2. Verify that the target environment correctly receives telemetry events and information about device state.
  3. Update the edge devices codes to receive commands and configuration updates from the target environment.
  4. Ensure that your edge devices report the result of command execution or configuration updates to the target environment.
  5. Send a command and a configuration update from the target environment to the edge devices.
  6. Verify that the execution of the command and of the configuration update are successful.

After you have run these steps for your use case, all of the categories of edge device do the following:

  • Connect to both the source and the target environment.
  • Send telemetry events and information about device state to both the source and the target environment.
  • Receive commands and configuration updates from the source environment only because you didn't migrate backend workloads yet.

It's best to avoid a scenario where backend workloads process the same message that edge devices send both to the source environment and the target environment. We recommend that you configure the message retention period in the target environment at the minimum value possible. This approach lets you verify that the target environment is working as expected. It also lets you check that messages in the target environment expire before migrating backend workloads. You can tune the message retention configuration in the target environment after the next migration step.

If your edge devices cannot connect to both the source and the target environment at the same time because of technical or regulatory reasons, you configure the edge device to disconnect from the source environment first. You can then connect to the target environment only. In this case, the backend workloads that are still connected to the source environment stop receiving telemetry events and information about device state from edge devices. The devices can no longer send commands and configuration updates to edge devices.

We also recommend that you provision and configure a buffer storage mechanism. This approach helps you to avoid losing data when your device sends telemetry events and information about device state to the target environment when your backend workloads are still connected to the source environment. The backend workloads can then consume this information when they connect to the target environment. For example, you can configure the message retention of the target environment, based on an MQTT broker, Pub/Sub, or on an IoT platform. This approach lets you keep unacknowledged messages available for the time necessary for you to complete the next phase of the migration, as described in the following section.

Migrate backend workloads from the source environment to the target environment

You migrate your backend workloads to the target environment. Depending on the architecture of your target environment, you must take different approaches to migrate the workload.

MQTT Broker on Google Cloud: If your target environment is based on an MQTT broker, the following factors guide your migration approach:

  • For backend workloads that do receive telemetry events or information about device state from edge devices, but don't send commands or device configuration updates: Configure your backend workloads to subscribe to MQTT topics to receive telemetry events and information about device state coming from edge devices.
  • Conversely, for backend workloads that don't receive telemetry events or information about device state, but do send commands or device configuration updates: Configure your backend workloads to publish messages to send commands and configuration updates to MQTT topics for command and configuration updates in the target environment.
  • For backend workloads that both receive telemetry events or information about device state from edge devices and send commands or device configuration updates: Configure your backend workloads to subscribe to MQTT topics to receive telemetry, then you configure your backend workloads to publish messages to send commands and configuration updates to the MQTT topics.

Pub/Sub: If your target environment is based on Pub/Sub, the following factors guide your migration approach:

  • For backend workloads that do receive telemetry events or information about device state from edge devices, but don't send commands or device configuration updates: Create new Pub/Sub subscriptions in the target environment, and update your backend workloads to consume the newly created subscriptions.
  • Conversely, for backend workloads that don't receive telemetry events or information about device state, but do send commands or device configuration updates: Create Pub/Sub topics and configure your backend workloads to publish messages to send commands and configuration updates to Pub/Sub topics.
  • For backend workloads that both receive telemetry events or information about device state from edge devices and send commands or device configuration updates: Configure your backend workloads to subscribe to Pub/Sub topics to receive telemetry events and information about device state. Then, configure your backend workloads to publish messages to send commands and configuration updates to Pub/Sub topics.

Third-party IoT platform: If your target environment is based on a third party IoT platform, you must follow the third-party IoT platform instructions to set up integrations between backend workloads and the IoT platform. You then verify that backend workloads are able to receive telemetry events and information about device state coming from edge devices. You also check that backend workloads can publish messages to send commands or device configuration updates to the edge device.

To verify that edge devices and backend workloads are working as expected, we recommend that you do the following:

  • Verify that backend workloads receive telemetry events and information about device state and react correctly. For example, if your backend workloads generate a near-real time dashboard to monitor specific telemetry data, verify that the dashboard is updated with the latest data period.
  • Verify that backend workloads send commands and configuration updates to edge devices as expected. Verify that edge devices also react as you expect.
  • Verify if the edge device reports telemetry events and information about device state to the target environment.

At this point, backend workloads do the following:

  • Connect to the target environment.
  • Receive telemetry events and information about device state from edge devices from the target environment.
  • Send commands and configuration updates to edge devices from the target environment.

You can now update the configuration of the message retention of the target environment that you set to a minimum value when you connect edge devices to both the source and the target environment, and set it according to your requirements.

When you update the configuration of backend workloads to receive telemetry events and information about device state from the target environment, backend workloads might need time to apply this updated configuration. During the transitory phase, backend workloads cannot consume telemetry events and information about device state that edge devices send. If your use cases require complete data integrity, you might need to configure the message retention period of the target environment before you update the configuration of backend workloads. This approach ensures that messages don't expire before backend workloads can apply the new configuration and consume the messages.

Update edge devices to connect to only the target environment

At this point, you have successfully migrated your edge devices to the target environment, however, they are also still using the source environment. To complete the migration step, update your edge devices to connect to the target environment only by removing connections to and integration with IoT Core. After this update is completed, your edge devices only connect to the target environment.

Decommission the source environment

After you migrate edge devices and backend workloads to the target environment, and you validate the target environment, you decommission the source environment.

To decommission the source environment, you do the following:

  1. Delete the Pub/Sub subscriptions that subscribe to the IoT Core topics.
  2. Delete unused Pub/Sub topics. If you reuse Pub/Sub topics, ensure you do not delete the topics created by IoT Core. You can find the Pub/Sub topics used by IoT Core by using the IoT Core console.
  3. Delete the IoT Core devices and registries.

Optimize your environment after migration

Optimization is the last phase of your migration. In this phase, you make your environment more efficient than it was before. In this phase, you execute multiple iterations of a repeatable loop until your environment meets your optimization requirements. The steps of this repeatable loop are as follows:

  1. Assessing your current environment, teams, and optimization loop.
  2. Establishing your optimization requirements and goals.
  3. Optimizing your environment and your teams.
  4. Tuning the optimization loop.

The following sections rely on Migration to Google Cloud: Optimizing your environment.

Assess your target environment, teams, and optimization loop

While the first assessment that you make focuses on the source environment, this phase of assessment is for the optimization phase. For more information about how to assess your target environment, teams, and optimization loop, see Measure your environment, teams, and optimization loop.

Establish your optimization requirements

Review the following optimization requirements that you might need to make for your target environment:

  • Set up automatic scaling: Use Google Cloud services such as Managed Instance Group or Google Kubernetes Engine to automatically horizontally or vertically scale your IoT solution and backend workloads when loads increase. This approach helps to ensure that your device registration and telemetry data storage can to handle a larger volume of data when you deploy a large device fleet. Because Spanner is a distributed, highly available and scalable transactional database, it's a good candidate to store your telemetry data and device registration information.
  • Enhance the logging and monitoring mechanism: Optimize and integrate your logging and monitoring mechanism to form a centralized solution. You might also want to improve certain monitoring metrics to help you understand how edge devices interact with your IoT solution. You should also log and correlate activities such as connect events, disconnect events, and telemetry events. We also recommend that you monitor system and application errors. If possible, set up an alert when certain critical failures occur at the system level.
  • Protect your workloads by using Google Cloud security services: Security Command Center is a centralized vulnerability and threat reporting service which you can use to help to strengthen your security posture by evaluating your security and data attack surface. It provides asset inventory and discovery, and can help you to identify misconfigurations, vulnerabilities, and threats. Security Command Center also helps you to mitigate and remediate risks. To learn how to help secure your workloads that run on Google Kubernetes Engine (GKE), see the Google Kubernetes Engine security overview to understand how to secure your GKE workloads.

Complete the optimization

After you populate the list of your optimization requirements, you complete the optimization phase. To learn how to do this, see Migration to Google Cloud: Optimizing your environment.

What's next