This document describes a reference architecture for technical decision makers who want to connect devices and build IoT apps on Google Cloud using Intelligent Products Essentials.
Manufacturers want to continuously improve their products by adding intelligent capabilities that delight customers and monetize new features. This document helps architects tasked with designing intelligent product systems on Google Cloud that is scalable, reliable, secure, and cost-effective.
Google Cloud provides capabilities for connecting, ingesting, storing, analyzing, and retrieving data from products to build in artificial intelligence and machine learning capabilities, such as a personalized product ownership app or a digital twin simulation. This document presents an overview of an architecture, its high-level components, integration topics, and general design considerations. The architecture includes the following:
- Firebase for user app hosting
- BigQuery and Looker for analytics and business intelligence (BI)
- Cloud Run and Firestore for microservices infrastructure
Architecture
The following diagram shows the overall architecture, including the following components:
- Device telemetry streaming
- Microservices
- Analytics and business intelligence
- User app
- Anomaly detection
The diagram uses the following terms:
- Edge: The constrained-device edge or smart device edge as defined in the Linux Foundation continuum.
- Telemetry: Read-only data about the environment, usually collected through sensors. This data is often referred to as eyes-and-ears data (data that comes from facial recognition, image recognition, gesture recognition, speech recognition, natural language processing, and emotion recognition).
- CRM: Customer relationship management.
- ERP: Enterprise resource planning.
The numbered components in this architecture diagram are discussed in detail in the following sections. The architecture provides the foundation for analysis of device data (and fleets of devices), product anomaly detection, predictive maintenance, and enhancing the product owner experience.
Although this document provides some integration recommendations, it doesn't provide details such as device power and hardware, local operating system, compute and storage, and LAN/WAN communications.
Component 1: Device telemetry streaming
The device telemetry streaming component handles the streaming of device telemetry data.
An optional global, high-frequency, low-latency MQTT/HTTP bridge hands off data to Pub/Sub, a global asynchronous messaging service. Cloud Run then streams the messages to BigQuery for storage, archival, and analytics.
The following diagram shows the subcomponents of the device telemetry streaming component:
Component 2: Microservices
The microservices component includes the device, customer, and user app microservices, as shown in the following diagram:
Device API microservice
The Device API microservice enables device interactions such as device provisioning by the manufacturer, updating device configuration, and viewing device status. The Device Registry API runs on Cloud Run and stores device data in a Firestore database instance.
Customer API microservice
The Customer API microservice provides an API for customer information relevant to the product ownership experience, such as a unique customer ID for reference and customer name. The Customer API runs on Cloud Run. Data is stored in a Firestore database instance.
User app API microservice
The User app API microservice provides an API for use by user apps. It runs on Cloud Run and combines ownership data which is stored in Firestore with customer and device data from the customer and device API microservices respectively.
Ingestion
The Ingestion component is a function that loads microservice data stored in Firestore into the data warehouse, BigQuery. It runs periodically in Cloud Run triggered by Cloud Scheduler.
Component 3: Analytics and business intelligence
The following diagram shows the analytics and business intelligence (BI) component:
Data is extracted from various sources such as device registry and device telemetry components and optionally other enterprise data sources such as CRM, ERP, and service management. The architecture uses BigQuery as the data warehouse and Looker for visualization.
Component 4: User app
The following diagram shows the subcomponents of the user app component, which is a web and mobile app for the product user:
Firebase provides app development capabilities such as hosting, messaging, authentication, application analytics, and A/B testing. The app communciates with the user app API microservice described above.
Component 5: Anomaly detection
An anomaly detection service runs in Cloud Run and uses Timeseries Insights API or other backends to analyze telemetry data. When an anomaly is detected, the services generates a records in the data warehouse as well as alerts in Pub/Sub that can be used by various subscribers.
Integrating with other systems
This section recommends options from Google Cloud partners and other parties for user edge functions such as IoT clients and OTA solutions.
MLOps with Intelligent products
With device telemetry and other data in BigQuery and Cloud Storage, implement MLOps with intelligent products with Vertex AI for an end-to-end machine-learning lifecycle: prepare, build, validate, deploy, and operate ML models.
Existing and third-party IoT providers
The architecture integrates with existing and third-party IoT deployments. For example, you might already have an IoT implementation with IoT hardware, edge software, and an IoT backend either on-premises or in the cloud.
The following diagram shows how you can integrate existing IoT services with the architecture:
Telemetry data is sent from the service provider edge to Pub/Sub messaging and pre-processed with a service like Dataflow. The device registry information is periodically ingested in batch (ETL/ELT) operations into the data warehouse (BigQuery) for analytics.
Edge IoT client
MQTT and HTTP messaging are common protocols to send telemetry from devices to third-party IoT cloud services. Clients can publish telemetry events with MQTT by using a standard MQTT client.
OTA updates
To implement OTA updates, the architecture integrates with various OTA solutions, such as open source Mender and Hawkbit.
What's next
- MLOps with Intelligent Products Essentials
- Need more information? Contact us.
- Explore reference architectures, diagrams, and best practices about Google Cloud. Take a look at our Cloud Architecture Center.