Monitoring Google Cloud with InfluxDB templates
Contributed by the Google Cloud community. Not official Google documentation.
Google Cloud allows businesses of all types to modernize their workloads on world-class infrastructure and help them to drive better decision making with intelligent analytics. Monitoring your Google Cloud infrastructure can help you to ensure that your technology is properly aligned with your goals as a business.
InfluxDB templates let you build and share comprehensive monitoring solutions for any technology in one open-source text file that can be imported into InfluxDB with a single command. After the templated monitoring solution is installed, you can modify the dashboards and configurations to meet your specific needs.
The Google Cloud Monitoring template for InfluxDB, provides a comprehensive view across your entire Google Cloud environment, containers, applications, and microservices. It provides pre-built and customizable dashboards to monitor the following:
- Cloud SQL instances
- Compute Engine instances
- Cloud Load Balancing instances
Some of the metrics that you can monitor using the template include the following:
- Basic status, including reserved cores, current state, and uptime
- Database server hardware utilization, including CPU utilization and memory utilization
- Pages read and written
- Sent and received rates
- Relative uptime
- Instance received and sent (measured in bytes)
- Backend latencies (mean)
- Platform total latencies (mean)
You can combine this with the Kubernetes template to get even more visibility into your GKE instances, as well as any metrics that you may have from other environments to get visibility across hybrid environments.
- Set up InfluxDB Cloud on Google Cloud.
- Install the Google Cloud Monitoring template for InfluxDB.
- Start the Google Cloud Monitoring solution.
This tutorial does not incur any costs on InfluxDB Cloud if using a Free-tier account.
This tutorial explains how to monitor existing Google Cloud resources but does not require any new resources on Google Cloud.
Before you begin
This tutorial assumes that you're using a Unix-like operating system, such as Linux or macOS.
You can also create InfluxDB Cloud usage-based plans through Google Marketplace.
Install Telegraf on your local machine with access to you Google Cloud environment, in one of your existing Google Cloud instances, or into a deployment if using a Kubernetes cluster.
Create or select a Google Cloud project in the Cloud Console. We recommend using an existing Google Cloud project with resources that you want to monitor.
Create a Google Cloud Monitoring stack in InfluxDB Cloud
- Generate a new token to be used by the Telegraf component by navigating in InfluxDB Cloud to Load Data > Tokens > Generate Read/Write Token for your Telegraf configuration.
- Use the token with the
influxcommand-line interface to install the Google Cloud Monitoring template, which creates a new stack in your InfluxDB Cloud account with dashboards for Google Cloud Monitoring.
Configure data collection with Telegraf
- In the InfluxDB Cloud UI, navigate to the Data page, and select the Telegraf tab.
- If the Google Cloud Monitoring template was installed successfully, a
GCP Stackdriverconfiguration appears as a Telegraf configuration.
GCP Stackdriverand copy the pre-populated Telegraf configuration, which should look something like this:
[agent] interval = "1m" flush_interval = "10s" metric_buffer_limit = 50000 debug = true omit_hostname = false [[outputs.influxdb_v2]] # Replace INFLUX_HOST,INFLUX_TOKEN, and INFLUX_ORG # with the values from your InfluxDB Cloud account urls = ["$INFLUX_HOST"] token = "$INFLUX_TOKEN" organization = "$INFLUX_ORG" [[inputs.stackdriver]] project = "$GCP_PROJECTNAME" #mygcpproject metric_type_prefix_include = [ "compute.googleapis.com/instance/", "loadbalancing.googleapis.com/", "cloudsql.googleapis.com/database/cpu/", "cloudsql.googleapis.com/database/disk/", "cloudsql.googleapis.com/database/disk/", "cloudsql.googleapis.com/database/network/", "cloudsql.googleapis.com/database/memory/", "cloudsql.googleapis.com/database/mysql/", ] # metric_type_prefix_exclude =  interval = "5m" # the more frequency the higher impact to your GCloud bills rate_limit = 20
Replace the contents of your
telegraf.confconfiguration file with the Telegraf configuration from the InfluxDB Cloud UI. By default, the Telegraf configuration file is located in the
Update the entries for
INFLUX_ORGwith the corresponding values from your InfluxDB Cloud instance. See the guide on manually configuring Telegraf to learn where to find these values.
After saving your changes, start the metric collection by starting (or restarting) the Telegraf service with the updated configuration file:
telegraf --config /etc/telegraf/telegraf.conf
Linux SysVinit and Upstart:
sudo service telegraf start
sudo systemctl telegraf start
Data will begin to flow to InfluxDB Cloud after a couple of minutes.
Navigate to one of the pre-built Google Cloud Monitoring dashboards in the InfluxDB Cloud UI.
Note: If you do not have
mysqlresources deployed, you won't receive any metrics for these services, so the corresponding dashboards will be empty.
You can also modify the pre-built dashboard layouts and underlying analysis.
Shut down the Telegraf agent
If you are on a usage-based plan on InfluxDB Cloud, you may be charged for collecting Google Cloud Monitoring metrics.
To stop collection of Google Cloud metrics, stop the Telegraf agent:
Linux SysVinit and Upstart:
sudo service telegraf stop
sudo systemctl telegraf stop
Remove the Google Cloud Monitoring resources in InfluxDB Cloud
Look up the Google Cloud Monitoring stack ID with the
In the output, the stack ID appears in the first column. For example:
ID OrgID Active ... 067f6daf0752b00x 5f57bfbb27a7decx true ...
Use the stack ID to delete the Google Cloud Monitoring stack:
influx stacks remove --stack-id 067f6daf0752b00x
When prompted to confirm removal of the stack and all associated resources, enter
If you created any Google Cloud resources specifically to monitor for this tutorial, shut them down, too.