This guide provides basic information about the Stackdriver Logging agent, an application based on fluentd that runs on your virtual machine (VM) instances.
In its default configuration, the Stackdriver Logging agent streams logs from common third-party applications and system software to Stackdriver Logging; see the list of default logs. You can configure the agent to stream additional logs; see Configuring the Stackdriver Logging agent for details on agent configuration and operation.
It is a best practice to run the Stackdriver Logging agent on all your VM instances. The agent runs under both Linux and Windows. To install the Stackdriver Logging agent, see Installing the Logging Agent.
Supported operating systems
You can run the Stackdriver Logging agent on the following operating systems on compatible virtual machine (VM) instances:
- CentOS 6 and 7
- Debian 7 "Wheezy", Debian-7-backports, Debian 8 "Jessie", and Debian 9 "Stretch"
- Red Hat Enterprise Linux 6 and 7
- Ubuntu LTS 14.04 "Trusty", 15.04 "Vivid", LTS 16.04 "Xenial", 17.10 "Artful", and LTS 18.04 "Bionic"
- SUSE Linux Enterprise Server 12 SP3, 12 SP2 for SAP, and 12 SP3 for SAP
- Windows Server 2008 R2, 2012 R2, and 2016
- Amazon Linux AMI (except Amazon Linux 2.0 AMI)
- Container-Optimized OS (supported for Kubernetes Engine nodes only)
Supported environments
The Stackdriver Logging agent is compatible with the following environments:
-
Compute Engine instances. The Stackdriver Logging agent sends the logs to the project associated with each VM instance.
For instances without external IP addresses, you must enable Private Google Access to allow the Stackdriver Logging agent to send logs.
-
Amazon Web Services Elastic Compute Cloud (AWS EC2) instances. The Stackdriver Logging agent sends the logs to the AWS connector project for your Workspace. Stackdriver creates this project for you when you connect your AWS account to a Workspace.
For the Stackdriver Logging agent to function correctly, the Amazon EC2 instance it runs on must be able to communicate with Google Cloud APIs, particularly the Stackdriver Logging API. This requires either an external IP address or a VPC internet gateway.
For the VM instances above, a minimum of 250 MiB of resident (RSS) memory is required to run the Stackdriver Logging agent, but 1 GiB is recommended. For example, at a rate of 100 1-KB-sized log entries per second, the Stackdriver Logging agent with default configurations consumes 5% CPU on one core and 150 MiB memory. At a peak rate of 3,000 1-KB-sized log entries per second, the Stackdriver Logging agent, uses 80% CPU on one core and 250 MiB memory.
The following VM instances support Stackdriver Logging using their own software, possibly including custom versions or configurations of the Stackdriver Logging agent. Manually installing the Stackdriver Logging agent on them is not supported:
-
App Engine standard environment VM instances. App Engine includes built-in support for Stackdriver Logging. For more information, see Stackdriver Logging in App Engine Apps.
-
App Engine flexible environment VM instances. Apps running in the App Engine flexible environment can write logs that are in addition to what is included in the App Engine standard environment. For more information, see Stackdriver Logging and the App Engine flexible environment.
-
Kubernetes Engine node instances. You can enable support for Stackdriver Logging on your new or existing container clusters. For more information, see Enabling Stackdriver Logging for Kubernetes Engine.
Stackdriver Logging agent source code
You do not need the information in this section unless you want to understand the source code or you have other special needs. The Stackdriver Logging agent is installed by the script described in the installation instructions.
The Stackdriver Logging agent, google-fluentd, is a modified version of the
fluentd log data collector.
google-fluentd is distributed in two separate packages. The source code is
available from the associated GitHub repositories:
- The GitHub repository named
google-fluentdwhich includes the corefluentdprogram, the custom packaging scripts, and the output plugin for the Stackdriver Logging API.- The output plugin is packaged as a Ruby gem and is included in the
google-fluentdpackage. It is also available separately at the Ruby gem hosting service at fluent-plugin-google-cloud.
- The output plugin is packaged as a Ruby gem and is included in the
- The GitHub repository named
google-fluentd-catch-all-configwhich includes the configuration files for the Stackdriver Logging agent for ingesting the logs from various third-party software packages.
