The Logging agent streams logs from your VM instances and from selected third-party software packages to Cloud Logging. It is a best practice to run the Logging agent on all your VM instances.
The VM images for Compute Engine and Amazon Elastic Compute Cloud (EC2) don't include the Logging agent, so you must complete these steps to install it on those instances. The agent runs under both Linux and Windows.
If your VMs are running in Google Kubernetes Engine or App Engine, the agent is already included in the VM image, so you can skip this page.
Before you begin
To install the agent, ensure that you have the following:
A supported VM instance in a Google Cloud project or Amazon Web Services (AWS) account.
- When installing the Logging agent, a minimum of 250 MiB memory is required, but 1 GiB is recommended.
Also ensure your VM is running a supported operating system.
Credentials on the VM instance that authorize communication with Cloud Logging or Cloud Monitoring. Compute Engine VM instances generally have the correct credentials by default. If either of the following scenarios applies to you, then you might not have the proper credentials and must complete the Authorize the Logging agent procedures:
Running AWS EC2 VM instances, you must install authorization credentials on your VMs before installing the agent.
Running very old Compute Engine instances or Compute Engine instances created without the default credentials.
To check if you have the proper credentials, run the Verifying Compute Engine credentials procedures.
For AWS users, do the following:
Connect your AWS account to a Google Cloud. For information about this process, see Collect metrics from AWS accounts.
Refer to the Google Cloud projects for AWS EC2 VM instances section for additional information.
For pricing information, go to Pricing for Google Cloud's operations suite.
If you're using VMs that don't have access to remote package repositories, refer to the VMs without remote package access section for more information.
Installing the agent using the command line
To install the agent using the command line, use the following instructions.
Installing the latest version of the agent
To install the latest version of the agent, complete the following steps.
Linux
Open a terminal connection to your VM instance using SSH or a similar tool and ensure you have
sudo
access.Change to a directory you have write access to, for example your home directory.
Download and run the agent-installation script by using the following commands:
curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh sudo bash add-logging-agent-repo.sh --also-install
After it is installed, the agent is started automatically.
Windows
Connect to your instance using RDP or a similar tool and login to Windows.
Open a PowerShell terminal with administrator privileges by right-clicking the PowerShell icon and selecting Run as Administrator.
Run the following PowerShell commands:
(New-Object Net.WebClient).DownloadFile("https://dl.google.com/cloudagents/windows/StackdriverLogging-v1-21.exe", "${env:UserProfile}\StackdriverLogging-v1-21.exe") & "${env:UserProfile}\StackdriverLogging-v1-21.exe"
Installing a specific version of the agent
To install a specific version of the agent, complete the following steps.
Linux
Open a terminal connection to your VM instance using SSH or a similar tool and ensure you have
sudo
access.Change to a directory you have write access to, for example your home directory.
Download the agent installation script:
curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh
When running the
add-logging-agent-repo.sh
script, you can also set the following flags:--verbose
: Turns on verbose logging during the script execution.--also-install
: Installs the agent after adding the agent package repository.--version
: Sets the agent version for the script to install.--uninstall
: Uninstalls the agent.--remove-repo
: Removes the corresponding agent package repository after installing or uninstalling the agent.--dry-run
: Triggers only a dry run of the script execution and prints out the commands that it is supposed to execute.
See the script comments for more information and example usage.
Add the agent's package repository and install the agent:
To list the available agent versions in order to select which version to install, refer to Listing all agent versions.
For production environments, you might want to pin to a major version to avoid installing major versions that might include backward incompatible changes. To pin to a major version, run:
sudo bash add-logging-agent-repo.sh --also-install \ --version=MAJOR_VERSION.*.*
For example, to pin to the 1.x.x of the agent, run:
sudo bash add-logging-agent-repo.sh --also-install \ --version=1.*.*
To install a specific version of the agent, run:
sudo bash add-logging-agent-repo.sh --also-install \ --version=MAJOR_VERSION.MINOR_VERSION.PATCH_VERSION
Restart the agent service
A restart is required for the configurations that are installed by the catch-all packages above to take effect.
sudo service google-fluentd restart
You can delete the installation script after it runs successfully.
To verify that the agent is working as expected, run:
sudo service google-fluentd status
The status of the agent should be OK.
You can also examine the logs and ensure there are no errors:
tail /var/log/google-fluentd/google-fluentd.log
If you have trouble with the installation, refer to the Troubleshooting page.
Windows
Connect to your instance using RDP or a similar tool and login to Windows.
Open a PowerShell terminal with administrator privileges by right-clicking the PowerShell icon and selecting Run as Administrator.
Run the following PowerShell commands:
(New-Object Net.WebClient).DownloadFile("https://dl.google.com/cloudagents/windows/StackdriverLogging-v1-21.exe", "${env:UserProfile}\StackdriverLogging-v1-21.exe") & "${env:UserProfile}\StackdriverLogging-v1-21.exe"
Alternatively, you can browse to the following URL to download and run the agent's installer:
https://dl.google.com/cloudagents/windows/StackdriverLogging-v1-21.exe
To install the agent silently, append the
/S
option to the invocation of the installer:& "${env:UserProfile}\StackdriverLogging-v1-21.exe" /S
In “silent” mode use the
/D
option to specify the installation directory, for example:& "${env:UserProfile}\StackdriverLogging-v1-21.exe" /S /D="C:\Stackdriver\Google Cloud's operations suite\"
You can delete the installer when it completes successfully.
If you have trouble with the installation, refer to the Troubleshooting page.
Installing the agent using the Google Cloud console
You can install the agent on one or more Compute Engine VMs from the pre-configured Monitoring VM Instances dashboard.
To reach this dashboard, do the following:
-
In the Google Cloud console, select Monitoring, and then select
Dashboards, or click the following button:
In the Categories menu, click GCP, and then select the VM Instances entry.
The List view on the Inventory tab on the dashboard lists all VMs and includes a status column for your agent, as shown in the following screenshot:
The Agent column reports the following values:
Not detected: Either you don't have an agent installed or it is not running. If you aren't sure if you've installed an agent, then you can check the installed version. If you've installed the agent, then you can restart the agent. Otherwise, you can install an agent by doing the following:
- Select the VM instances on which you want to install agents.
Click the Install/update agents option on the Instances table, which brings up an installation workflow in the dashboard, and follow the instructions.
You can also install or update agents from the VM Details page for a specific VM.
The Ops Agent collects both metrics and logs by default. You can change this default behavior by configuring the Ops Agent.
Ops Agent: You are running the Ops Agent. If you don't see a green checkmark beside the entry, then there is an agent upgrade available, based on the detected operating system of your VM.
When you hover over the Ops Agent indicator in the table, you see information about the version of the Ops Agent. If you are running an older version, you also see a recommendation to upgrade your agent.
Pending: The Ops Agent is being installed or upgraded.
Legacy Agent: You are running the legacy Monitoring or the Logging agent. See Migrating from the legacy agents to the Ops Agent for information on transitioning to the Ops Agent.
Not applicable: This VM is not a supported platform for running the agent.
Unknown: The VM is not running, so the agent's status is not known.
Optional tasks
This section describes how to perform common maintenance tasks.
Enable structured logging
When you run the installation script on a Linux VM, the script by default
installs the google-fluentd-catch-all
package when the --also-install
flag
is present. In order to switch to structured logging, complete the following
steps.
Back up any local changes to
/etc/google-fluentd/config.d/
.Run one of the following commands, depending on your Linux distribution:
YUM:
sudo yum remove -y google-fluentd-catch-all-config sudo yum install -y google-fluentd-catch-all-config-structured sudo service google-fluentd restart
APT:
sudo apt-get remove -y google-fluentd-catch-all-config sudo apt-get install -y google-fluentd-catch-all-config-structured sudo service google-fluentd restart
For information about structured logging, refer to the structured logging guide.
Configuring the Logging agent
The agent comes preconfigured to monitor certain known log locations. On Linux,
those locations are described in the package google-fluentd-catch-all-config
,
which is automatically pulled in by the installation script. On Windows, the
agent monitors the Windows Event Log by default.
To adjust the agent configuration, see Configure the Logging agent.
Configuring an HTTP proxy
If you use an HTTP proxy for proxying requests to the Logging and Monitoring APIs, do the following:
Linux
Edit the following configuration file (create the file if it doesn't already exist):
/etc/default/google-fluentd
Add the following to the file:
export http_proxy="http://proxy-ip:proxy-port" export https_proxy="http://proxy-ip:proxy-port" export no_proxy=169.254.169.254 # Skip proxy for the local Metadata Server.
Restart the agent by running the following command on your VM instance:
sudo service google-fluentd restart
Windows
If you use an HTTP proxy, run the following command from an administrator command prompt. This sets the
http_proxy
andhttps_proxy
environment variables so that the agent can send data using outbound HTTPS:setx http_proxy http://proxy-ip:proxy-port /m setx https_proxy http://proxy-ip:proxy-port /m setx no_proxy 169.254.169.254 /m
Determining the agent version
To determine the version of the Logging agent on your system, run the following commands on your VM instance:
AMAZON LINUX AMI / CENTOS / RHEL
Run the following command on Amazon Linux, Red Hat, or CentOS Linux:
rpm --query --queryformat '%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n' google-fluentd google-fluentd-catch-all-config google-fluentd-catch-all-config-structured
DEBIAN / UBUNTU
Run the following command on Debian or Ubuntu:
dpkg-query --show --showformat '${Package} ${Version} ${Architecture} ${Status}\n' google-fluentd google-fluentd-catch-all-config google-fluentd-catch-all-config-structured
SLES / SUSE
Run the following command on SUSE:
rpm --query --queryformat '%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n' google-fluentd google-fluentd-catch-all-config google-fluentd-catch-all-config-structured
WINDOWS
Connect to your instance using RDP or a similar tool and login to Windows.
Open a PowerShell terminal with administrator privileges by right-clicking the PowerShell icon and selecting Run as Administrator.
Run the following PowerShell command:
reg query HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\GoogleStackdriverLoggingAgent\ /v Version
Restarting the agent
You must restart the Logging agent to pick up changes in configuration files. To restart the agent, use the following instructions.
LINUX
Run the following command on your instance:
sudo service google-fluentd restart
After restarting the Logging agent, you might want to send a test message.
Windows
Connect to your instance using RDP or a similar tool and login to Windows.
Open a PowerShell terminal with administrator privileges by right-clicking the PowerShell icon and selecting Run as Administrator.
Run the following PowerShell command:
Restart-Service -Name StackdriverLogging
Upgrading the agent
To upgrade the Logging agent to the latest release, use the following instructions:
Linux
To upgrade the agent to the latest version, run the following command:
sudo bash add-logging-agent-repo.sh --also-install
To upgrade the agent to the latest point release of a specific major version, run the following command:
sudo bash add-logging-agent-repo.sh --also-install \
--version=MAJOR_VERSION.*.*
Windows
To upgrade to the latest agent release, install the newest agent as described in Installing on Windows on this page. The installer prompts you to uninstall the previous version of the agent.
Listing all agent versions
To list the available versions of the agent, run the following command:
AMAZON LINUX AMI / CENTOS / RHEL
List the available versions of the agent:
sudo yum list --showduplicates google-fluentd
DEBIAN / UBUNTU
List the available versions of the agent:
sudo apt-cache madison google-fluentd
SLES / SUSE
List the available versions of the agent:
sudo zypper search -s google-fluentd
WINDOWS
Installing earlier versions of the agent on Windows is not supported.
Uninstalling the agent
To remove the Logging agent and its configuration files, use the following instructions.
After you uninstall the agent, the Google Cloud console might take up to one hour to report this change.
Linux
Run the following command:
sudo bash add-logging-agent-repo.sh --uninstall
Optionally, to remove the repository in addition to uninstalling the agent,
append --remove-repo
to the previous command.
Windows
In the Windows Control Panel, choose Uninstall a program. You should see the
Logging agent in the list of programs that you can uninstall. You can also
run uninstall.exe
from the directory where you installed the
Logging agent.
Information about Google Cloud projects for AWS EC2 VM instances
When the documentation refers to the Google Cloud project associated with your VM instance, for EC2 VM instances, this phrase refers to the AWS Connector project linked to your AWS account.
When you connect your AWS account to a Google Cloud, you create an AWS Connector project. For information about this process, see Collect metrics from AWS accounts.
To access the AWS Connector project for an AWS account, do one of the following:
Use the Google Cloud console project selector to identify the projects that match your AWS Connector project naming conventions, and then select the specific project for your AWS account.
Identify the Google Cloud project whose metrics scope includes your AWS account metrics and select that project in the Google Cloud console project selector. For this Google Cloud project, go to the Monitoring page and then select the Settings page. The Settings page lists the AWS Connector projects. You can use the Google Cloud console project selector to access the AWS Connector project.
VMs without remote package access
Installing the Logging agent requires access to remote package repositories, for both the agent package and (on Linux) its dependencies.
If you are using VPC-SC or a private network, the network configuration might also affect your ability to install agent dependencies from upstream repositories. The agent packages themselves are accessible by using Private Google Access. This can be configured following Enable Private Google Access.
If your VM host's security policy denies access to remote package repositories, we recommend creating a custom VM image with the agent pre-installed and disabling package management in that image.
What's next
Learn about what logs the agent sends to Cloud Logging using its default configuration.
For information on viewing your logs, see Using the Logs Explorer.