The Logging agent streams logs from your VM instances and from selected third-party software packages to Cloud Logging. It is a best practice to run the Logging agent on all your VM instances.
The VM images for Compute Engine and Amazon Elastic Compute Cloud (EC2) don't include the Logging agent, so you must complete these steps to install it on those instances. The agent runs under both Linux and Windows.
If your VMs are running in Google Kubernetes Engine or App Engine, the agent is already included in the VM image, so you can skip this page.
If you are running specialized logging workloads that require higher throughput and/or improved resource-efficiency compared to the standard Cloud Logging agent, consider using the Ops agent.
Before you begin
To install the agent, ensure that you have the following:
A supported VM instance in a Google Cloud project or Amazon Web Services (AWS) account.
- When installing the Logging agent, a minimum of 250 MiB memory is required, but 1 GiB is recommended.
Also ensure your VM is running a supported operating system.
A Workspace monitoring the AWS account containing the VM instance. For pricing information, go to Pricing for Google Cloud's operations suite.
Credentials on the VM instance that authorize communication with Cloud Logging or Cloud Monitoring. Compute Engine VM instances generally have the correct credentials by default. If either of the following scenarios applies to you, then you might not have the proper credentials and must complete the Authorizing the agent procedures:
Running AWS EC2 VM instances, you must install authorization credentials on your VMs before installing the agent.
Running very old Compute Engine instances or Compute Engine instances created without the default credentials.
To check if you have the proper credentials, run the Verifying Compute Engine credentials procedures.
Google Cloud projects for AWS EC2 VM instances
When the documentation refers to the Google Cloud project associated with your VM instance, for EC2 VM instances, this phrase refers to the AWS connector project linked to your AWS account.
When you connect your AWS account to a Workspace, the AWS connector project is created. To identify the AWS connector project, go to the Workspace's Settings tab. The connector project are listed under the heading AWS Accounts. The Project ID column displays the Google Cloud projects associated with your Workspace.
Refer to the Workspace page for more information about AWS connector projects.
VMs without remote package access
Installing the Logging agent requires access to remote package repositories, for both the agent package and (on Linux) its dependencies.
If you are using VPC-SC or a private network, the network configuration might also affect your ability to install agent dependencies from upstream repositories. The agent packages themselves are accessible by using Private Google Access.
If your VM host's security policy denies access to remote package repositories, we recommend creating a custom VM image with the agent pre-installed and disabling package management in that image.
Installing the agent on a single VM
To install the agent, use the following instructions.
AMAZON LINUX AMI / CENTOS / RHEL
Open a terminal connection to your VM instance using SSH or a similar tool and ensure you have
sudo
access.Change to a directory you have write access to, for example your home directory.
Add the agent's package repository:
curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh sudo bash add-logging-agent-repo.sh
Install the agent:
List the available versions of the agent in order to select which version to install:
sudo yum list --showduplicates google-fluentd
For production environments, you might want to pin to a major version to avoid pulling in major versions that might include backward incompatible changes. To pin to a major version, run:
sudo yum install -y google-fluentd-major-version.*
For example, to pin to the 1.x.x of the agent, run:
sudo yum install -y google-fluentd-1.*
If you don't want to pin to a major version, you can select one of the other installation options:
To install a specific version of the agent, run:
sudo yum install -y google-fluentd-version-number
To install the latest version of the agent, run:
sudo yum install -y google-fluentd
Install the Configuration files.
For unstructured logging, run:
sudo yum install -y google-fluentd-catch-all-config
For structured logging, run:
sudo yum install -y google-fluentd-catch-all-config-structured
For information about structured logging, refer to the structured logging guide.
Restart the agent service
A restart is required for the configurations that are installed by the catch-all packages above to take effect.
sudo service google-fluentd restart
You can delete the installation script after it runs successfully.
To verify that the agent is working as expected, run:
sudo service google-fluentd status
The status of the agent should be OK.
You can also examine the logs and ensure there are no errors:
tail /var/log/google-fluentd/google-fluentd.log
If you have trouble with the installation, refer to the Troubleshooting page.
DEBIAN / UBUNTU
Open a terminal connection to your VM instance using SSH or a similar tool and ensure you have
sudo
access.Change to a directory you have write access to, for example your home directory.
Add the agent's package repository:
curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh && \ sudo bash add-logging-agent-repo.sh && \ sudo apt-get update
Install the agent:
List the available versions of the agent in order to select which version to install:
sudo apt-cache madison google-fluentd
For production environments, you might want to pin to a major version to avoid pulling in major versions that might include backward incompatible changes. To pin to a major version, run:
sudo apt-get install -y 'google-fluentd=major-version.*'
For example, to pin to the 1.x.x of the agent, run:
sudo apt-get install -y 'google-fluentd=1.*'
If you don't want to pin to a major version, you can select one of the other installation options:
To install a specific version of the agent, run:
sudo apt-get install -y google-fluentd=version-number
To install the latest version of the agent, run:
sudo apt-get install -y google-fluentd
Install the Configuration files.
For unstructured logging, run:
sudo apt-get install -y google-fluentd-catch-all-config
For structured logging, run:
sudo apt-get install -y google-fluentd-catch-all-config-structured
For information about structured logging, refer to the structured logging guide.
Restart the agent service
A restart is required for the configurations that are installed by the catch-all packages above to take effect.
sudo service google-fluentd restart
You can delete the installation script after it runs successfully.
To verify that the agent is working as expected, run:
sudo service google-fluentd status
The status of the agent should be OK.
You can also examine the logs and ensure there are no errors:
tail /var/log/google-fluentd/google-fluentd.log
If you have trouble with the installation, refer to the Troubleshooting page.
SLES / SUSE
Open a terminal connection to your VM instance using SSH or a similar tool and ensure you have
sudo
access.Change to a directory you have write access to, for example your home directory.
Add the agent's package repository:
curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh sudo bash add-logging-agent-repo.sh
Install the agent:
List the available versions of the agent in order to select which version to install:
sudo zypper search -s google-fluentd
For production environments, you might want to pin to a major version to avoid pulling in major versions that might include backward incompatible changes. To pin to a major version, run:
sudo zypper install -y 'google-fluentd<major-version.0.0'
For example, to pin to the 1.x.x of the agent, run:
sudo zypper install -y 'google-fluentd<2.0.0'
If you don't want to pin to a major version, you can select one of the other installation options:
To install a specific version of the agent, run:
sudo zypper install -y google-fluentd=version-number
To install the latest version of the agent, run:
sudo zypper install -y google-fluentd
Install the Configuration files.
For unstructured logging, run:
sudo zypper install -y google-fluentd-catch-all-config
For structured logging, run:
sudo zypper install -y google-fluentd-catch-all-config-structured
For information about structured logging, refer to the structured logging guide.
Restart the agent service
A restart is required for the configurations that are installed by the catch-all packages above to take effect.
sudo service google-fluentd restart
You can delete the installation script after it runs successfully.
To verify that the agent is working as expected, run:
sudo service google-fluentd status
The status of the agent should be OK.
You can also examine the logs and ensure there are no errors:
tail /var/log/google-fluentd/google-fluentd.log
If you have trouble with the installation, refer to the Troubleshooting page.
WINDOWS
To install the agent on a VM instance running Windows, perform the following steps:
Connect to your instance using RDP or a similar tool and login to Windows.
Open a PowerShell terminal with administrator privileges by right-clicking the PowerShell icon and selecting Run as Administrator.
Run the following PowerShell command:
(New-Object Net.WebClient).DownloadFile("https://dl.google.com/cloudagents/windows/StackdriverLogging-v1-14.exe", "${env:UserProfile}\StackdriverLogging-v1-14.exe") & "${env:UserProfile}\StackdriverLogging-v1-14.exe"
Alternatively, you can browse to the following URL to download and run the agent's installer:
https://dl.google.com/cloudagents/windows/StackdriverLogging-v1-14.exe
To install the agent silently, append the
/S
option to the invocation of the installer:.\StackdriverLogging-v1-14.exe /S
In “silent” mode use the
/D
option to specify the installation directory, for example:.\StackdriverLogging-v1-14.exe /S /D="C:\Google Cloud's operations suite\Logging\"
You can delete the installer when it completes successfully.
If you have trouble with the installation, refer to the Troubleshooting page.
Installing the agent on a Compute Engine VM
You can install the Logging agent on a single VM from the pre-configured Monitoring VM Instances dashboard.
To reach this dashboard, do the following:
- In the Cloud Console, select your Google Cloud project.
Go to Cloud Console - In the navigation pane, select Monitoring.
If you have never used Cloud Monitoring, then on your first access of Monitoring in the Google Cloud Console, a Workspace is automatically created and your project is associated with that Workspace. Otherwise, if your project isn't associated with a Workspace, then a dialog appears and you can either create a Workspace or add your project to an existing Workspace. We recommend that you create a Workspace. After you make your selection, click Add.
- In the Monitoring navigation pane, select Dashboards.
- In the table of dashboards, locate the VM Instances entry, and then click on the name.
The Inventory tab on the VM Instances dashboard lists all VMs and contains a status column for the agent, as shown in the following screenshot:
If the agent is not detected on a Compute Engine instance, you can click the "Not detected" link to bring up an installation workflow in the dashboard.
Optional tasks
This section describes how to perform common maintenance tasks.
Configuring the agent
The agent comes preconfigured to monitor certain known log locations. On Linux,
those locations are described in the package google-fluentd-catch-all-config
,
which is automatically pulled in by the installation script. On Windows, the
agent monitors the Windows Event Log by default.
To adjust the agent configuration, see Configuring the Logging agent.
Configuring an HTTP proxy
If you use an HTTP proxy for proxying requests to the Logging and Monitoring APIs, do the following:
LINUX
Edit the following configuration file (should be created if does not exist):
/etc/default/google-fluentd
Add the following lines:
export http_proxy="http://proxy-ip:proxy-port" export https_proxy="http://proxy-ip:proxy-port" export no_proxy=169.254.169.254 # Skip proxy for the local Metadata Server.
Restart the Logging agent by running the following command on your VM instance.
sudo service google-fluentd restart
WINDOWS
If you use an HTTP proxy, run the following command from an administrator command prompt. This sets the
http_proxy
andhttps_proxy
environment variables so that the agent can send data to Logging using outbound HTTPS:setx http_proxy http://proxy-ip:proxy-port /m setx https_proxy http://proxy-ip:proxy-port /m setx no_proxy 169.254.169.254 /m
Determining the agent version
To determine the version of the Logging agent on your system, run the following commands on your VM instance:
AMAZON LINUX AMI / CENTOS / RHEL
Run the following command on Amazon Linux, Red Hat, or CentOS Linux:
rpm --query --queryformat '%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n' \
google-fluentd
google-fluentd-catch-all-config
google-fluentd-catch-all-config-structured
DEBIAN / UBUNTU
Run the following command on Debian or Ubuntu:
dpkg-query --show --showformat \
'${Package} ${Version} ${Architecture} ${Status}\n' \
google-fluentd
google-fluentd-catch-all-config
google-fluentd-catch-all-config-structured
SLES / SUSE
Run the following command on SUSE:
rpm --query --queryformat '%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n' \
google-fluentd
google-fluentd-catch-all-config
google-fluentd-catch-all-config-structured
WINDOWS
Connect to your instance using RDP or a similar tool and login to Windows.
Open a PowerShell terminal with administrator privileges by right-clicking the PowerShell icon and selecting Run as Administrator.
Run the following PowerShell command:
reg query HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\GoogleStackdriverLoggingAgent\ /v Version
Restarting the agent
You must restart the Logging agent to pick up changes in configuration files. To restart the agent, use the following instructions.
LINUX
Run the following command on your instance:
sudo service google-fluentd restart
After restarting the Logging agent, you might want to send a test message.
WINDOWS
Connect to your instance using RDP or a similar tool and login to Windows.
Open a PowerShell terminal with administrator privileges by right-clicking the PowerShell icon and selecting Run as Administrator.
Run the following PowerShell command:
Restart-Service -Name StackdriverLogging
Upgrading the agent
To upgrade the Logging agent to the latest release, use the following instructions:
AMAZON LINUX AMI / CENTOS / RHEL
Run the following command on Amazon Linux, Red Hat, or CentOS Linux:
sudo yum upgrade google-fluentd
The previous command doesn't change the agent's configuration file. To get the latest default configuration and catch-all configuration files, run the following commands on your instance:
sudo yum upgrade google-fluentd google-fluentd-catch-all-config
DEBIAN / UBUNTU
Run the following commands on Debian or Ubuntu:
sudo apt-get update
sudo apt-get install -y --only-upgrade google-fluentd
The previous commands don't change the agent's configuration file. To get the latest default configuration and catch-all configuration files, run the following commands on your instance:
sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confnew" google-fluentd-catch-all-config
sudo apt-get install -y --only-upgrade google-fluentd
SLES / SUSE
Run the following command on SUSE:
sudo zypper update google-fluentd
The previous command doesn't change the agent's configuration file. To get the latest default configuration and catch-all configuration files, run the following commands on your instance:
sudo zypper update google-fluentd google-fluentd-catch-all-config
WINDOWS
To upgrade to the latest agent release, install the newest agent as described in Installing on Windows on this page. The installer prompts you to uninstall the previous version of the agent.
Uninstalling the agent
To remove the Logging agent and its configuration files, use the following instructions.
AMAZON LINUX AMI / CENTOS / RHEL
Run the following command on Amazon Linux, Red Hat, or CentOS Linux:
sudo service google-fluentd stop
sudo yum remove google-fluentd google-fluentd-catch-all-config
DEBIAN / UBUNTU
Run the following command on Debian or Ubuntu:
sudo service google-fluentd stop
sudo apt-get remove google-fluentd google-fluentd-catch-all-config
SLES / SUSE
Run the following command on SUSE Linux:
sudo service google-fluentd stop
sudo zypper remove google-fluentd google-fluentd-catch-all-config
WINDOWS
In the Windows Control Panel, choose Uninstall a program. You should see the Logging agent in the list of programs that you can uninstall.