Installing the Cloud Logging agent

The Logging agent streams logs from your VM instances and from selected third-party software packages to Cloud Logging. It is a best practice to run the Logging agent on all your VM instances.

The VM images for Compute Engine and Amazon Elastic Compute Cloud (EC2) don't include the Logging agent, so you must complete these steps to install it on those instances. The agent runs under both Linux and Windows.

Before you begin

To install the agent, ensure that you have the following:

  • A supported VM instance in a Google Cloud project or Amazon Web Services (AWS) account.

    • When installing the Logging agent, a minimum of 250 MiB memory is required, but 1 GiB is recommended.

    Also ensure your VM is running a supported operating system.

  • A Workspace monitoring the AWS account containing the VM instance. For pricing information, go to Pricing for Google Cloud's operations suite.

  • Credentials on the VM instance that authorize communication with Cloud Logging or Cloud Monitoring. Compute Engine VM instances generally have the correct credentials by default. If either of the following scenarios applies to you, then you might not have the proper credentials and must complete the Authorizing the agent procedures:

    • Running AWS EC2 VM instances, you must install authorization credentials on your VMs before installing the agent.

    • Running very old Compute Engine instances or Compute Engine instances created without the default credentials.

    To check if you have the proper credentials, run the Verifying Compute Engine credentials procedures.

Google Cloud projects for AWS EC2 VM instances

When the documentation refers to the Google Cloud project associated with your VM instance, for EC2 VM instances, this phrase refers to the AWS connector project linked to your AWS account.

When you connect your AWS account to a Workspace, the AWS connector project is created. The connector project has the following attributes:

  • A name that begins AWS Link

  • An ID that begins with aws-

    This ID is the Google Cloud project associated with your VM instance.

Refer to the Workspace page for more information about AWS connector projects.

VMs without remote package access

Installing the Logging agent requires access to remote package repositories, for both the agent package and (on Linux) its dependencies. If your VM host's security policy denies access to remote package repositories, we recommend creating a custom VM image with the agent pre-installed and disabling package management in that image.

Installing on Linux and Windows

To install the agent on VM instances running Linux or Microsoft Windows, use the following instructions.

AMAZON LINUX AMI / CENTOS / RHEL

  1. Open a terminal connection to your VM instance using SSH or a similar tool.

  2. Make sure you have sudo access.

  3. Change to a directory you have write access to, for example your home directory.

  4. Add the agent's package repository:

    curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh
    sudo bash add-logging-agent-repo.sh
    
  5. Install the agent:

    1. List the available versions of the agent in order to select which version to install:

      sudo yum list --showduplicates google-fluentd
      
    2. For production environments, you might want to pin to a major version to avoid pulling in major versions that might include backward incompatible changes. To pin to a major version, run:

      sudo yum install -y google-fluentd-[MAJOR_VERSION].*
      

      For example, to pin to the 1.x.x of the agent, run:

      sudo yum install -y google-fluentd-1.*
      
    3. If you don't want to pin to a major version, you can select one of the other installation options:

      1. To install a specific version of the agent, run:

          sudo yum install -y google-fluentd-[VERSION_NUMBER]
        
      2. To install the latest version of the agent, run:

          sudo yum install -y google-fluentd
        
  6. Install the Configuration files.

    1. For unstructured logging, run:

      sudo yum install -y google-fluentd-catch-all-config
      
    2. For structured logging, run:

      sudo yum install -y google-fluentd-catch-all-config-structured
      

      For information about structured logging, refer to the structured logging guide.

  7. Start the agent service

    sudo service google-fluentd start
    

You can delete the installation script after it runs successfully.

  • To verify that the agent is working as expected, run:

    sudo service google-fluentd status
    

    The status of the agent should be OK.

  • You can also examine the logs and ensure there are no errors:

    tail /var/log/google-fluentd/google-fluentd.log
    

If you have trouble with the installation, refer to the Troubleshooting page.

DEBIAN / UBUNTU

  1. Open a terminal connection to your VM instance using SSH or a similar tool.

  2. Make sure you have sudo access.

  3. Change to a directory you have write access to, for example your home directory.

  4. Add the agent's package repository:

    curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh
    sudo bash add-logging-agent-repo.sh
    sudo apt-get update
    
  5. Install the agent:

    1. List the available versions of the agent in order to select which version to install:

      sudo apt-cache madison google-fluentd
      
    2. For production environments, you might want to pin to a major version to avoid pulling in major versions that might include backward incompatible changes. To pin to a major version, run:

      sudo apt-get install -y 'google-fluentd=[MAJOR_VERSION].*'
      

      For example, to pin to the 1.x.x of the agent, run:

      sudo apt-get install -y 'google-fluentd=1.*'
      
    3. If you don't want to pin to a major version, you can select one of the other installation options:

      1. To install a specific version of the agent, run:

          sudo apt-get install -y google-fluentd=[VERSION_NUMBER]
        
      2. To install the latest version of the agent, run:

          sudo apt-get install google-fluentd
        
  6. Install the Configuration files.

    1. For unstructured logging, run:

      sudo apt-get install -y google-fluentd-catch-all-config
      
    2. For structured logging, run:

      sudo apt-get install -y google-fluentd-catch-all-config-structured
      

      For information about structured logging, refer to the structured logging guide.

  7. Start the agent service

    sudo service google-fluentd start
    

You can delete the installation script after it runs successfully.

  • To verify that the agent is working as expected, run:

    sudo service google-fluentd status
    

    The status of the agent should be OK.

  • You can also examine the logs and ensure there are no errors:

    tail /var/log/google-fluentd/google-fluentd.log
    

If you have trouble with the installation, refer to the Troubleshooting page.

SLES / SUSE

  1. Open a terminal connection to your VM instance using SSH or a similar tool.

  2. Make sure you have sudo access.

  3. Change to a directory you have write access to, for example your home directory.

  4. Add the agent's package repository:

    curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh
    sudo bash add-logging-agent-repo.sh
    
  5. Install the agent:

    1. List the available versions of the agent in order to select which version to install:

      sudo zypper search -s google-fluentd
      
    2. For production environments, you might want to pin to a major version to avoid pulling in major versions that might include backward incompatible changes. To pin to a major version, run:

      sudo zypper install -y 'google-fluentd<[MAJOR_VERSION].0.0'
      

      For example, to pin to the 1.x.x of the agent, run:

      sudo zypper install -y 'google-fluentd<1+1.0.0'
      
    3. If you don't want to pin to a major version, you can select one of the other installation options:

      1. To install a specific version of the agent, run:

          sudo zypper install -y google-fluentd=[VERSION_NUMBER]
        
      2. To install the latest version of the agent, run:

          sudo zypper install google-fluentd
        
  6. Install the Configuration files.

    1. For unstructured logging, run:

      sudo zypper install -y google-fluentd-catch-all-config
      
    2. For structured logging, run:

      sudo zypper install -y google-fluentd-catch-all-config-structured
      

      For information about structured logging, refer to the structured logging guide.

  7. Start the agent service

    sudo service google-fluentd start
    

You can delete the installation script after it runs successfully.

  • To verify that the agent is working as expected, run:

    sudo service google-fluentd status
    

    The status of the agent should be OK.

  • You can also examine the logs and ensure there are no errors:

    tail /var/log/google-fluentd/google-fluentd.log
    

If you have trouble with the installation, refer to the Troubleshooting page.

WINDOWS

To install the agent on a VM instance running Windows, perform the following steps:

  1. Connect to your instance using RDP or a similar tool and login to Windows.

  2. Open a PowerShell terminal, then run the following PowerShell commands. You don't need administrator privileges.

    cd $env:UserProfile;
    (New-Object Net.WebClient).DownloadFile("https://dl.google.com/cloudagents/windows/StackdriverLogging-v1-11.exe", ".\StackdriverLogging-v1-11.exe")
    .\StackdriverLogging-v1-11.exe
    

    Alternatively, you can browse to the following URL to download and run the agent's installer:

    https://dl.google.com/cloudagents/windows/StackdriverLogging-v1-11.exe

    To install the agent silently, append the /S option to the invocation of the installer:

    .\StackdriverLogging-v1-11.exe /S
    

    In “silent” mode use the /D option to specify the installation directory, for example:

    .\StackdriverLogging-v1-11.exe /S /D="C:\Google Cloud's operations suite\Logging\"
    

    You can delete the installer when it completes successfully.

    If you have trouble with the installation, refer to the Troubleshooting page.

Optional tasks

This section describes how to perform common maintenance tasks.

Configuring the agent

The agent comes preconfigured to monitor certain known log locations. On Linux, those locations are described in the package google-fluentd-catch-all-config, which is automatically pulled in by the installation script. On Windows, the agent monitors the Windows Event Log by default. If you want to adjust the agent configuration, see Configuring the Logging agent.

Configuring an HTTP proxy

If you use an HTTP proxy, do the following:

LINUX

  1. Edit the following configuration file:

     /etc/default/google-fluentd
    
  2. Add the following lines:

     export http_proxy="http://<PROXY_IP>:<PROXY_PORT>"
     export https_proxy="http://<PROXY_IP>:<PROXY_PORT>"
     export no_proxy=169.254.169.254  # Skip proxy for the local Metadata Server.
    
  3. Restart the Logging agent by running the following command on your VM instance.

     sudo service google-fluentd restart
    

WINDOWS

  1. If you use an HTTP proxy, run the following command from an administrator command prompt. This sets the http_proxy and https_proxy environment variables so that the agent can send data to Logging using outbound HTTPS:

    setx http_proxy http://<PROXY_IP>:<PROXY_PORT> /m
    setx https_proxy http://<PROXY_IP>:<PROXY_PORT> /m
    setx no_proxy 169.254.169.254 /m
    

Determining the agent version

To determine the version of the Logging agent on your system, run the following commands on your VM instance:

AMAZON LINUX AMI / CENTOS / RHEL

Run the following command Amazon Linux, Red Hat, or CentOS Linux:

rpm --query --queryformat '%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n' \
     google-fluentd \ 
     google-fluentd-catch-all-config \ 
     google-fluentd-catch-all-config-structured

DEBIAN / UBUNTU

Run the following command on Debian or Ubuntu:

dpkg-query --show --showformat \
    '${Package} ${Version} ${Architecture} ${Status}\n' \
     google-fluentd \ 
     google-fluentd-catch-all-config \ 
     google-fluentd-catch-all-config-structured

SLES / SUSE

Run the following command on SUSE:

rpm --query --queryformat '%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n' \
     google-fluentd \ 
     google-fluentd-catch-all-config \ 
     google-fluentd-catch-all-config-structured

WINDOWS

Run the following command on Windows:

reg query HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\GoogleStackdriverLoggingAgent\ /v Version

Restarting the agent

You must restart the Logging agent to pick up changes in configuration files. To restart the agent, use the following instructions.

LINUX

Run the following command on your instance:

 sudo service google-fluentd restart

After restarting the Logging agent, you might want to send a test message.

WINDOWS

Requires administrator privileges: To restart the agent, run the following command in a PowerShell terminal:

Restart-Service -Name StackdriverLogging

Upgrading the agent

To upgrade the Logging agent to the latest release, use the following instructions:

AMAZON LINUX AMI / CENTOS / RHEL

Run the following command on Amazon Linux, Red Hat, or CentOS Linux:

sudo yum upgrade google-fluentd

The previous command doesn't change the agent's configuration file. To get the latest default configuration and catch-all configuration files, run the following commands on your instance:

sudo yum upgrade google-fluentd google-fluentd-catch-all-config

DEBIAN / UBUNTU

Run the following commands on Debian or Ubuntu:

sudo apt-get update
sudo apt-get install --only-upgrade google-fluentd

The previous commands don't change the agent's configuration file. To get the latest default configuration and catch-all configuration files, run the following commands on your instance:

sudo apt-get install --only-upgrade -o Dpkg::Options::="--force-confnew" google-fluentd-catch-all-config
sudo apt-get install --only-upgrade google-fluentd

SLES / SUSE

Run the following command on SUSE:

sudo zypper update google-fluentd

The previous command doesn't change the agent's configuration file. To get the latest default configuration and catch-all configuration files, run the following commands on your instance:

sudo zypper update google-fluentd google-fluentd-catch-all-config

WINDOWS

To upgrade to the latest agent release, install the newest agent as described in Installing on Windows on this page. The installer prompts you to uninstall the previous version of the agent.

Uninstalling the agent

To remove the Logging agent and its configuration files, use the following instructions.

AMAZON LINUX AMI / CENTOS / RHEL

Run the following command on Amazon Linux, Red Hat, or CentOS Linux:

sudo service google-fluentd stop
sudo yum remove google-fluentd google-fluentd-catch-all-config

DEBIAN / UBUNTU

Run the following command on Debian or Ubuntu:

sudo service google-fluentd stop
sudo apt-get remove google-fluentd google-fluentd-catch-all-config

SLES / SUSE

Run the following command on SUSE Linux:

sudo service google-fluentd stop
sudo zypper remove google-fluentd google-fluentd-catch-all-config

WINDOWS

In the Windows Control Panel, choose Uninstall a program. You should see the Logging agent in the list of programs that you can uninstall.