Installing the Logging Agent

The Stackdriver Logging agent streams logs from your VM instances and from selected third party software packages to Stackdriver Logging. Using the agent is optional but we recommend it. The agent runs under both Linux and Microsoft Windows.

Before you begin

Check that the agent supports your VM instance and learn about Stackdriver accounts.

Supported VM instances

Some kinds of VM instances already contain the Logging agent or software similar to the Logging agent. For others, you should install the agent using the instructions on this page:

  • Google App Engine: You do not need the agent. Skip this page.
  • Google Container Engine: You do not need the agent. Skip this page.
  • Google Compute Engine: Install the agent on your VM instances. The VM instances already have the required authorization. See Installing on Linux and Windows.
  • Amazon EC2: Install authorization credentials on your VM instances, and then install the agent. See Authorizing the agent and then Installing on Linux and Windows.

For more information about the agent and the VM instances it supports, see Logging Agent.

Stackdriver accounts

Stackdriver accounts support Stackdriver Monitoring and Stackdriver billing.

If you are using GCP VM instances, then you can use Stackdriver Logging without a Stackdriver account. If your instances are associated with a Stackdriver account, then the account can have either Basic Tier or Premium Tier service.

If you are using AWS VM instances, then you must link your AWS account to a Premium Tier Stackdriver account in order to use Stackdriver Logging.

For more information, see Service Tiers and Creating a Stackdriver account.

Installing on Linux and Windows

Use the following instructions to install the agent on VM instances running Linux or Microsoft Windows. If you have problems with installation, see Troubleshooting.

Linux instance

  1. Open a terminal connection to your VM instance using SSH or a similar tool.

  2. Download the Logging agent's installation script by running the following two commands on your VM instance:

    curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh
    sha256sum install-logging-agent.sh

  3. Verify that the output of the preceding checksum command,sha256sum, is the same as the following:

    993a99ad560017aa561e94829f2dc0cbdf534e4aafed13b0716068e2162d0efe  install-logging-agent.sh
    

  4. Run the installation script with the following command:

    sudo bash install-logging-agent.sh

    You can delete the installation script when it runs successfully.

Windows instance

  1. Open a terminal connection to your instance using RDP or a similar tool and login to Windows.

  2. If you use an HTTP proxy, run the following command from an administrator command prompt. This sets the http_proxy environment variable so that the agent can send data to Stackdriver Logging:

    setx http_proxy http://YOUR-PROXY /m
    
  3. Download and run the agent's installer using the following three PowerShell commands. You do not need administrator privileges.

    cd C:\Users\[USERNAME]
    invoke-webrequest https://dl.google.com/cloudagents/windows/StackdriverLogging-v1-8.exe -OutFile StackdriverLogging-v1-8.exe;
    .\StackdriverLogging-v1-8.exe
    

    Notes:

    • You must download the installer to a non-system directory, such as C:\Users\[USERNAME]. For security reasons, the installer does not run from system directories, including the directory C:\.

      If you download the installer to a system directory, move it to another directory before running it.

    • Beginning with version v1-5, the installer places the agent in the following directory by default:

      C:\Program Files (x86)\Stackdriver\LoggingAgent\
      

      You can change that directory during installation.

    • You can delete the installer when it completes successfully.

Verifying the installation

To check that the installation was successful, look for the agent's test log entry in the Logs Viewer.

Go to the Logs Viewer

  1. At the top of the page, choose the project containing your VM instance:

    • For Compute Engine VM instances, choose the project that contains the VM instance.
    • For Amazon EC2 VM instances, choose the AWS LINK project that Stackdriver creates when you connect your AWS account to a Stackdriver account.
    • Do not choose a Stackdriver account project, unless that is the project containing your Compute Engine VM instance.
  2. In the logs menus, choose the resource for your VM instance:

    • For Compute Engine, choose GCE VM Instance.
    • For Amazon EC2, choose AWS EC2 Instance.
    • Select syslog (Linux), fluent.info (Windows), or All logs.

If you see a log entry, "Successfully sent gRPC to Stackdriver Logging API," then the agent installation is complete. Otherwise, see Troubleshooting.

For more information about the Logs Viewer, see Logs Viewer.

Next steps

Following are some optional tasks you might want to do:

Additional tasks

This section describes how to perform some common maintenance tasks.

Getting the version

Find out the version of the running Logging agent.

Linux instance

Run the following command on your instance:

# For Debian and Ubuntu systems:
dpkg -l google-fluentd

# For Amazon Linux, Red Hat and CentOS systems:
rpm -q google-fluentd

Windows instance

Run the following command.

reg query HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\GoogleStackdriverLoggingAgent\ /v Version

The preceding command returns an error if the agent's version is earlier than v1-5. Upgrade to the latest agent.

Configuring the agent

The Logging agent is pre-configured to collect custom logs. To add your own Fluentd configuration files, do the following:

Linux instance

  1. Copy your configuration files into the following directory:

    /etc/google-fluentd/config.d/
    
  2. Restart the agent by running the following command:

    sudo service google-fluentd reload
    

The Logging agent installation script populates this directory with the default catch-all configuration files. For more information, see Getting the logging agent source code.

Windows instance

  1. Copy your configuration files into the following directory:

    [AGENT_INSTALLATION_DIRECTORY]\config.d\
    

    If you accepted the default installation directory, then the preceding directory is:

    C:\Program Files (x86)\Stackdriver\LoggingAgent\config.d\
    
  2. Restart the agent by running the following commands in Command Prompt or PowerShell:

    net stop  StackdriverLogging
    net start StackdriverLogging
    

For more information on Fluentd configuration files, see Fluentd configuration.

Restarting the agent

Restarting the Logging agent causes it to pick up changes to its configuration files.

Linux instance

Run the following command on your instance:

    sudo service google-fluentd reload

To stop and then start the Logging agent, run the following command on your instance:

    sudo service google-fluentd restart

After reloading or restarting the Logging agent, you might want to send a test message.

Windows instance

Requires administrator privileges: To restart the agent, run the following commands on your VM instance in Command Prompt or PowerShell:

# For Logging agent version v1-5 and later.
net stop  StackdriverLogging
net start StackdriverLogging

# For Logging agents before v1-5.
net stop  fluentdwinsvc
net start fluentdwinsvc

Upgrading the agent

Upgrade the agent to the latest release.

Linux instance

To upgrade to the latest agent release, run the following command on your instance:

# For Debian and Ubuntu systems:
sudo apt-get install --only-upgrade google-fluentd

# For Amazon Linux, Red Hat and CentOS systems:
sudo yum upgrade google-fluentd

The preceding commands do not change the agent's configuration file. To get a clean configuration file, run the following command on your instance:

# For Debian and Ubuntu systems:
sudo apt-get install --only-upgrade google-fluentd google-fluentd-catch-all-config

# For Amazon Linux, Red Hat and CentOS systems:
sudo yum upgrade google-fluentd google-fluentd-catch-all-config

Alternatively, you can remove the current agent and then install the most recent agent.

Windows instance

To upgrade to the latest agent release, install the newest agent as described in Installing on Windows. The previous agent is optionally uninstalled for you.

Uninstalling the agent

Remove the Logging agent and its configuration files.

Linux instance

Uninstall the current Linux agent:

# For Debian and Ubuntu systems:
sudo service google-fluentd stop
sudo apt-get remove google-fluentd google-fluentd-catch-all-config

# For Amazon Linux, Red Hat and CentOS systems:
sudo service google-fluentd stop
sudo yum remove google-fluentd google-fluentd-catch-all-config

Windows instance

Uninstall the current agent by running the following command:

[AGENT_INSTALLATION_DIRECTORY]\uninstall.exe

If you are running the Logging agent versions v1-5 and later, use the directory that you chose when installing the agent. The default installation directory is:

C:\Program Files (x86)\Stackdriver\LoggingAgent\

If you are running a Logging agent before v1-5, the installation directory is:

C:\GoogleStackdriverLoggingAgent\

Testing the agent

If you suspect that the agent is not working, check that it is running and try to send a test message to Stackdriver Logging:

Linux instance

The following procedure works on both Compute Engine and Amazon EC2 VM instances running Linux:

  1. Verify that the Logging agent is running by executing the following commands on your VM instance:

    ps ax | grep fluentd
    

    You should see output similar to the following:

     2284 ?        Sl     0:00 /opt/google-fluentd/embedded/bin/ruby /usr/sbin/google-fluentd [...]
     2287 ?        Sl    42:44 /opt/google-fluentd/embedded/bin/ruby /usr/sbin/google-fluentd [...]
    
  2. Send a test log message by running the following command on your VM instance:

    logger "Some test message"
    

Windows instance

The Logging agent has two Windows service names:

  • StackdriverLogging for versions v1-5 and later
  • fluentdwinsvc for earlier versions

You should be running one agent service. Run the indicated commands on your VM instance using PowerShell:

  1. Ask for the status of both services. If you know which service should be running, you can use just that service name:

    Get-Service StackdriverLogging,fluentdwinsvc
    
  2. If a service is not running, you see an error message. If it is running, you see output like the following:

    Status    Name                DisplayName
    ------    ----                -----------
    Running  StackdriverLogging   Stackdriver Logging
    
  3. You should see one error message and one Running status:

    • If you do not see any Running status, then the Logging agent is not running.
    • If you see that StackdriverLogging is running, then you are running a recent agent version. To determine the specific version, see Getting the version.
    • If you see that fluentdwinsvc is running, then you should upgrade your agent to the latest version.
  4. Requires Administrator privileges: If any agent version is running, then send a test log message by running the following PowerShell commands:

    New-EventLog   -LogName Application -Source "Test Source"
    Write-EventLog -LogName Application -Source "Test Source" -EntryType Information -EventID 1 -Message "Testing 123 Testing."
    

Finding your test message

After sending a test message, look for it in the Logs Viewer:

Go to the Logs Viewer

  1. At the top of the page, choose the project containing your VM instance:

    • For Compute Engine VM instances, choose the project that contains the VM instance.
    • For Amazon EC2 VM instances, choose the AWS LINK project that Stackdriver creates when you connect your AWS account to a Stackdriver account.
    • Do not choose a Stackdriver account project, unless that is the project containing your Compute Engine VM instance.
  2. In the logs menus, choose the resource for your VM instance:

    • For Compute Engine, choose GCE VM Instance.
    • For Amazon EC2, choose AWS EC2 Instance.
    • Select syslog (Linux), fluent.info (Windows), or All logs.

Troubleshooting

The following table lists some common problems and tells you how to fix them.

On Linux, the Logging agent records errors in /var/log/google-fluentd/google-fluentd.log. The error class, Google::APIClient::ClientError, indicates there is a problem with permissions or API access.

You can start seeing errors after the agent has been running successfully. For example, someone might have revoked the required permissions from your project or your VM instance.

Error Cause Solution
The agent's installer on Windows fails to run You might have downloaded the installer to a system directory. Move the installer to a non-system directory, such as C:\Users\[USERID]\.
Project has not enabled the API You have not enabled the Stackdriver Logging API in your project. Go to the APIs console and change the status of the Stackdriver Logging API to ON.
Request had invalid credentials
or
Unable to fetch access token (no scopes configured?)
Your VM instance does not have suitable credentials. If you are using Amazon EC2, then you must install credentials on your VM instances before installing the agent. See Authorizing the Agent to install credentials.
Authorization failed Your private-key authorization credentials for the Logging agent are not configured correctly. See Verifying private key credentials.
The caller does not have permission The service account used for authorization in your project has insufficient permissions. It might be the default service account used within Compute Engine or App Engine, or it might be a user-defined service account used for private key authorization. The account must have Can edit permission. Change the permission of the service account in your project's Permissions page.
Cannot obtain project ID The Logging agent failed to get the project ID from a service account's private key credentials file. To add or override a project ID for the agent, edit the agent's configuration file, google-fluentd.conf, on your VM instance. It appears in directory /etc/default/google-fluentd or /etc/sysconfig/google-fluentd. In the <match **> section, add the following line:
project_id [YOUR_PROJECT_ID]
Otherwise, see Authorizing the Agent to fix or replace the credentials.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Stackdriver Logging