Collect on-premises and multi-cloud logs

Stay organized with collections Save and categorize content based on your preferences.

Google Cloud partners with Blue Medora to provide BindPlane to Cloud Logging users at no additional cost. Users can collect their own data from their on-premise and hybrid cloud platforms and send it to Cloud Logging for analysis. For information about the integration between BindPlane and Cloud Logging, see About Blue Medora.

You can activate BindPlane from the Google Cloud Marketplace. After activating BindPlane, you can manage the service from its dashboard. For more information about getting started with BindPlane logs, refer to the Blue Medora documentation.

If you don't want to use BindPlane, then you can follow the procedure described in the following section of this document to collect and view logs for your Amazon Elastic Compute Cloud (Amazon EC2) instances.

Collect and view logs from Amazon Elastic Compute Cloud instances

The procedure outlined in this section, which doesn't rely on BindPlane, configures your Amazon EC2 instance to send logs and metric data to Google Cloud.

Collect logs

To collect logs from Amazon EC2 instances, do the following:

  1. Connect your Amazon account to Google Cloud. If you haven't performed this configuration, then follow the instructions for how to add a monitored project.

  2. Create a service account for the AWS connector project, create a private key for it, and then transfer the private key to your Amazon EC2 instances whose logs you want to exported to Google Cloud. The private key is used to authenticate the Cloud Logging agent.

    To create a service account and private key, do the following:

    1. In the Google Cloud console, go to the Service Accounts page of IAM & Admin:

      Go to Service Accounts

    2. Click Select project and then select your AWS connector project. If you don't know the name of your AWS connector project, the first step in the View logs section of this document describes how to identify that project.

    3. Click Create service account and complete the dialog. Grant the following roles:

      • Logs writer
      • Monitoring Metric Writer
    4. After you create the service account, edit the service account and then create a private key. Select JSON as the KeyType. When you create a private key, it's downloaded to your workstation. Save a copy of that key to reuse across your Amazon EC2 instances—this is your only chance to save this particular key.

    For information about these steps, see Authorize the Logging agent.

  3. Install the Cloud Logging agent on those Amazon EC2 instances whose logs you want exported to Google Cloud. For information about installing the agent, see Installing the Cloud Logging agent on individual VMs.

  4. (Optional) If you also want to export metric data from your Amazon EC2 instances, then install the Cloud Monitoring agent on those instances. For information about installing the agent, see Installing the Cloud Monitoring agent on individual VMs.

View logs

To view the logs for your AWS account, do the following:

  1. In the Google Cloud console project picker, select the AWS connector project that you created when you connected your AWS account to Google Cloud.

    Go to Google Cloud console

    If you don't know the name of the connector project, then do one of the following:

    • Search through your projects by using the Google Cloud console project picker.

    • View the Cloud Monitoring configuration of the Google Cloud project that added the AWS account as a monitored project:

      1. Select the Google Cloud project that you used when you created the AWS connector project:
      2. In the Google Cloud console navigation pane, select Monitoring.
      3. In the Monitoring navigation pane, select Settings and then select the Summary tab.

        The AWS Accounts pane lists your AWS connector projects.

  2. Use the navigation pane to select Logging and then select Logs.

    For information about the Logs Explorer, see View logs by using the Logs Explorer.