Configuring the Ops Agent

Contact us at google-cloud-ops-agent@google.com if you have any questions, need support, or would like to offer feedback.

This page provides details about the Ops Agent's default and custom configurations.

Most users don't need to read this page. Read this page if the following applies to you:

  • You're interested in learning the technical details of the Ops Agent's configuration.

  • You want to change the configuration of the Ops Agent to achieve the following goals:

    • Customize the file path of the log files that the agent collects logs from.

    • Customize the structured log format (for example, json or regex) that the agent uses to process the log entries.

    • Customize which group or groups of metrics to enable. The agent collects all system metrics, like cpu and memory, by default.

Root configuration file location

The Ops Agent configuration file lives in /etc/google-cloud-ops-agent/config.yaml.

Default configuration

This section describe the default configuration for the Ops Agent. The configuration defines building blocks of receivers, processors, and exporters, and then defines a pipeline, or a set of pipelines, to tie those blocks together.

By default the agent collects file-based syslog and system metrics only.

logging:
  receivers:
    syslog:
      type: files
      include_paths:
      - /var/log/messages
      - /var/log/syslog
  exporters:
    google:
      type: google_cloud_logging
  service:
    pipelines:
      default_pipeline:
        receivers: [syslog]
        exporters: [google]

metrics:
  receivers:
    hostmetrics:
      type: hostmetrics
      collection_interval: 60s
  exporters:
    google:
      type: google_cloud_monitoring
  service:
    pipelines:
      default_pipeline:
        receivers: [hostmetrics]
        exporters: [google]

Logging configurations

The logging configuration has the following configuration sub-sections:

  • receivers: describes what data, for example, log files, to map into the model.

  • processors: a list of processor definitions. The processors in this list can be shared among multiple log input.

  • exporters: a list of output definitions. The output in this list can be shared among multiple receivers.

  • service: contains a pipelines section that is a list of pipeline definitions. A pipeline connects a list of receivers, a list of processors, and a list of exporters to form the data flow.

Logging receivers

This section contains a map from receiver IDs to receivers. A receiver describes where to retrieve the logs, for example by tailing files or via a tcp port. The receivers in this list can be shared among multiple pipelines. Each receiver must specify a unique receiver ID, the type, and additional parameters specific to that type.

Common parameters among all logging receivers:

  • type: Required. The type of the logging receiver. Supported types: files and syslog.

Parameters exclusive to files type receivers:

  • include_paths: Required. A list of filesystem paths to read by tailing each file. A wild card can be used as part of the path. For example, /var/log/*.log.

  • exclude_paths: Optional. A list of filesystem path patterns to exclude from those matched by include_paths.

Parameters exclusive to syslog type receivers:

  • transport_protocol: Optional. Supported values: tcp or udp. The default is tcp.

  • listen_host: Optional. An IP address to listen on. Only applicable when the transport_protocol is tcp. The default is 0.0.0.0.

  • listen_port: Optional. A port to listen on. Only applicable when the transport_protocol is tcp. The default is 5140.

Sample files type logging receiver configuration:

receivers:
  <receiver_id>:
    type: files

    include_paths: [/var/log/*.log]
    exclude_paths: [/var/log/not-this-one.log]

Sample syslog type logging receiver configuration:

receivers:
  <receiver_id>:
    type: syslog

    transport_protocol: tcp
    listen_host: 0.0.0.0
    listen_port: 5140

Logging processors

This section contains a map from processor IDs to processors. A processor describes additional processing of the logs, for example, parse to json or parse by regex. The processors in this list can be shared among multiple pipelines. Each processor must specify a unique processor ID, the type, and additional parameters specific to that type.

Common parameters for all logging processors:

  • type: Required. The type of the logging processor. Supported values: parse_json and parse_regex.

Common parameters for the parse_json and parse_regex logging processors:

  • field: Optional. The name of the field in the record to parse. If field isn't provided, then the processor parses the message field.

  • time_key: Optional. If the log entry provides a field with a timestamp, this option specifies the name of that field.

  • time_format: Only required when time_key is specified. This option specifies the format of the time_key field so it can be recognized and analyzed properly. For details of the format, refer to the strptime(3) guide.

Paramater exclusive to parse_regex type processors:

  • regex: Required. The regular expression for parsing the field. The expression must include key names for the matched subexpressions, for example, "^(<time>[^ ]*) (<severity>[^ ]*) (<msg>.*)$".

Sample parse_json type logging processor configuration:

processors:
  <processor_id>:
    type: parse_json

    field:       message
    time_key     time
    time_format  %Y-%m-%dT%H:%M:%S.%L%Z

Sample parse_regex type logging processor configuration:

processors:
  <processor_id>:
    type: parse_regex

    field:       message
    regex:       "^(<time>[^ ]*) (<severity>[^ ]*) (<msg>.*)$"
    time_key     time
    time_format  %Y-%m-%dT%H:%M:%S.%L%Z

Logging exporters

This section contains a map from ID to exporter. An exporter describes which backend to send the logs to. The exporters in this list can be shared among multiple pipelines. Each exporter must specify a unique exporter ID and type.

Common parameter among all logging exporters:

  • type: Required. The type of the logging exporter. Supported value: google_cloud_logging. When google_cloud_logging is specified, the Ops Agent translates the processed records into the Google LogEntry proto and sends them to the Logging API. Only one receiver with type google_cloud_logging is allowed.

When Ops Agent receives an unstructured log record, it maps it to textPayload.

For structured log records, Ops Agent treats the following fields specially. You can set specific fields in the LogEntry object that the agent writes to the Logging API.

All fields in the following table are stripped from the payload if present.

Record field LogEntry field

Option 1


"timestamp": {
  "seconds": CURRENT_SECONDS,
  "nanos": CURRENT_NANOS,
}

Option 2


{
  "timestampSeconds":
    CURRENT_SECONDS,
  "timestampNanos":
    CURRENT_NANOS,
}
 timestamp
receiver_id (not a record field) logName
logging.googleapis.com/httpRequest httpRequest
logging.googleapis.com/severity severity
logging.googleapis.com/labels labels
logging.googleapis.com/operation operation
logging.googleapis.com/sourceLocation sourceLocation

Any remaining structured record fields remain part of jsonPayload.

Sample google_cloud_logging type logging exporter configuration:

exporters:
  <exporter_id>:
    type: google_cloud_logging

Logging service

The preceding sections, receivers, processors, and exporters, define components that you can instantiate in the service section.

The service section has a single subsection, pipelines, that contains a map from user-defined pipeline IDs to pipeline definitions. Each pipeline definition consists of three parts:

  • receivers: a list of receiver IDs, as defined in the Logging receivers section. The pipeline collects data from all of the listed receivers. The order of the receivers doesn't matter.

  • processors: a list of processor IDs, as defined in the Logging processors section. The order of processors here matters. Each record is run through each processor in the listed order.

  • exporters: a list of exporter IDs, as defined in the Logging exporters section. The pipeline sends data to all of the listed exporters. The order of the exporters doesn't matter.

Example service section:

service:
  pipelines:
    <pipeline_id>:
      receivers:  [...]  # list of receiver_id
      processors: [...]  # list of processor_id
      exporters:  [...]  # list of exporter_id

Metrics configurations

The metrics configuration has the following configuration sub-sections:

  • receivers: a list of receiver definitions. A receiver describes where to retrieve the metrics (for example, system metrics like cpu or memory). The receivers in this list can be shared among multiple pipelines.

  • exporters: a list of exporter definitions. An exporter describes which backend to send the metrics to. The exporters in this list can be shared among multiple pipelines.

  • service: contains a pipelines section that is a list of pipeline definitions. A pipeline connects a list of receivers, a list of processors, and a list of exporters to form the data flow.

Metrics receivers

This section contains a map from receiver IDs to receivers. A receiver describes where to retrieve the metrics, such as like cpu and memory. The receivers in this list can be shared among multiple pipelines. Each receiver must specify a unique receiver ID, the type, and additional parameters specific to that type.

Common parameters for all metrics receivers:

  • type: Required. The type of the metrics receiver. Supported metrics receiver types: "hostmetrics". Only one receiver with type "hostmetrics" is allowed.

Parameter exclusive to hostmetrics type receivers:

  • collection_interval: Optional. The interval between each metrics collection in the format of a duration. For example, 30s or 2m. The default duration is 60s.

Sample syslog type logging receiver config:

receivers:
  <receiver_id>:
    type: hostmetrics
    collection_interval: 60s

hostmetrics ingests the following metric data. See details in the Agent Metrics page.

pattern Metric
cpu CPU load at 1 minute intervals
CPU load at 5 minute intervals
CPU load at 15 minute intervals
CPU usage, with labels for CPU number and CPU state
CPU usage percent, with labels for CPU number and CPU state
disk Disk bytes read, with label for device
Disk bytes written, with label for device
Disk I/O time, with label for device
Disk weighted I/O time, with label for device
Disk pending operations, with label for device
Disk merged operations, with labels for device and direction
Disk operations, with labels for device and direction
Disk operation time, with labels for device and direction
Disk usage, with labels for device and state
Disk utilization, with labels for device and state
memory Memory usage, with label for state (buffered, cached, free, slab, used)
Memory usage percent, with label for state (buffered, cached, free, slab, used)
network Network error counts, with labels for device and direction
Network packets sent, with labels for device and direction
Network bytes sent, with labels for device and direction
TCP connection count, with labels for port and TCP state
swap Swap I/O operations, with label for direction
Swap bytes used, with labels for device and state
Swap percent used, with labels for device and state
process Process count, with label for state
Processes forked count
perprocess Process CPU time, with labels for process name + others (omitted for brevity)
Process disk read I/O, with labels for process name + others
Process disk write I/O, with labels for process name + others
Process RSS usage, with labels for process name + others
Process VM usage, with labels for process name + others

Metrics exporters

This section contains a map from exporter IDs to exporters. An exporter describes which backend to send the metrics to. The exporters in this list can be shared among multiple pipelines. Each exporter must specify a unique exporter ID and type.

Common parameter for all metrics exporters:

  • type: Required. The type of the metrics exporter. Supported value: google_cloud_monitoring. When google_cloud_monitoring is specified, the Ops Agent translates the processed records into the Google TimeSeries and sends them to the Monitoring API. Only one receiver with type google_cloud_monitoring is allowed.

Sample google_cloud_monitoring type metrics exporter configuration:

exporters:
  <exporter_id>:
    type: google_cloud_monitoring

Metrics service

The preceding sections, receivers, processors, and exporters, define components that you can instantiate in the service section.

The service section has a single subsection, pipelines, that contains a map from user-defined pipeline IDs to pipeline definitions. Each pipeline definition consists of three parts:

  • receivers: a list of receiver IDs, as defined in the Metrics receivers section. The pipeline collects data from all of the listed receivers. The order of the receivers doesn't matter.

  • processors: a list of processor IDs, as defined in the Metrics processors section. The order of processors here matters. Each record is run through each processor in the listed order.

  • exporters: a list of exporter IDs, as defined in the Metrics exporters section. The pipeline sends data to all of the listed exporters. The order of the exporters doesn't matter.

Example service section:

service:
  pipelines:
    <pipeline_id>:
      receivers:  [...]  # list of receiver_id
      processors: [...]  # list of processor_id
      exporters:  [...]  # list of exporter_id