The Apache Solr integration collects core and cache metrics. Core metrics focus on requests, document count, and index size. Cache metrics focus on cache utilization. The integration also collects Solr logs and parses them into a JSON payload. The result includes fields for shard, replica, core, level, and message.
For more information about Solr, see solr.apache.org/.
Prerequisites
To collect and ingest Solr logs and metrics, you must install Ops Agent version 2.10.0 or higher.
This receiver supports Apache Solr versions 7.7 and 8.x.
Configure your Solr instance
To expose a JMX endpoint, you must set the com.sun.management.jmxremote.port
system property when starting the JVM. We also recommend setting the
com.sun.management.jmxremote.rmi.port
system property to the same port. To
expose a JMX endpoint remotely, you must also set the java.rmi.server.hostname
system property.
By default, these properties are set in a Solr deployment's
solr-env.sh
file.
To set system properties by using command-line arguments, prepend the property
name with -D
when starting the JVM. For example, to set
com.sun.management.jmxremote.port
to port 18983
, specify the following when
starting the JVM:
-Dcom.sun.management.jmxremote.port=18983
Configure the Ops Agent for Solr
Following the guide for Configuring the Ops Agent, add the required elements to collect logs and metrics from your Solr instances, and restart the agent.
Example configuration
The following command creates the configuration file to collect and ingest logs and metrics for Solr and restarts the Ops Agent on Linux.
sudo tee /etc/google-cloud-ops-agent/config.yaml > /dev/null << EOF
logging:
receivers:
solr_system:
type: solr_system
service:
pipelines:
solr:
receivers:
- solr_system
metrics:
receivers:
solr:
type: solr
service:
pipelines:
solr:
receivers:
- solr
EOF
sudo service google-cloud-ops-agent restart
Configure logs collection
To ingest logs from Solr, you must create receivers for the logs Solr produces
and then create a pipeline for the new receivers. To configure a receiver for
your solr_system
logs, specify the following fields:
Field | Default | Description |
---|---|---|
type |
The value must be solr_system . |
|
include_paths |
[/var/solr/logs/solr.log ] |
The log files to read. |
exclude_paths |
The log files to exclude, if include_paths contains a glob or directory. |
|
wildcard_refresh_interval |
60s |
The interval at which wildcard file paths in include_paths are refreshed. Given as a time duration, for example 30s or 2m . This property might be useful under high logging throughputs where log files are rotated faster than the default interval. |
What is logged
The logName
of the solr_system
logs are derived from the receiver IDs
specified in the configuration. Detailed fields inside the
LogEntry
are as follows.
Field | Type | Description |
---|---|---|
timestamp |
string (Timestamp ) |
Time the request was received. |
jsonPayload.collection |
string | Solr collection related to the log. |
jsonPayload.shard |
string | Solr shard related to the log. |
jsonPayload.replica |
string | Solr replica related to the log. |
jsonPayload.core |
string | Solr core related to the log. |
jsonPayload.source |
string | Source from where the log originated. |
jsonPayload.thread |
string | Thread from where the log originated. |
jsonPayload.message |
string | Log message. |
jsonPayload.exception |
string | Exception related to the log, including detailed stack trace where provided. |
severity |
string (LogSeverity ) |
Log entry level (translated). |
Configure metrics collection
To collect metrics from Solr, you must create a receiver for Solr metrics and then create a pipeline for the new receiver. To configure a receiver for your Solr metrics, specify the following fields:
Field | Default | Description |
---|---|---|
type |
The value must be solr . |
|
endpoint |
localhost:18983 |
The JMX Service URL or host and port used to construct the Service URL. Must be in the form of host:port . Values in host:port form will be used to create a Service URL of service:jmx:rmi:///jndi/rmi://<host>:<port>/jmxrmi . |
username |
The configured username if JMX is configured to require authentication. | |
password |
The configured password if JMX is configured to require authentication. | |
collection_interval |
60s |
A time.Duration value, such as 30s or 5m . |
What is monitored
The following table provides the list of metrics that the Ops Agent collects from the Solr instance.
Metric type | |
---|---|
Kind, Type Monitored resources |
Labels |
workload.googleapis.com/solr.cache.eviction.count
|
|
CUMULATIVE , INT64 gce_instance |
cache
core
|
workload.googleapis.com/solr.cache.hit.count
|
|
CUMULATIVE , INT64 gce_instance |
cache
core
|
workload.googleapis.com/solr.cache.insert.count
|
|
CUMULATIVE , INT64 gce_instance |
cache
core
|
workload.googleapis.com/solr.cache.lookup.count
|
|
CUMULATIVE , INT64 gce_instance |
cache
core
|
workload.googleapis.com/solr.cache.size
|
|
GAUGE , INT64 gce_instance |
cache
core
|
workload.googleapis.com/solr.document.count
|
|
GAUGE , INT64 gce_instance |
core
|
workload.googleapis.com/solr.index.size
|
|
GAUGE , INT64 gce_instance |
core
|
workload.googleapis.com/solr.request.count
|
|
CUMULATIVE , INT64 gce_instance |
handler
core
type
|
workload.googleapis.com/solr.request.error.count
|
|
CUMULATIVE , INT64 gce_instance |
handler
core
type
|
workload.googleapis.com/solr.request.time.average
|
|
GAUGE , DOUBLE gce_instance |
handler
core
type
|
workload.googleapis.com/solr.request.timeout.count
|
|
CUMULATIVE , INT64 gce_instance |
handler
core
type
|
Verify the configuration
You can use the Logs Explorer and Metrics Explorer to verify that you correctly configured the Solr receiver. It might take one or two minutes for the Ops Agent to begin collecting logs and metrics.
To verify the logs are ingested, go to the Logs Explorer and run the following query to view the Solr logs:
resource.type="gce_instance"
logName=("projects/PROJECT_ID/logs/solr_system")
To verify the metrics are ingested, go to
Metrics Explorer
and run the following query in the MQL tab.
fetch gce_instance
| metric 'workload.googleapis.com/solr.request.count'
| align rate(1m)
| every 1m