Sample queries using Logs Viewer (Preview)

Beta:

Logs Viewer (Preview) is currently in beta and might change or have limited support. For more information, see the product launch stages.

Google Cloud's operations suite offers two ways to view logs: Logs Viewer (Classic) and Logs Viewer (Preview). For an overview of the available features in Logs Viewer (Preview), go to Logs Viewer (Preview) overview.

This page provides you with suggested queries to make it easier to find important logs using the Logs Viewer (Preview) in the Google Cloud Console.

All listed queries can be applied in the Logs Viewer (both Classic and Preview versions), the Logging API, or the command-line interface, but this page focuses on using the queries in the Logs Viewer (Preview).

The Logs Viewer uses Boolean expressions to specify a subset of all the log entries in your project. You can use these queries to choose log entries from specific logs or log services, or that satisfy conditions on metadata or user-defined fields. For more information about querying, go to Building queries.

Getting started

To navigate to the Logs Viewer (Preview), do the following:

1. Go to the Google Cloud navigation menu and select Logging > Logs Viewer:
   Go to the Logs Viewer
2. Select a Google Cloud project.
3. From the version-picker menu, switch the Logs Viewer version from Classic to Preview the new Logs Viewer.

You're now in Logs Viewer (Preview).

Using the sample queries

To apply a query from the following tables, copy an expression by clicking the clipboard icon at the end of any expression's row and then paste the copied expression into the Query builder text box:

After you enter your expression, click Run query. Logs that match your query are listed under Query results.

Some of the queries listed later on this page include variables (indicated by brackets []) that you should replace with valid values. When a query includes logName, then the [PROJECT_ID] you supply must refer to the currently selected Google Cloud project; otherwise, the query won't work. For more information, go to Troubleshooting.

The following sections group queries by Google Cloud services.

App Engine queries

Filter name	Expression
App Engine logs from New Year's Eve (in UTC time)	resource.type="gae_app" AND severity>=ERROR AND timestamp>="2018-12-31T00:00:00Z" AND timestamp<="2019-01-01T00:00:00Z"
App Engine request logs with server errors	resource.type="gae_app" AND log_name="projects/[PROJECT_ID]/logs/appengine.googleapis.com%2Frequest_log" AND http_request.status>=500
Sampled HTTP error logs	resource.type="gae_app" AND proto_payload.status >= 400 AND sample(insertId, 0.1)
Search for App Engine trace ID	resource.type="gae_app" AND trace="projects/[PROJECT_ID]/traces/[TRACE_ID]"

BigQuery queries

Filter name	Expression
BigQuery audit logs	resource.type="bigquery_resource" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com"
BigQuery data transfer service jobs	resource.type="bigquery_resource" AND proto_payload.request_metadata.caller_supplied_user_agent="BigQuery Data Transfer Service" AND proto_payload.method_name="jobservice.insert"
BigQuery dataset updates	resource.type="bigquery_resource" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name="datasetservice.update"
BigQuery jobs completed	resource.type="bigquery_resource" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Fdata_access" AND proto_payload.method_name="jobservice.jobcompleted"
BigQuery large queries	resource.type="bigquery_resource" AND proto_payload.method_name="jobservice.jobcompleted" AND proto_payload.service_data.job_completed_event.job.job_statistics.total_billed_bytes>1073741824
BigQuery quota exceeded	resource.type="bigquery_resource" AND proto_payload.status.code=8 AND severity>=WARNING
BigQuery query started	resource.type="bigquery_resource" AND proto_payload.method_name="jobservice.insert"

Dataflow queries

Filter name	Expression
Errors and warnings in Dataflow workers	resource.type="dataflow_step" AND log_name="projects/[PROJECT_ID]/logs/dataflow.googleapis.com%2Fworker" AND severity>=WARNING

Dataproc queries

Filter name	Expression
Dataproc Apache Hadoop logs	resource.type="cloud_dataproc_cluster" AND json_payload.class:"org.apache.hadoop.mapreduce"

Cloud Deployment Manager

Filter name	Expression
Deployment Manager errors	resource.type="deployment" AND severity>=ERROR

Cloud Functions queries

Filter name	Expression
Cloud function errors	resource.type="cloud_function" AND log_name="projects/[PROJECT_ID]/logs/cloudfunctions.googleapis.com%2Fcloud-functions" AND severity>=ERROR

Cloud Identity and Access Management queries

Filter name	Expression
Service account creation logs	resource.type="service_account" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name="google.iam.admin.v1.CreateServiceAccount"
Service account creation key logs	resource.type="service_account" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name="google.iam.admin.v1.CreateServiceAccountKey"
Set access control policy logs	resource.type="project" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name="SetIamPolicy"
External member granted access to organization	resource.type="project" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.@type="type.googleapis.com/google.cloud.audit.AuditLog" AND proto_payload.request.@type:"IamPolicy" AND proto_payload.service_data.policy_delta.binding_deltas.member:* AND NOT proto_payload.service_data.policy_delta.binding_deltas.member:"@[DOMAIN_NAME].com"

Cloud Source Repositories queries

Filter name	Expression
Cloud Source Repository logs	resource.type="csr_repository" AND resource.labels.name="[REPOSITORY_NAME]"

Cloud Spanner queries

Filter name	Expression
Cloud Spanner logs for a specific spanner instance	resource.type="spanner_instance" AND resource.labels.instance_id="[SPANNER_INSTANCE]"

Cloud SQL queries

Filter name	Expression
Cloud SQL database	resource.type="cloudsql_database" AND resource.labels.database_id="[DATABASE_ID]"
Cloud SQL MySQL error logs	resource.type="cloudsql_database" AND log_name="projects/[PROJECT_ID]/logs/cloudsql.googleapis.com%2Fmysql.err"
Cloud SQL MySQL-based databases	resource.type="cloudsql_database" AND resource.labels.database_id="[DATABASE_ID]" AND log_name="projects/[PROJECT_ID]/logs/cloudsql.googleapis.com%2Fmysql"
Cloud SQL Postgres-based databases	resource.type="cloudsql_database" AND resource.labels.database_id="[DATABASE_ID]" AND log_name="projects/[PROJECT_ID]/logs/cloudsql.googleapis.com%2Fpostgres.log"

Compute Engine queries

Filter name	Expression
Google Compute Engine Admin Activity logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity"
Google Compute Engine firewall rule deletion	resource.type="gce_firewall_rule" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name:"firewalls.delete"
Google Compute Engine legacy activity logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Factivity_log"
Google Compute Engine VM syslogs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/syslog"

Cloud Storage queries

Filter name	Expression
GCS bucket logs	resource.type="gcs_bucket" AND resource.labels.bucket_name="[BUCKET_NAME]"
GCS bucket audit logs	resource.type="gcs_bucket" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com"
GCS bucket creation logs	resource.type="gcs_bucket" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name="storage.buckets.create"
GCS bucket deletion logs	resource.type="gcs_bucket" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name="storage.buckets.delete"

Cloud Tasks queries

Filter name	Expression
Cloud Tasks queue logs	resource.type="cloud_tasks_queue" AND resource.labels.queue_id="[QUEUE_ID]"

Kubernetes-related queries

Filter name	Expression
Google Kubernetes Engine cluster activity with errors	resource.type="gke_cluster" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND severity="ERROR"
Google Kubernetes Engine cluster creation	resource.type="gke_cluster" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name="google.container.v1.ClusterManager.CreateCluster"
Kubernetes cluster deployment	resource.type="k8s_cluster" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name:"deployments"
Kubernetes cluster authentication failure	resource.type="k8s_cluster" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.authentication_info.principal_email="system:anonymous"
Kubernetes cluster write-requests to a secret	resource.type="k8s_cluster" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name="io.k8s.core.v1.secrets" NOT proto_payload.method_name="get" NOT proto_payload.method_name="list" NOT proto_payload.method_name="watch"
Kubernetes clusters in us-central1-b	resource.type="k8s_cluster" AND resource.labels.location="us-central1-b"
Kubernetes container guestbook logs	resource.type="k8s_container" AND resource.labels.cluster_name="guestbook"
Kubernetes pod requests from users	resource.type="k8s_cluster" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name:"io.k8s.core.v1.pods" AND proto_payload.authentication_info.principal_email="[USER_EMAIL]"

Logging agent application queries

Filter name	Expression
Apache logs	resource.type="gce_instance" AND (log_name:"/apache-access" OR log_name:"/apache-error")
Cassandra logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/cassandra"
Chef logs	resource.type="gce_instance" AND log_name:"projects/[PROJECT_ID]/logs/chef-"
Gitlab logs	resource.type="gce_instance" log_name:"projects/[PROJECT_ID]/logs/gitlab-"
Jenkins logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/jenkins"
Jetty logs	resource.type="gce_instance" AND log_name:"projects/[PROJECT_ID]/logs/jetty-"
Joomla logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/joomla"
Linux syslogs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/syslog"
Magneto logs	resource.type="gce_instance" AND log_name:"projects/[PROJECT_ID]/logs/magneto-"
Mediawiki logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/mediawiki"
memcached logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/memcached"
MongoDB logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/mongodb"
MySQL logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/mysql"
Nginx logs	resource.type="gce_instance" AND log_name:"projects/[PROJECT_ID]/logs/nginx-"
Postgresql logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/postgresql"
Puppet logs	resource.type="gce_instance" AND log_name:"projects/[PROJECT_ID]/logs/puppet-"
RabbitMQ logs	resource.type="gce_instance" AND log_name:"projects/[PROJECT_ID]/logs/rabbitmq-"
Redmine logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/redmine"
Salt logs	resource.type="gce_instance" AND log_name:"projects/[PROJECT_ID]/logs/salt-"
Slow MySQL queries	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/mysql-slow"
Solr logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/solr"
SugarCRM logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/sugarcrm"
Tomcat logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/tomcat"
Zookeeper logs	resource.type="gce_instance" AND log_name="projects/[PROJECT_ID]/logs/zookeeper"

Networking queries

Filter name	Expression
Firewall- all logs	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Ffirewall"
Firewall logs by country	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Ffirewall" AND json_payload.remote_location.country=[COUNTRY_ISO_ALPHA_3]
Firewall logs by VM	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Ffirewall" AND json_payload.instance.vm_name="[INSTANCE_NAME]"
Firewall subnet logs	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Ffirewall" AND resource.labels.subnetwork_name="[SUBNET_NAME]"
Compute Engine subnetwork traffic logs to a subnet	resource.type="gce_subnetwork" AND ip_in_net(json_payload.connection.dest_ip, "[SUBNET_IP]")
VPC Flow logs	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fvpc_flows"
VPC Flow logs for specific port and protocol	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fvpc_flows" AND json_payload.connection.src_port="[PORT_ID]" AND json_payload.connection.protocol="[PROTOCOL]"
VPC Flow logs for specific subnet	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fvpc_flows" AND resource.labels.subnetwork_name"=[SUBNET_NAME]"
VPC Flow logs for specific subnet prefix	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fvpc_flows" AND ip_in_net(json_payload.connection.dest_ip,[SUBNET_IP])
VPC Flow logs for specific VMs	resource.type="gce_subnetwork" AND log_name="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fvpc_flows" AND json_payload.src_instance.vm_name="[VM_NAME]"
VPN gateway logs	resource.type="vpn_gateway" AND resource.labels.gateway_id="[GATEWAY_ID]"
HTTP Load Balancer 5xx errors	resource.type="http_load_balancer" AND http_request.status>=500
HTTP Load Balancer requests to PHPMyAdmin	resource.type="http_load_balancer" AND http_request.request_url:"phpmyadmin"

Security logging queries

Filter name	Expression
Audit logs- all	log_name:"projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com"
Audit logs- Access Transparency (AXT)	log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Faccess_transparency"
Audit logs- Admin Activity	log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity"
Audit logs- Data Access	log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Fdata_access"
Audit logs- System Event	log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Fsystem_event"

Google Cloud's operations suite queries

Filter name	Expression
Log sink activities	resource.type="logging_sink" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity"
Log-based metric create or update activities	resource.type="metric" AND log_name="projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity" AND proto_payload.method_name:(UpdateLogMetric OR CreateLogMetric)
Uptime URL checks for a host	resource.type="uptime_url" AND resource.labels.host="[URL]"

Troubleshooting

For more information about the query syntax and troubleshooting instructions, go to Building queries: Troubleshooting.

What's next

For more information about the query syntax, which you can use to customize these queries, review Building queries.