Cloud Logging for Compute Engine VMs
This document shows you how to view syslog
logs collected from an Apache Web
Server installed on a Compute Engine virtual machine (VM) instance. You can
use a process similar to the one in this quickstart to monitor third-party applications.
In this quickstart, you do the following:
- Create a Compute Engine VM instance.
- Install Apache Web Server.
- Install and configure the Ops Agent for Apache Web Server.
- View your logs in the Logs Explorer.
- Create a log-based alert.
- Clean up.
Before you begin
Some of the steps in this document might not work correctly if your organization applies constraints to your Google Cloud environment. In that case, you might not be able to complete tasks like creating public IP addresses or service account keys. If you make a request that returns an error about constraints, see how to Develop applications in a constrained Google Cloud environment.
You must have a Google Cloud project with billing enabled to complete this quickstart. If you don't have a Cloud project, or if you don't have billing enabled for your Cloud project, then do the following:
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.
If you have an email notification channel configured, then skip this step.
To set up an email notification channel, do the following:
In the console, select Cloud Monitoring or click the following button:
Go to Monitoring- Select Alerting.
- Click edit Edit notification channels.
- In the Email section, click Add new and complete the dialog.
Create a Compute Engine VM instance
We recommend that you create a Compute Engine VM instance for this quickstart because you install and configure software.
In the console, go to Compute and then select Compute Engine:
To create a VM instance, click Create instance.
Fill in the fields for your instance as follows:
- In the Name field, enter
quickstart-vm
. - In the Machine type field, select Small.
- Ensure the Boot disk is configured for Debian GNU/Linux.
- In the Firewall field, select both Allow HTTP traffic and Allow HTTPS traffic.
Leave the rest of the fields at their default values.
- In the Name field, enter
Click Create. When your VM is ready, it appears in the list of instances in the Instances tab.
Install Apache Web Server
To deploy an Apache Web Server on your Compute Engine VM instance, do the following:
To open a terminal to your instance, in the Connect column, click SSH.
To update the package lists on your instance, run the following command:
sudo apt-get update
To install an Apache2 HTTP Server, run the following command:
sudo apt-get install apache2 php7.0
Open your browser and connect to your Apache2 HTTP server by using the URL
http://EXTERNAL_IP
, whereEXTERNAL_IP
is the external IP address of your VM. You can find this address in the External IP column of your VM instance.You see the Apache2 default page:
Install and configure the Ops Agent
To collect logs and metrics information from your Apache Web Server, install the the Ops Agent by using the terminal:
To open a terminal to your VM instance, in the Connect column, click SSH.
To install the Ops Agent, run the following command:
curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh sudo bash add-google-cloud-ops-agent-repo.sh --also-install
You see
google-cloud-ops-agent installation succeeded.
Copy the following command, then paste it into the terminal:
The previous command creates the configuration to collect and ingest logs and metrics from the Apache Web Server. For more information about ingesting logs from the Apache Web Server, see Configure the Ops Agent for Apache Web Server.
View your logs
You can use the Logs Explorer to view logs in the console. To select certain log entries, you can write a query.
To view your logs in the console, do the following:
In the console, select Logging from the navigation menu, then click Logs Explorer:
In the Query pane, enable Show query. The query editor opens.
To locate your Cloud project ID, expand the list of Cloud projects from the Cloud project selector and copy the Cloud project ID in the ID column.
Replace
PROJECT_ID
with the project ID you copied in the previous step, copy the following query expression that displays onlyapache_access
andapache_error
logs, and paste it into the query editor:resource.type="gce_instance" logName=("projects/PROJECT_ID/logs/apache_access" OR "projects/PROJECT_ID/logs/apache_error")
Click Run query or press the Enter key. The results of the query are displayed in the Query results pane.
Create a log-based alert
To be notified when a specific message appears in your log entries, create a
log-based alert. In this section, you will create a log-based alert to be
notified when a 404 Not Found
message appears in your log entries after
running a command that searches for the fake page /help
.
To create a log-based alert, do the following:
Go to the Logs Explorer in the console:
In the Query results toolbar, click add_alert Create alert. The log-based alert policy pane opens. Fill in the fields as follows:
In Alert details, enter
404 Not Found
in the Alert Policy Name field.In Choose logs to include in this alert, copy the following query and paste it into the log filter text box:
severity>=DEFAULT /help httpRequest.status=404
The previous log filter searches for log entries with a
severity
of at least DEFAULT, that contain the text/help
, and that contain anhttpRequest
status of 404.In Set notification frequency and autoclose duration, select the following values:
Time between notifications: 5 min
Incident autoclose duration: 30 min
In Who should be notified?, select an email in the Notification Channels drop-down menu then click Save to configure the alert policy.
To test the log-based alert you just created, do the following:
In the console, go to Compute and then select Compute Engine:
In the Connect column, click SSH to open a terminal to your VM instance.
To search the server for
localhost/help
, run the following command:curl localhost/help
After you see a
404 Not Found
message in the terminal, an email notification will be sent. It might take several minutes for this process to complete.Notifications sent look similar to the following:
Clean up
To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.
If you created a new Cloud project for this quickstart, then delete the Cloud project:
- In the console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down to delete the project.
If you didn't create a project and if you created a new VM instance, then delete the VM instance:
- In the console, go to the VM instances page.
- Select the checkbox for the instance that you want to delete.
- To delete the instance, click More actions, click Delete, and then follow the instructions.
If you didn't create a project and if you used an existing VM instance for this quickstart, then uninstall the Ops Agent:
To uninstall the Ops Agent, open the terminal and run the following command:
sudo bash add-google-cloud-ops-agent-repo.sh --uninstall
What's next
For information about viewing logs in the Logs Explorer, see Using the Logs Explorer.
For information about querying in the Logs Explorer, see Build queries in the Logs Explorer.
For information on how to monitor a Compute Engine VM instance, see the Monitor a Compute Engine virtual machine quickstart.
For more information on Monitoring and its relation to Cloud Logging, see Monitoring.