Collect Apache web server metrics with the Ops Agent

Collect Apache web server metrics with the Ops Agent

Learn how to collect and monitor metrics from an Apache web server installed on a Compute Engine virtual machine (VM) instance by using the Ops Agent:

1. Create a Compute Engine VM instance and install the Ops Agent.
2. Install an Apache web server.
3. Configure the Ops Agent for the Apache web server.
4. Generate traffic to the Apache web server.
5. View metrics on the predefined Apache dashboard.
6. Create an alerting policy.
7. Test the alerting policy.
8. Clean up.

---

To follow step-by-step guidance for this task directly in the Google Cloud console, click Guide me:

Guide me

---

Before you begin

1. Security constraints defined by your organization might prevent you from completing the following steps. For troubleshooting information, see Develop applications in a constrained Google Cloud environment.

2. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

3. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

4. Make sure that billing is enabled for your Google Cloud project.

5. Enable the Compute Engine, Cloud Monitoring, and Cloud Logging APIs.

   Enable the APIs

6. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

7. Make sure that billing is enabled for your Google Cloud project.

8. Enable the Compute Engine, Cloud Monitoring, and Cloud Logging APIs.

   Enable the APIs

Create a VM instance

1. In the Google Cloud console, select Compute Engine, and then select VM instances, or click the following button:

   Go to VM instances

2. Create a VM by clicking Create instance.

3. In the Name field, enter a descriptive name.
4. In the Machine type field, select e2-small.
5. In the Boot disk section, keep the default setting of Debian GNU/Linux.
6. In the Firewall section, select both Allow HTTP traffic and Allow HTTPS traffic.
7. In the Observability - Ops Agent section, select Install Ops Agent for Monitoring and Logging.
8. Click Create.

Install an Apache web server

To install an Apache web server on your Compute Engine VM instance, do the following:

1. On the VM instances page, locate your new VM, go to the Connect column, and then click SSH.

   Having trouble connecting? Refer to Troubleshooting SSH.

2. To update the package lists, copy the following command to your clipboard, paste the command into the SSH terminal, and then press enter:

   sudo apt-get update
   

3. After you see the message "Reading package lists... Done", in the SSH terminal, run the following command to install an Apache2 web server:

   sudo apt-get install apache2 php7.0
   

   When asked to continue the installation, enter Y. If the install command fails, then use sudo apt-get install apache2 php.

4. When your command prompt returns, go to the VM instances page and copy the VM's external IP address into the following URL:

   http://EXTERNAL_IP
   

5. To connect to your Apache web server, open a new browser tab, and then enter the URL from the previous step.

   When the web server is successfully installed, the browser tab displays the Apache2 Debian default page.

   

Collect Apache web server logs and metrics

In these steps, you configure the Ops Agent to collect logs and metrics from your Apache web server:

1. Go to the SSH terminal for your VM instance. If you don't have a terminal open, then do the following:

   1. In the Google Cloud console, select Compute Engine, and then select VM instances, or click the following button:

      Go to VM instances

   2. Locate your new VM and then click SSH.

2. Copy the following command, then paste it into the terminal for your instance, and then press enter:

   # Configures Ops Agent to collect telemetry from the app and restart Ops Agent.
   
   set -e
   
   # Create a back up of the existing file so existing configurations are not lost.
   sudo cp /etc/google-cloud-ops-agent/config.yaml /etc/google-cloud-ops-agent/config.yaml.bak
   
   # Configure the Ops Agent.
   sudo tee /etc/google-cloud-ops-agent/config.yaml > /dev/null << EOF
   metrics:
     receivers:
       apache:
         type: apache
     service:
       pipelines:
         apache:
           receivers:
             - apache
   logging:
     receivers:
       apache_access:
         type: apache_access
       apache_error:
         type: apache_error
     service:
       pipelines:
         apache:
           receivers:
             - apache_access
             - apache_error
   EOF
   
   sudo service google-cloud-ops-agent restart
   sleep 60
   

   The previous command creates the configuration to collect and ingest logs and metrics from the Apache web server. For more information, see Configure the Ops Agent for Apache web server.

3. Wait until the command prompt is shown, which takes at least 60 seconds.

Generate traffic

To generate traffic to your Apache web server, do the following:

1. Go to the SSH terminal for your VM instance. If you don't have a terminal open, then do the following:

   1. In the Google Cloud console, select Compute Engine, and then select VM instances, or click the following button:

      Go to VM instances

   2. Locate your new VM and then click SSH.

2. In the SSH terminal, run the following command, which generates requests to your Apache web server:

   timeout 120 bash -c -- 'while true; do curl localhost; sleep $((RANDOM % 4)) ; done'

3. Wait two minutes for the command prompt to return or for the terminal to close. While the command is running, HTML text is shown in the terminal.

View Apache metrics

To view the Apache Overview dashboard, which is automatically created, do the following:

1. In the Google Cloud console, select Monitoring, and then select  Dashboards, or click the following button:

   Go to Dashboards

2. In the All Dashboards pane, select the Apache Overview dashboard from the list.

   

You've configured the Ops Agent to collect metrics from your Apache web server, and you've viewed those metrics. The next step is to create an alerting policy so that you're notified when load on your Apache web server exceeds a threshold.

Create an email notification channel

Before you create an alerting policy, configure the notification channels that you want the alerting policy to use. Cloud Monitoring supports many different types of notification channels, including email, Slack, PagerDuty, and Pub/Sub. For more information, see Create and manage notification channels. To get notifications by e-mail, do the following:

1. In the Google Cloud console, select Monitoring, and then select  Alerting, or click the following button:

   Go to Alerting

2. In the toolbar, click Edit Notification Channels.
3. On the Notification channels page, scroll to Email, and then click Add new.
4. Enter your email address, a display name such as My email, and then click Save.

Create an alerting policy

In this section, you create an alerting policy so that you are notified when the traffic to your Apache web server exceeds a defined threshold:

1. In the Google Cloud console, select Monitoring, and then select  Alerting, or click the following button:

   Go to Alerting

2. Click Create policy.

3. Select the time series to be monitored:

   1. Click Select a metric and select VM instance.
   2. In the Active metric categories list, select Apache.
   3. In the Active metrics list, select workload/apache.traffic.
   4. Click Apply.

   The chart for Apache traffic is shown.

4. Advance to the Configure trigger fields, and then set the Threshold value field to 1500.

   The chart displays the threshold as a dashed line. Ensure that the dashed line is much less than the peak traffic level.

5. Advance to the Notifications and name fields, and then use the Notification channels menu to select your email address.

6. For the policy name, enter Apache traffic above threshold.

7. Advance to the Review alert fields, review the alerting policy, and then click Create policy.

Test the alerting policy

To test the alerting policy, generate traffic that exceeds the threshold:

1. Go to the SSH terminal for your VM instance. If you don't have a terminal open, then do the following:

   1. In the Google Cloud console, select Compute Engine, and then select VM instances, or click the following button:

      Go to VM instances

   2. Locate your new VM and then click SSH.

2. In the SSH terminal, run the following command, which generates requests to your Apache web server:

   timeout 120 bash -c -- 'while true; do curl localhost; sleep $((RANDOM % 4)) ; done'

3. Wait two minutes for the command prompt to return or for the terminal to close. While the command is running, HTML text is shown in the terminal.

4. When the command prompt returns, check your email for a message whose subject line begins with [ALERT].

   If you don't see an email, then examine the chart on the alerting policy to verify that the traffic level exceeded the threshold. You might need to run the previous command again. Otherwise, wait a minute or two for the email to arrive.

   The notification provides a summary of the alerting policy and a link to an incident. Each incident contains a record of the failure, and these are typically helpful when troubleshooting.

   

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

If you created a new project and you no longer need the project, then delete the project.

If you used an existing project, then do the following:

1. If you created a VM, then delete it:

   1. In the Google Cloud console, go to the VM instances page.

      Go to VM instances

   2. Select the checkbox for the instance that you want to delete.
   3. To delete the instance, click More actions, click Delete, and then follow the instructions.

2. Delete the alerting policy that you created:

   1. In the Google Cloud console, select Monitoring, and then select  Alerting, or click the following button:

      Go to Alerting

   2. Select the alerting policy that you created, and then click Delete.

What's next

• To learn about the Ops Agent and supported integrations, see:

  • Ops Agent overview.
  • Monitor third-party applications

• To learn how to test whether a URL is responding to requests and to simulate a failure, see the quickstart Get notified if your application stops responding.

• To view logs from your Apache web server, see the quickstart Cloud Logging for Compute Engine VMs.

• To learn about charting and viewing performance data, see Dashboards overview.

• For a list of metrics supported by Google Cloud, see Metrics list.

• To create your own Monitoring metrics, see Custom metrics.