Collecting custom data with the DataCapture policy

This page applies to Apigee and Apigee hybrid.

View Apigee Edge documentation.

In addition to the usual API data collected by Apigee, you can also collect custom data using the DataCapture policy. There are two main steps required, as described in the following sections:

  1. Create a Data Collector REST resource.
  2. Specify the data you want to collect using the DataCapture policy's <Collect> element.

Create a Data Collector REST resource

To use the DataCapture policy, you must first create a Data Collector REST resource. To do so, send an API request like the following:

curl -H "Authorization: Bearer $(gcloud auth print-access-token)" -X POST -H "content-type:application/json" \
  -d '
{
  "name": "dc_my_custom_data",
  "description": "Collects data for analysis.",
  "type": "STRING",
}' \
  "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/datacollectors"

This creates a resource named dc_my_custom_data, which you can use with the DataCapture policy.

Specify the data to collect

You can specify the data to collect using the DataCapture policy's <Collect> element. There are various ways to do this, as shown in the Examples section in the DataCapure reference page. One way is to specify a flow variable, which stores data passed in API requests and responses. For example, you could collect data that customers enter in a web page form using the flow variable message.formparam.param_name.values, where param_name is the name of a parameter in the form. Suppose the form has a parameter named payment_amount. The corresponding flow variable would be message.formparam.payment_amount.values.

The following code specifies this variable by the ref attribute of the <Collect> element.

<DataCapture name="capturepayment">
    <Capture>
        <DataCollector>dc_my_custom_data</DataCollector>
        <Collect ref="message.form_param.payment_amount.values" >
    </Capture>
</DataCapture>

With this setting, the data entered in the form's payment_amount field will be collected by DataCapture and stored in a variable named dc_my_custom_data.

Creating a custom report from data collected with DataCapture

Once you have set up data collection with the DataCapture policy, you can use the data to create a custom report by the procedure described in Adding a custom report. To do so, when you select a metric for the report, as shown in step 4 of the procedure, choose the variable that has the same name as the Data Collector REST resource you created for DataCapture—for example, dc_my_custom_data— as shown below.

Add custom metric for DataCapture.

You can also use the same variable as a dimension in a custom report. Since dimensions group data by the values of the variable, it is usually preferable for the variable to be a string that only takes on a limited number of values.