Unpacking API Management policies [Part 1]: How to transform or reformat your payload

Today’s application and integration landscape is filled with APIs. With the rapid increase of APIs over the last few years, more than 9 in 10 developers today use APIs. Although most developers work with APIs, in most cases that might not be their prime focus day to day. A recent Statista report found that API developers spend 29.5% of their time programming APIs and 17% of their time manually debugging them. To help reduce the time you spend in building APIs on Apigee, we will unpack a key aspect of API development – policies – in a series of blogs. 

In this blog, we’ll start with the basic concepts of policies and then explore a few policies that transform the messages flowing through your APIs. In part 2 of this series, we will explore how other policies help secure, manage traffic, and implement custom behavior into the APIs.

What are policies in API management?

A policy is a rule that an API gateway enforces when processing incoming requests. The key to adoption of APIs and popularity of API management platforms is the extent of consistency and automation available in enforcing these policies. 

Consider the security check at an airport. The security agents will validate your identity details and travel information to allow any traveler to pass through the security gates. This behavior is analogous to a policy in the world of API management – which requires all incoming requests to adhere to a certain behavior if they are to be permitted. Specifically, they would be referred to as authentication and authorization policies.

What else can Apigee policies help with?

In Apigee, policies are designed to let you add common types of management capabilities to an API without having to write or maintain any code. Apigee policies can be grouped into the following 4 broad functional categories:

• Traffic management policies (Ex: Rate limiting) that enable you to control the flow of request and response messages through an API. 

• Security policies (Ex: OAuth) that authenticate and authorize every request, and protect your content

• Mediation policies (Ex: CORS) allow you actively manipulate the request and response payloads

• Extension policies (Ex: MessageLogging) enable you to implement custom behavior in multiple programming languages 

Apigee offers 50+ policies out of the box. In this blog post, we will explore the functionality of some mediation policies, when to use them, and how you can implement them based on your application needs

You're not limited to the set of policy types provided by Apigee. You can also write custom scripts and code (such as JavaScript), that extend API proxy functionality and enable you to innovate on top of the basic management capabilities supported by Apigee policies.

Mediation policies: What are they and when can I use them?

Mediation policies in Apigee are incredibly powerful. There are many situations when an API developer can take advantage of mediation policies to adjust the shape and values of a request and response to cleanly fit their API. Let us look at some of the examples where they can be helpful

#1 Extract the data from a request/response message and store it in variables

Let’s say in a weather application, you want to tailor the information being presented to a consumer based on a user’s zip code – which is being passed through the XML payload of an API. In use cases like these, Apigee’s ExtractVariables policy extracts content from a request or response and sets the value of a variable to that content. You can extract any part of the message, including headers, URI paths, JSON/XML payloads, form parameters, and query parameters (even including entities created by AccessEntity policy, XML objects, or JSON objects). The policy works by applying a text pattern to the message content and, upon finding a match, sets a variable with the specified message content.

Let us look at the sample implementation of the same trading application example. Given below, the ExtractVariables Policy labeled "Extract-Temperature" extracts the value of the "Temperature" field value from either the XML payload of the API response, and stores it in a new context variable named "getcityweather.zipcode".

Configuration of ExtractVariables policy - from XML:

#2 Modify the request/response message

As an API developer you will frequently encounter use cases where you are required to 

• add custom headers or modify the payload of a message before it is sent or received (or)

• perform advanced transformations on message content like extracting the data from a message payload, or to convert data between different formats (or)

• Pass data between different policies or steps in the flow by using variables in a message

• Make calls to different services and mashup their responses

For example Verify API key policy sets a lot of variables but those variables can have quite lengthy names to access. In such use cases, the AssignMessage policy is very helpful. The AssignMessage policy can change an existing request or response message, or create a new request or response message during the API proxy flow. The policy lets you perform the following actions on those messages:

• Add new form parameters, headers, or query parameters to a message

• Copy existing properties from one message to another

• Remove headers, query parameters, form parameters, and message payloads from a message

• Set the value of existing properties in a message

With AssignMessage, you can add, change, or remove properties of either the request or response. Alternatively you can also use it to create a custom request or response message and pass it to an alternative target, as described in Create custom request messages.

Here is an example showcasing how to access the variables in AssignMessage policy. In the example, we are accessing the flow variable set by verify API key in AssignMessage policy and assigning the value to another variable.

What other mediation policies are available in Apigee?

Apigee provides a wide range of mediation policies that you can use to modify and manipulate the content of request and response messages as they pass through an API. Here is a recap of the examples discussed above and a few other mediation policies that are available in Apigee:

1. ExtractVariables policy allows you to extract data from the request or response message and store it in variables. You can then use these variables in other policies or steps in the API flow, or pass them to the client application.

2. AssignMessage policy allows you to add, modify, or remove message content, and set or retrieve message variables. You can use this policy to customize the content and behavior of request and response messages.

3. JSONToXML policy allows you to convert a JSON-formatted message to XML. You can use this policy if you need to process a JSON message using an XML-based API, or convert the message to XML.

4. XMLToJSON: The XMLToJSON policy allows you to convert an XML-formatted message to JSON. You can use this policy if you need to process an XML message using a JSON-based API, or convert the message to JSON.

As mentioned above, these are just a few examples and there are many other mediation policies that you can use. You can even write custom scripts that extend API proxy functionalities for specialized use cases. 

Get started with Apigee today or check out our documentation for additional information. Or if you are interested in continuing the conversation, join our Google Cloud Innovator community.