KeyValueMapOperations policy

You're viewing Apigee X documentation.
View Apigee Edge documentation.

Key Value Map Operations icon from the Apigee UI

What

Provides policy-based access to a key value map (KVM) store available in Apigee. Key/value pairs can be stored, retrieved, and deleted from named existing maps by configuring KeyValueMapOperations policies that specify PUT, GET, or DELETE operations, respectively. (At least one of these operations must be performed by the policy.)

Video: The following video provides a high-level introduction to KVMs.

Samples

PUT KVM with a literal

When the following policy runs, it creates an encrypted KVM named FooKVM, then creates a key named FooKey_1 with two values set with literal strings (not set with values extracted from variables): foo,bar. When you GET the key in the next example, you specify an index number to retrieve the value you want.

<KeyValueMapOperations async="false" continueOnError="false" enabled="true" name="CreateFooKVM" mapIdentifier="FooKVM">
  <DisplayName>CreateFooKVM</DisplayName>
  <ExpiryTimeInSecs>86400</ExpiryTimeInSecs>
  <Scope>environment</Scope>
  <Put>
    <Key>
      <Parameter>FooKey_1</Parameter>
    </Key>
    <Value>foo</Value>
    <Value>bar</Value>
  </Put>
</KeyValueMapOperations>

Notice that the scope is environment. That means you can see the KVM in the management UI under APIs > Environment Configuration > Key Value Maps. The KVMs shown on that page are all scoped to the selected environment.

GET KVM from a literal

This policy looks at the FooKVM map from the previous example, gets the second value (index="2") from the FooKey_1 key, and stores it in a variable called foo_variable.

<KeyValueMapOperations mapIdentifier="FooKVM" async="false" continueOnError="false" enabled="true" name="GetKVM">
  <DisplayName>GetKVM</DisplayName>
  <ExpiryTimeInSecs>86400</ExpiryTimeInSecs>
  <Scope>environment</Scope>
  <Get assignTo="foo_variable" index="2">
    <Key>
      <Parameter>FooKey_1</Parameter>
    </Key>
  </Get>
</KeyValueMapOperations>

Access a KVM dynamically

This policy uses the <MapName> element to reference a KVM dynamically with a flow variable. The element obtains the KVM identifier from the flow variable. Note that the mapIdentifier attribute is omitted; you cannot use <MapName> with the mapIdentifier attribute.

<KeyValueMapOperations async="false" continueOnError="false" enabled="true" name="GetKVM">
  <DisplayName>GetKVM</DisplayName>
  <MapName>ref="flow.variable"</MapName>
  <ExpiryTimeInSecs>86400</ExpiryTimeInSecs>
  <Scope>environment</Scope>
  <Get assignTo="foo_variable" index="2">
    <Key>
      <Parameter>FooKey_1</Parameter>
    </Key>
  </Get>
</KeyValueMapOperations>

PUT KVM with a variable

A simple example of a useful KVM is a URL shortening service. The KVM could be configured to store shortened URLs along with corresponding full URLs.

This policy sample creates a KVM. The policy uses the PUT command to place a key with two associated values into a KVM named urlMapper.

<KeyValueMapOperations name="putUrl" mapIdentifier="urlMapper">
  <Scope>apiproxy</Scope>
  <Put override="true">
    <Key>
      <Parameter ref="urlencoding.requesturl.hashed"/>
    </Key>
    <Value ref="urlencoding.longurl.encoded"/>
    <Value ref="request.queryparam.url"/>
  </Put>
</KeyValueMapOperations>

The key in this example, urlencoding.requesturl.hashed, is an example of a custom variable. The hashed request URL would be generated by code (JavaScript or Java, for example) and then stored in this variable, where the KeyValueMapOperations policy can access it.

For each key, requesturl.hashed, two values are stored:

  • The contents of the custom variable named urlencoding.longurl.encoded
  • The contents of the predefined variable request.queryparam.url

For example, when the policy executes at runtime, the values of the variables may be as follows:

  • urlencoding.requesturl.hashed: ed24e12820f2f900ae383b7cc4f2b31c402db1be
  • urlencoding.longurl.encoded: http://tinyurl.com/38lwmlr
  • request.queryparam.url: http://apigee.com

The following KVM and entry would be generated in Apigee's key/value store and scoped to the API proxy to which the policy is attached:

{
    "entry" :[
        {
            "name" : "ed24e12820f2f900ae383b7cc4f2b31c402db1be",
            "value" : "http://tinyurl.com/38lwmlr,http://apigee.com"
        }
    ],
    "name" : "urlMapper"
}

The entry will persist until it is deleted. Key/value store entries are distributed across instances of Apigee that are running the cloud.

GET KVM from a variable

A simple example of a useful KVM is a URL shortening service. The KVM could be configured to store shortened URLs along with corresponding full URLs.

To retrieve the value of KVM entry, such as the one covered on the KeyValueMapOperations PUT tab, configure a policy to GET the KVM:

<KeyValueMapOperations name="getUrl" mapIdentifier="urlMapper">
  <Scope>apiproxy</Scope>
  <Get assignTo="urlencoding.shorturl" index='1'>
    <Key>
      <Parameter ref="urlencoding.requesturl.hashed"/>
    </Key>
  </Get>
</KeyValueMapOperations>

When this policy is executed, if the value of the urlencoding.requesturl.hashed variable is ed24e12820f2f900ae383b7cc4f2b31c402db1be, then the custom variable named urlencoding.shorturl will be set with the value http://tinyurl.com/38lwmlr.

Now that the data has been retrieved, other policies and code can access it by extracting the value from those variables.

GET value from KVM

Use the private. attribute with all variables when accessing a KVM with the GET command to hide the KVM information in a Debug session. If the private. attribute is not used, the KVM is still encrypted; however, the KVM information will appear decrypted in the Debug session and no exception will be thrown.

In this example, the variable private.encryptedVar holds the decrypted value of the KVM's foo key.

<KeyValueMapOperations name="getEncrypted" mapIdentifier="encrypted_map">
  <Scope>apiproxy</Scope>
  <Get assignTo="private.encryptedVar" index='1'>
    <Key>
      <Parameter>foo</Parameter>
    </Key>
  </Get>
</KeyValueMapOperations>

Now that the data has been retrieved, other policies and code can access it by extracting the value from that variable.

<KeyValueMapOperations>

Provides policy-based access to a key value map (KVM).

Syntax

<KeyValueMapOperations async="false" continueOnError="false"
    enabled="true" name="Key-Value-Map-Operations-1"
    mapIdentifier="urlMapper" >
  <DisplayName>Key Value Map Operations 1</DisplayName>
  <Scope>environment</Scope>
  <ExpiryTimeInSecs>300</ExpiryTimeInSecs>
  <InitialEntries>
    <Entry>
      <Key>
        <Parameter>KEY_NAME_LITERAL</Parameter>
      </Key>
      <Value>VALUE_LITERAL</Value>
    </Entry>
    <Entry>
      <Key>
         <Parameter>VARIABLE_NAME</Parameter>
      </Key>
   </Get>
   <Delete>
      <Key>
         <Parameter>KEY_NAME_LITERAL</Parameter>
      </Key>
   </Delete>
    <MapName ref="flow.variable"></MapName>
</KeyValueMapOperations>

<KeyValueMapOperations> attributes

The following example shows the attributes of the <KeyValueMapOperations> element:

<KeyValueMapOperations async="false" continueOnError="false" enabled="true" name="Key-Value-Map-Operations-1" mapIdentifier="map_name">

The following table describes the attributes of the <KeyValueMapOperations> element:

Attribute Description Default Presence
mapIdentifier

Specifies an identifier that tells the policy which KVM it should access.

If both this identifier and <MapName> are missing, a KVM named kvmap is used. You cannot use this attribute if you specify the <MapName> element.

N/A Optional

The following table describes attributes that are common to all policy parent elements:

Attribute Description Default Presence
name

The internal name of the policy. The value of the name attribute can contain letters, numbers, spaces, hyphens, underscores, and periods. This value cannot exceed 255 characters.

Optionally, use the <DisplayName> element to label the policy in the management UI proxy editor with a different, natural-language name.

N/A Required
continueOnError

Set to false to return an error when a policy fails. This is expected behavior for most policies.

Set to true to have flow execution continue even after a policy fails.

false Optional
enabled

Set to true to enforce the policy.

Set to false to turn off the policy. The policy will not be enforced even if it remains attached to a flow.

true Optional
async

This attribute is deprecated.

false Deprecated

<DisplayName> element

Use in addition to the name attribute to label the policy in the management UI proxy editor with a different, natural-language name.

<DisplayName>Policy Display Name</DisplayName>
Default

N/A

If you omit this element, the value of the policy's name attribute is used.

Presence Optional
Type String

Child elements

This section describes the elements and attributes of the KeyValueMapOperations policy:

<Delete> element

Deletes the specified key/value pair. At least one of <Get>, <Put>, or <Delete> must be used.

Be sure to specify the name of the KVM with the mapIdentifier attribute on the root element or with the <MapName> element. For example:

<Delete>
   <Key>
      <Parameter>KEY_NAME_LITERAL</Parameter>
   </Key>
</Delete>
Default N/A
Presence Required if <Get> or <Put> are not present.
Type N/A

<Entry> element

Seed values for KVMs, which are populated in the KVM when it is initialized. For Apigee, the key size is limited to 2 KB.

For example:

<InitialEntries>
   <Entry>
      <Key>
         <Parameter>KEY_NAME_LITERAL</Parameter>
      </Key>
      <Value>v1</Value>
   </Entry>
   <Entry>
      <Key>
         <Parameter>key_name_variable</Parameter>
      </Key>
      <Value>v3</Value>
      <Value>v4</Value>
   </Entry>
</InitialEntries>
Default N/A
Presence Optional
Type N/A

<ExclusiveCache> element

Deprecated. Use the <Scope> element instead.

<ExpiryTimeInSecs> element

Specifies the duration in seconds after which Apigee refreshes its cached value from the specified KVM.

A value of 0 or -1, or excluding this element, means the default value of 300 seconds is used. For example:

<ExpiryTimeInSecs>600</ExpiryTimeInSecs>
Default 300 (5 minutes)
Presence Optional
Type Integer

A KVM is a long-term persistence mechanism that stores keys and values in a NoSQL database. Because of this, reading from a KVM at runtime can potentially slow proxy performance. To improve performance, Apigee has a built-in mechanism for caching KVM keys/values in memory during runtime. This KVM Operations policy always reads from cache for GET operations.

The <ExpiryTimeInSecs> element lets you control how long the keys/values used in the policy are stored in cache before they're refreshed again from the KVM. However, there are some differences between how GET and PUT operations affect cache expiration.

GET - The very first time a KVM GET operation executes, the requested keys/values from the KVM (whose name is specified in the policy's root mapIdentifier attribute or the <MapName> element) are loaded into cache, where they remain for subsequent GET operations until one of the following occurs:

  • The number of seconds specified in <ExpiryTimeInSecs> expires.
    or
  • A PUT operation in a KVM policy overwrites the existing values (explained next).

PUT - A PUT operation writes keys/values to the specified KVM. If the PUT writes to a key that already exists in cache, that cache is immediately refreshed and now holds the new value for the number of seconds specified in the policy's <ExpiryTimeInSecs> element.

Example - Caching a KVM

  1. A GET operation retrieves the value of "rating," which adds the value "10" to cache. The <ExpiryTimeInSecs> on the policy is 60.
  2. 30 seconds later, the GET policy executes again and retrieves 10 from the cache.
  3. 5 seconds later, a PUT policy updates the value of rating to 10, and the <ExpiryTimeInSecs> on the PUT policy is 20. The cache is immediately refreshed with the new value, which is now set to remain in cache for 20 seconds. (If the PUT hadn't happened, the cache originally populated by the first GET would still exist for another 30 seconds, left over from the original 60 seconds.)
  4. 15 seconds later, another GET executes and retrieves a value of 8.

<Get> element

Retrieves the value for the key specified. At least one of <Get>, <Put>, or <Delete> must be used.

Be sure to specify the name of the KVM either with the mapIdentifier attribute on the root element or with the <MapName> element.

You can include multiple Get blocks in the policy to retrieve multiple items from a KVM.

Default N/A
Presence Required if <Put> or <Delete> are not present.
Type N/A

private. attribute

Apigee encrypts all KVMs by default. Use the private. attribute with all variables when accessing a KVM with the GET command to hide the KVM information in a Debug session. If the private. attribute is not used, the KVM is still encrypted; however, the KVM information will appear decrypted in the Debug session and no exception will be thrown.

To get a single item from a KVM using the private. attribute:

<Get assignTo="private.myvar" index="1">
  <Key>
    <Parameter>key_name_literal</Parameter>
  </Key>
</Get>

Get a single item from a KVM

<Get assignTo="myvar" index="1">
  <Key>
    <Parameter>key_name_literal</Parameter>
  </Key>
</Get>

Get multiple items from a KVM

In the following example, assume a KVM with the following keys and values. In addition to storing a running list of the most popular movies of all time, the KVM stores the director name for all major films.

Key Value
top_movies Princess Bride,The Godfather,Citizen Kane
Citizen Kane Orson Welles
Princess Bride Rob Reiner
The Godfather Francis Ford Coppola

Here's a KVM policy configuration we can use to retrieve the current most popular movie and the name of its director:

<Get assignTo="top.movie.pick" index="1">
  <Key>
    <Parameter>top_movies</Parameter>
  </Key>
</Get>
<Get assignTo="movie.director">
  <Key>
    <Parameter ref="top.movie.pick"/>
  </Key>
</Get>

When the API proxy is called, Apigee creates the following variables that you can use in the API proxy flow:

  • top.movie.pick=Princess Bride
  • movie.director=Rob Reiner

Attributes

The following table describes the attributes of the <Get> element:

Attribute Description Default Presence
assignTo

The variable to which the retrieved value should be assigned.

N/A Required
index

The index number (in a 1-based index) of the item to fetch from a multi-valued key. For example, specifying index=1 will return the first value and assign it to the assignTo variable. If no index value is specified, all the values of that entry are assigned to the variable as a java.util.List.

For an example, see the Get Value from KVM tab in Samples.

N/A Optional

<InitialEntries> element

Seed values for KVMs, which are populated in the KVM when it is initialized. Be sure to specify the name of the KVM with the mapIdentifier attribute on the root element.

For example:

<InitialEntries>
   <Entry>
      <Key>
         <Parameter>KEY_NAME_LITERAL</Parameter>
      </Key>
      <Value>v1</Value>
   </Entry>
   <Entry>
      <Key>
         <Parameter>KEY_NAME_VARIABLE</Parameter>
      </Key>
      <Value>v3</Value>
      <Value>v4</Value>
   </Entry>
</InitialEntries>

When using this element, when you save the policy in the Apigee UI on a deployed version of the API proxy, or deploy the API proxy bundle containing the policy with this element, the key(s) are automatically created in the KVM. If the values in the policy are different than the values in the KVM, the values in the KVM are overwritten when the API proxy is deployed. Any new keys/values are added to the existing KVM alongside the existing keys/values.

Keys and values populated by this element must be literals. For example, <Parameter ref="request.queryparam.key"> isn't supported within this element.

The key size is limited to 2 KB.

Default N/A
Presence Optional
Type N/A

<Key> element

Specifies the key in a KVM entry. A key can be composite, which means that more than one parameter can be appended to create the key. For example, userID and role might be combined to create a key. For example:

<Key>
  <Parameter>KEY_NAME_LITERAL</Parameter>
</Key>

Be sure to see the <Parameter> element for specifics about how to set the key name.

The key size is limited to 2 KB.

Default N/A
Presence Optional
Type N/A

<MapName> element

The <MapName> element enables the policy to identify which KVM to use dynamically, at runtime. The ability to select KVMs dynamically gives you the flexibility to design one KVMOperations policy that can access multiple KVMs.

Some examples:

<!-- use one of the following forms -->

<MapName>literal_string</MapName>

<MapName ref="flow.variable"></MapName>

<MapName ref="flow.variable">literal_string</MapName>

The first line specifies the KVM name as a literal string. The second line gets the name from a flow variable. In the third line, if the flow variable resolves to an empty value, the literal string is used instead.

Do not specify the mapIdentifier attribute if you use <MapName> in your policy. See policy attributes for more information.

If the map does not exist when the proxy is deployed, the map won't be created and Apigee throws a runtime error when the policy is executed. If the flow variable is provided, the <InitialEntries> element is not allowed. You will get a validation error during deployment.

<Parameter> element

Specifies the key in a key/value pair. This element specifies the name when creating, updating, retrieving, or deleting the key/value pair.

You can specify the name by using:

  • A literal string

    <Key>
      <Parameter>literal</Parameter>
    </Key>
  • A variable to be retrieved at run time, using the ref attribute

    <Key>
      <Parameter ref="variable_name"/>
    </Key>
    
  • A combination of literals and variable references

    <Key>
      <Parameter>targeturl</Parameter>
      <Parameter ref="apiproxy.name"/>
      <Parameter>weight</Parameter>
    </Key>

When the <Key> element includes multiple <Parameter> elements, the effective key string is the concatenation of the values of each parameter, joined with a double underscore. For example, in the above example, if the apiproxy.name variable has the value abc1, then the effective key will be targeturl__abc1__weight.

Whether you're getting, updating, or deleting a key/value entry, the key name must match the name of the key in the KVM. See Specifying and retrieving key names for guidelines.

Default N/A
Presence Required
Type String

Attributes

The following table describes the attributes of the <Parameter> element:

Attribute Description Default Presence
ref Specifies the name of a variable whose value contains the exact name of the key you want to create, get, or delete. N/A Required if no literal value is given between the opening and closing tags.

<Put> element

Writes a key/value pair to a KVM. If the KVM specified in the mapIdentifier attribute on the root element doesn't exist and if the <MapName> element is not used, the map is automatically created. If the key value map already exists, the key/value are added to it.

To create a KVM using the UI or API, see:

<Put override="false">
  <Key>
    <Parameter ref="mykeyvar"/>
  </Key>
  <Value ref="myvalvar1"/>
</Put>
Default N/A
Presence Required if <Get> or <Delete> are not present.
Type N/A

Attributes

The following table describes the attributes of the <Put> element:

Attribute Description Default Presence
override

If set to true, it overrides the value for a key.

<Put override="false"> will write if and only if there is no entry currently persisted in the KVM store for that key.

true Optional

<Scope> element

Defines the boundary of accessibility for KVMs. The default scope is environment, meaning that, by default, maps entries are shared by all API proxies running in an environment (for example, test or prod). If you set the scope to apiproxy, then entries in the KVM are accessible only by the API proxy that writes the values to the map.

Note that when accessing a map or map entry, you must specify the same scope value you used when the map was created. For example, if the map was created with a scope of apiproxy, you must use the apiproxy scope when retrieving its values, putting changes, or deleting entries.

<Scope>environment</Scope>
Default environment
Presence Optional
Type String
Valid values
  • organization
  • environment
  • apiproxy
  • policy (API proxy revision)

<Value> element

Specifies the value of a key. You can specify the value as either a literal string or, using the ref attribute, as a variable to be retrieved at run time:

<!-- Specify a literal value -->
<Value>literal<Value>

or:

<!-- Specify the name of variable value to be populated at run time. -->
<Value ref="variable_name"/>

You can also include multiple <Value> elements to specify a multi-part value. Values are combined at run time.

In the following example, two keys are added to the KVM:

  • Key k1 with values v1,v2
  • Key k2 with values v3,v4
<InitialEntries>
  <Entry>
    <Key>
      <Parameter>k1</Parameter>
    </Key>
    <Value>v1</Value>
    <Value>v2</Value>
  </Entry>
  <Entry>
    <Key>
      <Parameter>k2</Parameter>
    </Key>
    <Value>v3</Value>
    <Value>v4</Value>
  </Entry>
</InitialEntries>

In the following example, one key is created with two values. Let's assume the organization name is foo_org, the API proxy name is bar, and the environment is test:

  • Key foo_org with values bar,test
<Put>
  <Key>
    <Parameter ref="organization.name"/>
  </Key>
  <Value ref="apiproxy.name"/>
  <Value ref="environment.name"/>
</Put>
Default N/A
Presence Required
Type String

Attributes

The following table describes the attributes of the <Value> element:

Attribute Description Default Presence
ref Specifies the name of a variable whose value contains the key value(s) you want to set. N/A Required if no literal value is given between the opening and closing tags. Prohibited if a literal value is given.

Error reference

This section describes the fault codes and error messages that are returned and fault variables that are set by Apigee when this policy triggers an error. This information is important to know if you are developing fault rules to handle faults. To learn more, see What you need to know about policy errors and Handling faults.

Runtime errors

These errors can occur when the policy executes.

Fault code HTTP status Cause Fix
steps.keyvaluemapoperations.UnsupportedOperationException 500

This error occurs if the mapIdentifier attribute is set to empty string in the KeyValueMapOperations policy.

Deployment errors

These errors can occur when you deploy a proxy containing this policy.

Error name Cause Fix
InvalidIndex If the index attribute specified in the <Get> element of KeyValueMapOperations policy is zero or a negative number, then the deployment of the API proxy fails. The index starts from 1, so an index of zero or negative integer is considered as invalid.
KeyIsMissing This error occurs if the <Key> element is completely missing or <Parameter> element is missing within <Key> element underneath the <Entry> of the <InitialEntries> element of the KeyValueMapOperations policy.
ValueIsMissing This error occurs if the <Value> element is missing underneath the <Entry> element of the <InitialEntries> element of the KeyValueMapOperations policy.

Schemas

Usage notes

For an overview of KVMs, see Using key value maps.

Using the Apigee UI, you can define KVMs at the environment scope only, as described in Using KVMs with the Apigee UI. Using the Apigee APIs, you can define KVMs at the organization, environment, or API proxy scope, as described in the following sections:

A KVM store provides a lightweight persistence mechanism for data formatted as key/value pairs. You can access these at runtime through policies or code. A map contains any arbitrary data in the format key=value.

For example localhost=127.0.0.1, zip_code=94110, or first_name=felix. In the first example, localhost is a key, and 127.0.0.1 is a value. Each key/value pair is stored as an entry in a key value map. A KVM can store many entries.

For example, say you need to store a list of IP addresses associated with various backend environments. You could create a KVM called ipAddresses that contains a list of key/value pairs as entries. For example, this JSON can represent such a map:

{
  "entry" : [ {
    "name" : "Development",
    "value" : "65.87.18.18"
  }, {
    "name" : "Staging",
    "value" : "65.87.18.22"
  } ],
  "name" : "ipAddresses"
}

You could use this structure to create a store of IP addresses that could be used by policies at runtime to enforce IP allowlisting/denylisting, to dynamically select a backend target address, and so on. Typically, the KeyValueMapOperations policy is used to store or retrieve long-lived information that needs to be reused over multiple request/response transactions.

KVMs can be manipulated via the KeyValueMapOperations policy, or directly via the Apigee API. You can use the API to, for example, upload large data sets to the key/value store, or create scripts to manage key value map entries. You will need to create a KVM with the API before accessing it with the KeyValueMapOperations policy.

Specifying and retrieving key names

With the <Parameter> and <Value> elements, you can specify either a literal value (where the value is between the opening and closing tags) or use the ref attribute to specify the name of a variable whose value should be used at runtime.

The <Parameter> element deserves special mention, because it determines the name of the key that gets created, as well as the key name you want to retrieve or delete. Following are two examples. The first specifies a key name literally, and the second specifies a key name using a variable. Let's assume the following are used to create keys in a KVM:

<Parameter>KEY_NAME_LITERAL</Parameter>
<Parameter ref="key.name.variable"/>

In the first instance, the literal value of KEY_NAME_LITERAL is stored in the KVM as the key name. In the second instance, whatever value is in the key.name.variable becomes the name of the key in the KVM. For example, if the key.name.variable contained the value foo, the key would be named foo.

To retrieve the key and a key value with a GET operation (or remove with a DELETE operation), the value of the <Parameter> element must match the key name in the KVM. For example, if the key name in the KVM is my_key, you can either specify the literal value with <Parameter>my_key</Parameter> or specify a variable that contains the exact value mny_key, like this: <Parameter ref="variable.containing.foo"/>.

Related topics

Refer to the following topics for more information about KVMs: