Hide
Python

Memcache Python API Overview

Python |Java |PHP |Go

High performance scalable web applications often use a distributed in-memory data cache in front of or in place of robust persistent storage for some tasks. App Engine includes a memory cache service for this purpose.

  1. When to use a memory cache
  2. Limits
  3. Caching data in Python
  4. How cached data expires
  5. Configuring memcache
  6. Monitoring memcache
  7. Best practices
  8. Sharing memcache between different programming languages
  9. Using compare and set in Python

When to use a memory cache

One use of a memory cache is to speed up common datastore queries. If many requests make the same query with the same parameters, and changes to the results do not need to appear on the web site right away, the app can cache the results in the memcache. Subsequent requests can check the memcache, and only perform the datastore query if the results are absent or expired. Session data, user preferences, and any other queries performed on most pages of a site are good candidates for caching.

Memcache may be useful for other temporary values. However, when considering whether to store a value solely in the memcache and not backed by other persistent storage, be sure that your application behaves acceptably when the value is suddenly not available. Values can expire from the memcache at any time, and may be expired prior to the expiration deadline set for the value. For example, if the sudden absence of a user's session data would cause the session to malfunction, that data should probably be stored in the datastore in addition to the memcache.

The memcache service provides best-effort cache space by default. Apps with billing enabled may opt to use dedicated memcache, which provides a fixed cache size assigned exclusively to your app.

Limits

The following limits apply to the use of the memcache service:

  • The maximum size of a cached data value is 1 MB minus the size of the key minus an implementation-dependent overhead, which is approximately 96 bytes.
  • A key cannot be larger than 250 bytes. In the Python runtime, keys that are strings longer than 250 bytes will be hashed. (Other runtimes behave differently.)
  • The "multi" batch operations can have any number of elements. The total size of the call and the total size of the data fetched must not exceed 32 megabytes.
  • A memcache key cannot contain a null byte.

Caching data in Python

The following example demonstrates several ways to set values in memcache using the Python API.

from google.appengine.api import memcache

# Add a value if it doesn't exist in the cache, with a cache expiration of 1 hour.
memcache.add(key="weather_USA_98105", value="raining", time=3600)

# Set several values, overwriting any existing values for these keys.
memcache.set_multi({ "USA_98105": "raining",
                     "USA_94105": "foggy",
                     "USA_94043": "sunny" },
                     key_prefix="weather_", time=3600)

# Atomically increment an integer value.
memcache.set(key="counter", value=0)
memcache.incr("counter")
memcache.incr("counter")
memcache.incr("counter")

How cached data expires

By default, values stored in memcache are retained as long as possible. Values may be evicted from the cache when a new value is added to the cache if the cache is low on memory. When values are evicted due to memory pressure, the least recently used values are evicted first.

The app can provide an expiration time when a value is stored, as either a number of seconds relative to when the value is added, or as an absolute Unix epoch time in the future (a number of seconds from midnight January 1, 1970). The value will be evicted no later than this time, though it may be evicted for other reasons.

Under rare circumstances, values may also disappear from the cache prior to expiration for reasons other than memory pressure. While memcache is resilient to server failures, memcache values are not saved to disk, so a service failure may cause values to become unavailable.

In general, an application should not expect a cached value to always be available.

You can erase an application's entire cache via the API or via the Google Developers Console's memcache page.

Configuring memcache

App Engine supports two classes of the memcache service:

  • Shared memcache is the free default for App Engine applications. It provides cache capacity on a best-effort basis and is subject to the overall demand of all the App Engine applications using the shared memcache service.

  • Dedicated memcache provides a fixed cache capacity assigned exclusively to your application. It's billed by the GB-hour of cache size. Having control over cache size means your app can perform more predictably and with fewer accesses to more costly durable storage.

Both memcache service classes use the same API. Use the Google Developers Console's memcache page to select the memcache service class for an app.

Whether shared or dedicated, memcache is not durable storage. Keys may be evicted when the cache fills up, according to the cache's LRU policy. Changes in the cache configuration or datacenter maintenance events may also flush some or all of the cache.

The following table summarizes the differences between the two classes of memcache service:

Feature Dedicated Memcache Shared Memcache
Price $0.06 per GB per hour Free
Capacity 1 to 100GB No guaranteed capacity
Performance Up to 10k operations per second per GB (items < 1KB) Not guaranteed
Durable store No No
SLA None None

Dedicated memcache billing is charged in 15 minute increments. When charging in local currency, Google will convert the prices listed into applicable local currency pursuant to the conversion rates published by leading financial institutions.

If your app needs more than 100GB of cache, please contact us at cloud-accounts-team@google.com

Monitoring memcache

Memcache collects information about the amount of data cached for an application, the cache hit rate, and the age of cache items. Note that the age of an item is reset every time it is used, either read or written. You can retrieve this information using the API or view it with the Google Developers Console's memcache page.

Dedicated memcache is rated in operations per second per GB, where an operation is defined as an individual cache item access. The operation rate varies by item size approximately according to the following table. Exceeding these ratings may result in increased API latency or errors.

Item size (KB) Rated ops/s
per GB of cache
0.1 11,000
1 10,000
2 8,500
10 4,000
100 1,500
512 500

An app configured for multiple GB of cache can in theory achieve an aggregate operation rate computed as the number of GB multiplied by the per-GB rate. For example, an app configured for 5GB of cache could reach 50,000 small-item memcache operations/sec. Achieving this level requires a good distribution of load across the memcache keyspace (see Best practices below).

Memcache compute units

A memcache compute unit (MCU) is an alternate way to measure cache traffic capacity, rather than using operations per second. Dedicated memcache is rated at 10,000 MCU per second per GB. Each cache operation has its own corresponding MCU cost. For a get that returns a value, the MCU depends on the size of the returned value; it is calculated as follows:

Get returned value size (KB) MCU
<=11.0
21.3
101.7
1005.0
51220.0
102450.0

The MCU for a set depends on the value size. It is 2 times the cost of a successful get-hit operation. Other operations are assigned MCU as follows:

Operation MCU
get-miss1.0
delete2.0
increment2.0
flush100.0
status100.0

Best practices

Following are some best practices for using memcache:

  • Handle memcache API failures gracefully. Memcache operations can fail for various reasons. Applications should be designed to catch failed operations without exposing these errors to end users. This applies especially to Set operations.
  • Use the batching capability of the API when possible, especially for small items. This will increase the performance and efficiency of your app.
  • Distribute load across your memcache keyspace. Having a single or small set of memcache items represent a disproportionate amount of traffic will hinder your app from scaling. This applies to both operations/sec and bandwidth. The problem can often be alleviated by explicit sharding of your data. For example, a frequently updated counter can be split among several keys, reading them back and summing only when a total is needed. Likewise, a 500K piece of data that must be read on every HTTP request can be split across multiple keys and read back using a single batch API call. (Even better would be to cache the value in instance memory.) For dedicated memcache, the peak access rate on a single key should be 1-2 orders of magnitude less than the per-GB rating.

Sharing memcache between different programming languages

An App Engine app can be factored into one or more modules and versions. Sometimes it is convenient to write modules and versions in different programming languages. You can share the data in your memcache between any of your app's modules and versions. Because the memcache API serializes its parameters, and the API may be implemented differently in different languages, you need to code memcache keys and values carefully if you intend to share them between langauges.

Key Compatibility

To ensure language-independence, memcache keys should be bytes:

  • In Python use plain strings (not Unicode strings)
  • In Java use byte arrays (not strings)
  • In Go use byte arrays
  • In PHP use strings

Remember that memcache keys cannot be longer than 250 bytes, and they cannot contain null bytes.

Value Compatibility

For memcache values that can be written and read in all languages, follow these guidelines:

  • Byte-arrays and ASCII strings can be safely passed between languages.
  • Unicode strings are compatible, but you must encode and decode them properly in Go and PHP.
  • Be careful using integers, they are safe for increment/decrement operations, but Go does not directly support integers and PHP cannot handle 64-bit integers.
  • Avoid using floating point values and complex types like lists, maps, structs, and classes, because each language serializes them in a different way. If you need to use types like these we recommend that you implement your own language-independent serialization that uses a format such as JSON or protocol buffers.

Example

The example code below operates on two memcache items in Python, Java, Go, and PHP. It reads and writes an item with the key “who” and increments an item with the key “count”. If you create a single app with separate modules using these four code snippets, you will see that the values set or incremented in one language will be read by the other languages.

Python

self.response.headers['Content-Type'] = 'text/plain'

who = memcache.get('who')
self.response.write('Previously incremented by %s\n' % who)
memcache.set('who', 'Python')

count = memcache.incr('count', 1, initial_value=0)
self.response.write('Count incremented by Python = %s\n' % count)

Java

resp.setContentType("text/plain");

byte[] whoKey = "who".getBytes();
byte[] countKey = "count".getBytes();

byte[] who = (byte[]) memcache.get(whoKey);
String whoString = who == null ? "nobody" : new String(who);
resp.getWriter().print("Previously incremented by " + whoString + "\n");
memcache.put(whoKey, "Java".getBytes());

Long count = memcache.increment(countKey, 1L, 0L);
resp.getWriter().print("Count incremented by Java = " + count + "\n");

Go

w.Header().Set("Content-Type", "text/plain")

whoItem, err := memcache.Get(c, "who")
var who = "nobody"
if err == nil {
  who = string(whoItem.Value)
}
fmt.Fprintf(w, "Previously incremented by %s\n", who)
memcache.Set(c, &memcache.Item{
  Key:   "who",
  Value: []byte("Go"),
})

count, _ := memcache.Increment(c, "count", 1, 0)
fmt.Fprintf(w, "Count incremented by Go = %d\n", count)

PHP

header('Content-Type: text/plain');

$who = $memcache->get('who');
echo 'Previously incremented by ' . $who . "\n";
$memcache->set('who', 'PHP');

$count = $memcache->increment('count', 1, 0);
echo 'Count incremented by PHP = ' .  $count . "\n";

Using compare and set in Python

What is compare and set?

The compare and set feature provides a way to safely make key-value updates to memcache in scenarios where multiple requests are being handled concurrently that need to update the same memcache key in an atomic fashion. Without using the compare and set feature, it is possible to get race conditions in those scenarios.

Key logical components of compare and set

The Client object is required for compare and set because certain state information is stored away in it by the methods that support compare and set. (You cannot use the memcache functions, which are stateless.) The Client class itself is not thread-safe, so you should not use the same Client object in more than one thread.

When you retrieve keys, you must use the memcache Client methods that support compare and set: gets() or get_multi() with the for_cas param set to True. The gets() operation internally receives two values from the memcache service: the value stored for the key and a timestamp (also known as the cas_id). The timestamp is an opaque number; only the memcache service knows what it means. The important thing is that each time the value associated with a memcache key is updated, the associated timestamp is changed. The gets() operation stores this timestamp in a Python dict on the Client object, using the key passed to gets() as the dict key.

When you update a key, you must use the memcache Client methods that support compare and set: cas() or cas_multi(). The cas() operation internally adds the timestamp to the request it sends to the memcache service. The service then compares the timestamp received with a cas() operation to the timestamp currently associated with the key. If they match, it updates the value and the timestamp, and returns success. If they don't match, it leaves the value and timestamp alone, and returns failure. By the way, it does not send the new timestamp back with a successful response. The only way to retrieve the timestamp is to call gets().

The other key logical component is the App Engine memcache service and its behavior with regard to compare and set. The App Engine memcache service itself behaves atomically. That is, when two concurrent requests (for the same app id) use memcache, they will go to the same memcache service instance (for historic reasons called a shard), and the memcache service has enough internal locking so that concurrent requests for the same key are properly serialized. In particular this means that two cas() requests for the same key do not actually run in parallel -- the service handles the first request that came in until completion (i.e., updating the value and timestamp) before it starts handling the second request.

Using compare and set

To use the compare and set feature,

  1. Instantiate a memcache Client object.
  2. Use a Retry loop
    1. Within the Retry loop, get the key using gets() (or get_multi() with the for_cas param set to True).
    2. Within the Retry loop, Update the key value using cas() or cas_multi().

The following snippet shows one way to use this feature:

def bump_counter(key):
   client = memcache.Client()
   while True: # Retry loop
     counter = client.gets(key)
     assert counter is not None, 'Uninitialized counter'
     if client.cas(key, counter+1):
        break

The retry loop is necessary because without the loop this code doesn't actually avoid race conditions, it just detects them! The memcache service guarantees that when used in the pattern shown here (i.e. using gets() and cas(), if two (or more) different client instances happen to be involved a race condition, only the first one to execute the cas() operation will succeed (return True), while the second one (and later ones) will fail (return False).

Another refinement you should add to this sample code is to set a limit on the number of retries, to avoid an infinite loop in worst-case scenarios where there is a lot of contention for the same counter (meaning more requests are trying to update the counter than the memcache service can process in real time).