Note: Developers building new applications are strongly encouraged to use the NDB Client Library, which has several benefits compared to this client library, such as automatic entity caching via the Memcache API. If you are currently using the older DB Client Library, read the DB to NDB Migration Guide
Google Cloud Datastore supports transactions. A transaction is an operation or set of operations that is atomic—either all of the operations in the transaction occur, or none of them occur. An application can perform multiple operations and calculations in a single transaction.
A transaction is a set of Cloud Datastore operations on one or more entities. Each transaction is guaranteed to be atomic, which means that transactions are never partially applied. Either all of the operations in the transaction are applied, or none of them are applied. Transactions have a maximum duration of 60 seconds with a 10 second idle expiration time after 30 seconds.
An operation might fail when:
- Too many concurrent modifications are attempted on the same entity group.
- The transaction exceeds a resource limit.
- Cloud Datastore encounters an internal error.
In all these cases, the Cloud Datastore API raises an exception.
Transactions are an optional feature of Cloud Datastore; you're not required to use transactions to perform Cloud Datastore operations.
An application can execute a set of statements and data store operations in a
single transaction, such that if any statement or operation raises an exception,
none of the Cloud Datastore operations in the set are applied. The
application defines the actions to perform in the transaction using a Python
function. The application starts the transaction using one of the
run_in_transaction methods, depending on whether the transaction accesses
entities within a single entity group or whether the transaction is a
For the common use case of a function that is only used within transactions, use
from google.appengine.ext import db class Accumulator(db.Model): counter = db.IntegerProperty(default=0) @db.transactional def increment_counter(key, amount): obj = db.get(key) obj.counter += amount obj.put() q = db.GqlQuery("SELECT * FROM Accumulator") acc = q.get() increment_counter(acc.key(), 5)
If the function is sometimes called without a transaction, then instead of
decorating it, call
with the function as an argument:
from google.appengine.ext import db class Accumulator(db.Model): counter = db.IntegerProperty(default=0) def increment_counter(key, amount): obj = db.get(key) obj.counter += amount obj.put() q = db.GqlQuery("SELECT * FROM Accumulator") acc = q.get() db.run_in_transaction(increment_counter, acc.key(), 5)
db.run_in_transaction() takes the function object, and positional and keyword
arguments to pass to the function. If the function returns a value,
db.run_in_transaction() returns that value.
If the function returns, the transaction is committed, and all effects of Cloud Datastore operations are applied. If the function raises an exception, the transaction is "rolled back," and the effects are not applied. See the note above about exceptions.
When one transaction function is called from within another transaction,
db.run_in_transaction() have different default behavior.
@db.transactional will allow this, and the inner transaction becomes the same transaction as the outer transaction. Calling
db.run_in_transaction() attempts to "nest" another transaction within the existing transaction; but this behavior is not yet supported, and it raises
db.BadRequestError. You can specify other behavior; see the function reference on transaction options for details.
Using Cross-Group (XG) Transactions
Cross-group transactions, which operate across multiple entity groups, behave like single-group transactions, but don't fail if code tries to update entities from more than one entity group. To invoke a cross-group transaction, use transaction options.
from google.appengine.ext import db @db.transactional(xg=True) def make_things(): thing1 = Thing(a=3) thing1.put() thing2 = Thing(a=7) thing2.put() make_things()
from google.appengine.ext import db xg_on = db.create_transaction_options(xg=True) def my_txn(): x = MyModel(a=3) x.put() y = MyModel(a=7) y.put() db.run_in_transaction_options(xg_on, my_txn)
What can be done in a transaction
Google Cloud Datastore imposes restrictions on what can be done inside a single transaction.
All Cloud Datastore operations in a transaction must operate on entities in the same entity group if the transaction is a single-group transaction, or on entities in a maximum of twenty-five entity groups if the transaction is a cross-group transaction. This includes querying for entities by ancestor, retrieving entities by key, updating entities, and deleting entities. Notice that each root entity belongs to a separate entity group, so a single transaction cannot create or operate on more than one root entity unless it is a cross-group transaction.
When two or more transactions simultaneously attempt to modify entities in one or more common entity groups, only the first transaction to commit its changes can succeed; all the others will fail on commit. Because of this design, using entity groups limits the number of concurrent writes you can do on any entity in the groups. When a transaction starts, Google Cloud Datastore uses optimistic concurrency control by checking the last update time for the entity groups used in the transaction. Upon commiting a transaction for the entity groups, Google Cloud Datastore again checks he last update time for the entity groups used in the transaction. If it has hanged since the initial check, an exception is thrown . For an explanation of entity groups, see the Cloud Datastore Overview page.
An app can perform a query during a transaction, but only if it includes an ancestor filter. An app can also get Datastore entities by key during a transaction. You can prepare keys prior to the transaction, or you can build keys inside the transaction with key names or IDs.
All other Python code is allowed inside a transaction function. You can
determine if the current scope is nested in a transaction function using
The transaction function should not have side effects other than the
Cloud Datastore operations. The transaction function can be called
multiple times if a Cloud Datastore operation fails due to another
user updating entities in the entity group at the same time. When this happens,
the Cloud Datastore API retries the transaction a fixed number of
times. If they all fail,
db.run_in_transaction() raises a
You can adjust the number of times the transaction is retried using
instead of db.run_in_transaction().
Similarly, the transaction function should not have side effects that depend on the success of the transaction, unless the code that calls the transaction function knows to undo those effects. For example, if the transaction stores a new Cloud Datastore entity, saves the created entity's ID for later use, then the transaction fails, the saved ID does not refer to the intended entity because the entity's creation was rolled back. The calling code would have to be careful not to use the saved ID in this case.
Isolation and consistency
Outside of transactions, Cloud Datastore's isolation level is closest to read committed. Inside of transactions, serializable isolation is enforced. This means that another transaction cannot concurrently modify the data that is read or modified by this transaction. Read the serializable isolation wiki and the Transaction Isolation article for more information on isolation levels.
In a transaction, all reads reflect the current, consistent state of Cloud Datastore at the time the transaction started. Queries and gets inside a transaction are guaranteed to see a single, consistent snapshot of Cloud Datastore as of the beginning of the transaction. Entities and index rows in the transaction's entity group are fully updated so that queries return the complete, correct set of result entities, without the false positives or false negatives described in Transaction Isolation that can occur in queries outside of transactions.
This consistent snapshot view also extends to reads after writes inside transactions. Unlike with most databases, queries and gets inside a Cloud Datastore transaction do not see the results of previous writes inside that transaction. Specifically, if an entity is modified or deleted within a transaction, a query or get returns the original version of the entity as of the beginning of the transaction, or nothing if the entity did not exist then.
Uses for transactions
This example demonstrates one use of transactions: updating an entity with a new property value relative to its current value.
def increment_counter(key, amount): obj = db.get(key) obj.counter += amount obj.put()
This requires a transaction because the value might be updated by another user
after this code fetches the object, but before it saves the modified object.
Without a transaction, the user's request uses the value of
count prior to the
other user's update, and the save overwrites the new value. With a
transaction, the application is told about the other user's update.
If the entity is updated during the
transaction, then the transaction is retried until all steps are completed
Another common use for transactions is to fetch an entity with a named key, or create it if it doesn't yet exist:
class SalesAccount(db.Model): address = db.PostalAddressProperty() phone_number = db.PhoneNumberProperty() def get_or_create(parent_key, account_id, address, phone_number): obj = db.get(db.Key.from_path("SalesAccount", account_id, parent=parent_key)) if not obj: obj = SalesAccount(key_name=account_id, parent=parent_key, address=address, phone_number=phone_number) obj.put() else: obj.address = address obj.phone_number = phone_number
As before, a transaction is necessary to handle the case where another user is attempting to create or update an entity with the same string ID. Without a transaction, if the entity does not exist and two users attempt to create it, the second overwrites the first without knowing that it happened. With a transaction, the second attempt retries, notices that the entity now exists, and updates the entity instead.
When a transaction fails, you can have your app retry the transaction until it succeeds, or you can let your users deal with the error by propagating it to your app's user interface level. You do not have to create a retry loop around every transaction.
Get-or-create is so useful that there is a built-in method for it:
takes a key name, an optional parent, and arguments to pass to the model
constructor if an entity of that name and path does not exist. The get attempt
and the create happen in one transaction, so (if the transaction is successful)
the method always returns a model instance that represents an actual entity.
Finally, you can use a transaction to read a consistent snapshot of Cloud Datastore. This can be useful when multiple reads are needed to render a page or export data that must be consistent. These kinds of transactions are often called read-only transactions, since they perform no writes. Read-only single-group transactions never fail due to concurrent modifications, so you don't have to implement retries upon failure. However, cross-group transactions can fail due to concurrent modifications, so these should have retries. Committing and rolling back a read-only transaction are both no-ops.
class Customer(db.Model): user = db.StringProperty() class Account(db.Model): """An Account has a Customer as its parent.""" address = db.PostalAddressProperty() balance = db.FloatProperty() def get_all_accounts(): """Returns a consistent view of the current user's accounts.""" accounts =  for customer in Customer.all().filter('user =', users.get_current_user().user_id()): accounts.extend(Account.all().ancestor(customer)) return accounts
Transactional task enqueuing
You can enqueue a task as part of a Cloud Datastore transaction, so
that the task is only enqueued if the transaction is committed successfully. If
the transaction does not get committed, the task is not enqueued. If the
transaction does get committed, the task is enqueued. Once enqueued, the task
will not execute immediately, so the task is not atomic with the transaction.
Still, once enqueued, the task will retry until it succeeds. This applies to any
task enqueued during a
Transactional tasks are useful because they allow you to combine non-Cloud Datastore actions to a transaction that depends on the transaction succeeding (such as sending an email to confirm a purchase). You can also tie Cloud Datastore actions to the transaction, such as to commit changes to entity groups outside of the transaction if and only if the transaction succeeds.
def do_something_in_transaction(...) taskqueue.add(url='/path/to/my/worker', transactional=True) ... db.run_in_transaction(do_something_in_transaction, ....)