Using JPA with App Engine

Java Persistence API (JPA) is a standard interface for accessing databases in Java, providing an automatic mapping between Java classes and database tables. There is an open-source plugin available for using JPA with Datastore, and this page provides information on how to get started with it.

Warning: We think most developers will have a better experience using the low-level Datastore API, or one of the open-source APIs developed specifically for Datastore, such as Objectify. JPA was designed for use with traditional relational databases, and so has no way to explicitly represent some of the aspects of Datastore that make it different from relational databases, such as entity groups and ancestor queries. This can lead to subtle issues that are difficult to understand and fix.

The App Engine Java SDK includes version 2.x of the DataNucleus plugin for Datastore. This plugin corresponds to version 3.0 of the DataNucleus Access Platform, which enables you to use the App Engine Datastore via JPA 2.0.

See the Access Platform 3.0 documentation for more information about JPA. In particular, see JPA Documentation and JPA API.

Warning: Version 2.x of the DataNucleus plugin for App Engine uses DataNucleus v3.x. The 2.x plugin is not fully backwards-compatible with the previous 1.x plugin. If you upgrade to the new version, be sure to update and test your application.

  1. Build tools and IDEs supporting JPA 2.x and 3.0
  2. Migrating to Version 2.x of the DataNucleus Plugin
  3. Setting Up JPA 2.0
  4. Enhancing Data Classes
  5. Getting an EntityManager Instance
  6. Class and Field Annotations
  7. Inheritance
  8. Unsupported Features of JPA 2.0

Build tools and IDEs supporting JPA 2.x and 3.0

You can use Apache Ant, the App Engine Eclipse plugin, or Maven to use version 2.x or 3.0 of the DataNucleus plugin for App Engine:

  • For Ant users: The SDK includes an Ant task that performs the enhancement step. You must copy the JARs and create the configuration file when you set up your project. See Using Apache Ant for more information about the Ant task.
  • For Eclipse users: The App Engine Eclipse plugin supports DataNucleus plugin 2.0: you can select which version to use in the GUI configuration window.
  • For Maven users: You can enhance the classes with the following configurations in your pom.xml file:
                <plugin>
                    <groupId>org.datanucleus</groupId>
                    <artifactId>maven-datanucleus-plugin</artifactId>
                    <version>3.2.0-m1</version>
                    <configuration>
                        <api>JDO</api>
                        <props>${basedir}/datanucleus.properties</props>
                        <verbose>true</verbose>
                        <enhancerName>ASM</enhancerName>
                    </configuration>
                    <executions>
                        <execution>
                            <phase>process-classes</phase>
                            <goals>
                                <goal>enhance</goal>
                            </goals>
                        </execution>
                    </executions>
                    <dependencies>
                        <dependency>
                            <groupId>org.datanucleus</groupId>
                            <artifactId>datanucleus-api-jdo</artifactId>
                            <version>3.1.3</version>
                        </dependency>
                    </dependencies>
                </plugin>

Migrating to Version 2.x of the DataNucleus Plugin

This section provides instructions for upgrading your app to use version 2.x of the DataNucleus plugin for App Engine, which corresponds to DataNucleus Access Platform 3.0 and JPA 2.0. The 2.x plugin version is not fully backwards-compatible with version 1.x and may change without warning. If you upgrade, be sure to update and test your application code.

New Default Behaviors

Version 2.x of the App Engine DataNucleus plugin has some different defaults than the previous 1.x version:

  • JPA "persistence provider" is now org.datanucleus.api.jpa.PersistenceProviderImpl.
  • Level2 Caching is enabled by default. To get the previous default behavior, set the persistence property datanucleus.cache.level2.type to none. (Alternatively include the datanucleus-cache plugin in the classpath, and set the persistence property datanucleus.cache.level2.type to javax.cache to use Memcache for L2 caching.
  • Datastore IdentifierFactory now defaults to datanucleus2. To get the previous behavior, set the persistence property datanucleus.identifierFactory to datanucleus1.
  • Non-transactional calls to EntityManager.persist(), EntityManager.merge(), and EntityManager.remove() are now executed atomically. (Previously, execution occurred at the next transaction or upon EntityManager.close().
  • JPA has retainValues enabled, which means the values of loaded fields are retained in objects after a commit.
  • javax.persistence.query.chunkSize is no longer used. Use datanucleus.query.fetchSize instead.
  • There is now no longer an exception on duplicate EMF allocation. If you have the persistence property datanucleus.singletonEMFForName set to true, then it will return the currently allocated singleton EMF for that name.
  • Unowned relationships are now supported.
  • Datastore Identity is now supported.

For a full list of new features, see the release notes.

Changes to Configuration Files

To upgrade your app to use version 2.0 of the DataNucleus plugin for App Engine, you need to change some configuration settings in build.xml and persistence.xml. If you are setting up a new application and wish to use the latest version of the DataNucleus plugin, proceed to Setting up JPA 2.0.

Warning! After updating your configuration, you need to test your application code to ensure backwards-compatibility.

In build.xml

The copyjars target needs to change in order to accommodate DataNucleus 2.x:

  1. The copyjars target has changed. Update this section:
      <target name="copyjars"
          description="Copies the App Engine JARs to the WAR.">
        <mkdir dir="war/WEB-INF/lib" />
        <copy
            todir="war/WEB-INF/lib"
            flatten="true">
          <fileset dir="${sdk.dir}/lib/user">
            <include name="**/*.jar" />
          </fileset>
        </copy>
      </target>

    to:
      <target name="copyjars"
          description="Copies the App Engine JARs to the WAR.">
        <mkdir dir="war/WEB-INF/lib" />
        <copy
            todir="war/WEB-INF/lib"
            flatten="true">
          <fileset dir="${sdk.dir}/lib/user">
            <include name="**/appengine-api-1.0-sdk*.jar" />
          </fileset>
          <fileset dir="${sdk.dir}/lib/opt/user">
            <include name="appengine-api-labs/v1/*.jar" />
            <include name="jsr107/v1/*.jar" />
            <include name="datanucleus/v2/*.jar" />
          </fileset>
        </copy>
      </target>
  2. The datanucleusenhance target has changed. Update this section:
      <target name="datanucleusenhance" depends="compile"
          description="Performs enhancement on compiled data classes.">
        <enhance_war war="war" />
      </target>

    to:
      <target name="datanucleusenhance" depends="compile"
          description="Performs enhancement on compiled data classes.">
          <enhance_war war="war">
                  <args>
                  <arg value="-enhancerVersion"/>
                  <arg value="v2"/>
              </args>
          </enhance_war>
      </target>

In persistence.xml

The <provider> target has changed. Update this section:

        <provider>org.datanucleus.store.appengine.jpa.DatastorePersistenceProvider</provider>

to:

        <provider>org.datanucleus.api.jpa.PersistenceProviderImpl</provider>

Setting Up JPA 2.0

To use JPA to access the datastore, an App Engine app needs the following:

  • The JPA and datastore JARs must be in the app's war/WEB-INF/lib/ directory.
  • A configuration file named persistence.xml must be in the app's war/WEB-INF/classes/META-INF/ directory, with configuration that tells JPA to use the App Engine datastore.
  • The project's build process must perform a post-compilation "enhancement" step on the compiled data classes to associate them with the JPA implementation.

Copying the JARs

The JPA and datastore JARs are included with the App Engine Java SDK. You can find them in the appengine-java-sdk/lib/opt/user/datanucleus/v2/ directory.

Copy the JARs to your application's war/WEB-INF/lib/ directory.

Make sure the appengine-api.jar is also in the war/WEB-INF/lib/ directory. (You may have already copied this when creating your project.) The App Engine DataNucleus plugin uses this JAR to access the datastore.

Creating the persistence.xml File

The JPA interface needs a configuration file named persistence.xml in the application's war/WEB-INF/classes/META-INF/ directory. You can create this file in this location directly, or have your build process copy this file from a source directory.

Create the file with the following contents:

<?xml version="1.0" encoding="UTF-8" ?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
        http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="1.0">

    <persistence-unit name="transactions-optional">
        <provider>org.datanucleus.api.jpa.PersistenceProviderImpl</provider>
        <properties>
            <property name="datanucleus.NontransactionalRead" value="true"/>
            <property name="datanucleus.NontransactionalWrite" value="true"/>
            <property name="datanucleus.ConnectionURL" value="appengine"/>
            <property name="datanucleus.singletonEMFForName" value="true"/>
        </properties>

    </persistence-unit>

</persistence>

Datastore Read Policy and Call Deadline

As described on the Datastore Queries page, you can set the read policy (strong consistency vs. eventual consistency) and the datastore call deadline for a EntityManagerFactory in the persistence.xml file. These settings go in the <persistence-unit> element. All calls made with a given EntityManager instance use the configuration selected when the manager was created by the EntityManagerFactory. You can also override these options for an individual Query (described below).

To set the read policy, include a property named datanucleus.appengine.datastoreReadConsistency. Its possible values are EVENTUAL (for reads with eventual consistency) and STRONG (for reads with strong consistency). If not specified, the default is STRONG.

            <property name="datanucleus.appengine.datastoreReadConsistency" value="EVENTUAL" />

You can set separate datastore call deadlines for reads and for writes. For reads, use the JPA standard property javax.persistence.query.timeout. For writes, use datanucleus.datastoreWriteTimeout. The value is an amount of time, in milliseconds.

            <property name="javax.persistence.query.timeout" value="5000" />
            <property name="datanucleus.datastoreWriteTimeout" value="10000" />

If you want to use cross-group (XG) transactions, add the following property:

            <property name="datanucleus.appengine.datastoreEnableXGTransactions" value="true" />

You can have multiple <persistence-unit> elements in the same persistence.xml file, using different name attributes, to use EntityManager instances with different configurations in the same app. For example, the following persistence.xml file establishes two sets of configuration, one named "transactions-optional" and another named "eventual-reads-short-deadlines":

<?xml version="1.0" encoding="UTF-8" ?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
        http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="1.0">

    <persistence-unit name="transactions-optional">
        <provider>org.datanucleus.api.jpa.PersistenceProviderImpl</provider>
        <properties>
            <property name="datanucleus.NontransactionalRead" value="true"/>
            <property name="datanucleus.NontransactionalWrite" value="true"/>
            <property name="datanucleus.ConnectionURL" value="appengine"/>
        </properties>
    </persistence-unit>

    <persistence-unit name="eventual-reads-short-deadlines">
        <provider>org.datanucleus.api.jpa.PersistenceProviderImpl</provider>
        <properties>
            <property name="datanucleus.NontransactionalRead" value="true"/>
            <property name="datanucleus.NontransactionalWrite" value="true"/>
            <property name="datanucleus.ConnectionURL" value="appengine"/>

            <property name="datanucleus.appengine.datastoreReadConsistency" value="EVENTUAL" />
            <property name="javax.persistence.query.timeout" value="5000" />
            <property name="datanucleus.datastoreWriteTimeout" value="10000" />
            <property name="datanucleus.singletonEMFForName" value="true"/>
        </properties>
    </persistence-unit>
</persistence>

See Getting an EntityManager Instance below for information on creating an EntityManager with a named configuration set.

You can override the read policy and call deadline for an individual Query object. To override the read policy for a Query, call its setHint() method as follows:

        Query q = em.createQuery("select from " + Book.class.getName());
        q.setHint("datanucleus.appengine.datastoreReadConsistency", "EVENTUAL");

As above, the possible values are "EVENTUAL" and "STRONG".

To over