Local Unit Testing for Java 8

Unit testing allows you to check the quality of your code after you've written it, but you can also use unit testing to improve your development process as you go along. Instead of writing tests after you finish developing your application, consider writing the tests as you go. This helps you design small, maintainable, reusable units of code. It also makes it easier for you to test your code thoroughly and quickly.

When you do local unit testing, you run tests that stay inside your own development environment without involving remote components. App Engine provides testing utilities that use local implementations of datastore and other App Engine services. This means you can exercise your code's use of these services locally, without deploying your code to App Engine, by using service stubs.

A service stub is a method that simulates the behavior of the service. For example, the datastore service stub shown in Writing Datastore and Memcache Tests allows you to test your datastore code without making any requests to the real datastore. Any entity stored during a datastore unit test is held in memory, not in the datastore, and is deleted after the test run. You can run small, fast tests without any dependency on datastore itself.

This document provides some information about setting up a testing framework, then describes how to write unit tests against several local App Engine services.

Setting up a testing framework

Even though the SDK's testing utilities are not tied to any specific framework, this guide uses JUnit for the examples so you have something concrete and complete to work from. Before you begin writing tests, you'll need to add the appropriate JUnit 4 JAR to your testing classpath. Once that's done, you're ready to write a very simple JUnit test.

import static org.junit.Assert.assertEquals;

import org.junit.Test;

public class MyFirstTest {
  @Test
  public void testAddition() {
    assertEquals(4, 2 + 2);
  }
}

If you're running Eclipse, select the source file of the test to run. Select the Run menu > Run As > JUnit Test. The results of the test appear in the Console window.

Note: It is good practice to store your unit tests in a different location than your application code. Also, avoid deploying JUnit and other testing packages with your application.

Introducing the Java 8 testing utilities

MyFirstTest demonstrates the simplest possible test setup, and for tests that have no dependency on App Engine APIs or local service implementations, you might not need anything more. However, if your tests or code under test have these dependencies, add the following JAR files to your testing classpath:

• ${SDK_ROOT}/lib/impl/appengine-api.jar
• ${SDK_ROOT}/lib/impl/appengine-api-stubs.jar
• ${SDK_ROOT}/lib/appengine-tools-api.jar

These JARs make the runtime APIs and the local implementations of those APIs available to your tests.

App Engine services expect a number of things from their execution environment, and setting these things up involves a fair amount of boilerplate code. Rather than set it up yourself, you can use the utilities in the com.google.appengine.tools.development.testing package. To use this package, add the following JAR file to your testing classpath:

• ${SDK_ROOT}/lib/testing/appengine-testing.jar

Take a minute to browse the javadoc for the com.google.appengine.tools.development.testing package. The most important class in this package is LocalServiceTestHelper, which handles all of the necessary environment setup and gives you a top-level point of configuration for all the local services you might want to access in your tests.

To write a test that accesses a specific local service:

• Create an instance of LocalServiceTestHelper with a LocalServiceTestConfig implementation for that specific local service.
• Call setUp() on your LocalServiceTestHelper instance before each test and tearDown() after each test.

Writing Datastore and memcache tests

The following example tests the use of the datastore service.

import static com.google.appengine.api.datastore.FetchOptions.Builder.withLimit;
import static org.junit.Assert.assertEquals;

import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.Query;
import com.google.appengine.tools.development.testing.LocalDatastoreServiceTestConfig;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class LocalDatastoreTest {

  private final LocalServiceTestHelper helper =
      new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig());

  @Before
  public void setUp() {
    helper.setUp();
  }

  @After
  public void tearDown() {
    helper.tearDown();
  }

  // Run this test twice to prove we're not leaking any state across tests.
  private void doTest() {
    DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
    assertEquals(0, ds.prepare(new Query("yam")).countEntities(withLimit(10)));
    ds.put(new Entity("yam"));
    ds.put(new Entity("yam"));
    assertEquals(2, ds.prepare(new Query("yam")).countEntities(withLimit(10)));
  }

  @Test
  public void testInsert1() {
    doTest();
  }

  @Test
  public void testInsert2() {
    doTest();
  }
}

In this example, LocalServiceTestHelper sets up and tears down the parts of the execution environment that are common to all local services, and LocalDatastoreServiceTestConfig sets up and tears down the parts of the execution environment that are specific to the local datastore service. If you read the javadoc you'll learn that this involves configuring the local datastore service to keep all data in memory (as opposed to flushing to disk at regular intervals) and wiping out all in-memory data at the end of every test. This is just the default behavior for a datastore test, and if this behavior isn't what you want you can change it.

Changing the example to access memcache instead of datastore

To create a test that accesses the local memcache service, you can use the code that is shown above, with a few small changes.

Instead of importing classes related to datastore, import those related to memcache. You still need to import LocalServiceTestHelper.

Change the name of the class that you are creating, and change the instance of LocalServiceTestHelper, so that they are specific to memcache.

public class LocalMemcacheTest {

  private final LocalServiceTestHelper helper =
      new LocalServiceTestHelper(new LocalMemcacheServiceTestConfig());

And finally, change the way you actually run the test so that it is relevant to memcache.

private void doTest() {
  MemcacheService ms = MemcacheServiceFactory.getMemcacheService();
  assertFalse(ms.contains("yar"));
  ms.put("yar", "foo");
  assertTrue(ms.contains("yar"));
}

As in the datastore example, the LocalServiceTestHelper and the service- specific LocalServiceTestConfig (in this case LocalMemcacheServiceTestConfig) manage the execution environment.

Writing Cloud Datastore tests

If your app uses Cloud Datastore, you might want to write tests that verify your application's behavior in the face of eventual consistency. LocalDatastoreServiceTestConfig exposes options that make this easy:

import static com.google.appengine.api.datastore.FetchOptions.Builder.withLimit;
import static org.junit.Assert.assertEquals;

import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.KeyFactory;
import com.google.appengine.api.datastore.Query;
import com.google.appengine.tools.development.testing.LocalDatastoreServiceTestConfig;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class LocalHighRepDatastoreTest {

  // Maximum eventual consistency.
  private final LocalServiceTestHelper helper =
      new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig()
          .setDefaultHighRepJobPolicyUnappliedJobPercentage(100));

  @Before
  public void setUp() {
    helper.setUp();
  }

  @After
  public void tearDown() {
    helper.tearDown();
  }

  @Test
  public void testEventuallyConsistentGlobalQueryResult() {
    DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
    Key ancestor = KeyFactory.createKey("foo", 3);
    ds.put(new Entity("yam", ancestor));
    ds.put(new Entity("yam", ancestor));
    // Global query doesn't see the data.
    assertEquals(0, ds.prepare(new Query("yam")).countEntities(withLimit(10)));
    // Ancestor query does see the data.
    assertEquals(2, ds.prepare(new Query("yam", ancestor)).countEntities(withLimit(10)));
  }
}

By setting the unapplied job percentage to 100, we are instructing the local datastore to operate with the maximum amount of eventual consistency. Maximum eventual consistency means writes will commit but always fail to apply, so global (non-ancestor) queries will consistently fail to see changes. This is of course not representative of the amount of eventual consistency your application will see when running in production, but for testing purposes, it's very useful to be able to configure the local datastore to behave this way every time.

If you want more fine-grained control over which transactions fail to apply, you can register your own HighRepJobPolicy:

import static com.google.appengine.api.datastore.FetchOptions.Builder.withLimit;
import static org.junit.Assert.assertEquals;

import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.Query;
import com.google.appengine.api.datastore.dev.HighRepJobPolicy;
import com.google.appengine.tools.development.testing.LocalDatastoreServiceTestConfig;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class LocalCustomPolicyHighRepDatastoreTest {
  private static final class CustomHighRepJobPolicy implements HighRepJobPolicy {
    static int newJobCounter = 0;
    static int existingJobCounter = 0;

    @Override
    public boolean shouldApplyNewJob(Key entityGroup) {
      // Every other new job fails to apply.
      return newJobCounter++ % 2 == 0;
    }

    @Override
    public boolean shouldRollForwardExistingJob(Key entityGroup) {
      // Every other existing job fails to apply.
      return existingJobCounter++ % 2 == 0;
    }
  }

  private final LocalServiceTestHelper helper =
      new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig()
          .setAlternateHighRepJobPolicyClass(CustomHighRepJobPolicy.class));

  @Before
  public void setUp() {
    helper.setUp();
  }

  @After
  public void tearDown() {
    helper.tearDown();
  }

  @Test
  public void testEventuallyConsistentGlobalQueryResult() {
    DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
    ds.put(new Entity("yam")); // applies
    ds.put(new Entity("yam")); // does not apply
    // First global query only sees the first Entity.
    assertEquals(1, ds.prepare(new Query("yam")).countEntities(withLimit(10)));
    // Second global query sees both Entities because we "groom" (attempt to
    // apply unapplied jobs) after every query.
    assertEquals(2, ds.prepare(new Query("yam")).countEntities(withLimit(10)));
  }
}

The testing APIs are useful for verifying that your application behaves properly in the face of eventual consistency, but please keep in mind that the local High Replication read consistency model is an approximation of the production High Replication read consistency model, not an exact replica. In the local environment, performing a get() of an Entity that belongs to an entity group with an unapplied write will always make the results of the unapplied write visible to subsequent global queries. In production this is not the case.

Writing task queue tests

Tests that use the local task queue are a bit more involved because, unlike datastore and memcache, the task queue API does not expose a facility for examining the state of the service. We need to access the local task queue itself to verify that a task has been scheduled with the expected parameters. To do this, we need com.google.appengine.api.taskqueue.dev.LocalTaskQueue.

import static org.junit.Assert.assertEquals;

import com.google.appengine.api.taskqueue.QueueFactory;
import com.google.appengine.api.taskqueue.TaskOptions;
import com.google.appengine.api.taskqueue.dev.LocalTaskQueue;
import com.google.appengine.api.taskqueue.dev.QueueStateInfo;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
import com.google.appengine.tools.development.testing.LocalTaskQueueTestConfig;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class TaskQueueTest {

  private final LocalServiceTestHelper helper =
      new LocalServiceTestHelper(new LocalTaskQueueTestConfig());

  @Before
  public void setUp() {
    helper.setUp();
  }

  @After
  public void tearDown() {
    helper.tearDown();
  }

  // Run this test twice to demonstrate we're not leaking state across tests.
  // If we _are_ leaking state across tests we'll get an exception on the
  // second test because there will already be a task with the given name.
  private void doTest() throws InterruptedException {
    QueueFactory.getDefaultQueue().add(TaskOptions.Builder.withTaskName("task29"));
    // Give the task time to execute if tasks are actually enabled (which they
    // aren't, but that's part of the test).
    Thread.sleep(1000);
    LocalTaskQueue ltq = LocalTaskQueueTestConfig.getLocalTaskQueue();
    QueueStateInfo qsi = ltq.getQueueStateInfo().get(QueueFactory.getDefaultQueue().getQueueName());
    assertEquals(1, qsi.getTaskInfo().size());
    assertEquals("task29", qsi.getTaskInfo().get(0).getTaskName());
  }

  @Test
  public void testTaskGetsScheduled1() throws InterruptedException {
    doTest();
  }

  @Test
  public void testTaskGetsScheduled2() throws InterruptedException {
    doTest();
  }
}

Notice how we ask the LocalTaskqueueTestConfig for a handle to the local service instance, and then we investigate the local service itself to make sure the task was scheduled as expected. All LocalServiceTestConfig implementations expose a similar method. You may not always need it, but sooner or later you'll be glad it's there.

Setting the queue.xml configuration file

The task queue test libraries allow any number of queue.xml configurations to be specified on a per-LocalServiceTestHelper basis via the LocalTaskQueueTestConfig.setQueueXmlPath method. Currently any queue's rate limit settings are ignored by the local development server. It is not possible to run simultaneous tasks at a time locally.

For example, a project may need to test against the queue.xml file that will be uploaded and used by the App Engine application. Assuming that the queue.xml file is in the standard location, the above sample code could be modified as follows to grant the test access to the queues specified in the src/main/webapp/WEB-INF/queue.xml file:

private final LocalServiceTestHelper helper =
    new LocalServiceTestHelper(new LocalTaskQueueTestConfig()
        .setQueueXmlPath("src/main/webapp/WEB-INF/queue.xml"));

Modify the path to the queue.xml file to fit your project's file structure.

Use the QueueFactory.getQueue method to access queues by name:

QueueFactory.getQueue("my-queue-name").add(TaskOptions.Builder.withTaskName("task29"));

Writing deferred task tests

If your application code uses Deferred Tasks, the Java Testing Utilities make it easy to write an integration test that verifies the results of these tasks.

import static org.junit.Assert.assertTrue;

import com.google.appengine.api.taskqueue.DeferredTask;
import com.google.appengine.api.taskqueue.QueueFactory;
import com.google.appengine.api.taskqueue.TaskOptions;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
import com.google.appengine.tools.development.testing.LocalTaskQueueTestConfig;
import java.util.concurrent.TimeUnit;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class DeferredTaskTest {

  // Unlike CountDownLatch, TaskCountDownlatch lets us reset.
  private final LocalTaskQueueTestConfig.TaskCountDownLatch latch =
      new LocalTaskQueueTestConfig.TaskCountDownLatch(1);

  private final LocalServiceTestHelper helper =
      new LocalServiceTestHelper(new LocalTaskQueueTestConfig()
          .setDisableAutoTaskExecution(false)
          .setCallbackClass(LocalTaskQueueTestConfig.DeferredTaskCallback.class)
          .setTaskExecutionLatch(latch));

  private static class MyTask implements DeferredTask {
    private static boolean taskRan = false;

    @Override
    public void run() {
      taskRan = true;
    }
  }

  @Before
  public void setUp() {
    helper.setUp();
  }

  @After
  public void tearDown() {
    MyTask.taskRan = false;
    latch.reset();
    helper.tearDown();
  }

  @Test
  public void testTaskGetsRun() throws InterruptedException {
    QueueFactory.getDefaultQueue().add(
        TaskOptions.Builder.withPayload(new MyTask()));
    assertTrue(latch.await(5, TimeUnit.SECONDS));
    assertTrue(MyTask.taskRan);
  }
}

As with our first Local Task Queue example, we are using a LocalTaskqueueTestConfig, but this time we are initializing it with some additional arguments that give us an easy way to verify not just that the task was scheduled but that the task was executed: We call setDisableAutoTaskExecution(false) to tell the Local Task Queue to automatically execute tasks. We call setCallbackClass(LocalTaskQueueTestConfig.DeferredTaskCallback.class) to tell the Local Task Queue to use a callback that understands how to execute Deferred tasks. And finally we call setTaskExecutionLatch(latch) to tell the Local Task Queue to decrement the latch after each task execution. This configuration allows us to write a test in which we enqueue a Deferred task, wait until that task runs, and then verify that the task behaved as expected when it ran.

Writing local service capabilities tests

Capabilities testing involves changing the status of some service, such as datastore, blobstore, memcache, and so forth, and running your application against that service to determine whether your application is responding as expected under different conditions. The capability status can be changed using the LocalCapabilitiesServiceTestConfig class.

Note: The capability status can also be changed in the dev app server console, or via command line options. For complete details, see Capabilities Service Test Configuration.

The following code snippet changes the capability status of the datastore service to disabled and then runs a test on the datastore service. You can substitute other services for datastore as needed.

import static org.junit.Assert.assertEquals;

import com.google.appengine.api.capabilities.Capability;
import com.google.appengine.api.capabilities.CapabilityStatus;
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.FetchOptions;
import com.google.appengine.api.datastore.Query;
import com.google.appengine.tools.development.testing.LocalCapabilitiesServiceTestConfig;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
import com.google.apphosting.api.ApiProxy;
import org.junit.After;
import org.junit.Test;

public class ShortTest {
  private LocalServiceTestHelper helper;

  @After
  public void tearDown() {
    helper.tearDown();
  }

  @Test(expected = ApiProxy.CapabilityDisabledException.class)
  public void testDisabledDatastore() {
    Capability testOne = new Capability("datastore_v3");
    CapabilityStatus testStatus = CapabilityStatus.DISABLED;
    // Initialize the test configuration.
    LocalCapabilitiesServiceTestConfig config =
        new LocalCapabilitiesServiceTestConfig().setCapabilityStatus(testOne, testStatus);
    helper = new LocalServiceTestHelper(config);
    helper.setUp();
    FetchOptions fo = FetchOptions.Builder.withLimit(10);
    DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
    assertEquals(0, ds.prepare(new Query("yam")).countEntities(fo));
  }
}

The sample test first creates a Capability object initialized to datastore, then creates a CapabilityStatus object set to DISABLED. The LocalCapabilitiesServiceTestConfig is created with the capability and status set using the Capability and CapabilityStatus objects just created.

The LocalServiceHelper is then created using the LocalCapabilitiesServiceTestConfig object. Now that the test has been set up, the DatastoreService is created and a Query is sent to it to determine whether the test generates the expected results, in this case, a CapabilityDisabledException.

Writing tests for other services

Testing utilities are available for blobstore and other App Engine services. For a list of all the services that have local implementations for testing, see the LocalServiceTestConfig documentation.

Writing tests with authentication expectations

This example shows how to write tests that verify logic that uses UserService to determine if a user is logged on or has admin privileges. Note that any user with the Viewer, Editor, or Owner basic role, or the App Engine App Admin predefined role has admin privileges.

import static org.junit.Assert.assertTrue;

import com.google.appengine.api.users.UserService;
import com.google.appengine.api.users.UserServiceFactory;
import com.google.appengine.tools.development.testing.LocalServiceTestHelper;
import com.google.appengine.tools.development.testing.LocalUserServiceTestConfig;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class AuthenticationTest {

  private final LocalServiceTestHelper helper =
      new LocalServiceTestHelper(new LocalUserServiceTestConfig())
          .setEnvIsAdmin(true).setEnvIsLoggedIn(true);

  @Before
  public void setUp() {
    helper.setUp();
  }

  @After
  public void tearDown() {
    helper.tearDown();
  }

  @Test
  public void testIsAdmin() {
    UserService userService = UserServiceFactory.getUserService();
    assertTrue(userService.isUserAdmin());
  }
}

In this example, we're configuring the LocalServiceTestHelper with the LocalUserServiceTestConfig so we can use the UserService in our test, but we're also configuring some authentication-related environment data on the LocalServiceTestHelper itself.

In this example, we're configuring the LocalServiceTestHelper with the LocalUserServiceTestConfig so we can use the OAuthService.