Local Unit Testing for Python 2

Unit testing allows you to check the quality of your code after you've written it, but you can also use unit testing to improve your development process as you go along. Instead of writing tests after you finish developing your application, consider writing the tests as you go. This helps you design small, maintainable, reusable units of code. It also makes it easier for you to test your code thoroughly and quickly.

When you do local unit testing, you run tests that stay inside your own development environment without involving remote components. App Engine provides testing utilities that use local implementations of datastore and other App Engine services. This means you can exercise your code's use of these services locally, without deploying your code to App Engine, by using service stubs.

A service stub is a method that simulates the behavior of the service. For example, the datastore service stub shown in Writing Datastore and Memcache Tests allows you to test your datastore code without making any requests to the real datastore. Any entity stored during a datastore unit test is held in memory, not in the datastore, and is deleted after the test run. You can run small, fast tests without any dependency on datastore itself.

This document describes how to write unit tests against several local App Engine services, then gives some information about setting up a testing framework.

Introducing the Python 2 testing utilities

An App Engine Python module called testbed makes service stubs available for unit testing.

Service stubs are available for the following services:

  • App Identity init_app_identity_stub
  • Blobstore (use init_blobstore_stub)
  • Capability (use init_capability_stub)
  • Datastore (use init_datastore_v3_stub)
  • Files (use init_files_stub)
  • Images (only for dev_appserver; use init_images_stub)
  • LogService (use init_logservice_stub)
  • Mail (use init_mail_stub)
  • Memcache (use init_memcache_stub)
  • Task Queue (use init_taskqueue_stub)
  • URL fetch (use init_urlfetch_stub)
  • User service (use init_user_stub)

To initialize all stubs at the same time, you can use init_all_stubs.

Writing Datastore and memcache tests

This section shows an example of how to write code that tests the use of the datastore and memcache services.

Make sure your test runner has the appropriate libraries on the Python load path, including the App Engine libraries, yaml (included in the App Engine SDK), the application root, and any other modifications to the library path expected by application code (such as a local ./lib directory, if you have one). For example:

import sys
sys.path.insert(1, 'google-cloud-sdk/platform/google_appengine')
sys.path.insert(1, 'google-cloud-sdk/platform/google_appengine/lib/yaml/lib')
sys.path.insert(1, 'myapp/lib')

Import the Python unittest module and the App Engine modules that are relevant to the services being tested—in this case memcache and ndb, which uses both the datastore and memcache. Also import the testbed module.

import unittest

from google.appengine.api import memcache
from google.appengine.ext import ndb
from google.appengine.ext import testbed

Then create a TestModel class. In this example, a function checks to see whether an entity is stored in memcache. If no entity is found, it checks for an entity in the datastore. This can often be redundant in real life, since ndb uses memcache itself behind the curtains, but it's still an OK pattern for a test.

class TestModel(ndb.Model):
    """A model class used for testing."""
    number = ndb.IntegerProperty(default=42)
    text = ndb.StringProperty()


class TestEntityGroupRoot(ndb.Model):
    """Entity group root"""
    pass


def GetEntityViaMemcache(entity_key):
    """Get entity from memcache if available, from datastore if not."""
    entity = memcache.get(entity_key)
    if entity is not None:
        return entity
    key = ndb.Key(urlsafe=entity_key)
    entity = key.get()
    if entity is not None:
        memcache.set(entity_key, entity)
    return entity

Next, create a test case. No matter what services you are testing, the test case must create a Testbed instance and activate it. The test case must also initialize the relevant service stubs, in this case using init_datastore_v3_stub and init_memcache_stub. The methods for initializing other App Engine service stubs are listed in Introducing the Python Testing Utilities.

class DatastoreTestCase(unittest.TestCase):

    def setUp(self):
        # First, create an instance of the Testbed class.
        self.testbed = testbed.Testbed()
        # Then activate the testbed, which prepares the service stubs for use.
        self.testbed.activate()
        # Next, declare which service stubs you want to use.
        self.testbed.init_datastore_v3_stub()
        self.testbed.init_memcache_stub()
        # Clear ndb's in-context cache between tests.
        # This prevents data from leaking between tests.
        # Alternatively, you could disable caching by
        # using ndb.get_context().set_cache_policy(False)
        ndb.get_context().clear_cache()

The init_datastore_v3_stub() method with no argument uses an in-memory datastore that is initially empty. If you want to test an existing datastore entity, include its pathname as an argument to init_datastore_v3_stub().

In addition to setUp(), include a tearDown() method that deactivates the testbed. This restores the original stubs so that tests do not interfere with each other.

def tearDown(self):
    self.testbed.deactivate()

Then implement the tests.

def testInsertEntity(self):
    TestModel().put()
    self.assertEqual(1, len(TestModel.query().fetch(2)))

Now you can use TestModel to write tests that use the datastore or memcache service stubs instead of using the real services.

For example, the method shown below creates two entities: the first entity uses the default value for the number attribute (42), and the second uses a non-default value for number (17). The method then builds a query for TestModel entities, but only for those with the default value of number.

After retrieving all matching entities, the method tests that exactly one entity was found, and that the number attribute value of that entity is the default value.

def testFilterByNumber(self):
    root = TestEntityGroupRoot(id="root")
    TestModel(parent=root.key).put()
    TestModel(number=17, parent=root.key).put()
    query = TestModel.query(ancestor=root.key).filter(
        TestModel.number == 42)
    results = query.fetch(2)
    self.assertEqual(1, len(results))
    self.assertEqual(42, results[0].number)

As another example, the following method creates an entity and retrieves it using the GetEntityViaMemcache() function that we created above. The method then tests that an entity was returned, and that its number value is the same as for the previously created entity.

def testGetEntityViaMemcache(self):
    entity_key = TestModel(number=18).put().urlsafe()
    retrieved_entity = GetEntityViaMemcache(entity_key)
    self.assertNotEqual(None, retrieved_entity)
    self.assertEqual(18, retrieved_entity.number)

And finally, invoke unittest.main().

if __name__ == '__main__':
    unittest.main()

To run the tests, see Running tests.

Writing Cloud Datastore tests

If your app uses Cloud Datastore, you might want to write tests that verify your application's behavior in the face of eventual consistency. db.testbed exposes options that make this easy:

from google.appengine.datastore import datastore_stub_util  # noqa


class HighReplicationTestCaseOne(unittest.TestCase):

    def setUp(self):
        # First, create an instance of the Testbed class.
        self.testbed = testbed.Testbed()
        # Then activate the testbed, which prepares the service stubs for use.
        self.testbed.activate()
        # Create a consistency policy that will simulate the High Replication
        # consistency model.
        self.policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(
            probability=0)
        # Initialize the datastore stub with this policy.
        self.testbed.init_datastore_v3_stub(consistency_policy=self.policy)
        # Initialize memcache stub too, since ndb also uses memcache
        self.testbed.init_memcache_stub()
        # Clear in-context cache before each test.
        ndb.get_context().clear_cache()

    def tearDown(self):
        self.testbed.deactivate()

    def testEventuallyConsistentGlobalQueryResult(self):
        class TestModel(ndb.Model):
            pass

        user_key = ndb.Key('User', 'ryan')

        # Put two entities
        ndb.put_multi([
            TestModel(parent=user_key),
            TestModel(parent=user_key)
        ])

        # Global query doesn't see the data.
        self.assertEqual(0, TestModel.query().count(3))
        # Ancestor query does see the data.
        self.assertEqual(2, TestModel.query(ancestor=user_key).count(3))

The PseudoRandomHRConsistencyPolicy class lets you control the likelihood of a write applying before each global (non-ancestor) query. By setting the probability to 0%, we are instructing the datastore stub to operate with the maximum amount of eventual consistency. Maximum eventual consistency means writes will commit but always fail to apply, so global (non-ancestor) queries will consistently fail to see changes. This is of course not representative of the amount of eventual consistency your application will see when running in production, but for testing purposes, it's very useful to be able to configure the local datastore to behave this way every time. If you use a non-zero probability, PseudoRandomHRConsistencyPolicy makes a deterministic sequence of consistency decisions so test outcomes are consistent:

def testDeterministicOutcome(self):
    # 50% chance to apply.
    self.policy.SetProbability(.5)
    # Use the pseudo random sequence derived from seed=2.
    self.policy.SetSeed(2)

    class TestModel(ndb.Model):
        pass

    TestModel().put()

    self.assertEqual(0, TestModel.query().count(3))
    self.assertEqual(0, TestModel.query().count(3))
    # Will always be applied before the third query.
    self.assertEqual(1, TestModel.query().count(3))

The testing APIs are useful for verifying that your application behaves properly in the face of eventual consistency, but please keep in mind that the local High Replication read consistency model is an approximation of the production High Replication read consistency model, not an exact replica. In the local environment, performing a get() of an Entity that belongs to an entity group with an unapplied write will always make the results of the unapplied write visible to subsequent global queries. In production this is not the case.

Writing mail tests

You can use the mail service stub to test the mail service. Similar to other services supported by testbed, at first you initialize the stub, then invoke the code which uses the mail API, and finally test whether the correct messages were sent.

import unittest

from google.appengine.api import mail
from google.appengine.ext import testbed


class MailTestCase(unittest.TestCase):

    def setUp(self):
        self.testbed = testbed.Testbed()
        self.testbed.activate()
        self.testbed.init_mail_stub()
        self.mail_stub = self.testbed.get_stub(testbed.MAIL_SERVICE_NAME)

    def tearDown(self):
        self.testbed.deactivate()

    def testMailSent(self):
        mail.send_mail(to='alice@example.com',
                       subject='This is a test',
                       sender='bob@example.com',
                       body='This is a test e-mail')
        messages = self.mail_stub.get_sent_messages(to='alice@example.com')
        self.assertEqual(1, len(messages))
        self.assertEqual('alice@example.com', messages[0].to)

Writing task queue tests

You can use the taskqueue stub to write tests that use the taskqueue service. Similar to other services supported by testbed, at first you initialize the stub, then invoke the code which uses the taskqueue API, and finally test whether the tasks were properly added to the queue.

import operator
import os
import unittest

from google.appengine.api import taskqueue
from google.appengine.ext import deferred
from google.appengine.ext import testbed


class TaskQueueTestCase(unittest.TestCase):
    def setUp(self):
        self.testbed = testbed.Testbed()
        self.testbed.activate()

        # root_path must be set the the location of queue.yaml.
        # Otherwise, only the 'default' queue will be available.
        self.testbed.init_taskqueue_stub(
            root_path=os.path.join(os.path.dirname(__file__), 'resources'))
        self.taskqueue_stub = self.testbed.get_stub(
            testbed.TASKQUEUE_SERVICE_NAME)

    def tearDown(self):
        self.testbed.deactivate()

    def testTaskAddedToQueue(self):
        taskqueue.Task(name='my_task', url='/url/of/my/task/').add()
        tasks = self.taskqueue_stub.get_filtered_tasks()
        self.assertEqual(len(tasks), 1)
        self.assertEqual(tasks[0].name, 'my_task')

Setting the queue.yaml configuration file

If you want to run tests on code that interacts a non-default queue, you will need to create and specify a queue.yaml file for your application to use. Below is an example queue.yaml:

For more information on the queue.yaml options available, see task queue configuration.

queue:
- name: default
  rate: 5/s
- name: queue-1
  rate: 5/s
- name: queue-2
  rate: 5/s

The location of the queue.yaml is specified when initializing the stub:

self.testbed.init_taskqueue_stub(root_path='.')

In the sample, queue.yaml is in the same directory as the tests. If it were in another folder, that path would need to be specified in root_path.

Filtering tasks

The taskqueue stub's get_filtered_tasks allows you to filter queued tasks. This makes it easier to write tests that need to verify code that enqueues multiple tasks.

def testFiltering(self):
    taskqueue.Task(name='task_one', url='/url/of/task/1/').add('queue-1')
    taskqueue.Task(name='task_two', url='/url/of/task/2/').add('queue-2')

    # All tasks
    tasks = self.taskqueue_stub.get_filtered_tasks()
    self.assertEqual(len(tasks), 2)

    # Filter by name
    tasks = self.taskqueue_stub.get_filtered_tasks(name='task_one')
    self.assertEqual(len(tasks), 1)
    self.assertEqual(tasks[0].name, 'task_one')

    # Filter by URL
    tasks = self.taskqueue_stub.get_filtered_tasks(url='/url/of/task/1/')
    self.assertEqual(len(tasks), 1)
    self.assertEqual(tasks[0].name, 'task_one')

    # Filter by queue
    tasks = self.taskqueue_stub.get_filtered_tasks(queue_names='queue-1')
    self.assertEqual(len(tasks), 1)
    self.assertEqual(tasks[0].name, 'task_one')

    # Multiple queues
    tasks = self.taskqueue_stub.get_filtered_tasks(
        queue_names=['queue-1', 'queue-2'])
    self.assertEqual(len(tasks), 2)

Writing deferred task tests

If your application code uses the deferred library, you can use the taskqueue stub along with deferred to verify that deferred functions are queued and executed correctly.

def testTaskAddedByDeferred(self):
    deferred.defer(operator.add, 1, 2)

    tasks = self.taskqueue_stub.get_filtered_tasks()
    self.assertEqual(len(tasks), 1)

    result = deferred.run(tasks[0].payload)
    self.assertEqual(result, 3)

Changing the default environment variables

App Engine services often depend on environment variables. The activate() method of class testbed.Testbed uses default values for these, but you can set custom values based on your testing needs with the setup_env method of class testbed.Testbed.

For example, let's say you have a test that stores several entities in datastore, all of them linked to the same application ID. Now you want to run the same tests again, but using an application ID different from the one that is linked to the stored entities. To do this, pass the new value into self.setup_env() as app_id.

For example:

import os
import unittest

from google.appengine.ext import testbed


class EnvVarsTestCase(unittest.TestCase):
    def setUp(self):
        self.testbed = testbed.Testbed()
        self.testbed.activate()
        self.testbed.setup_env(
            app_id='your-app-id',
            my_config_setting='example',
            overwrite=True)

    def tearDown(self):
        self.testbed.deactivate()

    def testEnvVars(self):
        self.assertEqual(os.environ['APPLICATION_ID'], 'your-app-id')
        self.assertEqual(os.environ['MY_CONFIG_SETTING'], 'example')

Simulating login

Another frequent use for setup_env is to simulate a user being logged in, either with or without admin privileges, to check if your handlers operate properly in each case.

import unittest

from google.appengine.api import users
from google.appengine.ext import testbed


class LoginTestCase(unittest.TestCase):
    def setUp(self):
        self.testbed = testbed.Testbed()
        self.testbed.activate()
        self.testbed.init_user_stub()

    def tearDown(self):
        self.testbed.deactivate()

    def loginUser(self, email='user@example.com', id='123', is_admin=False):
        self.testbed.setup_env(
            user_email=email,
            user_id=id,
            user_is_admin='1' if is_admin else '0',
            overwrite=True)

    def testLogin(self):
        self.assertFalse(users.get_current_user())
        self.loginUser()
        self.assertEquals(users.get_current_user().email(), 'user@example.com')
        self.loginUser(is_admin=True)
        self.assertTrue(users.is_current_user_admin())

Now, your test methods can call, for example, self.loginUser('', '') to simulate no user being logged in, self.loginUser('test@example.com', '123') to simulate a non-admin user being logged in, self.loginUser('test@example.com', '123', is_admin=True) to simulate an admin user being logged in.

Setting up a testing framework

The SDK's testing utilities are not tied to a specific framework. You can run your unit tests with any available App Engine testrunner, for example nose- gae or ferrisnose. You can also write a simple testrunner of your own, or use the one shown below.

The following scripts use Python's unittest module.

You can name the script anything you want. When you run it, provide the path to your Google Cloud CLI or Google App Engine SDK installation and the path to your test modules. The script will discover all tests in the path provided and will print results to the standard error stream. Test files follow the convention of having test prefixed to their name.

"""App Engine local test runner example.

This program handles properly importing the App Engine SDK so that test modules
can use google.appengine.* APIs and the Google App Engine testbed.

Example invocation:

    $ python runner.py ~/google-cloud-sdk
"""

import argparse
import os
import sys
import unittest


def fixup_paths(path):
    """Adds GAE SDK path to system path and appends it to the google path
    if that already exists."""
    # Not all Google packages are inside namespace packages, which means
    # there might be another non-namespace package named `google` already on
    # the path and simply appending the App Engine SDK to the path will not
    # work since the other package will get discovered and used first.
    # This emulates namespace packages by first searching if a `google` package
    # exists by importing it, and if so appending to its module search path.
    try:
        import google
        google.__path__.append("{0}/google".format(path))
    except ImportError:
        pass

    sys.path.insert(0, path)


def main(sdk_path, test_path, test_pattern):
    # If the SDK path points to a Google Cloud SDK installation
    # then we should alter it to point to the GAE platform location.
    if os.path.exists(os.path.join(sdk_path, 'platform/google_appengine')):
        sdk_path = os.path.join(sdk_path, 'platform/google_appengine')

    # Make sure google.appengine.* modules are importable.
    fixup_paths(sdk_path)

    # Make sure all bundled third-party packages are available.
    import dev_appserver
    dev_appserver.fix_sys_path()

    # Loading appengine_config from the current project ensures that any
    # changes to configuration there are available to all tests (e.g.
    # sys.path modifications, namespaces, etc.)
    try:
        import appengine_config
        (appengine_config)
    except ImportError:
        print('Note: unable to import appengine_config.')

    # Discover and run tests.
    suite = unittest.loader.TestLoader().discover(test_path, test_pattern)
    return unittest.TextTestRunner(verbosity=2).run(suite)


if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description=__doc__,
        formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument(
        'sdk_path',
        help='The path to the Google App Engine SDK or the Google Cloud SDK.')
    parser.add_argument(
        '--test-path',
        help='The path to look for tests, defaults to the current directory.',
        default=os.getcwd())
    parser.add_argument(
        '--test-pattern',
        help='The file pattern for test modules, defaults to *_test.py.',
        default='*_test.py')

    args = parser.parse_args()

    result = main(args.sdk_path, args.test_path, args.test_pattern)

    if not result.wasSuccessful():
        sys.exit(1)

Running the tests

You can run these tests simply by running the script runner.py, which is described in detail under Setting up a testing framework:

python runner.py <path-to-appengine-or-gcloud-SDK> .