Python 2 Runtime Environment

With App Engine, you can build web applications using the Python programming language, and take advantage of the many libraries, tools and frameworks for Python that professional developers use to build world- class web applications. Your Python application runs on Google's scalable infrastructure, and uses large-scale persistent storage and services.

Introduction

App Engine executes your Python application code using a pre-loaded Python interpreter in a safe "sandboxed" environment. Your app receives web requests, performs work, and sends responses by interacting with this environment.

A Python web app interacts with the App Engine web server using the WSGI protocol, so apps can use any WSGI-compatible web application framework. App Engine includes a simple web application framework, called webapp2, to make it easy to get started. For larger applications, mature third-party frameworks, such as Django, work well with App Engine.

The Python interpreter can run any Python code, including Python modules you include with your application, as well as the Python standard library. The interpreter cannot load Python modules with C code; it is a "pure" Python environment.

The secured "sandbox" environment isolates your application for service and security. It ensures that apps can only perform actions that do not interfere with the performance and scalability of other apps. For instance, an app cannot write data to the local file system or make arbitrary network connections. Instead, apps use scalable services provided by App Engine to store data and communicate over the Internet. The Python interpreter raises an exception when an app attempts to import a Python module from the standard library known to not work within the sandbox restrictions.

The App Engine platform provides many services that your code can call. Your application can also configure scheduled tasks that run at specified intervals.

Selecting the Python 2 runtime

You specify the Python runtime environment in the app.yaml configuration file, which is used to deploy your application to App Engine. For example, you add the following to the app.yaml file to use Python version 2.7:

runtime: python27
api_version: 1
threadsafe: true
...

The first element, runtime, selects the Python runtime environment.

The second element, api_version, selects which version of the Python runtime environment to use. As of this writing, App Engine only has one version of the Python environment, 1. If the App Engine team ever needs to release changes to the environment that may not be compatible with existing code, they will do so with a new version identifier. Your app will continue to use the selected version until you change the api_version setting and upload your app.

For more information about the app.yaml file and how to deploy your app to App Engine, see the app.yaml Reference, Migrating to Python 2.7, and Deploying a Python App topics.

The sandbox

To allow App Engine to distribute requests for applications across multiple web servers, and to prevent one application from interfering with another, the application runs in a restricted "sandbox" environment. In this environment, the application can execute code, store and query the data in Datastore, use the App Engine mail, URL fetch and users services, and examine the user's web request and prepare the response.

An App Engine application cannot:

• write to the filesystem. Applications must use Datastore for storing persistent data. Reading from the filesystem is allowed, and all application files uploaded with the application are available.

• respond slowly. A web request to an application must be handled within a few seconds. Processes that take a very long time to respond are terminated to avoid overloading the web server.

• make other kinds of system calls.

Sandboxing in Python

You can upload and use .pyc files when using the Python 2.7 runtime, but you cannot upload a .py and a .pyc version of the same file. You can upload .zip files containing .py or .pyc files (or a combination). A number of important caveats apply if you upload .pyc files:

• For a CGI script, the script handler should still use the .py file extension, even if you upload a .pyc file.
• By default, .pyc files are skipped during deployment. You must override the skip_files element in your app.yaml file so that the new value does not cause .pyc files to be skipped.
• You must use Python 2.7 to build the .pyc file. If you have a different version of Python (such as Python 2.6) on your development machine, you will need to obtain version 2.7 to build a compatible .pyc file.

Pure Python 2

All code for the Python runtime environment must be pure Python, and not include any C extensions or other code that must be compiled.

The environment includes the Python standard library. Some modules have been disabled because their core functions are not supported by App Engine, such as networking or writing to the filesystem. In addition, the os module is available, but with unsupported features disabled. An attempt to import an unsupported module or use an unsupported feature will raise an exception.

A few modules from the standard library have been replaced or customized to work with App Engine. These modules vary between the two Python runtimes, as described below.

Customized libraries in Python version 2.7

In the Python version 2.7 runtime, the following modules have been replaced or customized:

• tempfile is disabled, except for TemporaryFile which is aliased to StringIO.

• logging is available and its use is highly encouraged! See Logging.

In addition to the Python standard library and the App Engine libraries, the Python version 2.7 runtime includes several third-party libraries.

Adding Third Party Python Libraries

You can include third party Python libraries with your application by putting the code in your application directory. If you make a symbolic link to a library's directory in your application directory, that link is followed and the library gets included in the app that you deploy to App Engine.

The include path of the Python module includes your application's root directory, which is the directory containing the app.yaml file. Python modules that you create in your application's root directory are available using a path from the root. Don't forget to create the required __init__.py files in your sub-directories so that Python recognizes those sub-directories as packages. Also ensure that your libraries do not need any C extensions.

Threads

Threads can be created in Python version 2.7 using the thread or threading modules. Note that threads will be joined by the runtime when the request ends so the threads cannot run past the end of the request.

Background threads

Code running on an instance with manual or basic scaling can start a background thread that can outlive the request that spawns it. This allows an instance to perform arbitrary periodic or scheduled tasks, or to continue working in the background after a request has returned to the user.

A background thread's os.environ and logging entries are independent of those of the spawning thread.

You must import the google.appengine.api.background_thread module from the SDK for App Engine.

from google.appengine.api import background_thread

The BackgroundThread class is like the regular Python threading.Threadclass, but can "outlive" the request that spawns it. There is also function start_new_background_thread() which creates a background thread and starts it:

# sample function to run in a background thread
def change_val(arg):
    global val
    val = arg

if auto:
    # Start the new thread in one command
    background_thread.start_new_background_thread(change_val, ['Cat'])
else:
    # create a new thread and start it
    t = background_thread.BackgroundThread(
        target=change_val, args=['Cat'])
    t.start()

The maximum number of concurrent background threads created by the App Engine API is 10 per instance. (This limit doesn't apply to regular Java threads unrelated to the App Engine API.)

Tools

The SDK for App Engine includes tools for testing your application, uploading your application files, managing Datastore indexes, downloading log data, and uploading large amounts of data to the Datastore.

The development server runs your application on your local computer for testing your application. The server simulates the Datastore services and sandbox restrictions. The development server can also generate configuration for Datastore indexes based on the queries the app performs during testing.

The gcloud tool handles all command-line interaction with your application running on App Engine. You use gcloud app deploy to upload your application to App Engine, or to update individual configuration files like the Datastore index configuration, which allows you to build new indexes before deploying your code. You can also view your app's log data, so you can analyze your app's performance using your own tools.

Note: Python functions, methods, or classes that begin with _ or __ are private and can change without notice. Apps using these interface names might break in the future.

Concurrency and latency

Your application's latency has the biggest impact on the number of instances needed to serve your traffic. If you process requests quickly, a single instance can handle a lot of requests.

Single-threaded instances can handle one concurrent request. Therefore, there is a direct relationship between the latency and number of requests that can be handled on the instance per second. For example, 10ms latency equals 100 request/second/instance.

Multi-threaded instances can handle many concurrent requests. Therefore, there is a direct relationship between the CPU consumed and the number of requests/second.

Python version 2.7 apps support concurrent requests, so a single instance can handle new requests while waiting for other requests to complete. Concurrency significantly reduces the number of instances your app requires, but you need to design your app for multithreading.

For example, if a B4 instance (approx 2.4GHz) consumes 10 Mcycles/request, you can process 240 requests/second/instance. If it consumes 100 Mcycles/request, you can process 24 requests/second/instance. These numbers are the ideal case but are fairly realistic in terms of what you can accomplish on an instance.

Environment variables

The following environment variables are set by the runtime:

Environment variable	Description
GAE_APPLICATION	The ID of your App Engine application. This ID is prefixed with 'region code~' such as 'e~' for applications deployed in Europe.
GAE_DEPLOYMENT_ID	The ID of the current deployment.
GAE_ENV	The App Engine environment. Set to standard.
GAE_INSTANCE	The ID of the instance on which your service is currently running.
GAE_RUNTIME	The runtime specified in your app.yaml file.
GAE_SERVICE	The service name specified in your app.yaml file. If no service name is specified, it is set to default.
GAE_VERSION	The current version label of your service.
GOOGLE_CLOUD_PROJECT	The Google Cloud project ID associated with your application.
PORT	The port that receives HTTP requests.

You can define additional environment variables in your app.yaml file, but the above values cannot be overridden.