Python Runtime Environment

With App Engine, you can build web applications using the Python programming language, and take advantage of the many libraries, tools and frameworks for Python that professional developers use to build world- class web applications. Your Python application runs on Google's scalable infrastructure, and uses large-scale persistent storage and services.

Introduction

App Engine executes your Python application code using a pre-loaded Python interpreter in a safe "sandboxed" environment. Your app receives web requests, performs work, and sends responses by interacting with this environment.

A Python web app interacts with the App Engine web server using the WSGI protocol, so apps can use any WSGI-compatible web application framework. App Engine includes a simple web application framework, called webapp2, to make it easy to get started. For larger applications, mature third-party frameworks, such as Django, work well with App Engine.

The Python interpreter can run any Python code, including Python modules you include with your application, as well as the Python standard library. The interpreter cannot load Python modules with C code; it is a "pure" Python environment.

The secured "sandbox" environment isolates your application for service and security. It ensures that apps can only perform actions that do not interfere with the performance and scalability of other apps. For instance, an app cannot write data to the local file system or make arbitrary network connections. Instead, apps use scalable services provided by App Engine to store data and communicate over the Internet. The Python interpreter raises an exception when an app attempts to import a module from the standard library known to not work within the sandbox restrictions.

The App Engine platform provides many services that your code can call. Your application can also configure scheduled tasks that run at specified intervals.

If you haven't already, see the Python 2.7 Getting Started Guide for an interactive introduction to developing web applications with Python and Google App Engine.

Selecting the Python runtime

You specify the Python runtime environment in the app.yaml configuration file, which is used to deploy your application to App Engine. For example, you add the following to the app.yaml file to use Python 2.7:

runtime: python27
api_version: 1
threadsafe: true
...

The first element, runtime, selects the Python runtime environment.

The second element, api_version, selects which version of the Python runtime environment to use. As of this writing, App Engine only has one version of the Python environment, 1. If the App Engine team ever needs to release changes to the environment that may not be compatible with existing code, they will do so with a new version identifier. Your app will continue to use the selected version until you change the api_version setting and upload your app.

For more information about the app.yaml file and how to deploy your app to App Engine, see the app.yaml Reference, Migrating to Python 2.7, and Deploying a Python App topics.

The sandbox

To allow App Engine to distribute requests for applications across multiple web servers, and to prevent one application from interfering with another, the application runs in a restricted "sandbox" environment. In this environment, the application can execute code, store and query data in the App Engine datastore, use the App Engine mail, URL fetch and users services, and examine the user's web request and prepare the response.

An App Engine application cannot:

  • write to the filesystem. Applications must use the App Engine datastore for storing persistent data. Reading from the filesystem is allowed, and all application files uploaded with the application are available.

  • respond slowly. A web request to an application must be handled within a few seconds. Processes that take a very long time to respond are terminated to avoid overloading the web server.

  • make other kinds of system calls.

Sandboxing in Python

The Python 2.7 runtime does not restrict access to Python bytecode. Libraries that generate or manipulate bytecode (e.g. the jinja2 templating library) can do so in this runtime. The Python 2.5 runtime does not allow bytecode manipulation.

You can upload and use .pyc files when using the Python 2.7 runtime, but you cannot upload a .py and a .pyc version of the same file. You can upload zip files containing .py or .pyc files (or a combination). A number of important caveats apply if you upload .pyc files:

  • For a CGI script, the script handler should still use the .py file extension, even if you upload a .pyc file.
  • By default, .pyc files are skipped during deployment by appcfg. You must override the skip_files element of app.yaml so that the new value does not cause .pyc files to be skipped.
  • You must use Python 2.7 to build the .pyc file. If you have a different version of Python (such as Python 2.6) on your development machine, you will need to obtain version 2.7 to build a compatible .pyc file.

Pure Python

All code for the Python runtime environment must be pure Python, and not include any C extensions or other code that must be compiled.

The environment includes the Python standard library. Some modules have been disabled because their core functions are not supported by App Engine, such as networking or writing to the filesystem. In addition, the os module is available, but with unsupported features disabled. An attempt to import an unsupported module or use an unsupported feature will raise an exception.

A few modules from the standard library have been replaced or customized to work with App Engine. These modules vary between the two Python runtimes, as described below.

Customized libraries in Python 2.7

In the Python 2.7 runtime, the following modules have been replaced or customized:

In addition to the Python standard library and the App Engine libraries, the Python 2.7 runtime includes several third-party libraries.

Adding Third Party Python Libraries

You can include third party Python libraries with your application by putting the code in your application directory. If you make a symbolic link to a module's directory in your application directory, appcfg.py will follow the link and include the module in your app.

The Python module include path includes your application's root directory (the directory containing the app.yaml file). Modules you create in your application's root directory are available using a path from the root. Don't forget to create __init__.py files in sub-directories, so Python will recognize the sub-directories as packages.

Threads

Threads can be created in Python 2.7 using the thread or threading modules. Note that threads will be joined by the runtime when the request ends so the threads cannot run past the end of the request.

Background threads

Code running on manual or basic scaling instances can start a background thread that can outlive the request that spawns it. This allows instances to perform arbitrary periodic or scheduled tasks or to continue working in the background after a request has returned to the user.

A background thread's os.environ and logging entries are independent of those of the spawning thread.

You must import the google.appengine.api.background_thread module from the SDK.

from google.appengine.api import background_thread

The BackgroundThread class is like the regular Python threading.Threadclass, but can "outlive" the request that spawns it. There is also function start_new_background_thread() which creates a background thread and starts it:

# sample function to run in a background thread
def change_val(arg):
    global val
    val = arg

if auto:
    # Start the new thread in one command
    background_thread.start_new_background_thread(change_val, ['Cat'])
else:
    # create a new thread and start it
    t = background_thread.BackgroundThread(
        target=change_val, args=['Cat'])
    t.start()
The maximum number of concurrent background threads is 10 per instance.

The Python SDK and tools

The App Engine Python SDK includes tools for testing your application, uploading your application files, managing datastore indexes, downloading log data, and uploading large amounts of data to the datastore.

The development server runs your application on your local computer for testing your application. The server simulates the App Engine datastore, services and sandbox restrictions. The development server can also generate configuration for datastore indexes based on the queries the app performs during testing.

A multipurpose tool called appcfg.py handles all command- line interaction with your application running on App Engine. appcfg.py can upload your application to App Engine, or just update the datastore index configuration so you can build new indexes before updating the code. It can also download the app's log data, so you can analyze your app's performance using your own tools.

Concurrency and latency

Your application's latency has the biggest impact on the number of instances needed to serve your traffic. If you process requests quickly, a single instance can handle a lot of requests.

Single-threaded instances can currently handle one concurrent request. Therefore, there is a direct relationship between the latency and number of requests that can be handled on the instance per second. For example, 10ms latency equals 100 request/second/instance.

Multi-threaded instances can handle many concurrent requests. Therefore, there is a direct relationship between the CPU consumed and the number of requests/second.

Python 2.7 apps support concurrent requests, so a single instance can handle new requests while waiting for other requests to complete. Concurrency significantly reduces the number of instances your app requires, but you need to design your app specifically with multithreading in mind.

For example, if a B4 instance (approx 2.4GHz) consumes 10 Mcycles/request, you can process 240 requests/second/instance. If it consumes 100 Mcycles/request, you can process 24 requests/second/instance. These numbers are the ideal case but are fairly realistic in terms of what you can accomplish on an instance.

Enviar comentarios sobre...

App Engine standard environment for Python