With App Engine, you can build web applications using the Python programming language, and take advantage of the many libraries, tools and frameworks for Python that professional developers use to build world- class web applications. Your Python application runs on Google's scalable infrastructure, and uses large-scale persistent storage and services.
- Selecting the Python runtime
- The sandbox
- Pure Python
- Concurrency and latency
- The Python SDK and tools
App Engine executes your Python application code using a pre-loaded Python interpreter in a safe "sandboxed" environment. Your app receives web requests, performs work, and sends responses by interacting with this environment.
A Python web app interacts with the App Engine web server using the WSGI protocol, so apps can use any WSGI-compatible web application framework. App Engine includes a simple web application framework, called webapp2, to make it easy to get started. For larger applications, mature third-party frameworks, such as Django, work well with App Engine.
The Python interpreter can run any Python code, including Python modules you include with your application, as well as the Python standard library. The interpreter cannot load Python modules with C code; it is a "pure" Python environment.
The secured "sandbox" environment isolates your application for service and security. It ensures that apps can only perform actions that do not interfere with the performance and scalability of other apps. For instance, an app cannot write data to the local file system or make arbitrary network connections. Instead, apps use scalable services provided by App Engine to store data and communicate over the Internet. The Python interpreter raises an exception when an app attempts to import a module from the standard library known to not work within the sandbox restrictions.
If you haven't already, see the Python 2.7 Getting Started Guide for an interactive introduction to developing web applications with Python and Google App Engine.
Selecting the Python runtime
App Engine knows to use the Python runtime environment for your application
code when you use the tool named
appcfg.py from the Python SDK with a
configuration file named
You specify the
runtime element in
app.yaml. To use Python 2.7, add the
runtime: python27 api_version: 1 threadsafe: true ...
The first element,
runtime, selects the Python runtime environment.
The second element,
api_version, selects which version of the Python runtime
environment to use. As of this writing, App Engine only has one version of the
1. If the App Engine team ever needs to release changes
to the environment that may not be compatible with existing code, they will do
so with a new version identifier. Your app will continue to use the selected
version until you change the
api_version setting and upload your app.
To allow App Engine to distribute requests for applications across multiple web servers, and to prevent one application from interfering with another, the application runs in a restricted "sandbox" environment. In this environment, the application can execute code, store and query data in the App Engine datastore, use the App Engine mail, URL fetch and users services, and examine the user's web request and prepare the response.
An App Engine application cannot:
write to the filesystem. Applications must use the App Engine datastore for storing persistent data. Reading from the filesystem is allowed, and all application files uploaded with the application are available.
respond slowly. A web request to an application must be handled within a few seconds. Processes that take a very long time to respond are terminated to avoid overloading the web server.
make other kinds of system calls.
Sandboxing in Python
The Python 2.7 runtime does not restrict access to Python bytecode. Libraries that generate or manipulate bytecode (e.g. the jinja2 templating library) can do so in this runtime. The Python 2.5 runtime does not allow bytecode manipulation.
You can upload and use
.pyc files when using the Python 2.7 runtime, but you
cannot upload a
.py and a
.pyc version of the same file. You can upload zip
.pyc files (or a combination). A number of
important caveats apply if you upload
- For a CGI script, the script handler should still use
.pyfile extension, even if you upload a
- By default,
.pycfiles are skipped during deployment by
appcfg. You must override the
app.yamlso that the new value does not cause
.pycfiles to be skipped.
- You must use Python 2.7 to build the
.pycfile. If you have a different version of Python (such as Python 2.6) on your development machine, you will need to obtain version 2.7 to build a compatible
All code for the Python runtime environment must be pure Python, and not include any C extensions or other code that must be compiled.
The environment includes the Python standard library. Some modules have been disabled because their core functions
are not supported by App Engine, such as networking or writing to the
filesystem. In addition, the
os module is available, but with unsupported
features disabled. An attempt to import an unsupported module or use an
unsupported feature will raise an exception.
A few modules from the standard library have been replaced or customized to work with App Engine. These modules vary between the two Python runtimes, as described below.
Customized libraries in Python 2.7
In the Python 2.7 runtime, the following modules have been replaced or customized:
In addition to the Python standard library and the App Engine libraries, the Python 2.7 runtime includes several third-party libraries.
Adding Third Party Python Libraries
You can include third party Python libraries with your application by putting the code in your application directory. If you make a symbolic link to a module's directory in your application directory, appcfg.py will follow the link and include the module in your app.
The Python module include path includes your application's root directory (the
directory containing the
app.yaml file). Modules you create in your
application's root directory are available using a path from the root. Don't
forget to create
__init__.py files in sub-directories, so Python will
recognize the sub-directories as packages.
ThreadsThreads can be created in Python 2.7 using the
threadingmodules. Note that threads will be joined by the runtime when the request ends so the threads cannot run past the end of the request.
Code running on a manual scaling instance can start a background thread that can outlive the request that spawns it. This allows instances to perform arbitrary periodic or scheduled tasks or to continue working in the background after a request has returned to the user.
A background thread's
os.environ and logging entries are independent of those
of the spawning thread.
You must import the
google.appengine.api.background_thread module from the SDK.
from google.appengine.api import background_thread
class is like the regular Python
threading.Threadclass, but can "outlive" the request that spawns it.
There is also function
start_new_background_thread() which creates a background
thread and starts it:
# sample function to run in a background thread def change_val(arg): global val val = arg if auto: # Start the new thread in one command background_thread.start_new_background_thread(change_val, ['Cat']) else: # create a new thread and start it t = background_thread.BackgroundThread( target=change_val, args=['Cat']) t.start()
The Python SDK and tools
The App Engine Python SDK includes tools for testing your application, uploading your application files, managing datastore indexes, downloading log data, and uploading large amounts of data to the datastore.
The development server runs your application on your local computer for testing your application. The server simulates the App Engine datastore, services and sandbox restrictions. The development server can also generate configuration for datastore indexes based on the queries the app performs during testing.
A multipurpose tool called
appcfg.py handles all command-
line interaction with your application running on App Engine.
upload your application to App Engine, or just update the datastore index
configuration so you can build new indexes before updating the code. It can
also download the app's log data, so you can analyze your app's performance
using your own tools.
Concurrency and latency
Your application's latency has the biggest impact on the number of instances needed to serve your traffic. If you process requests quickly, a single instance can handle a lot of requests.Single-threaded instances can currently handle one concurrent request. Therefore, there is a direct relationship between the latency and number of requests that can be handled on the instance per second. For example, 10ms latency equals 100 request/second/instance.
Multi-threaded instances can handle many concurrent requests. Therefore, there is a direct relationship between the CPU consumed and the number of requests/second.Python 2.7 apps support concurrent requests, so a single instance can handle new requests while waiting for other requests to complete. Concurrency significantly reduces the number of instances your app requires, but you need to design your app specifically with multithreading in mind.
For example, if a B4 instance (approx 2.4GHz) consumes 10 Mcycles/request, you can process 240 requests/second/instance. If it consumes 100 Mcycles/request, you can process 24 requests/second/instance. These numbers are the ideal case but are fairly realistic in terms of what you can accomplish on an instance.