This page covers the measures necessary to upgrade your Python application to the Python 2.7 runtime. After following this walkthrough, your application can take advantage of the Python 2.7 runtime's many new capabilities, including multithreading, the Jinja2 templating engine, bytecode access and uploading, and several new included third-party libraries.
Prerequisites and Considerations
In order to use Python 2.7, an application must meet the following requirements:
- If the application uses Django, it must use version 1.2 or later. See the Django documentation for upgrade details.
- If you wish to use concurrent requests, the application must use Web Server Gateway Interface (WSGI) script handlers, as covered in Using WSGI.
In addition to meeting these general prerequisites, you must use specific versions of some App Engine features and third-party libraries. Be sure to update the versions that you include and import into your application, and test the application extensively after upgrading. The following list identifies the primary compatibility issues and points to further resources to resolve them:
- Performance: Application performance may change after upgrading to Python 2.7. If you experience increased response latency, you can increase the frontend instance class and enable concurrent requests. Concurrent requests allow your application to perform faster with a lower instance cost on larger instances. For more information, see Request Handling in Instances.
- Django: You must use Django 1.2 or later with Python 2.7. For information on upgrading, see the Django Release Notes.
- PyCrypto:
Crypto.Util.randpool
has been deprecated in favor ofCrypto.Random
. For more information, see What to do about RandomPool. webapp
: webapp templates are deprecated in Python 2.7. Instead, you can use Django templates directly, jinja2, or a different template engine of your choice.- WebOb: Python 2.7 supports WebOb version 1.1. This version is not fully backward compatible with the old version (0.9). If the application uses WebOb, you need to test it extensively to catch any errors resulting from the upgrade.
- zipimport: Python 2.7 does not support zipimport, but Python 2.7 can natively import from .zip files.
- simplejson: Python 2.7 does not support simplejson, but Python 2.7 includes the equivalent, and much faster, standard library json module.
Concurrent Requests and WSGI
The Python 2.7 feature that most affects your application's design and performance is its support for multithreaded applications that can handle concurrent requests. The ability to handle concurrent requests results in improved utilization that can significantly enhance your application's performance, especially for applications utilizing higher instance classes that exploit multiple CPU cores.
In order to enable multithreading, applications must move from the Common Gateway Interface (CGI)-based approach of the previous Python runtimes to a Web Server Gateway Interface (WSGI)-based approach. This is because CGI scripts, which were designed to handle requests serially, rely on environment variables for access to the input and output streams.
While the WSGI model for handling requests gives applications more direct access to the input and output streams (enabling concurring requests), handling multiple requests in parallel can cause race conditions when a request handler's logic relies on or interacts with data with a greater-than-local scope, such as application state. For this reason, it is important to code defensively to deal with race conditions in order to ensure your new WSGI application is threadsafe.
See Making Your Application Threadsafe for more details.
Updating app.yaml
Python 2.7 requires a special
runtime
configuration element in the header of app.yaml
.
Note that the threadsafe: [ true | false ]
element is required for Python 2.7
applications. If true
, App Engine sends requests concurrently; if false
, App Engine sends them serially. The following
app.yaml
header enables concurrent requests:
application: myapp version: 1 runtime: python27 api_version: 1 threadsafe: true ...
Using WSGI
The Python 2.7 runtime optionally allows you to directly run a Web Server
Gateway Interface (WSGI) application, instead of using the
run_wsgi_app
adapter to run the program as a CGI script.
To do this, replace the CGI handler (e.g. myapp.py
) in
app.yaml
with a WSGI
application name (e.g. myapp.app
).
... handlers: - url: /.* script: myapp.app ...
You also need to move your WSGI application object to the global scope:
import webapp2 class MainPage(webapp2.RequestHandler): def get(self): self.response.headers['Content-Type'] = 'text/plain' self.response.out.write('Hello, WebApp World!') app = webapp2.WSGIApplication([('/', MainPage)]) """ Old code: def main(): run_wsgi_app(app) if __name__ == '__main__': main() """
You can still specify CGI script handlers in
app.yaml
; however, requests handled by CGI scripts are processed
serially, not concurrently. Also, you can not have an
app.yaml
file that mixes CGI scripts and WSGI applications, and you can not set threadsafe
to true
if you define any CGI handlers.
Some conventions from previous Python runtimes, such as the use of
main()
and the check for
__name__ == 'main'
,
are now deprecated. These measures helped the CGI scripts of the past
stay cached, but now that you're executing WSGI applications directly, these
steps are no longer necessary.
Using the App Root Directory
In Python 2.5, CGI scripts ran with the current working directory set to the directory that contained the script. This has changed in Python 2.7. With WSGI, the current working directory at the beginning of the request handler's lifetime is the application root directory.
Review your application code, and make sure that all handlers are written to expect the current working directory to be the application root.
Configuring Libraries
The Python 2.7 runtime includes some third-party modules. Some of these are available by default; others are only available if configured. You can specify which version you want to use.
libraries: - name: PIL version: "1.1.7" - name: webob version: "1.1.1"
You can specify that the application should use the latest version of the module. This is useful if you're developing an application that doesn't have users yet: you don't need to track new versions. But if your application is being actively used, beware: you might be surprised that your application starts using a new not-backward-compatible library version. To use the latest version:
libraries: - name: PIL version: latest
For a list of supported libraries, please see Third-party Libraries.
Making Your Application Threadsafe
Handling concurrent requests is simple if each handler only interacts with variables within its scope. But it gets tricky quickly if one handler modifies resources as another is reading them. Making sure that your application behaves as expected—even though multiple requests might be manipulating the same data and interfering with one another—is known as making your application "threadsafe."
The primary rule when designing an application to be threadsafe is to limit use of shared resources (such as state information or global variables) as often as possible. However, it's usually not possible to completely rule out their use, and that's where synchronization mechanisms like lock objects come in.
In Python 2.7, you have access to Python's threading library, which allows you to declare a lock on a block of logic that forces the code inside to be executed serially instead of concurrently. Consider the following code:
class Configuration(ndb.Model): some_config_data = ndb.StringProperty() _config_cache = None _config_lock = threading.Lock() @classmethod def get_config(cls): with cls._config_lock: if not cls._config_cache: cls._config_cache = cls.get_by_id('config') return cls._config_cache
This code shows the creation of a cache of
some global configuration variables into a variable called _config_cache
. Here, the use of a lock object named _config_lock
ensures that the check for a pre-existing _config_cache
behaves reliably. Otherwise, this variable could waste time making
multiple trips to the Datastore to set the same variable multiple
times with the same data, because the competing requests all found
that _config_cache
was empty.
Do not choose lightly to use locks. A lock forces any other threads executing this method to block. This can be a bottleneck to performance.
Updating Your Application to webapp2
Note: if you are not using webapp
as your request
handler, you can skip this section.
The web framework included with the Python 2.7 runtime has been
upgraded from webapp
to webapp2. Among other
things, webapp2
adds improved URI routing and exception handling, a
full-featured response object, and a more flexible dispatching
mechanism.
webapp
templates are
now deprecated. In their place, you can use Jinja2, Django,
or a templating system of your choice (as long as it's written in pure
Python).
In App Engine, webapp2
has been aliased to webapp
, and
webapp2
is backward compatible. However, after upgrading, you still need to
thoroughly test your application and familiarize yourself with the
new syntax and capabilities of webapp2
,
rather than continue to rely on backwards compatibility.
And your upgrade is complete!
Once you've uploaded the application, you need to test it extensively to ensure backward compatibility. If you are experiencing issues, check out the forums.