Migrating to the Python 3 runtime

Migrating to the Python 3 runtime allows you to use up-to-date language features and build apps that are more portable, with idiomatic code. The Python 3 runtime uses the latest version of the open source Python interpreter provided by the Python Software Foundation. Apps built in the Python 3 runtime can use Python's rich ecosystem of packages and frameworks in your app, including those that use C code, by declaring dependencies in a requirements.txt file.

Overview of the runtime migration process

We recommend the following incremental approach to the runtime migration, in which you maintain a functioning and testable application throughout the process:

Upgrade your app to be compatible with both Python 2 and Python 3.

Several solutions are available to help with this upgrade. For example, use Six, Python-Future, or Python-Modernize.

For more information about this step of the runtime migration process, see Porting Python 2 Code to Python 3 on the Python Software Foundation documentation site.
Pick one of these implementation strategies for any App Engine bundled service your app uses:
1. Migrate the App Engine bundled services in your Python 2 app to unbundled Google Cloud services, third-party services, or other recommended replacements.
2. Continue using legacy bundled services in your Python 3 apps. This approach gives you flexibility to move to unbundled services later in the migration cycle.
  
  Note: Not all the App Engine bundled services available for Python 2 have a corresponding service in Python 3, so those will need to migrated to either a Google Cloud or third-party service before porting to Python 3.
Make sure to test your app after migrating each service.
Prepare App Engine configuration files for the Python 3 runtime. Several important changes affect the configuration settings in app.yaml, including but not limited to:
- Apps are now assumed to be threadsafe. If your application isn't threadsafe, you should set max_concurrent_requests in your app.yaml to 1. This setting may result in more instances being created than you need for a threadsafe app, and lead to unnecessary cost.
- The app.yaml file no longer routes requests to your scripts. Instead, you are required to use a web framework with in-app routing, and update or remove all script handlers in app.yaml. For an example of how to do this with the Flask framework, see the App Engine migration guide code sample in GitHub.
  
  To learn more about changing this and other configuration files, see the Configuration files section.
Test and deploy your upgraded app in a Python 3 environment.

After all of your tests pass, deploy the upgraded app to App Engine but prevent traffic from automatically routing to the new version. Use traffic splitting to slowly migrate traffic from your app in the Python 2 runtime to the app in the Python 3 runtime. If you run into problems, you can route all traffic to a stable version until the problem is fixed.

For examples of how to convert your Python 2 apps to Python 3, you can refer to these additional resources.

Key differences between the Python 2 and Python 3 runtimes

Most of the changes you need to make during the runtime migration come from the following differences between the Python 2 and Python 3 runtimes:

Compatibility issues between Python 2 and Python 3
App Engine bundled services in the Python 3 runtime
Configuration files differences
A web framework is required to route requests for dynamic content
Testing differences
Deployment differences

Compatibility issues between Python 2 and Python 3

When Python 3 was first released in 2008, several backward incompatible changes were introduced to the language. Some of these changes require only minor updates to your code, such as changing the print statement to a print() function. Other changes may require significant updates to your code, such as updating the way that you handle binary data, text, and strings.

Many popular open source libraries, including the Python standard libraries, also changed when they moved from Python 2 to Python 3.

App Engine bundled services in the Python 3 runtime

To reduce migration effort and complexity, App Engine standard environment allows you to access many of legacy bundled services and APIs in the Python 3 runtime, such as Memcache. Your Python 3 app can call the bundled services APIs through language idiomatic libraries, and access the same functionality as on the Python 2 runtime.

You also have the option to use Google Cloud products that offer similar functionality as the legacy bundled services. We recommend that you consider migrating to the unbundled Google Cloud products, as doing so lets you take advantage of ongoing improvements and new features.

For the bundled services that are not available as separate products in Google Cloud, such as image processing, search, and messaging, you can use our suggested third-party providers or other workarounds.

Configuration files

Before you can run your app in the Python 3 runtime of the App Engine standard environment, you may need to change some of the configuration files that App Engine uses:

app.yaml. The behavior of some fields in your app.yaml configuration file has been modified. Remove any deprecated fields and update other fields as described in the migration guide.
requirements.txt. Create this file to install third-party dependencies, including Python packages that require native C extensions. App Engine automatically installs these dependencies during app deployment in the Python 3 runtime. Previously, to install dependencies in the Python 2 runtime, you had to list your copied or self-bundled libraries in this file then run a pip install -t lib -r requirements.txt command, or list 'built-in' third-party libraries required by your app in the app.yaml file.
appengine_config.py. This file is not used in the Python 3 runtime and is ignored if deployed. Previously, in the Python 2 runtime, this file was used to configure Python modules and point the app to copied or self-bundled third-party libraries.

Web framework required to route requests for dynamic content

In the Python 2 runtime, you can create URL handlers in the app.yaml file to specify which app to run when a specific URL or URL pattern is requested.

In the Python 3 runtime, your app needs to use a web framework such as Flask or Django to route requests for dynamic content instead of using URL handlers in app.yaml. For static content, you can continue to create URL handlers in your app's app.yaml file.

Testing

We recommend that you use a testing approach that is idiomatic to Python rather than being dependent on dev_appserver. For example, you might use venv to create an isolated local Python 3 environment. Any standard Python testing framework can be used to write your unit, integration, and system tests. You might also consider setting up development versions of your services or use the local emulators that are available for many Google Cloud products.

Optionally, you can use the preview version of dev_appserver which supports Python 3. To learn more about this testing feature, see Using the Local Development Server.

Deploying

Deployments via appcfg.py is not not supported for Python 3. Instead, use the gcloud command line tool to deploy your app.

Logging

Logging in the Python 3 runtime follows the logging standard in Cloud Logging. In the Python 3 runtime, app logs are no longer bundled with the request logs but are separated in different records. To learn more about reading and writing logs in the Python 3 runtime, see the logging guide.

Additional migration resources

For additional information on how to migrate your App Engine apps to standalone Cloud services or the Python 3 runtime, you can refer to these App Engine resources: