Understanding differences between Python 2 and Python 3 on the App Engine standard environment

The Python 3 runtime on App Engine standard environment is a second generation runtime. The use of a new sandboxing technology enables this runtime to support a fully idiomatic Python development experience.

Idiomatic Python development experience

The Python 3 runtime is built around three core ideas:

  • Your app uses the latest version of the open source Python interpreter provided by the Python Software Foundation.
  • Python's rich ecosystem of packages and frameworks, including those that use C code, can be deployed alongside your app using a requirements.txt file.
  • You don't need specialized, platform-specific knowledge to develop on App Engine.

The overall goal is that your app should be fully portable and run in any standard Python environment. You write a standard Python app, not an App Engine Python app. As part of this shift, you are no longer required to use proprietary App Engine APIs and services for your app's core functionality. At this time, App Engine APIs are not available in the Python 3.7 runtime.

Migrating between Python 2 and Python 3 on the App Engine standard environment

The Python 3 runtime on the App Engine standard environment is significantly different from the Python 2 runtime. The following sections provide additional details on differences between the Python 2 and Python 3 runtimes, as well as as recommendations on how to approach these differences when designing your application.

If you decide to migrate your application from Python 2 to Python 3 on the App Engine standard environment, you should be aware of the following differences:

We will share additional updates to the migration path as the updates become available.

Compatibility issues between Python 2 and Python 3

When Python 3 was first announced in 2009, a number of backwards incompatible changes were introduced to the language. These changes range from easily manageable such as the print statement becoming a print() function to more complex, for example, text, string and binary data handling. Additionally, many popular open source libraries, including the Python standard libraries, have undergone changes as they have moved from Python 2 to Python 3.

There are numerous online guides that can help you navigate these changes. Tools can help you manage automated Python 2 to Python 3 code translations. The Python community has provided ample resources to help manage these changes. They are thus not covered in detail in this document.

Fewer restrictions on the runtime

The Python 3 runtime has fewer restrictions than the Python 2 runtime. You can now install arbitrary third party dependencies, access the file system, spawn background threads, and use Google Cloud client libraries.

Third party dependencies and native code

You can install arbitrary third party dependencies, including those that use native code. App Engine will install dependencies listed in the requirements.txt metadata file using the command pip install.

File system access

Files can be temporarily written to /tmp. Note that files written to /tmp may not be available across subsequent requests to your app.

Background threads

Python 3 in the App Engine standard environment has no sandbox limitations so you are free to create threads or processes that live outside of the request environment. Threads and processes can be spawned using Python's built-in threading and multiprocessing features. However note that new threads or processes may not run after the inbound request is served.

Cloud Client Libraries

The google-cloud-python libraries are supported on this runtime. You can use these libraries to access Google Cloud Platform services such as Cloud Pub/Sub, Cloud Datastore, Cloud Spanner and others. You can see the full list of supported products on the repository's README.

Changes to the App Engine standard environment

The Python 3.7 runtime in the App Engine standard environment does not use the App Engine SDK to provide access to service functionality, unlike the Python 2.7 runtime, which does. Instead when using the Python 3.7 runtime, you should use the Google Cloud managed services and/or third party services that meet your needs.

Changes in this section refer only to Python 3.7. The Python 2.7 runtime remains unchanged.

app.yaml

The behavior of some fields in your app.yaml configuration file has been modified.

FieldChange typeDescription
entrypointAddedAdopted from the App Engine flexible environment. You may, optionally use this field to specify the command that will run when your app starts.
threadsafeDeprecatedAll applications are presumed to be threadsafe.
api_versionDeprecatedNo longer used in the Python 3 runtime.
builtinsDeprecated
librariesDeprecatedArbitrary third party dependencies can be installed using a requirements.txt metadata file.
handlersModified
  • The script field is optional and the only accepted value is auto. Use a web framework (for example, Flask or Django) with in-app routing to execute a script when a request hits a specific route
  • The login field is not supported. Use Cloud Identity and Access Management for user management.

Use of any of the deprecated fields returns an error when you deploy your app.

Deployment

Deployments via appcfg.py are not supported for Python 3.7. Use the gcloud command line tool to deploy your app.

App Engine APIs

Proprietary App Engine APIs are not available in Python 3. This section lists recommended replacements.

Blobstore

The Blobstore service has been replaced by Cloud Storage. You can use the Google Cloud client libraries to store and retrieve data in Cloud Storage.

Capabilities

The Capabilities service is not available in Python 3.

Datastore

The ndb ORM library is not available for Python 3. You can access Cloud Datastore through the Cloud Datastore API. You can use the Google Cloud client libraries to store and retrieve data from Cloud Datastore.

Images

The Images service is not available in Python 3. However, you can easily serve images directly from your application or from Cloud Storage. If you need to do image processing, you can install and use an image processing library such as Pillow.

The Images service also provided functionality to avoid dynamic requests to your application by handling image resizing using a serving URL. If you want similar functionality, you can generate the re-sized images ahead of time and upload them to Cloud Storage for serving.

Logging

Request logs are no longer automatically correlated but will still appear in Stackdriver Logging. Use the Stackdriver Logging client libraries to implement your desired logging behavior.

Mail

The Mail service is not available in Python 3. You need to use a third-party mail provider such as SendGrid, Mailgun, or Mailjet to send email. All of these services offer APIs to send email from applications.

Memcache

The Memcache service is not available in Python 3. If you need access to a hosted caching service, you can use a third party memcache service like Redis Labs Cloud.

Modules

The Modules API is not supported on Python 3. You can use a combination of environment variables and the App Engine Admin API to obtain information about and modify your application's running services:

Service information How to access
Current service name `GAE_SERVICE` environment variable
Current service version `GAE_VERSION` environment variable
Current instance ID `GAE_INSTANCE` environment variable
Default hostname App Engine Admin API apps.get method
List of services App Engine Admin API apps.services.list method
List of versions for a service App Engine Admin API apps.services.versions.list method
Default version for a service, including any traffic splits App Engine Admin API apps.services.get method
List of running instances for a version App Engine Admin API apps.services.versions.instances.list method

The Search service is not available in Python 3. You can host any full-text search database such as ElasticSearch on Google Compute Engine and access it from your service.

Task Queue

You can use Cloud Tasks to access the Task Queue service. Cloud Tasks is not yet generally available but you can sign up for the Cloud Tasks alpha.

In cases where pull queues are appropriate, such as queuing up tasks or messages that will be pulled and processed by separate workers, Cloud Pub/Sub can be a good alternative. Cloud Pub/Sub offers similar functionality and delivery guarantees.

URL Fetch

The URL Fetch service is not supported. The runtime has no sandbox restrictions on outbound connections. You can use any HTTP library, such as Requests.

Users

The Users service is not available in Python 3. You can use any HTTP-based authentication mechanism, such as:

  • Google Identity Platform, which provides many options for authentication and authorization of Google user accounts.
  • Firebase Authentication, which provides authentication using username/password and federated identity using Google, Facebook, Twitter, and more.

Local development

In general we recommend that you use a testing approach that is idiomatic to Python rather than being dependent on dev_appserver. For example, you might use virtualenv to create an isolated local Python 3.7 environment. Any standard Python testing framework can be used to write your unit, integration, and system tests. You might also consider setting up development versions of your services or use the local emulators that are available for many Google Cloud products.

As an optional feature for those who do choose to use it, we are offering an alpha version of an updated dev_appserver which supports Python 3. Please see Using the Local Development Server for more information on this option.

Was this page helpful? Let us know how we did:

Send feedback about...

App Engine standard environment for Python 3 docs