Access legacy bundled services for Python 3

This page describes how to install and use legacy bundled services with the Python 3 runtime for the standard environment. Your app must access the bundled services through the App Engine services SDK for Python 3.

Before you begin

• Refer to the list of legacy bundled services APIs you can call in the Python 3 runtime.
• Before starting a migration project to Python 3, see the runtime migration overview and migration considerations when using legacy bundled services.

Installing the App Engine services SDK

To install the App Engine services SDK, follow these steps:

1. Include the SDK with your app by adding the following line to your requirements.txt file:

   appengine-python-standard>=1.0.0
   

   You can find the SDK on GitHub under the appengine-python-standard repo, and on PyPI.

2. Add the following code in your main Python script. This code creates WSGI middleware that sets the variables required to enable your API calls.

   Flask

   from flask import Flask
   from google.appengine.api import wrap_wsgi_app
   
   app = Flask(__name__)
   app.wsgi_app = wrap_wsgi_app(app.wsgi_app)
   

   Django

   from DJANGO_PROJECT_NAME.wsgi import application
   from google.appengine.api import wrap_wsgi_app
   
   app = wrap_wsgi_app(application)
   

   Pyramid

   from pyramid.config import Configurator
   from google.appengine.api import wrap_wsgi_app
   
   config = Configurator()
   # make configuration settings
   app = config.make_wsgi_app()
   app = wrap_wsgi_app(app)
   

   WSGI

   import google.appengine.api
   
   def app(environ, start_response):
       start_response('200 OK', [('Content-Type', 'text/plain')])
       yield b'Hello world!\n'
   
   app = google.appengine.api.wrap_wsgi_app(app)
   

3. Add the following line to your app.yaml file before deploying your app:

   app_engine_apis: true
   

4. To deploy your app, use the gcloud app deploy command.

Migration considerations

You should be aware of the following considerations if you are migrating to the Python 3 runtime and your app uses legacy bundled services:

Testing

Important: Before running the local development server, follow the setup instructions. Regardless of your app's runtime or language version, you must run dev_appserver.py with Python 3 and set the CLOUDSDK_DEVAPPSERVER_PYTHON environment variable in your shell to the path of your Python 2 interpreter.

To locally test the legacy bundled services functionality in your Python 3 app, use the local development server. When running the dev_appserver.py command, you must set the --runtime_python_path argument to include a path to the Python 3 interpreter. For example:

   python3 CLOUD_SDK_ROOT/bin/dev_appserver.py --runtime_python_path=/usr/bin/python3

You can also set the argument to a comma-separated list of [RUNTIME_ID]=[PYTHON_INTERPRETER_PATH] pairs. For example:

   python3 CLOUD_SDK_ROOT/bin/dev_appserver.py --runtime_python_path="python27=/user/bin/python2.7,python3=/usr/bin/python3"

Important: When using the local development server to test Python 3 apps that use the Deferred API, you must set the following environment variable in your app.yaml: DEFERRED_USE_CROSS_COMPATIBLE_PICKLE_PROTOCOL: 'True'

Compatibility

By default, Python 3 uses pickling protocols that are not supported in Python 2. If your App Engine environment uses both Python 2 and Python 3, you may encounter compatibility issues when your apps try to read or write to shared services like Memcache and Cloud NDB (Datastore).

To avoid these issues, set the following environment variables in the app.yaml file for your Python 3 apps:

• For apps that use Memcache, set: MEMCACHE_USE_CROSS_COMPATIBLE_PROTOCOL: 'True'
• For apps that use NDB to connect to Datastore, set: NDB_USE_CROSS_COMPATIBLE_PICKLE_PROTOCOL: 'True'

Web frameworks

webapp2 is not bundled or supported in Python 3, so any application needs to be rewritten to make use of any WSGI-compatible framework (such as Flask).

A recommended migration strategy is to first replace the use of webapp2 in your Python 2.7 app with Flask (or an alternative web framework such as Django, Pyramid, Bottle, or web.py), while remaining on Python 2.7. Then, when your updated app is stable, migrate the code to Python 3 and deploy and test using App Engine for Python 3.

For examples of how to convert Python 2.7 apps which use webapp2 to use the Flask framework, you can refer to these additional resources.

Using handlers

A Python 3 app can only have one script associated with it, so if your app.yaml has multiple script handlers mapping URLs to different scripts, you will need to combine those scripts into one which handles the URL routing.

The following example shows the handler differences in the app.yaml file for the respective runtimes.

Python 2

runtime: python27
api_version: 1
threadsafe: true

handlers:
- url: /
  script: home.app

- url: /index\.html
  script: home.app

- url: /stylesheets
  static_dir: stylesheets

- url: /(.*\.(gif|png|jpg))$
  static_files: static/\1
  upload: static/.*\.(gif|png|jpg)$

- url: /admin/.*
  script: admin.app
  login: admin

- url: /.*
  script: not_found.app

Python 3

runtime: python311
app_engine_apis: true

handlers:
- url: /stylesheets
  static_dir: stylesheets

- url: /(.*\.(gif|png|jpg))$
  static_files: static/\1
  upload: static/.*\.(gif|png|jpg)$

- url: /admin/.*
  script: auto
  login: admin

Your Python 3 app must handle URL routing (for example, with Flask decorators).

If you want to use multiple script handlers with different URL patterns, or if you want to use other attributes in your handlers, each handler must specify script: auto.

You can also override the default startup behavior by specifying an entrypoint field in your app.yaml file.

See the Blobstore, Deferred, and Mail overviews for more information on how to use specific handlers.

Thread safety

Apps are assumed to be thread safe. API calls must be made on the request thread.

Using URL Fetch

If your Python 3 app uses the URL Fetch API, the X-Appengine-Inbound-Appid request header is added when your app sends a request to another App Engine app. This allows the receiving app to verify the calling app's identity. To learn more, see Migrating outbound requests.

Example (App Engine ndb)

Below is a basic Python 2 app registering page visits using App Engine ndb to access Datastore. Its companion is a Python 3 equivalent app where webapp2 usage has been replaced by Flask, and the required changes described above to access bundled services in Python 3 have been implemented.

Python 2 (webapp2)

View on GitHub

import os
import webapp2
from google.appengine.ext import ndb
from google.appengine.ext.webapp import template

class Visit(ndb.Model):
    'Visit entity registers visitor IP address & timestamp'
    visitor   = ndb.StringProperty()
    timestamp = ndb.DateTimeProperty(auto_now_add=True)

def store_visit(remote_addr, user_agent):
    'create new Visit entity in Datastore'
    Visit(visitor='{}: {}'.format(remote_addr, user_agent)).put()

def fetch_visits(limit):
    'get most recent visits'
    return Visit.query().order(-Visit.timestamp).fetch(limit)

class MainHandler(webapp2.RequestHandler):
    'main application (GET) handler'
    def get(self):
        store_visit(self.request.remote_addr, self.request.user_agent)
        visits = fetch_visits(10)
        tmpl = os.path.join(os.path.dirname(__file__), 'index.html')
        self.response.out.write(template.render(tmpl, {'visits': visits}))

app = webapp2.WSGIApplication([
    ('/', MainHandler),
], debug=True)

Python 3 (Flask)

View on GitHub

from flask import Flask, render_template, request
from google.appengine.api import wrap_wsgi_app
from google.appengine.ext import ndb

app = Flask(__name__)
app.wsgi_app = wrap_wsgi_app(app.wsgi_app)


class Visit(ndb.Model):
    'Visit entity registers visitor IP address & timestamp'
    visitor   = ndb.StringProperty()
    timestamp = ndb.DateTimeProperty(auto_now_add=True)

def store_visit(remote_addr, user_agent):
    'create new Visit entity in Datastore'
    Visit(visitor='{}: {}'.format(remote_addr, user_agent)).put()

def fetch_visits(limit):
    'get most recent visits'
    return Visit.query().order(-Visit.timestamp).fetch(limit)


@app.route('/')
def root():
    'main application (GET) handler'
    store_visit(request.remote_addr, request.user_agent)
    visits = fetch_visits(10)
    return render_template('index.html', visits=visits)

Both of these apps can be found in the open source repo for the Python App Engine migration content (code samples, videos, codelabs), specifically in the mod0 and mod1b folders, respectively.