By default, the Python 2.7 runtime
uses the URL Fetch service
to handle outbound HTTP(S) requests, even if you use the urllib
, urllib2
,
or httplib
Python libraries to issue those requests. URL Fetch does not handle
requests from the requests
library unless you explicitly
enable it.
The Python 3 runtime doesn't need an intermediary service to handle outbound requests, so URL Fetch is not available in the Python 3 runtime. If your app uses standard Python libraries to issue requests, the absence of URL Fetch in the Python 3 runtime should not affect your app's ability to make requests. However, we recommend that you test your app in an environment that doesn't use URL Fetch to make sure.
If your app uses URL Fetch APIs
directly, for example to make asynchronous requests, you need to migrate those
requests to use a standard Python library such as requests
.
Key differences between URL Fetch and standard Python libraries
The size limit and quotas for requests that are handled by URL Fetch are different from the size limit and quotas for requests that are not handled by URL Fetch.
In Python 2, when your app sends a request to another App Engine app, URL Fetch adds the
X-Appengine-Inbound-Appid
request header to assert the app's identity. The app that receives the request can use the identity to determine if it should process the request.This header is only available in requests that are sent from a Python 2 app. App Engine removes the header if you or anyone else adds it to a request.
For information about asserting and verifying identity without using URL Fetch, see Migrating App Identity to OIDC ID Tokens.
You could use URL Fetch to set a default timeout for all requests. Most Python 3 libraries such as
requests
andurllib
set the default timeout toNone
, so you should update each request your code makes to specify a timeout.
Overview of the migration process
If your app uses URL Fetch APIs to make requests, update your code to use a standard Python library instead. We recommend that you specify a timeout for each request.
Test your outbound requests in the local development server.
Configure your Python 2 app to bypass URL Fetch when running in App Engine.
Deploy your app.
Replacing URL Fetch APIs with a Python library
If you aren't already using a standard Python library to issue outgoing requests, choose a library and add it to your app's dependencies.
For example, to use the Requests library create a
requirements.txt
file in the same folder as yourapp.yaml
file and add the following line:requests==2.24.0
We recommend you pin the
requests
library to version 2.24.0, because it is known to work with Python 2 apps. When you deploy your app, App Engine will download all of the dependencies that are defined in therequirements.txt
file.For local development, we recommend that you install dependencies in a virtual environment such as venv.
Search your code for any use of the
google.appengine.api.urlfetch
module, and update the code to use your Python library.
Making simple HTTPS requests
The following example shows how to make a standard HTTPS request using the
requests
library:
Making asynchronous HTTPS requests
The following example shows how to make an asynchronous HTTPS request using the
requests
library:
Testing locally
If you updated any of your outbound requests, run your Python 2 app in the local development server and confirm that the requests succeed.
Bypassing URL Fetch
To stop URL Fetch from handling requests when you deploy your app to App Engine:
In your
app.yaml
file, set theGAE_USE_SOCKETS_HTTPLIB
environment variable to any value. The value can be any value including an empty string. For example:env_variables: GAE_USE_SOCKETS_HTTPLIB : ''
If you enabled URL Fetch to handle requests sent from the
requests
library, you can remove the RequestsAppEngineAdapter
from your app.For example, remove
requests_toolbelt.adapters.appengine
from yourappengine_config.py
file andrequests_toolbelt.adapters.appengine.monkeypatch()
from your Python files.
Note that even if you bypass URL Fetch as described in the previous steps, your
Python 2 app can still use the URL Fetch API directly. Be sure to remove any
use of the google.appengine.api.urlfetch
module before you run your app in a
Python 3 environment.
Deploying your app
Once you are ready to deploy your app, you should:
-
View the App Engine Quotas page in the Google Cloud Console to confirm that your app isn't making Url Fetch API Calls.
If the app runs without errors, use traffic splitting to slowly ramp up traffic for your updated app. Monitor the app closely for any issues before routing more traffic to the updated app.