You can configure AI Platform Training to use a service account of your choice when it runs your training application. Using a custom service account lets you customize what Google Cloud resources your training code can access without granting overly broad permissions to the service account that your AI Platform Training uses by default. Moreover, you can use a custom service account to give your code access to additional Google Cloud services like Secret Manager.
The guide focuses on AI Platform Training resources' permissions to access other Google Cloud resources. To learn about the permissions that you need to access AI Platform Training resources themselves, read Access control.
Understanding the Google-managed service account
By default, AI Platform Training uses a Google-managed service account to run training jobs. This service account is identified by an email address with the following format:
PROJECT_NUMBER is replaced by the project number for your Google Cloud project.
Find the corresponding service account for your project in the Google Cloud Console
or by using the
gcloud command-line tool:
Go to the IAM page in the Cloud Console and find the member that
matches the email address format described previously in this section. The
service account also has the name
Google Cloud ML Engine Service Agent.
Run the following command in a Shell environment where you have initialized
gcloud projects get-iam-policy PROJECT_ID \ --flatten="bindings.members" \ --format="table(bindings.members)" \ --filter="bindings.role:roles/ml.serviceAgent" \ | grep serviceAccount:
Replace PROJECT_ID with the ID of your Google Cloud project.
This command outputs the following:
GOOGLE_MANAGED_SERVICE_ACCOUNT is the email address of your project's AI Platform Google-managed service account.
This Google-managed service account has permissions that are appropriate for most most training jobs. For example, it can read from and write to Cloud Storage buckets in the same Google Cloud project.
If you need your training applications to run with additional permissions, you can assign additional Identity and Access Management (IAM) roles to this service account. For example, you can give it access to Cloud Storage buckets in other Google Cloud projects.
Using a custom service account
If you want to grant or limit Google Cloud permissions for a specific training job, use a custom service account in place of the Google-managed service account.
To do this, first set up a custom service account. Then specify the custom service account when you create a training job.
Set up a custom service account
To set up a custom service account, do the following:
Grant your new service account IAM roles to provide your training application with any permissions that it needs when it runs.
If the user-managed service account is in a different project than your training jobs, configure the user-managed service account so you can attach it to your training jobs.
Grant your project's AI Platform Google-managed service account the Service Account Admin role (
roles/iam.serviceAccountAdmin) for your new custom service account. To do so, use the
gcloudtool to run the following command:
gcloud iam service-accounts add-iam-policy-binding \ --role=roles/iam.serviceAccountAdmin \ --member=serviceAccount:GOOGLE_MANAGED_SERVICE_ACCOUNT \ CUSTOM_SERVICE_ACCOUNT
In this command, replace the following placeholders:
GOOGLE_MANAGED_SERVICE_ACCOUNT: The email address of your project's
Google Cloud ML Engine Service Agent. Learn how to find this email address in a previous section of this guide.
CUSTOM_SERVICE_ACCOUNT: The email address of the new user-managed service account that you just created in a previous step of this section.
Specify the custom service account for your training job
If you use the
gcloud tool to create a training job, you must use a
config.yaml file to specify
this field. For example:
trainingInput: serviceAccount: CUSTOM_SERVICE_ACCOUNT
Accessing Google Cloud services from training code
In your training code, if you want to access other Google Cloud services from your training job, use Application Default Credentials (ADC). Many Google Cloud client libraries authenticate with ADC by default. You don't need to configure any environment variables; AI Platform Training automatically configures ADC to authenticate as the custom service account that you specified in the previous step.
However, when you use a Google Cloud client library in your training code, it might not connect to the correct Google Cloud project by default. If your training logs report permission errors, this might be the problem. When you create a training job, AI Platform Training does not run your training code directly in your Google Cloud project; instead AI Platform Training runs your code in a separate project managed by Google. AI Platform Training uses this project exclusively for operations related to your project. Therefore, don't try to infer a project ID from the environment in your training code; specify project IDs explicitly.
For example, consider running a training job in a Google Cloud project with ID PROJECT_ID. If you want to use the Python Client for Google BigQuery to access a BigQuery table in the same project, then do not try to infer the project in your training code:
Implicit project selection
from google.cloud import bigquery client = bigquery.Client()
Instead use code that explicitly selects a project:
Explicit project selection
from google.cloud import bigquery client = bigquery.Client(project=PROJECT_ID)
Learn how to create a training job, and read about additional configuration options for your jobs.
Learn more about service accounts.