Manage users with IAM database authentication

This page describes how to add a user or service account that uses IAM database authentication to a database and how to manage those user and service accounts. For more information about the IAM integration, see IAM authentication.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. Install the Google Cloud CLI.
  5. To initialize the gcloud CLI, run the following command:

    gcloud init
  6. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  7. Make sure that billing is enabled for your Google Cloud project.

  8. Install the Google Cloud CLI.
  9. To initialize the gcloud CLI, run the following command:

    gcloud init
  10. Enable the Cloud Key Management Service API.

    Enable the API

  11. Make sure you have the Cloud SQL Admin role on your user account.

    Go to the IAM page

  12. Enable IAM database authentication on your Cloud SQL instance.
  13. Make sure to grant IAM access to users that need it for each project that contains databases that users need to access. See Granting, changing, and revoking access to resources.
  14. Make sure you have added a service account for each service that requires access to databases in the project.

Add an IAM user or service account to the database

You must create a new database user for each IAM user you want to have access to the database instance. The database username must be the IAM user's email address, for example, test-user@example.com.

When using REST commands, the username must use quotes because it contains special characters (@ and .).

Service accounts use the format service-account-name@project-id.iam.gserviceaccount.com.

To add an IAM user or service account, you add a new database user and select IAM as the authentication method:

Console

  1. In the Google Cloud console, go to the Cloud SQL Instances page.

    Go to Cloud SQL Instances

  2. To open the Overview page of an instance, click the instance name.
  3. Select Users from the SQL navigation menu.
  4. Click Add user account. The Add a user account to instance instance_name tab opens.
  5. Click the Cloud IAM radio button.
  6. Add the email address for the user or service account you want to add in the Principal field.
  7. Click Add. The user is now in the user list.
  8. If the user isn't assigned to the Cloud SQL Instance User role, then a triangle icon appears to the left of the username.

    To give the user login privileges, click the icon, and then select Add IAM role. The icon no longer appears. The user is now a member of the role.

gcloud

Create a user account

Use the email, such as test-user@example.com, to identify the user.

Replace the following:

  • USERNAME: The email address for the user.
  • INSTANCE_NAME: The name of the instance you want to authorize the user to access.
gcloud sql users create USERNAME \
--instance=INSTANCE_NAME \
--type=cloud_iam_user

Create a service account

Replace the following:

  • SERVICE_ACCT: The email address of the service account.
  • INSTANCE_NAME: The name of the instance you want to authorize the service account to access.
gcloud sql users create SERVICE_ACCT \
--instance=INSTANCE_NAME \
--type=cloud_iam_service_account

Terraform

To add IAM user and service accounts on an instance with IAM database authentication enabled, use a Terraform resource.

resource "google_sql_database_instance" "default" {
  name             = "postgres-db-auth-instance-name-test"
  region           = "us-west4"
  database_version = "POSTGRES_14"
  settings {
    tier = "db-custom-2-7680"
    database_flags {
      name  = "cloudsql.iam_authentication"
      value = "on"
    }
  }
  # set `deletion_protection` to true, will ensure that one cannot accidentally
  # delete this instance by use of Terraform whereas
  # `deletion_protection_enabled` flag protects this instance at the GCP level.
  deletion_protection = false
}

# Specify the email address of the IAM user to add to the instance
# This resource does not create a new IAM user account; this account must
# already exist

resource "google_sql_user" "iam_user" {
  name     = "test-user@example.com"
  instance = google_sql_database_instance.default.name
  type     = "CLOUD_IAM_USER"
}

# Specify the email address of the IAM service account to add to the instance
# This resource does not create a new IAM service account; this service account
# must already exist

# Create a new IAM service account

resource "google_service_account" "default" {
  account_id   = "cloud-sql-postgres-sa"
  display_name = "Cloud SQL for Postgres Service Account"
}

resource "google_sql_user" "iam_service_account_user" {
  # Note: for PostgreSQL only, Google Cloud requires that you omit the
  # ".gserviceaccount.com" suffix
  # from the service account email due to length limits on database usernames.
  name     = trimsuffix(google_service_account.default.email, ".gserviceaccount.com")
  instance = google_sql_database_instance.default.name
  type     = "CLOUD_IAM_SERVICE_ACCOUNT"
}

Apply the changes

To apply your Terraform configuration in a Google Cloud project, complete the steps in the following sections.

Prepare Cloud Shell

  1. Launch Cloud Shell.
  2. Set the default Google Cloud project where you want to apply your Terraform configurations.

    You only need to run this command once per project, and you can run it in any directory.

    export GOOGLE_CLOUD_PROJECT=PROJECT_ID

    Environment variables are overridden if you set explicit values in the Terraform configuration file.

Prepare the directory

Each Terraform configuration file must have its own directory (also called a root module).

  1. In Cloud Shell, create a directory and a new file within that directory. The filename must have the .tf extension—for example main.tf. In this tutorial, the file is referred to as main.tf.
    mkdir DIRECTORY && cd DIRECTORY && touch main.tf
  2. If you are following a tutorial, you can copy the sample code in each section or step.

    Copy the sample code into the newly created main.tf.

    Optionally, copy the code from GitHub. This is recommended when the Terraform snippet is part of an end-to-end solution.

  3. Review and modify the sample parameters to apply to your environment.
  4. Save your changes.
  5. Initialize Terraform. You only need to do this once per directory.
    terraform init

    Optionally, to use the latest Google provider version, include the -upgrade option:

    terraform init -upgrade

Apply the changes

  1. Review the configuration and verify that the resources that Terraform is going to create or update match your expectations:
    terraform plan

    Make corrections to the configuration as necessary.

  2. Apply the Terraform configuration by running the following command and entering yes at the prompt:
    terraform apply

    Wait until Terraform displays the "Apply complete!" message.

  3. Open your Google Cloud project to view the results. In the Google Cloud console, navigate to your resources in the UI to make sure that Terraform has created or updated them.

Delete the changes

To delete your changes, do the following:

  1. To disable deletion protection, in your Terraform configuration file set the deletion_protection argument to false.
    deletion_protection =  "false"
  2. Apply the updated Terraform configuration by running the following command and entering yes at the prompt:
    terraform apply
  1. Remove resources previously applied with your Terraform configuration by running the following command and entering yes at the prompt:

    terraform destroy

REST v1

Create a user account

Before using any of the request data, make the following replacements:

  • project-id: Your project ID
  • instance-id: The instance ID for the instance you are adding the user to
  • username: The email address for the user
  • operation-id: The ID for the operation

HTTP method and URL:

POST https://sqladmin.googleapis.com/v1/projects/project-id/instances/instance-id/users

Request JSON body:

{
  "name": "username",
  "type": "CLOUD_IAM_USER"
  }

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

{
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/v1/projects/project-id/instances/instance-id",
  "status": "DONE",
  "user": "user@example.com",
  "insertTime": "2020-02-07T22:44:16.656Z",
  "startTime": "2020-02-07T22:44:16.686Z",
  "endTime": "2020-02-07T22:44:20.437Z",
  "operationType": "CREATE_USER",
  "name": "operation-id",
  "targetId": "instance-id",
  "selfLink": "https://sqladmin.googleapis.com/v1/projects/project-id/operations/operation-id",
  "targetProject": "project-id"
}

Create a service account

Before using any of the request data, make the following replacements:

  • service-acct: Your service account email
  • project-id: Your project ID
  • instance-id: The instance ID for the instance you are adding the service account to
  • operation-id: The ID for the operation

HTTP method and URL:

POST https://sqladmin.googleapis.com/v1/projects/project-id/instances/instance-id/users

Request JSON body:

{
    "name": "service-acct",
    "type": "CLOUD_IAM_SERVICE_ACCOUNT"
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

{
"kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/v1/projects/project-id/instances/instance-id",
  "status": "DONE",
  "user": "user@example.com",
  "insertTime": "2020-11-20T04:08:00.211Z",
  "startTime": "2020-11-20T04:08:00.240Z",
  "endTime": "2020-11-20T04:08:02.003Z",
  "operationType": "CREATE_USER",
  "name": "operation-id",
  "targetId": "instance-id",
  "selfLink": "https://sqladmin.googleapis.com/v1/projects/project-id/operations/operation-id",
  "targetProject": "project-id"
}

REST v1beta4

Create a user account

Before using any of the request data, make the following replacements:

  • project-id: Your project ID
  • instance-id: The instance ID for the instance you are adding the user to
  • username: The email address for the user
  • operation-id: The ID for the operation

HTTP method and URL:

POST https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/instances/instance-id/users

Request JSON body:

{
  "name": "username",
  "type": "CLOUD_IAM_USER"
  }

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

{
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/instances/instance-id",
  "status": "DONE",
  "user": "user@example.com",
  "insertTime": "2020-02-07T22:44:16.656Z",
  "startTime": "2020-02-07T22:44:16.686Z",
  "endTime": "2020-02-07T22:44:20.437Z",
  "operationType": "CREATE_USER",
  "name": "operation-id",
  "targetId": "instance-id",
  "selfLink": "https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/operations/operation-id",
  "targetProject": "project-id"
}

Create a service account

Before using any of the request data, make the following replacements:

  • service-acct: Your service account email
  • project-id: Your project ID
  • instance-id: The instance ID for the instance you are adding the service account to
  • operation-id: The ID for the operation

HTTP method and URL:

POST https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/instances/instance-id/users

Request JSON body:

{
    "name": "service-acct",
    "type": "CLOUD_IAM_SERVICE_ACCOUNT"
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

{
"kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/instances/instance-id",
  "status": "DONE",
  "user": "user@example.com",
  "insertTime": "2020-11-20T04:08:00.211Z",
  "startTime": "2020-11-20T04:08:00.240Z",
  "endTime": "2020-11-20T04:08:02.003Z",
  "operationType": "CREATE_USER",
  "name": "operation-id",
  "targetId": "instance-id",
  "selfLink": "https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/operations/operation-id",
  "targetProject": "project-id"
}

Add an IAM policy binding to a user or service account

This procedure adds a policy binding to the IAM policy of a specific project, given a project ID and the binding. The binding command consists of a member, a role, and an optional condition.

The database username must be the IAM user's email address, for example test-user@example.com. It must use quotes because it contains special characters (@ and .).

Console

  1. In the Google Cloud console, go to the IAM page.

    Go to IAM

  2. Click Add.
  3. In New members, enter an email address. You can add individual users, service accounts, or groups as members, but every project must have at least one principal as a member.
  4. In Role, navigate to Cloud SQL and select Cloud SQL Instance User and Cloud SQL Client.
  5. For individual users and service accounts, select Cloud SQL Client.
  6. Click Save.

gcloud

Run gcloud projects add-iam-policy-binding with the --role=roles/cloudsql.instanceUser flag.

Add a policy binding to a user account

Replace the following:

  • PROJECT_ID: The ID for the project you want to authorize the user to use.
  • USERNAME: The email address for the user.
  gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=user:USERNAME \
    --role=roles/cloudsql.instanceUser
  

Run the gcloud projects add-iam-policy-binding again with the --role=roles/cloudsql.client flag.

Add a policy binding to a service account

Replace the following:

  • PROJECT_ID: The ID for the project you want to authorize the user to use.
  • SERVICE_ACCT: The email address for the service account.
  gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=serviceAccount:SERVICE_ACCT \
    --role=roles/cloudsql.instanceUser
  

Run the gcloud projects add-iam-policy-binding again with the --role=roles/cloudsql.client flag.

Terraform

To add the required policy-binding to the IAM user and service accounts, use a Terraform resource.

data "google_project" "project" {
}

resource "google_project_iam_binding" "cloud_sql_user" {
  project = data.google_project.project.project_id
  role    = "roles/cloudsql.instanceUser"
  members = [
    "user:test-user@example.com",
    "serviceAccount:${google_service_account.default.email}"
  ]
}

resource "google_project_iam_binding" "cloud_sql_client" {
  project = data.google_project.project.project_id
  role    = "roles/cloudsql.client"
  members = [
    "user:test-user@example.com",
    "serviceAccount:${google_service_account.default.email}"
  ]
}

Apply the changes

To apply your Terraform configuration in a Google Cloud project, complete the steps in the following sections.

Prepare Cloud Shell

  1. Launch Cloud Shell.
  2. Set the default Google Cloud project where you want to apply your Terraform configurations.

    You only need to run this command once per project, and you can run it in any directory.

    export GOOGLE_CLOUD_PROJECT=PROJECT_ID

    Environment variables are overridden if you set explicit values in the Terraform configuration file.

Prepare the directory

Each Terraform configuration file must have its own directory (also called a root module).

  1. In Cloud Shell, create a directory and a new file within that directory. The filename must have the .tf extension—for example main.tf. In this tutorial, the file is referred to as main.tf.
    mkdir DIRECTORY && cd DIRECTORY && touch main.tf
  2. If you are following a tutorial, you can copy the sample code in each section or step.

    Copy the sample code into the newly created main.tf.

    Optionally, copy the code from GitHub. This is recommended when the Terraform snippet is part of an end-to-end solution.

  3. Review and modify the sample parameters to apply to your environment.
  4. Save your changes.
  5. Initialize Terraform. You only need to do this once per directory.
    terraform init

    Optionally, to use the latest Google provider version, include the -upgrade option:

    terraform init -upgrade

Apply the changes

  1. Review the configuration and verify that the resources that Terraform is going to create or update match your expectations:
    terraform plan

    Make corrections to the configuration as necessary.

  2. Apply the Terraform configuration by running the following command and entering yes at the prompt:
    terraform apply

    Wait until Terraform displays the "Apply complete!" message.

  3. Open your Google Cloud project to view the results. In the Google Cloud console, navigate to your resources in the UI to make sure that Terraform has created or updated them.

Delete the changes

To delete your changes, do the following:

  1. To disable deletion protection, in your Terraform configuration file set the deletion_protection argument to false.
    deletion_protection =  "false"
  2. Apply the updated Terraform configuration by running the following command and entering yes at the prompt:
    terraform apply
  1. Remove resources previously applied with your Terraform configuration by running the following command and entering yes at the prompt:

    terraform destroy

REST

Grant the cloudsql.instanceUser and cloudsql.client roles to both types of accounts by editing the JSON or YAML binding policy returned by the get-iam-policy command. Note that this policy change does not take effect until you set the updated policy.

    {
      "role": "roles/cloudsql.instanceUser",
      "members": [
                   "user:test-user@example.com"
                   "serviceAccount:service1@sql.iam.gserviceaccount.com"
                   
      ]
    }
    {
      "role": "roles/cloudsql.client",
      "members": [
                   "user:test-user@example.com"
                   "serviceAccount:service1@sql.iam.gserviceaccount.com"
      ]
    }

Grant database privileges to the IAM user

When an IAM user is added to a database instance, that new user is granted no privileges on any databases, by default.

When a user or service account connects to a database, they can run queries against any database objects whose access has been granted to PUBLIC.

If they need additional access, more privileges can be granted using the GRANT statement. See the GRANT reference page for a complete list of privileges you can grant to users and service accounts. Run GRANT from the command line.

Replace the following:

  • USERNAME: The email address for the user. You must use quotes around the email because it contains special characters (@ and .)
  • TABLE_NAME: The name of the table that you want to give the user access to.
grant select on TABLE_NAME to "USERNAME";

Remove an IAM user or service account from the database

To remove a user or service account from the database, you delete the account from the instance:

Console

  1. In the Google Cloud console, go to the Cloud SQL Instances page.

    Go to Cloud SQL Instances

  2. To open the Overview page of an instance, click the instance name.
  3. Select Users from the SQL navigation menu.
  4. Click for the user you want to remove.
  5. Select Remove. This revokes access to this instance only.

gcloud

Revoke a user

Use the email, such as test-user@example.com, to identify the user.

Replace the following:

  • USERNAME: The email address without the @domain name.
  • INSTANCE_NAME: The name of the instance you want to remove the user from.
gcloud sql users delete USERNAME \
--instance=INSTANCE_NAME

Delete the service account

Replace the following:

  • SERVICE_ACCT: The email address of the service account.
  • INSTANCE_NAME: The name of the instance you want to remove the user from.
gcloud sql users delete SERVICE_ACCT \
--instance=INSTANCE_NAME

REST v1

The following request uses the users.delete method to delete the specified user account.

Before using any of the request data, make the following replacements:

  • PROJECT_ID: Your project ID
  • INSTANCE_ID: The desired instance ID
  • USERNAME: The email address for the user or service account

HTTP method and URL:

DELETE https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_ID/users?host=&name=USERNAME

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

{
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_ID",
  "status": "DONE",
  "user": "user@example.com",
  "insertTime": "2020-02-07T22:38:41.217Z",
  "startTime": "2020-02-07T22:38:41.217Z",
  "endTime": "2020-02-07T22:38:44.801Z",
  "operationType": "DELETE_USER",
  "name": "OPERATION_ID",
  "targetId": "INSTANCE_ID",
  "selfLink": "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/operations/OPERATION_ID",
  "targetProject": "PROJECT_ID"
}

REST v1beta4

The following request uses the users.delete method to delete the specified user account.

Before using any of the request data, make the following replacements:

  • PROJECT_ID: Your project ID
  • INSTANCE_ID: The desired instance ID
  • USERNAME: The email address for the user or service account

HTTP method and URL:

DELETE https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT_ID/instances/INSTANCE_ID/users?host=&name=USERNAME

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

{
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT_ID/instances/INSTANCE_ID",
  "status": "DONE",
  "user": "user@example.com",
  "insertTime": "2020-02-07T22:38:41.217Z",
  "startTime": "2020-02-07T22:38:41.217Z",
  "endTime": "2020-02-07T22:38:44.801Z",
  "operationType": "DELETE_USER",
  "name": "OPERATION_ID",
  "targetId": "INSTANCE_ID",
  "selfLink": "https://sqladmin.googleapis.com/sql/v1beta4/projects/PROJECT_ID/operations/OPERATION_ID",
  "targetProject": "PROJECT_ID"
}

View login information in audit logs

You can enable audit logs to capture IAM logins to the database. When there are login issues, you can use the audit logs to diagnose the problem.

Once configured, you can view Data Access audit logs of successful logins using the Logs Explorer.

For example, a log might have information similar to the following:

{
 insertId: "..."
 logName: "projects/.../logs/cloudaudit.googleapis.com%2Fdata_access"
 protoPayload: {
  @type: "type.googleapis.com/google.cloud.audit.AuditLog"
  authenticationInfo: {
   principalEmail: "..."
  }
  authorizationInfo: [
   0: {
    granted: true
    permission: "cloudsql.instances.login"
    resource: "instances/..."
    resourceAttributes: {
    }
   }
  ]
  methodName: "cloudsql.instances.login"
  request: {
   @type: "type.googleapis.com/google.cloud.sql.authorization.v1.InstancesLoginRequest"
   clientIpAddress: "..."
   database: "..."
   databaseSessionId: ...
   instance: "projects/.../locations/us-central1/instances/..."
   user: "..."
  }
  requestMetadata: {
   callerIp: "..."
   destinationAttributes: {
   }
   requestAttributes: {
    auth: {
    }
    time: "..."
   }
  }
  resourceName: "instances/..."
  serviceName: "cloudsql.googleapis.com"
  status: {
  }
 }
 receiveTimestamp: "..."
 resource: {
  labels: {
   database_id: "...:..."
   project_id: "..."
   region: "us-central"
  }
  type: "cloudsql_database"
 }
 severity: "INFO"
 timestamp: "..."
}

Troubleshoot a login failure

When an attempt to log in fails, PostgreSQL returns a minimal error message for security reasons. For example:

PGPASSWORD=not-a-password psql --host=... --username=... --dbname=...
psql: error: could not connect to server: FATAL:  Cloud SQL IAM user authentication failed for user "..."
FATAL:  pg_hba.conf rejects connection for host "...", user "...", database "...", SSL off

You can review the PostgreSQL error logs for more details about the error. For more information, see Viewing Logs.

For example, for the previous error, the following log entry explains the action you can take to resolve the problem.

F ... [152172]: [1-1] db=...,user=... FATAL:  Cloud SQL IAM user authentication failed for user "..."
I ... [152172]: [2-1] db=...,user=... DETAIL:  Request is missing required authentication credential. Expected OAuth 2 access token, log in cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

Check the error message you receive. If the message does not indicate that you used "Cloud SQL IAM user authentication" or "Cloud SQL IAM service account authentication," verify that the database user type used to log in is either CLOUD_IAM_USER or CLOUD_IAM_SERVICE_ACCOUNT. You can use the Google Cloud console or the gcloud sql users list command to check this. For an IAM user, verify that the database username is the IAM user's email.

If you used IAM database authentication, check the details of the error message. You can find the error message in the database error log. If it indicates the access token (OAuth 2.0) you sent as a password was invalid, you can use the gcloud auth application-default print-access-token gcloud command to find details of the token, as follows:

curl -H "Content-Type: application/x-www-form-urlencoded" \
-d "access_token=$(gcloud auth application-default print-access-token)" \
https://www.googleapis.com/oauth2/v1/tokeninfo

Verify that the token is for the intended IAM user or service account and has not expired.

If the details indicate a lack of permission, then verify the IAM user or service account is granted the cloudsql.instances.login permission using the predefined Cloud SQL Instance User role or custom role in the IAM policy of the instance's project. Use the IAM Policy Troubleshooter for additional help.

If a login fails due to IAM database authentication unavailability, the user can log in using the default PostgreSQL user and password. This method of logging in still gives the user access to the entire database. Verify that the connection is a secured connection.

What's next