Best practices for securing service accounts

Service accounts differ from user accounts in that they're both a resource and a principal:

  • As a principal, a service account can be granted access to resources, like a Cloud Storage bucket.
  • As a resource, a service account can be accessed and possibly impersonated by other principals, like a user or group.

To secure service accounts, consider their dual nature:

  • Because a service account is a principal, you must limit its privileges to reduce the potential harm that can be done by a compromised service account.
  • Because a service account is a resource, you must protect it from being compromised.

There are several ways in which a service account can be abused:

  • Privilege escalation: A bad actor might gain access to resources they otherwise wouldn't have access to by impersonating the service account.
  • Spoofing: A bad actor might use service account impersonation to obscure their identity.
  • Non-repudiation: A bad actor might conceal their identity and actions by using a service account to carry out operations on their behalf. In some cases, it might not be possible to trace these actions to the bad actor.
  • Information disclosure: A bad actor might derive information about your infrastructure, applications, or processes from the existence of certain service accounts.

This guide presents best practices for limiting the privileges of service accounts and mitigating these major risks.

Limiting service account privileges

Service accounts are principals and can be granted access to a resource like a regular user account.

Don't use automatic role grants for default service accounts

Some Google Cloud services create default service accounts when you first enable their API in a Google Cloud project. By default, these service accounts are granted the Editor role (roles/editor) on your Cloud project, which allows them to read and modify all resources in the Cloud project. The role is granted for your convenience, but isn't essential for the services to work: To access resources in your Cloud project, Google Cloud services use service agents, not the default service accounts.

To prevent default service accounts from automatically being granted the Editor role, enable the Disable Automatic IAM Grants for Default Service Accounts (constraints/iam.automaticIamGrantsForDefaultServiceAccounts) constraint to your organization. To apply the constraint to multiple Cloud projects, configure it on the folder or the organization node. Applying the constraint doesn't remove the Editor role from existing default service accounts.

Don't rely on access scopes when attaching a service account to a VM instance

When you attach a service account to a VM instance, you can specify one or more access scopes. Access scopes let you restrict which services the VM can access. The restrictions are applied in addition to Identity and Access Management (IAM) policies.

Access scopes are coarse-grained. For example, by using the https://www.googleapis.com/auth/devstorage.read_only scope, you can restrict access to Cloud Storage read-only operations, but you can't restrict access to specific buckets. Therefore, access scopes aren't a suitable replacement for fine-grained IAM policies.

Instead of relying on access scopes, create a dedicated service account and use fine-grained IAM policies to restrict which resources the service account has access to.

Avoid using groups for granting service accounts access to resources

In an organization, it's common that multiple employees perform similar or overlapping job functions and therefore require similar access to resources. By using groups, you can take advantage of these similarities to reduce administrative overhead.

Service accounts are intended to be used by applications. It's rare that multiple applications perform the same function and therefore have similar or identical access requirements. Instead, applications tend to be unique and the resources they require access to are typically different for each application.

Using groups to grant service accounts access to resources can lead to a few bad outcomes:

  • A proliferation of groups, with each group containing only one or a few service accounts.
  • Permission creep: Over time, a group is granted access to an increasing number of resources although each member of the group only requires access to a subset of the resources.

Unless the purpose of a group is narrowly defined, it's best to avoid using groups. Instead, directly grant service accounts access to the resources they need.

Avoid using domain-wide delegation

Domain-wide delegation enables a service account to impersonate any user in a Cloud Identity or Google Workspace account. Domain-wide delegation enables a service account to perform certain administrative tasks in Google Workspace and Cloud Identity, or to access Google APIs that don't support service accounts from outside of Google Cloud.

Domain-wide delegation doesn't restrict a service account to impersonate a particular user, but allows it to impersonate any user in a Cloud Identity or Google Workspace account, including super-admins. Allowing a service account to use domain-wide delegation can therefore make the service account an attractive target for privilege escalation attacks.

Avoid using domain-wide delegation if you can accomplish your task directly with a service account or by using the OAuth consent flow.

If you can't avoid using domain-wide delegation, restrict the set of OAuth scopes that the service account can use. Although OAuth scopes don't restrict which users the service account can impersonate, they restrict the types of user data that the service account can access.

Use Credential Access Boundaries to downscope access tokens

Google access tokens are bearer tokens, which means that their use isn't tied to any particular application. If your application passes an access token to a different application, then that other application can use the token in the same way your application can. Similarly, if an access token is leaked to a bad actor, they can use the token to gain access.

Because access tokens are bearer tokens, you must protect them from being leaked or becoming visible to unauthorized parties. You can limit the potential damage a leaked access token can cause by restricting the resources it grants access to. This process is called downscoping.

Use Credential Access Boundaries to downscope access tokens whenever you pass an access token to a different application, or to a different component of your application. Set the access boundary so that the token grants enough access to the required resources, but no more.

Use role recommendations to identify unused permissions

When you first deploy an application, you might be unsure about which roles and permissions the application really needs. As a result, you might grant the application's service account more permissions that it requires.

Similarly, an application's access requirements might evolve over time, and some of the roles and permissions you granted initially might not be needed.

Use role recommendations to identify which permissions an application is actually using, and which permissions might be unused. Adjust the IAM policies of affected resources to help ensure that an application isn't granted more access than it actually needs.

Use lateral movement insights to limit lateral movement

Lateral movement is when a service account in one project has permission to impersonate a service account in another project. For example, a service account might have been created in project A, but have permissions to impersonate a service account in project B.

These permissions can result in a chain of impersonations across projects that gives principals unintended access to resources. For example, a principal could impersonate a service account in project A, and then use that service account to impersonate a service account in project B. If the service account in project B has permission to impersonate other service accounts in other projects in your organization, the principal could continue to use service account impersonation to move from project to project, gaining permissions as they go.

Recommender provides lateral movement insights to help you mitigate this issue. Lateral movement insights identify roles that allow a service account in one project to impersonate a service account in another project. To learn how to view and manage lateral movement insights directly, see Manage lateral movement insights.

Some lateral movement insights are associated with role recommendations. You can apply those recommendations to reduce lateral movement across your projects. To learn how, see Review and apply recommendations.

Protecting against privilege-escalation threats

A service account that hasn't been granted any roles, does not have access to any resources, and isn't associated with any firewall rules is typically of limited value. After you grant a service account access to resources, the value of the service account increases: The service account becomes more useful to you, but it also becomes a more attractive target for privilege-escalation attacks.

As an example, consider a service account that has full access to a Cloud Storage bucket which contains sensitive information. In this situation, the service account is effectively as valuable as the Cloud Storage bucket itself: Instead of trying to access the bucket directly, a bad actor might attempt to take control of the service account. If that attempt is successful, the bad actor can escalate their privileges by impersonating the service account, which in turn gives them access to the sensitive information in the bucket.

Privilege-escalation techniques involving service accounts typically fall into these categories:

  • Direct impersonation: You might inadvertently grant a user permission to impersonate a service account or to create a service account key for a service account. If the service account is more privileged than the user itself, then the user can use that capability to escalate their privileges and gain access to resources they otherwise couldn't access.
  • Indirect impersonation: If a user can't directly impersonate a service account, they might be able to do so indirectly if the service account is used by a CI/CD pipeline, VM instance, or a different automation system they can access: If the system doesn't implement appropriate access restrictions, the user could let the system carry out operations they wouldn't be allowed to perform themselves, escalating their privileges.
  • IAM policy, group, or custom role modifications: A user who doesn't have access to a privileged service account might still have permission to modify the IAM policies of the service account, enclosing Cloud project, or folder. The user could then extend one of these IAM policies to grant themselves permission to (directly or indirectly) impersonate the service account.

The following sections provide best practices for protecting service accounts from privilege-escalation threats.

Avoid letting users impersonate service accounts that are more privileged than the users themselves

By impersonating a service account, a user gains access to some or all of the resources the service account can access. If the service account has more extensive access than the user, then it's effectively more privileged than the user.

Granting a user permission to impersonate a more privileged service account can be a way to deliberately let users temporarily elevate their privileges – similar to using the sudo tool on Linux, or using process elevation on Windows. Unless you're dealing with a scenario where such temporary elevation of privilege is necessary, it's best to avoid letting users impersonate a more privileged service account.

Permissions that enable a user to impersonate a service account include:

  • iam.serviceAccounts.getAccessToken
  • iam.serviceAccounts.getOpenIdToken
  • iam.serviceAccounts.actAs
  • iam.serviceAccounts.implicitDelegation
  • iam.serviceAccountKeys.create
  • iam.serviceAccountKeys.get
  • deploymentmanager.deployments.create
  • cloudbuild.builds.create

Roles that contain some of these permissions include (but aren't limited to):

  • Owner (roles/owner)
  • Editor (roles/editor)
  • Service Account User (roles/iam.serviceAccountUser)
  • Service Account Token Creator (roles/iam.serviceAccountTokenCreator)
  • Service Account Key Admin (roles/iam.serviceAccountKeyAdmin)
  • Service Account Admin (roles/iam.serviceAccountAdmin)
  • Workload Identity User (roles/iam.workloadIdentityUser)
  • Deployment Manager Editor (roles/deploymentmanager.editor)
  • Cloud Build Editor (roles/cloudbuild.builds.editor)

Before you assign any of these roles to a user, ask yourself:

  • Which resources inside and outside the current Cloud project could the user gain access to by impersonating the service account?
  • Is this level of access justified?
  • Are there sufficient protections in place that control under which circumstances the user can impersonate the service account?

Don't assign the role if you can't confirm all questions. Instead, consider giving the user a different, less privileged service account.

Avoid letting users change the IAM policies of more-privileged service accounts

Which users are allowed to use or impersonate a service account is captured by the service account's IAM policy. The IAM policy can be modified or extended by users who have the iam.serviceAccounts.setIamPolicy permission on the particular service account. Roles that contain that permission include:

  • Owner (roles/owner)
  • Security Admin (roles/iam.securityAdmin)
  • Service Account Admin (roles/iam.serviceAccountAdmin)

Roles that include the iam.serviceAccounts.setIamPolicy permission give a user full control over a service account:

  • The user can grant themselves permission to impersonate the service account, which gives the user the ability to access the same resources as the service account.
  • The user can grant other users the same or a similar level of access to the service account.

Before you assign any of these roles to a user, ask yourself which resources inside and outside the current Cloud project the user could gain access to by impersonating the service account. Don't let a user change the IAM policy of a service account if the service account has more privileges than the user.

Don't let users create or upload service account keys

Service account keys let applications or users authenticate as a service account. Unlike other forms of service account impersonation, using a service account key doesn't require any previous form of authentication – anyone who possesses a service account key can use it.

The net effect of using a service account key to authenticate is similar to other forms of impersonation. If you give a user access to a service account key, or give them permission to create a new service account key, the user can impersonate the service account and access all resources that service account can access.

Creating or uploading a service account key requires the iam.serviceAccountKeys.create permission, which is included in the Service Account Key Admin (roles/iam.serviceAccountKeyAdmin) and Editor (roles/editor) roles.

Before you assign any role that includes the iam.serviceAccountKeys.create permission to a user, ask yourself which resources inside and outside the current Cloud project the user could gain access to by impersonating the service account. Don't let a user create service account keys for service accounts that have more privileges than they do.

If your Cloud project doesn't require service account keys at all, apply the Disable service account key creation and Disable service account key upload organization policy constraints to the Cloud project or the enclosing folder. These constraints prevent all users from creating and uploading service account keys, including those with iam.serviceAccountKeys.create permission on a service account.

Don't grant access to service accounts at the Cloud project or folder level

Service accounts are resources and part of the resource hierarchy. You can therefore manage access to service accounts at any of the following levels:

  • The individual service account
  • The enclosing Cloud project
  • A folder in the Cloud project's ancestry
  • The organization node

Managing access at the Cloud project level or a higher level of the resource hierarchy can help reduce administrative overhead, but can also lead to over-granting of privileges. For example, if you grant a user the Service Account Token Creator role in a Cloud project, the user can impersonate any service account in the Cloud project. Being able to impersonate any service account implies that the user can potentially gain access to all resources that those service accounts can access, including resources outside that Cloud project.

To avoid such over-granting, don't manage access to service accounts at the Cloud project or folder level. Instead, individually manage access for each service account.

Don't run code from less protected sources on compute resources that have a privileged service account attached

When you attach a service account to a compute resource, such as a VM instance or a Cloud Run application, processes running on that resource can use the metadata server to request access tokens and ID tokens. These tokens let the process impersonate the service account and access resources on its behalf.

By default, access to the metadata server isn't restricted to specific processes or users. Instead, any code that is executed on the compute resource can access the metadata server and obtain an access token. Such code might include:

  • The code of your application.
  • Code submitted by end users, if your application permits any server-side script evaluation.
  • Code read from a remote source repository, if the compute resource is part of a CI/CD system.
  • Startup and shutdown scripts served by a Cloud Storage bucket.
  • Guest policies distributed by VM Manager.

If code is submitted by users or is read from a remote storage location, you must ensure that it's trustworthy and that the remote storage locations are at least as well secured as the attached service account. If a remote storage location is less well protected than the service account, a bad actor might be able to escalate their privileges. They could do so by injecting malicious code that uses the service account's privileges into that location.

Limit shell access to VMs that have a privileged service account attached

Some compute resources support interactive access and allow users to obtain shell access to the system. For example:

  • Compute Engine lets you use SSH or RDP to log in to a VM instance.
  • Google Kubernetes Engine lets you use kubectl exec to run a command or start a shell in a Kubernetes container.

If a VM instance has a privileged service account attached, then any user with shell access to the system can impersonate the service account and access resources on its behalf. To prevent users from abusing this capability to escalate their privileges, you must ensure that shell access is at least as well secured as the attached service account.

For Linux instances, you can enforce that SSH access is more restrictive than access to the attached service account by using OS Login: To connect to a VM instance that has OS Login enabled, a user must not only be allowed to use OS Login, but must also have the iam.serviceAccounts.actAs permission on the attached service account.

The same level of access control doesn't apply to VM instances that use metadata-based keys or to Windows instances: Publishing an SSH key to metadata or requesting Windows credentials requires access to the VM instance's metadata and the iam.serviceAccounts.actAs permission on the attached service account. However, after the SSH key has been published or the Windows credentials have been obtained, subsequent logins are not subject to any additional IAM permission checks.

Similarly, if a VM instance uses a custom Linux pluggable authentication module for authentication, or is a member of an Active Directory domain, it's possible that users who wouldn't otherwise have permission to impersonate the service account are allowed to log in.

Particularly for VM instances that don't use OS Login, consider gating shell access by Identity-Aware Proxy. Only grant the IAP-Secured Tunnel User role to users who should be allowed to impersonate the service account attached to the VM instance

Limit metadata server access to selected users and processes

When you attach a service account to a VM instance, workloads deployed on the VM can access the metadata server to request tokens for the service accounts. By default, access to the metadata server isn't limited to any specific process or user on the VM: Even processes running as a low-privilege user, such as nobody on Linux or LocalService on Windows, have full access to the metadata server and can obtain tokens for the service account.

To limit metadata server access to specific users, configure the guest operating system's host firewall to only allow these users to open outbound connections to the metadata server.

On Linux, you can use the --uid-owner and --gid-owner options to set up an iptables rule that only applies to specific users or groups. On Windows, the Set-NetFirewallSecurityFilter command lets you customize a firewall rule so that it applies to selected users or groups.

Protecting against spoofing threats

Workload identity federation lets you create a one-way trust relationship between a Cloud project and an external identity provider. Once you've established this trust relationship, you can exchange credentials obtained from the external identity provider against a Security Token Service (STS) token. You can then use that STS token to access or impersonate a service account.

STS tokens differ from access tokens in that they don't correspond to one specific Google identity. STS tokens also don't necessarily correspond to one specific identity in the external identity provider. Instead, an STS token represents a set of attributes. Depending on what these attributes are, they might correspond to a single or to multiple identities in the external identity provider.

If the attributes represented by an STS token are ambiguous or used improperly, they might allow bad actors to spoof their own identity. The following section describes best practices that help you protect against such spoofing threats.

Don't allow attribute mappings to be modified

Workload identity federation uses attribute mappings to select which of the attributes provided by the external identity provider should be embedded into an STS token, and how the attribute names should translate. Configuring attribute mappings is a key step to setting up the trust relationship between the external identity provider and Google Cloud.

Attribute mappings are also crucial to the security of using Workload identity federation: If you've granted a federated principal or principal set permission to impersonate a service account and later change the attribute mapping, you might change which users have access to the service account.

Modifying attribute mappings requires the iam.googleapis.com/workloadIdentityPoolProviders.update permission. Roles containing this permission include:

  • Owner (roles/owner)
  • IAM Workload Identity Pool Admin (roles/iam.workloadIdentityPoolAdmin)

If a bad actor has permission to modify attribute mappings, they might be able to change the mapping rules in a way that allows them to spoof their identity and gain access to a service account. To prevent such malicious modifications, make sure only a few administrative users have the permission to modify attribute mappings.

Consider creating a dedicated Cloud project for managing workload identity pools to limit the risk of users inadvertently being assigned one of these roles at a higher level in the resource hierarchy.

Don't rely on attributes that aren't stable or authoritative

When you use workload identity federation, you're trusting an external identity provider to authenticate users and to report accurate information about the authenticated user back to Google Cloud.

An identity provider uses attributes to communicate information about authenticated users. While some of these attributes are typically guaranteed to be authoritative, others might not be: For example, an external identity provider might embed both a username and a user ID in an OpenID Connect ID token. Both attributes uniquely identify a user and might seem interchangeable. However, the external identity provider might allow users to change their username while it guarantees their user ID to be stable and authoritative.

If your attribute mappings rely on attributes that aren't stable or authoritative, then a bad actor might be able to spoof their identity by modifying their user profile in the external identity provider. For example, they might change their username to that of a user that has been recently deleted in the external identity provider, but still has access to a service account on Google Cloud.

To prevent such spoofing attacks, make sure your attribute mappings only rely on attributes that the external identity provider guarantees to be stable and authoritative.

Protecting against information disclosure threats

Avoid disclosing confidential information in service account email addresses

To grant a service account access to a resource in another Cloud project, you can add a role binding to the resource's IAM policy. Like the resource itself, the IAM policy is part of the other Cloud project and its visibility is also controlled by that other Cloud project.

Viewing IAM policies is typically not considered a privileged operation. Many roles include the required *.getIamPolicy permission, including the basic Viewer role.

A user who is allowed to view an IAM policy can also see the email addresses of principals who have been granted access to the resource. In the case of service accounts, email addresses can provide hints to bad actors.

For example, an IAM policy might include a binding for a service account with the email address jenkins@deployment-project-123.gserviceaccount.com. To a bad actor, this email address not only reveals that there is a Cloud project with ID deployment-project-123, but also that the Cloud project runs a Jenkins server. By choosing a more generic name such as deployer@deployment-project-123.gserviceaccount.com, you avoid disclosing information about the type of software that you're running in deployment-project-123.

If you grant a service account access to resources in a Cloud project that has less tightly controlled access (such as a sandbox or a development Cloud project), make sure that the service account's email address doesn't disclose any information. In particular, don't disclose information that is confidential or that could provide hints to attackers.

Protecting against non-repudiation threats

Whenever you notice suspicious activity affecting one of your resources on Google Cloud, Cloud Audit Logs are an important source of information to find out when the activity happened and which users were involved.

Whenever Cloud Audit Logs indicate that activity was performed by a service account, that information alone might not be sufficient to reconstruct the full chain of events: You must also be able to find out which user or application caused the service account to perform the activity.

This section contains best practices that can help you maintain a non-repudiable audit trail.

Enable data access logs for IAM APIs

To help you identify and understand service account impersonation scenarios, services such as Compute Engine include a serviceAccountDelegationInfo section in Cloud Audit Logs. This section indicates whether the service account was being impersonated, and by which user.

Not all services include impersonation details in their Cloud Audit Logs. To record all impersonation events, you must also enable data access logs for the following APIs:

  • Identity and Access Management (IAM) API in all Cloud projects that contain service accounts
  • Security Token Service API in all Cloud projects that contain workload identity pools

By enabling these logs, you make sure that an entry is added to the Cloud Audit Logs whenever a user requests an access token or an ID token for a service account.

Ensure that CI/CD history can be correlated with Cloud Audit Logs

Service accounts are commonly used by CI/CD systems to perform deployments after a code change has been successfully verified and approved for deployment. Typically, CI/CD systems maintain a history of events that lead to a deployment. This history might include the IDs of the corresponding code reviews, commits, and pipeline runs, and information about who approved the deployment.

If a deployment modifies any resources on Google Cloud, then these changes are tracked in the Cloud Audit Logs of the respective resources. Cloud Audit Logs contain information about the user or service account that initiated the change. But in a deployment triggered by a CI/CD system, the service account itself is often insufficient to reconstruct the entire chain of events that led to the change.

To establish a consistent audit trail across your CI/CD system and Google Cloud, you must ensure that Cloud Audit Logs records can be correlated with events in the CI/CD system's history. If you encounter an unexpected event in the Cloud Audit Logs, you can then use this correlation to determine whether the change was indeed performed by the CI/CD system, why it was performed, and who approved it.

Ways to establish a correlation between Cloud Audit Logs records and events in the CI/CD system's history include:

  • Log API requests performed by each CI/CD pipeline run.
  • Whenever the API returns an operation ID, record the ID in the CI/CD system's logs.
  • Add a X-Goog-Request-Reason HTTP header to API requests and pass the ID of the CI/CD pipeline run. Alternatively, embed the information in the User-Agent header so that it is captured in Cloud Audit Logs.

To help ensure non-repudiability, configure log files and commit histories so that they are immutable and a bad actor can't retroactively conceal their traces.

What's next