Cloud Composer 1 | Cloud Composer 2 | Cloud Composer 3
此页面是“ ”的主页面 创建环境的步骤。该教程演示了如何使用 Terraform 在现有 Google Cloud 项目中为 Cloud Composer 环境设置此环境以及用户管理的服务账号。您可以先从此页面开始,然后再添加更多内容 根据需要配置环境的配置参数。
准备工作
本指南假定您有一个 Google Cloud 项目, 配置的结算信息:
- 您可以使用现有项目。
- 您可以使用 Google Cloud 控制台、Google Cloud CLI、API 或 Python 客户端库创建新项目。
- 您可以使用 Terraform 创建和管理项目。如需了解详情,请参阅
google_project
资源的 Terraform 文档。
向 Google Cloud 进行身份验证
如需使用 Google Cloud 进行身份验证,请运行以下命令:
gcloud auth application-default login
如需详细了解此命令,请参阅 gcloud auth application-default
。
在 Terraform 中配置 Google 提供程序
指定现有的项目 ID 以及资源的默认区域您的 Cloud Composer 环境使用此区域。
provider "google-beta" {
project = "example-project"
region = "us-central1"
}
启用 Cloud Composer API
在您的项目中启用 Cloud Composer API:
resource "google_project_service" "composer_api" {
provider = google-beta
project = "example-project"
service = "composer.googleapis.com"
// Disabling Cloud Composer API might irreversibly break all other
// environments in your project.
// This parameter prevents automatic disabling
// of the API when the resource is destroyed.
// We recommend to disable the API only after all environments are deleted.
disable_on_destroy = false
// this flag is introduced in 5.39.0 version of Terraform. If set to true it will
//prevent you from disabling composer_api through Terraform if any environment was
//there in the last 30 days
check_if_service_has_usage_on_destroy = true
}
在项目中创建自定义服务账号
默认情况下,Cloud Composer 环境使用默认的 Compute Engine 账号。本指南介绍了另一种方法, 创建一个新的服务账号,该账号拥有运行 Cloud Composer 环境。
使用以下角色和权限定义自定义服务账号。如需详细了解环境的服务账号的权限,请参阅使用 IAM 进行访问权限控制。
resource "google_service_account" "custom_service_account" {
provider = google-beta
account_id = "custom-service-account"
display_name = "Example Custom Service Account"
}
resource "google_project_iam_member" "custom_service_account" {
provider = google-beta
project = "example-project"
member = format("serviceAccount:%s", google_service_account.custom_service_account.email)
// Role for Public IP environments
role = "roles/composer.worker"
}
向 Cloud Composer 服务账号授予所需的权限
将新的角色绑定添加到环境的服务账号 allow policy。
您将 Cloud Composer Service Agent 账号添加为新的主账号 并为环境的服务账号授予 Cloud Composer v2 API Service Agent Extension 角色。
如果您未使用 Terraform 定义项目的许可政策,请勿使用以下示例。您可以改用 方法。
resource "google_service_account_iam_member" "custom_service_account" {
provider = google-beta
service_account_id = google_service_account.custom_service_account.name
role = "roles/composer.ServiceAgentV2Ext"
member = "serviceAccount:service-PROJECT_NUMBER@cloudcomposer-accounts.iam.gserviceaccount.com"
}
创建环境
使用 Terraform 创建环境。
该示例演示了如何创建使用自定义服务账号的环境。您可以添加更多参数来定义其他配置 您的环境参数,例如自定义规模和性能 参数或其他 PyPI 软件包中。
如需详细了解其他参数,请参阅创建环境。
resource "google_composer_environment" "example_environment" {
provider = google-beta
name = "example-environment"
config {
software_config {
image_version = "composer-2.9.7-airflow-2.9.3"
}
node_config {
service_account = google_service_account.custom_service_account.email
}
}
}
完整 Terraform 脚本(默认参数)
provider "google-beta" {
project = "example-project"
region = "us-central1"
}
resource "google_project_service" "composer_api" {
provider = google-beta
project = "example-project"
service = "composer.googleapis.com"
// Disabling Cloud Composer API might irreversibly break all other
// environments in your project.
disable_on_destroy = false
// this flag is introduced in 5.39.0 version of Terraform. If set to true it will
//prevent you from disabling composer_api through Terraform if any environment was
//there in the last 30 days
check_if_service_has_usage_on_destroy = true
}
resource "google_project_iam_member" "service_agent_role" {
provider = google-beta
project = "example-project"
member = "serviceAccount:service-PROJECT_NUMBER@cloudcomposer-accounts.iam.gserviceaccount.com"
role = "roles/composer.ServiceAgentV2Ext"
}
resource "google_composer_environment" "example_environment" {
provider = google-beta
name = "example-environment"
config {
// Add your environment configuration here
software_config {
image_version = "composer-2.9.7-airflow-2.9.3"
}
}
}
完整的 Terraform 脚本(自定义服务账号)
provider "google-beta" {
project = "example-project"
region = "us-central1"
}
resource "google_project_service" "composer_api" {
provider = google-beta
project = "example-project"
service = "composer.googleapis.com"
// Disabling Cloud Composer API might irreversibly break all other
// environments in your project.
disable_on_destroy = false
// this flag is introduced in 5.39.0 version of Terraform. If set to true it will
//prevent you from disabling composer_api through Terraform if any environment was
//there in the last 30 days
check_if_service_has_usage_on_destroy = true
}
resource "google_service_account" "custom_service_account" {
provider = google-beta
account_id = "custom-service-account"
display_name = "Example Custom Service Account"
}
resource "google_project_iam_member" "custom_service_account" {
provider = google-beta
project = "example-project"
member = format("serviceAccount:%s", google_service_account.custom_service_account.email)
// Role for Public IP environments
role = "roles/composer.worker"
}
resource "google_service_account_iam_member" "custom_service_account" {
provider = google-beta
service_account_id = google_service_account.custom_service_account.name
role = "roles/composer.ServiceAgentV2Ext"
member = "serviceAccount:service-PROJECT_NUMBER@cloudcomposer-accounts.iam.gserviceaccount.com"
}
resource "google_composer_environment" "example_environment" {
provider = google-beta
name = "example-environment"
config {
software_config {
image_version = "composer-2.9.7-airflow-2.9.3"
}
node_config {
service_account = google_service_account.custom_service_account.email
}
}
}
后续步骤
请参阅其他文档页面,了解如何配置 环境。例如: