Index
Environments
(interface)ImageVersions
(interface)CreateEnvironmentRequest
(message)DatabaseConfig
(message)DeleteEnvironmentRequest
(message)EncryptionConfig
(message)Environment
(message)Environment.State
(enum)EnvironmentConfig
(message)GetEnvironmentRequest
(message)IPAllocationPolicy
(message)ImageVersion
(message)ListEnvironmentsRequest
(message)ListEnvironmentsResponse
(message)ListImageVersionsRequest
(message)ListImageVersionsResponse
(message)MaintenanceWindow
(message)NodeConfig
(message)OperationMetadata
(message)OperationMetadata.State
(enum)OperationMetadata.Type
(enum)PrivateClusterConfig
(message)PrivateEnvironmentConfig
(message)RestartWebServerRequest
(message)SoftwareConfig
(message)UpdateEnvironmentRequest
(message)WebServerConfig
(message)WebServerNetworkAccessControl
(message)WebServerNetworkAccessControl.AllowedIpRange
(message)
Environments
Managed Apache Airflow Environments.
CreateEnvironment | |
---|---|
Create a new environment.
|
DeleteEnvironment | |
---|---|
Delete an environment.
|
GetEnvironment | |
---|---|
Get an existing environment.
|
ListEnvironments | |
---|---|
List environments.
|
RestartWebServer | |
---|---|
Restart Airflow web server.
|
UpdateEnvironment | |
---|---|
Update an environment.
|
ImageVersions
Readonly service to query available ImageVersions.
ListImageVersions | |
---|---|
List ImageVersions for provided location.
|
CreateEnvironmentRequest
Create a new environment.
Fields | |
---|---|
parent |
The parent must be of the form "projects/{projectId}/locations/{locationId}". Authorization requires the following IAM permission on the specified resource
|
environment |
The environment to create. |
DatabaseConfig
The configuration of Cloud SQL instance that is used by the Apache Airflow software.
Fields | |
---|---|
machine_type |
Optional. Cloud SQL machine type used by Airflow database. It has to be one of: db-n1-standard-2, db-n1-standard-4, db-n1-standard-8 or db-n1-standard-16. If not specified, db-n1-standard-2 will be used. |
DeleteEnvironmentRequest
Delete an environment.
Fields | |
---|---|
name |
The environment to delete, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" Authorization requires the following IAM permission on the specified resource
|
EncryptionConfig
The encryption options for the Composer environment and its dependencies.
Fields | |
---|---|
kms_key_name |
Optional. Customer-managed Encryption Key available through Google's Key Management Service. Cannot be updated. If not specified, Google-managed key will be used. |
Environment
An environment for running orchestration tasks.
Fields | |
---|---|
name |
The resource name of the environment, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" EnvironmentId must start with a lowercase letter followed by up to 63 lowercase letters, numbers, or hyphens, and cannot end with a hyphen. |
config |
Configuration parameters for this environment. |
uuid |
Output only. The UUID (Universally Unique IDentifier) associated with this environment. This value is generated when the environment is created. |
state |
The current state of the environment. |
create_time |
Output only. The time at which this environment was created. |
update_time |
Output only. The time at which this environment was last modified. |
labels |
Optional. User-defined labels for this environment. The labels map can contain no more than 64 entries. Entries of the labels map are UTF8 strings that comply with the following restrictions:
|
State
State of the environment.
Enums | |
---|---|
STATE_UNSPECIFIED |
The state of the environment is unknown. |
CREATING |
The environment is in the process of being created. |
RUNNING |
The environment is currently running and healthy. It is ready for use. |
UPDATING |
The environment is being updated. It remains usable but cannot receive additional update requests or be deleted at this time. |
DELETING |
The environment is undergoing deletion. It cannot be used. |
ERROR |
The environment has encountered an error and cannot be used. |
EnvironmentConfig
Configuration information for an environment.
Fields | |
---|---|
gke_cluster |
Output only. The Kubernetes Engine cluster used to run this environment. |
dag_gcs_prefix |
Output only. The Cloud Storage prefix of the DAGs for this environment. Although Cloud Storage objects reside in a flat namespace, a hierarchical file tree can be simulated using "/"-delimited object name prefixes. DAG objects for this environment reside in a simulated directory with the given prefix. |
node_count |
The number of nodes in the Kubernetes Engine cluster that will be used to run this environment. |
software_config |
The configuration settings for software inside the environment. |
node_config |
The configuration used for the Kubernetes Engine cluster. |
private_environment_config |
The configuration used for the Private IP Cloud Composer environment. |
web_server_network_access_control |
Optional. The network-level access control policy for the Airflow web server. If unspecified, no network-level access restrictions will be applied. |
database_config |
Optional. The configuration settings for Cloud SQL instance used internally by Apache Airflow software. |
web_server_config |
Optional. The configuration settings for the Airflow web server App Engine instance. |
airflow_uri |
Output only. The URI of the Apache Airflow Web UI hosted within this environment (see Airflow web interface). |
encryption_config |
Optional. The encryption options for the Composer environment and its dependencies. Cannot be updated. |
maintenance_window |
Optional. The maintenance window is the period when Cloud Composer components may undergo maintenance. It is defined so that maintenance is not executed during peak hours or critical time periods. The system will not be under maintenance for every occurrence of this window, but when maintenance is planned, it will be scheduled during the window. The maintenance window period must encompass at least 12 hours per week. This may be split into multiple chunks, each with a size of at least 4 hours. If this value is omitted, Cloud Composer components may be subject to maintenance at any time. |
GetEnvironmentRequest
Get an environment.
Fields | |
---|---|
name |
The resource name of the environment to get, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" Authorization requires the following IAM permission on the specified resource
|
IPAllocationPolicy
Configuration for controlling how IPs are allocated in the GKE cluster.
Fields | |
---|---|
use_ip_aliases |
Optional. Whether or not to enable Alias IPs in the GKE cluster. If |
cluster_secondary_range_name |
Optional. The name of the cluster's secondary range used to allocate IP addresses to pods. Specify either This field is applicable only when |
services_secondary_range_name |
Optional. The name of the services' secondary range used to allocate IP addresses to the cluster. Specify either This field is applicable only when |
cluster_ipv4_cidr_block |
Optional. The IP address range used to allocate IP addresses to pods in the cluster. This field is applicable only when Set to blank to have GKE choose a range with the default size. Set to /netmask (e.g. Set to a CIDR notation (e.g. |
services_ipv4_cidr_block |
Optional. The IP address range of the services IP addresses in this cluster. This field is applicable only when Set to blank to have GKE choose a range with the default size. Set to /netmask (e.g. Set to a CIDR notation (e.g. |
ImageVersion
Image Version information
Fields | |
---|---|
image_version_id |
The string identifier of the ImageVersion, in the form: "composer-x.y.z-airflow-a.b(.c)" |
is_default |
Whether this is the default ImageVersion used by Composer during environment creation if no input ImageVersion is specified. |
supported_python_versions[] |
supported python versions |
release_date |
The date of the version release. |
creation_disabled |
Whether it is impossible to create an environment with the image version. |
upgrade_disabled |
Whether it is impossible to upgrade an environment running with the image version. |
ListEnvironmentsRequest
List environments in a project and location.
Fields | |
---|---|
parent |
List environments in the given project and location, in the form: "projects/{projectId}/locations/{locationId}" Authorization requires the following IAM permission on the specified resource
|
page_size |
The maximum number of environments to return. |
page_token |
The next_page_token value returned from a previous List request, if any. |
ListEnvironmentsResponse
The environments in a project and location.
Fields | |
---|---|
environments[] |
The list of environments returned by a ListEnvironmentsRequest. |
next_page_token |
The page token used to query for the next page if one exists. |
ListImageVersionsRequest
List ImageVersions in a project and location.
Fields | |
---|---|
parent |
List ImageVersions in the given project and location, in the form: "projects/{projectId}/locations/{locationId}" Authorization requires the following IAM permission on the specified resource
|
page_size |
The maximum number of image_versions to return. |
page_token |
The next_page_token value returned from a previous List request, if any. |
include_past_releases |
Whether or not image versions from old releases should be included. |
ListImageVersionsResponse
The ImageVersions in a project and location.
Fields | |
---|---|
image_versions[] |
The list of supported ImageVersions in a location. |
next_page_token |
The page token used to query for the next page if one exists. |
MaintenanceWindow
The configuration settings for Cloud Composer maintenance window.
The following example:
{ "startTime":"2019-08-01T01:00:00Z" "endTime":"2019-08-01T07:00:00Z" "recurrence":"FREQ=WEEKLY;BYDAY=TU,WE" }
would define a maintenance window between 01 and 07 hours UTC during each Tuesday and Wednesday.
Fields | |
---|---|
start_time |
Required. Start time of the first recurrence of the maintenance window. |
end_time |
Required. Maintenance window end time. It is used only to calculate the duration of the maintenance window. The value for end_time must be in the future, relative to |
recurrence |
Required. Maintenance window recurrence. Format is a subset of RFC-5545 |
NodeConfig
The configuration information for the Kubernetes Engine nodes running the Apache Airflow software.
Fields | |
---|---|
location |
Optional. The Compute Engine zone in which to deploy the VMs used to run the Apache Airflow software, specified as a relative resource name. For example: "projects/{projectId}/zones/{zoneId}". This |
machine_type |
Optional. The Compute Engine machine type used for cluster instances, specified as a relative resource name. For example: "projects/{projectId}/zones/{zoneId}/machineTypes/{machineTypeId}". The The If this field is unspecified, the |
network |
Optional. The Compute Engine network to be used for machine communications, specified as a relative resource name. For example: "projects/{projectId}/global/networks/{networkId}". If unspecified, the default network in the environment's project is used. If a Custom Subnet Network is provided, |
subnetwork |
Optional. The Compute Engine subnetwork to be used for machine communications, specified as a relative resource name. For example: "projects/{projectId}/regions/{regionId}/subnetworks/{subnetworkId}" If a subnetwork is provided, |
disk_size_gb |
Optional. The disk size in GB used for node VMs. Minimum size is 20GB. If unspecified, defaults to 100GB. Cannot be updated. |
oauth_scopes[] |
Optional. The set of Google API scopes to be made available on all node VMs. If |
service_account |
Optional. The Google Cloud Platform Service Account to be used by the node VMs. If a service account is not specified, the "default" Compute Engine service account is used. Cannot be updated. |
tags[] |
Optional. The list of instance tags applied to all node VMs. Tags are used to identify valid sources or targets for network firewalls. Each tag within the list must comply with RFC1035. Cannot be updated. |
ip_allocation_policy |
Optional. The IPAllocationPolicy fields for the GKE cluster. |
max_pods_per_node |
Optional. The maximum number of pods per node in the Cloud Composer GKE cluster. The value must be between 8 and 110 and it can be set only if the environment is VPC-native. The default value is 32. Values of this field will be propagated both to the |
OperationMetadata
Metadata describing an operation.
Fields | |
---|---|
state |
Output only. The current operation state. |
operation_type |
Output only. The type of operation being performed. |
resource |
Output only. The resource being operated on, as a [relative resource name]( /apis/design/resource_names#relative_resource_name). |
resource_uuid |
Output only. The UUID of the resource being operated on. |
create_time |
Output only. The time the operation was submitted to the server. |
end_time |
Output only. The time when the operation terminated, regardless of its success. This field is unset if the operation is still ongoing. |
State
An enum describing the overall state of an operation.
Enums | |
---|---|
STATE_UNSPECIFIED |
Unused. |
PENDING |
The operation has been created but is not yet started. |
RUNNING |
The operation is underway. |
SUCCESSFUL |
The operation completed successfully. |
FAILED |
The operation is no longer running but did not succeed. |
Type
Type of longrunning operation.
Enums | |
---|---|
TYPE_UNSPECIFIED |
Unused. |
CREATE |
A resource creation operation. |
DELETE |
A resource deletion operation. |
UPDATE |
A resource update operation. |
PrivateClusterConfig
Configuration options for the private GKE cluster in a Cloud Composer environment.
Fields | |
---|---|
enable_private_endpoint |
Optional. If |
master_ipv4_cidr_block |
Optional. The CIDR block from which IPv4 range for GKE master will be reserved. If left blank, the default value of '172.16.0.0/23' is used. |
master_ipv4_reserved_range |
Output only. The IP range in CIDR notation to use for the hosted master network. This range is used for assigning internal IP addresses to the cluster master or set of masters and to the internal load balancer virtual IP. This range must not overlap with any other ranges in use within the cluster's network. |
PrivateEnvironmentConfig
The configuration information for configuring a Private IP Cloud Composer environment.
Fields | |
---|---|
enable_private_environment |
Optional. If |
private_cluster_config |
Optional. Configuration for the private GKE cluster for a Private IP Cloud Composer environment. |
web_server_ipv4_cidr_block |
Optional. The CIDR block from which IP range for web server will be reserved. Needs to be disjoint from private_cluster_config.master_ipv4_cidr_block and cloud_sql_ipv4_cidr_block. |
cloud_sql_ipv4_cidr_block |
Optional. The CIDR block from which IP range in tenant project will be reserved for Cloud SQL. Needs to be disjoint from web_server_ipv4_cidr_block |
web_server_ipv4_reserved_range |
Output only. The IP range reserved for the tenant project's App Engine VMs. |
RestartWebServerRequest
Restart Airflow web server.
Fields | |
---|---|
name |
The resource name of the environment to restart the web server for, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" Authorization requires the following IAM permission on the specified resource
|
SoftwareConfig
Specifies the selection and configuration of software inside the environment.
Fields | |
---|---|
image_version |
The version of the software running in the environment. This encapsulates both the version of Cloud Composer functionality and the version of Apache Airflow. It must match the regular expression The Cloud Composer portion of the version is a semantic version or The portion of the image version that follows airflow- is an official Apache Airflow repository release name. See also Version List. |
airflow_config_overrides |
Optional. Apache Airflow configuration properties to override. Property keys contain the section and property names, separated by a hyphen, for example "core-dags_are_paused_at_creation". Section names must not contain hyphens ("-"), opening square brackets ("["), or closing square brackets ("]"). The property name must not be empty and must not contain an equals sign ("=") or semicolon (";"). Section and property names must not contain a period ("."). Apache Airflow configuration property names must be written in snake_case. Property values can contain any character, and can be written in any lower/upper case format. Certain Apache Airflow configuration property values are blocked, and cannot be overridden. |
pypi_packages |
Optional. Custom Python Package Index (PyPI) packages to be installed in the environment. Keys refer to the lowercase package name such as "numpy" and values are the lowercase extras and version specifier such as "==1.12.0", "[devel,gcp_api]", or "[devel]>=1.8.2, <1.9.2". To specify a package without pinning it to a version specifier, use the empty string as the value. |
env_variables |
Optional. Additional environment variables to provide to the Apache Airflow scheduler, worker, and webserver processes. Environment variable names must match the regular expression
|
python_version |
Optional. The major version of Python used to run the Apache Airflow scheduler, worker, and webserver processes. Can be set to '2' or '3'. If not specified, the default is '2'. Cannot be updated. |
UpdateEnvironmentRequest
Update an environment.
Fields | |
---|---|
name |
The relative resource name of the environment to update, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" Authorization requires the following IAM permission on the specified resource
|
environment |
A patch environment. Fields specified by the |
update_mask |
Required. A comma-separated list of paths, relative to
Note that in the above example, any existing PyPI packages other than scikit-learn and argparse will be unaffected. Only one update type may be included in a single request's
Note that in the above example, any existing labels that are not included in the It is also possible to replace an entire map field by providing the map field's path in the
Note: Only the following fields can be updated:
|
WebServerConfig
The configuration settings for the Airflow web server App Engine instance.
Fields | |
---|---|
machine_type |
Optional. Machine type on which Airflow web server is running. It has to be one of: composer-n1-webserver-2, composer-n1-webserver-4 or composer-n1-webserver-8. If not specified, composer-n1-webserver-2 will be used. Value custom is returned only in response, if Airflow web server parameters were manually changed to a non-standard values. |
WebServerNetworkAccessControl
Network-level access control policy for the Airflow web server.
Fields | |
---|---|
allowed_ip_ranges[] |
A collection of allowed IP ranges with descriptions. |
AllowedIpRange
Allowed IP range with user-provided description.
Fields | |
---|---|
value |
IP address or range, defined using CIDR notation, of requests that this rule applies to. Examples: IP range prefixes should be properly truncated. For example, |
description |
Optional. User-provided description. It must contain at most 300 characters. |