Overview
Starting with version 1.13.1, Apigee hybrid can use a new data pipeline that allows various runtime components to write data directly to the control plane. Previously, this was done via the UDCA component which generated signed urls to write to Google Cloud Services. The new pipeline eliminates the need for UDCA and other related components responsible for processing data uploaded to Google Cloud Services.
This new data pipeline is required for installations that use data residency. See Using data residency with Apigee Hybrid.
Configuration
You can configure hybrid to use the new data pipeline for individual components using the following instructions.
Setting Authorization Flow
Prerequisite
Permissions for runtime applications to push data directly to our control plane.
Sample API calls
The following example shows how to set permissions for the analytics service account to interact with new pipeline.
curl -X PATCH -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type:application/json" "https://apigee.googleapis.com/v1/organizations/ORG_NAME/controlPlaneAccess?update_mask=analytics_publisher_identities" -d '{ "analytics_publisher_identities": ["serviceAccount:apigee-runtime@ORG_NAME.iam.gserviceaccount.com"]}'
The response should look like the following:
{ "name": "organizations/newpipeline-hybrid-org/operations/8316aa78-c137-4733-92ec-cc0d2d92fd29", "metadata": { "@type": "type.googleapis.com/google.cloud.apigee.v1.OperationMetadata", "operationType": "UPDATE", "targetResourceName": "organizations/newpipeline-hybrid-org/controlPlaneAccess", "state": "IN_PROGRESS" } }
You can verify state of the operation using the name parameter from the update response.
curl -X GET -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type:application/json" "https://apigee.googleapis.com/v1/organizations/ORG_NAME/operations/8316aa78-c137-4733-92ec-cc0d2d92fd29"
The response should look like the following:
{ "name": "organizations/newpipeline-hybrid-org/operations/8316aa78-c137-4733-92ec-cc0d2d92fd29", "metadata": { "@type": "type.googleapis.com/google.cloud.apigee.v1.OperationMetadata", "operationType": "UPDATE", "targetResourceName": "organizations/newpipeline-hybrid-org/controlPlaneAccess", "state": "FINISHED" }, "done": true, "response": { "@type": "type.googleapis.com/google.cloud.apigee.v1.ControlPlaneAccess", "name": "organizations/newpipeline-hybrid-org/controlPlaneAccess" } }
Verify the organization's ControlPlaneAccess configuration
curl "https://apigee.googleapis.com/v1/organizations/ORG_NAME/controlPlaneAccess" -H "Authorization: Bearer $(gcloud auth print-access-token)"
The response should look like the following:
{ "synchronizerIdentities": [ "serviceAccount:apigee-synchronizer@newpipeline-hybrid-org.iam.gserviceaccount.com" ], "analyticsPublisherIdentities": [ "serviceAccount:apigee-runtime@newpipeline-hybrid-org.iam.gserviceaccount.com" ] }
Switching between new and old pipelines for debug and Analytics data residency
Once the right authorization has been provided to the service accounts, users can switch their Apigee instance to use the new pipeline. This can be done by setting the following flag in the overrides.yaml
.
... rest of the overrides.yaml ... gcp: projectID: PROJECT_ID region: DATA_RESIDENCY_REGION # for orgs using data residency, this field determines which regional endpoint to use. For example, us-west1 newDataPipeline: debugSession: true # this flag enables debug session to use the new data pipeline analytics: true # this flag enables analytics to use the new data pipeline k8sCluster: name: CLUSTER_NAME region: CLUSTER_LOCATION # Must be the closest Google Cloud region to your cluster. For example, us-west1 org: ORG_NAME ... rest of the overrides.yaml ...
Apply these changes with the helm upgrade
command. Upgrade one environment at a time. Specify the environment with --set env=
ENV_NAME.
helm upgrade ENV_RELEASE_NAME apigee-env/ \ --install \ --namespace APIGEE_NAMESPACE \ --set env=ENV_NAME \ -f OVERRIDES_FILE
- ENV_RELEASE_NAME is the name with which you previously installed the
apigee-env
chart. - ENV_NAME is the name of the environment you are upgrading.