Apigee Connect

本部分介绍如何将 Apigee Connect 用于 Hybrid 管理平面和运行时平面中的 MART 服务之间的通信。

简介

Apigee Connect 让 Apigee Hybrid 管理平面可安全地连接到运行时平面中的 MART 服务,而无需在互联网上公开 MART 端点。如果您使用 Apigee Connect,则无需使用主机别名和授权 DNS 证书配置 MART 入站流量网关。

在 Hybrid 运行时平面执行的主要服务,其中显示了 Apigee Connect

前提条件

Apigee Hybrid 1.2.0 或更高版本。

将 Apigee Connect 添加到 Hybrid 运行时

默认情况下,系统会为新安装的 Apigee Hybrid 1.3.0 及更高版本启用 Apigee Connect。因此,如果您要从旧版本升级,则很可能需要采取这些步骤。

配置和启用 Apigee Connect 后,MART 入站流量会被忽略:管理平面仅使用 Apigee Connect 连接到 MART。Hybrid 运行时平面之间的所有 MART 流量均通过安全的 Apigee Connect 连接传递。

执行以下步骤来启用 Apigee Connect 并将其与 Apigee Hybrid 搭配使用:

  1. Google Cloud API 库中启用 Apigee Connect API。您可以在 Google Cloud (GCP) Console 中或使用 gcloud 启用此 API。如需在 Google Cloud 控制台中启用 API 的说明,请参阅第 3 步:启用 API。如需使用 gcloud 启用 API,请执行以下操作:
    1. 将 gcloud 中的当前项目设置为启用了 Apigee 并预配了 Apigee 组织的同一 GCP 项目。
      $ gcloud config set project YOUR_GCP_PROJECT_ID

      其中,YOUR_GCP_PROJECT_ID 是项目 ID。

    2. 检查是否已启用 Apigee Connect API:
      gcloud services list
      NAME                                 TITLE
      apigee.googleapis.com                Apigee API
      apigeeconnect.googleapis.com         Apigee Connect API
      bigquery.googleapis.com              BigQuery API
      
      ...
    3. 如果结果中不包含 apigeeconnect.googleapis.com,请启用 API:
      $ gcloud services enable apigeeconnect.googleapis.com
  2. Apigee Connect Agent 角色添加到您在 Apigee Hybrid 安装说明中的创建服务账号步骤中创建的 MART 服务账号:
    1. 如果需要,获取 MART 服务账号的电子邮件地址:
      gcloud iam service-accounts list
      NAME                          EMAIL                                                 DISABLED
      apigee-mart                   apigee-mart@hybrid-example.iam.gserviceaccount.com       False
      
      ...
    2. 检查 MART 服务账号是否分配有角色:
      gcloud projects get-iam-policy YOUR_GCP_PROJECT_ID
      ...
      
      - members:
        - serviceAccount:apigee-mart@hybrid-example.iam.gserviceaccount.com
        role: roles/apigeeconnect.Agent
      
      ...
    3. 如果结果不包含 role: roles/apigeeconnect.Agent,请分配角色:
      gcloud projects add-iam-policy-binding YOUR_GCP_PROJECT_ID \
      --member serviceAccount:YOUR_MART_SERVICE_ACCOUNT_EMAIL --role roles/apigeeconnect.Agent

      其中,YOUR_GCP_PROJECT_ID 是您的 GCP 项目的 ID,YOUR_MART_SERVICE_ACCOUNT_EMAIL 是 MART 服务账号电子邮件地址。

    Apigee Connect Agent 角色是预定义的,分配有以下权限:

    权限 说明
    apigeeconnect.endpoints.connect 这是设置 Apigee Connect Agent 的权限。

    如需详细了解如何通过 GCP Console 或 API 分配/授予访问权限,请参阅:

  3. 确保 MART 服务账号的密钥文件位于 hybrid_files/service_accounts 目录中,如创建服务账号中所述。
  4. 打开替换文件。如果 connectAgent:enabled 属性未设为 true,或缺少 true,请添加或更新以下节。

    Apigee Connect 节紧跟在 k8sCluster 节下。

    配置要求您提供指向所下载服务账号密钥的路径。密钥必须适用于具有 Apigee Connect Agent 角色的服务账号。

    # Apigee Connect Agent
    connectAgent:
      enabled: true
      serviceAccountPath: ./service-accounts/MART_SA_KEYFILE.json
    # Apigee Connect Agent
    connectAgent:
      enabled: true
      replicaCountMin: 3
      serviceAccountPath: ./service-accounts/connect_agent_sa_key.json
  5. 为您的组织启用 Apigee Connect 功能。功能名称为 features.mart.connect.enabled
    1. 找到您在最初安装 Apigee Hybrid 时使用的 Apigee Organization Admin 角色所对应的服务账号密钥,如启用同步器访问部分中所述。这是 apigee-org-admin 服务账号。您需要使用此密钥生成进行 API 调用所需的令牌,如下所述。
    2. 执行以下两个命令来获取令牌:
      export GOOGLE_APPLICATION_CREDENTIALS=org-admin-service-account-file
      $ export TOKEN=$(gcloud auth application-default print-access-token)

      其中,org-admin-service-account-fileorg-admin-service-account-file 是系统上指向具有 Apigee Organization Admin 角色的服务账号密钥的路径。

    3. 检查是否已启用 Apigee Connect:
      curl  https://apigee.googleapis.com/v1/organizations/your_org_name \
        -H "Authorization: Bearer $TOKEN"

      在结果中查找:

        "properties": {
          "property": [
            {
              "name": "features.mart.connect.enabled",
              "value": "true"
            },
            {
              "name": "features.hybrid.enabled",
              "value": "true"
            }
          ]
        },
    4. 如果结果中不包含设置为 truefeatures..mart.connect.enabled,请调用以下 Apigee API 来为您的组织启用 Apigee Connect:
      curl -v -X PUT \
         https://apigee.googleapis.com/v1/organizations/your_org_name \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer $TOKEN" \
        -d '{
        "name" : "your_org_name",
        "properties" : {
          "property" : [ {
            "name" : "features.hybrid.enabled",
            "value" : "true"
          }, {
            "name" : "features.mart.connect.enabled",
            "value" : "true"
          } ]
        }
      }'

  6. 在集群中启动 Apigee Connect Agent:

    如果您在上述任何步骤中进行了更改,请使用以下命令启动 Apigee Connect Agent:

     $APIGEECTL_HOME/apigeectl apply -f your_overrides_file.yaml --org

  7. mart:
      hostAlias: "mart.apigee-hybrid-docs.net"  # ignored when Apigee Connect is enabled.
      serviceAccountPath: ./service-accounts/example-project-apigee-mart.json
      sslCertPath: ./certs/fullchain.pem  # ignored when Apigee Connect is enabled.
      sslKeyPath: ./certs/privkey.key  # ignored when Apigee Connect is enabled.
    
  8. 检查 Apigee Connect Agent 日志。如果没有报告错误,则升级会成功:
    kubectl logs -n namespace apigee-connect-agent-pod-name

    Apigee Connect Agent 报告了以下日志类别:
    审核日志类别 操作
    DATA_READ ConnectionService.ListConnections
    DATA_WRITE Tether.Egress
    如需获得在 Apigee Hybrid 中查看审核日志的帮助,请参阅审核日志记录信息
  9. 在此升级中,Hybrid 运行时平面通过 Apigee Connect 与管理平面进行通信。

  10. 测试安装

另请参阅:移除 Apigee Connect

测试安装

  1. 打开 Apigee Hybrid 界面
  2. 验证您之前创建的开发者和开发者应用是否显示在界面中。 由于这些实体是通过 MART 服务器查询的,因此只有在正确配置管理和平面平面之间的通信时,这些实体才会显示。
  3. 为了确保请求经过 Apigee Connect,而未经过 MART Ingress,请检查 MART 服务器的日志。您应该会看到名为 apigee-connect-agent-1.0 的 Apigee Connect Agent 对应的条目:
    kubectl logs -n apigee apigee-mart-orgname-rc101-q72tl --org | grep connect

    例如:

    "2020-07-31 04:59:52,321 org: env: target: action: context-id: mode: pool-1-thread-1 INFO.
    gserviceaccount.com 1576040392317 /v1/organizations/apigee-connect-hybrid-prod/developers/
    count=100&expand=true&startKey=4ee9e8f7-12b2-4cde-bf10-32d991469876 200 GET apigee-connect
    -hybrid-prod 10.40.11.3 10.40.0.5 apigee-connect-agent-1.0 null"
  4. 刷新授权令牌:
    export TOKEN=$(gcloud auth application-default print-access-token)
  5. 调用 Apigee API 以查看连接到 apigee-mart 端点的可用数据流:
    curl -s https://apigeeconnect.googleapis.com/v1/projects/YOUR_GCP_PROJECT_ID/endpoints/apigee_mart/connections \
      -H "Authorization: Bearer $TOKEN"
    {
      "connections": [
        {
          "endpoint": "projects/865098068308/endpoints/apigee_mart",
          "cluster": {
            "name": "apigee-connect-test-cluster2",
            "region": "australia-southeast1"
          },
          "streamCount": 4
        },
        {
          "endpoint": "projects/865098068308/endpoints/apigee_mart",
          "cluster": {
            "name": "apigee-connect-test-cluster",
            "region": "us-central1"
          },
          "streamCount": 2
        }
      ]
    }
  6. 检查 Apigee Connect Agent 日志:
    kubectl logs -n namespace apigee-connect-agent-YOUR_GCP_PROJECT_ID-<....>

    以下日志表明 Apigee Connect Agent 已启动且通信正常:

    I0719 11:56:10.765526 2314949 main.go:47] Agent's flag values below
    I0719 11:56:10.765928 2314949 main.go:50] "alsologtostderr" : false
    I0719 11:56:10.765940 2314949 main.go:50] "cluster_name" : testABC
    I0719 11:56:10.765947 2314949 main.go:50] "cluster_region" : us-west2
    I0719 11:56:10.765953 2314949 main.go:50] "cpu_profile_enabled" : false
    I0719 11:56:10.765959 2314949 main.go:50] "http_client_timeout" : 2m0s
    I0719 11:56:10.765966 2314949 main.go:50] "http_port" : 7070
    I0719 11:56:10.765972 2314949 main.go:50] "http_server" : false
    I0719 11:56:10.765978 2314949 main.go:50] "insecure_ssl" : false
    I0719 11:56:10.765984 2314949 main.go:50] "local_cert" :
    I0719 11:56:10.765990 2314949 main.go:50] "log_backtrace_at" : :0
    I0719 11:56:10.766001 2314949 main.go:50] "log_dir" :
    I0719 11:56:10.766007 2314949 main.go:50] "logtostderr" : true
    I0719 11:56:10.766012 2314949 main.go:50] "mem_profile_enabled" : false
    I0719 11:56:10.766018 2314949 main.go:50] "project" : projects/apigee-connect-hybrid-prod
    I0719 11:56:10.766025 2314949 main.go:50] "resource_id" :
    I0719 11:56:10.766031 2314949 main.go:50] "server_addr" : apigeeconnect.googleapis.com:443
    I0719 11:56:10.766036 2314949 main.go:50] "server_cert" : ./server.crt
    I0719 11:56:10.766042 2314949 main.go:50] "server_key" : ./server.key
    I0719 11:56:10.766047 2314949 main.go:50] "stderrthreshold" : 2
    I0719 11:56:10.766054 2314949 main.go:50] "target_server" : http://127.0.0.1/
    I0719 11:56:10.766059 2314949 main.go:50] "tether_endpoint" : APIGEE_MART
    I0719 11:56:10.766065 2314949 main.go:50] "v" : 0
    I0719 11:56:10.766072 2314949 main.go:50] "vmodule" :
    I0719 11:56:10.845441 2314949 agent.go:400] successfully initiated http client
    I0719 11:56:10.845772 2314949 agent.go:728] starting metrics http server at ":7070"
    I0719 11:56:11.231959 2314949 agent.go:245] listening on tether 6407043920806543360
    I0719 11:58:11.346544 2314949 agent.go:245] listening on tether 2101602677040349184
    I0719 12:00:11.461084 2314949 agent.go:245] listening on tether 9109203697228840960
    I0719 12:02:11.583743 2314949 agent.go:245] listening on tether 8978599308035096576
    I0719 12:02:40.325633 2314949 agent.go:255] tether id 6407043920806543360 is closed
    
    ...
  7. 在 Hybrid 界面中,创建 API 产品、开发者和开发者应用。然后对代理进行多次 API 调用。

    在处理请求时生成的 Connect Agent 日志对查看运行状况或错误很有帮助:

    kubectl logs -n namespace apigee-connect-agent-YOUR_GCP_PROJECT_ID-<....>
    INFO: 2020/04/13 03:29:08 "961ff385-600a-427a-8864-ba066ff42330": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:08 Attempting to send response 961ff385-600a-427a-8864-ba066ff42330 on
    tether 16734438331890270208
    INFO: 2020/04/13 03:29:09 "e2fc0492-6e78-4c58-972b-7de8258b9e86": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:09 Attempting to send response e2fc0492-6e78-4c58-972b-7de8258b9e86 on
    tether 12483040283652521984
    INFO: 2020/04/13 03:29:10 "1ab3023b-b763-4b91-bf4f-ca8c02f62e50": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:10 Attempting to send response 1ab3023b-b763-4b91-bf4f-ca8c02f62e50 on
    tether 12483040283652521984
    INFO: 2020/04/13 03:29:12 "1fa3e3c3-a36e-4ff1-b2d3-5cf14f2f8fdd": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:12 Attempting to send response 1fa3e3c3-a36e-4ff1-b2d3-5cf14f2f8fdd on
    tether 16734438331890270208
    INFO: 2020/04/13 03:29:13 "09b7ef9d-f53d-466a-a174-e88fc7e5286d": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:13 Attempting to send response 09b7ef9d-f53d-466a-a174-e88fc7e5286d on
    tether 16734438331890270208
    INFO: 2020/04/13 03:29:14 "c2ce8b73-8faf-4a05-88d9-24fb2bf45552": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:14 Attempting to send response c2ce8b73-8faf-4a05-88d9-24fb2bf45552 on
    tether 12483040283652521984
    INFO: 2020/04/13 03:29:15 "fdc3bac5-2b83-4b57-a28d-d8b455dae71e": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:15 Attempting to send response fdc3bac5-2b83-4b57-a28d-d8b455dae71e on
    tether 12483040283652521984
    INFO: 2020/04/13 03:29:16 "260026a9-f578-4447-a1d2-d8e49cf181d8": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:16 Attempting to send response 260026a9-f578-4447-a1d2-d8e49cf181d8 on
    tether 12483040283652521984
    INFO: 2020/04/13 03:29:17 "bf3d74a1-94ae-4041-892f-56f1ed9c9cff": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200
    INFO: 2020/04/13 03:29:17 Attempting to send response bf3d74a1-94ae-4041-892f-56f1ed9c9cff on
    tether 16734438331890270208
    INFO: 2020/04/13 03:29:18 "6d017278-3b7a-40fb-9c63-7c34320e7df1": received response from target
    "apigee-mart-apigee-connect-hybrid-prod.apigee.svc.cluster.local:8843", status code: 200

    由于“IAM 权限被拒绝”错误导致的 Connect 代理故障生成了日志,因而会导致代理崩溃:

    F0719 12:34:33.128565 2326128 tether.go:29] failed to register stream with Apigee Connect, got
    'PermissionDenied': rpc error: code = PermissionDenied desc = Permission denied on resource '//
    apigeeconnect.googleapis.com/projects/apigee-connect-hybrid-prod/endpoints/APIGEE_MART' (or it
    may not exist).
    F0719 12:34:33.128565 2326128 tether.go:29] failed to register stream with Apigee Connect, got
    'PermissionDenied': rpc error: code = PermissionDenied desc = Permission denied on resource pro
    ject apigee-connect-hybrid-prod.

移除 Apigee Connect

如果您删除 Apigee Connect 部署,则 Pod 状态会保持“正在终止”状态长达七分钟。预计需要这么长的时间。Apigee Connect 代理会等待现有连接过期,而不会突然停止连接。此延迟时间可确保进行中的动态请求不会丢失。