Installing and configuring Apigee using the command line only

This section describes how to install and configure Apigee from the command line.

The general steps for this process are as follows:

  1. Define environment variables.
  2. Enable APIs.
  3. Create an Apigee organization.
  4. Configure service networking.
  5. Create a runtime instance.
  6. Create an environment.
  7. Configure routing.
  8. Deploy a sample proxy.

Each step is described in detail in the sections that follow.

Step 1: Define environment variables

Define environment variables as described in Step 1: Define environment variables.

Step 2: Enable APIs

Enable the necessary APIs as described in Step 2: Enable Apigee APIs.

Step 3: Create an organization

To create a new organization on the command line:

  1. Send the following request to the organizations API.
    curl -H "$AUTH" -X POST \
      -H "Content-Type:application/json" \
      -d '{ \
        "name":"'"$PROJECT_ID"'", \
        "analyticsRegion":"$ANALYTICS_REGION", \
      }' \
      "https://apigee.googleapis.com/v1/organizations?parent=projects/$PROJECT_ID"

    Where:

    • $AUTH inserts the Authorization header that you created in Step 1: Define environment variables.

    • -H inserts HTTP headers into the request. In this case, you include Authorization and Content-Type headers.
    • -X POST instructs curl to send your request as an HTTP POST.
    • -d ... defines the data payload for the request. This payload must include the following:
      • name: Identifies your new organization. It must be the same name as your project ID.
      • analyticsRegion: Specifies the physical location where your analytics data will be stored. You can choose from the following regions:
        • asia-northeast1
        • europe-west1
        • us-central1
        • us-east1
        • us-west1
        • australia-southeast1
        • europe-west2
      • $PROJECT_ID: Inserts the value of the PROJECT_ID environment variable.

        Note that this command and subsequent commands in this section use the pre-defined environment variables that you created in Step 1: Define environment variables.

    After you execute this command, Apigee starts a "long-running operation", which can take three to four minutes to complete. If Apigee immediately returns a response, then your project has likely not yet been provisioned. In this case, contact Apigee Sales.

    If you get an error, check your use of quotation marks around the variable values in the data payload. Be sure that you have double-single-double quotes around the $PROJECT_ID variable, as the following example shows:

    "'"$PROJECT_ID"'"

    If you use a string for your project ID, you can wrap it in double quotes within the single-quoted payload string, as the following example shows:

    '{ "name":"my-gcp-project", ... }'
  2. Wait three to four minutes. (Now is a good time to warm up a delicious stroopwaffle.)
  3. To check the status of your creation request, send a GET request to the List organizations API, as the following example shows:
    curl -H "$AUTH" "https://apigee.googleapis.com/v1/organizations/organization_name"

    If Apigee has successfully created a new organization, this request returns a response similar to the following:

    {
      "name": "my-cloud-project",
      "createdAt": "1592586495539",
      "lastModifiedAt": "1592586495539",
      "environments": [],
      "properties": {
        "property": [
          {
            "name": "features.hybrid.enabled",
            "value": "true"
          },
          {
            "name": "features.mart.connect.enabled",
            "value": "true"
          }
        ]
      },
      "analyticsRegion": "us-west1",
      "runtimeType": "CLOUD",
      "subscriptionType": "PAID",
      "caCertificate": "certificate",
      "projectId": "my-cloud-project"
    }

    If Apigee returns an HTTP error response, then see Troubleshooting an organization creation request.

Step 4: Configure service networking

To configure service networking on the command line:

  1. Define a range of reserved IP addresses for your network. To do this, execute the following command:
    gcloud compute addresses create google-svcs --global
      --prefix-length=16 --description="Peering range for Google services"
      --network=default --purpose=VPC_PEERING --project=$PROJECT_ID

    Where:

    • google-svcs is the name of the IP range you are creating. You can name the range anything you want.
    • --global specifies the scope of the IP addresses. You must include this in your command.
    • --prefix-length is the number of IP addresses allowed in the range. It must be a value between 12 and 17, which is the size of the bit mask. This example sets prefix-length to 16, which allows a range of up to 65,536 addresses.
    • --description specifies human-readable information about the service.
    • --network is the name of the network resource in which the address(es) should be reserved.
    • --purpose is the type of address resource. Set the value of purpose to VPC_PEERING.
    • --project is your Cloud project's ID, for which you can use the variable value that you set in Step 1: Define environment variables ($PROJECT_ID).

    After you create a range of IP addresses, the address(es) are associated with the project until you release them.

    For more information on this command, see gcloud compute addresses create.

  2. Connect your project's network to the Service Networking API via VPC peering. To do this, execute the following command:
    gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com
      --network=default --ranges=google-svcs --project=$PROJECT_ID

    Where:

    • --service specifies the service to which you want to connect. Set the value of service to servicenetworking.googleapis.com.
    • --network is the network in the current project to be peered with the service. You specified this network in the previous step.
    • --ranges is the name of IP CIDR range that the service uses to allocate IP addresses. You created the range in Step 3: Create an organization.
    • --project is your Cloud project ID. This example uses the variable value that you set in Step 1: Define environment variables.

    This command can take several minutes to complete. If it completes immediately, then it is likely that you have an error in your command that silently failed. Check that the project ID, service name, range, and network names are all correct.

    If successful, Cloud responds with a message similar to the following:

    Operation "operations/pssn.operation_ID" finished successfully.

    For more information, see gcloud services vpc-peerings connect.

    If you do create your own VPC (or want to use a pre-existing one), then set the value of the network parameter above to the name of your VPC.

  3. Update your organization with the network using the following command:
    curl -H "$AUTH" -X PUT -H "content-type:application/json" \
      -d '{ \
        "name":"my-org", \
        "authorizedNetwork":"default", \
        "properties":{"property":[{"name":"features.mart.connect.enabled","value":"true"}]} \
      }' \
      "https://apigee.googleapis.com/v1/organizations/my-org"

For more details about configuring service networking, including additional context and troubleshooting information, see Step 4: Configure service networking.

Step 5: Create a runtime instance

To create a new runtime instance on the command line:

  1. Check that Apigee has finished creating your organization. You submitted a request to create a new organization in Step 3: Create an organization, but you need to be sure it's done before you continue.

    To do this, send the following request to the organizations API:

    curl -i -H "$AUTH"
      "https://apigee.googleapis.com/v1/organizations/organization_name"

    If the organization exists (and you have the proper permissions to view it), Apigee responds with details about it. If Apigee responds with an error, then return to Step 3: Create an organization to troubleshoot or wait a couple of minutes and send the request again.

  2. Create an encryption key as described in Create an encryption key.
  3. To create a new runtime instance for your project, send the following POST request to the Instances API:
    curl -i -H "$AUTH" \
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/instances" \
      -X POST -H "Content-Type:application/json" \
      -d '{ \
        "name":"my-runtime-instance", \
        "location":"$LOCATION", \
        "diskEncryptionKeyName":"'"key_path"'" \
      }

    Where:

    • -i instructs curl to output the request and response headers. This makes debugging easier.
    • -H "$AUTH"inserts the value of your AUTH environment variable for the Authorization header.
    • -X POST sets the request type to an HTTP POST.
    • -H "Content-Type:application/json" inserts a header that instructs curl to send your data payload formatted as JSON.
    • -d defines the request payload; it must contain the following:
      • name: Your new instance's name. The name must start with a lowercase letter, can be up to 32 characters long, and can include only lowercase letters, numbers and hyphens. It cannot start or end with a hyphen and must be at least 2 characters long.
      • location is the physical place in which your cluster is hosted. Valid values are any location allowed by Compute Engine (listed here). This example uses us-west1.
      • key_path is the location of the key that you created in Create an encryption key.

    This request can take up to 20 minutes to complete because Apigee must create and launch a new Kubernetes cluster, install the Apigee resources on that cluster, and set up load balancing.

    On a successful request, Apigee initially responds to let you know that the creation is in progress. The response looks like the following:

    {
      "name": "organizations/your_project_id/operations/d811fef5-2e57-4660-845e-5ab4cbe4b4d3",
      "metadata": {
        "@type": "type.googleapis.com/google.cloud.apigee.v1.OperationMetadata",
        "operationType": "INSERT",
        "targetResourceName": "organizations/your_project_id/instances/$LOCATION",
        "state": "IN_PROGRESS"
      }
    }

    If Apigee returns an error, see Troubleshooting.

  4. Wait about 20 minutes. Now is a good time to go warm up another delicious stroopwaffle.
  5. To check the status of your runtime instance creation request, you can do one of the following:
    • (Preferred) Use the Operations API to get the status of the operation. This is preferred because it does not return an error if the operation is in progress. The next API call, however, does return an error in some circumstances.
    • Get a list of existing runtime instances by sending a GET request to the Instances API:
      curl -i -X GET -H "$AUTH" -H "Content-Type:application/json" \
        "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/instances"

      If there are any available runtime instances associated with your Cloud account, Apigee responds to the request with details about each one.

      The response looks like the following:

      {
        "instances": [
          {
            "name": "my-runtime-instance",
            "location": "us-west1",
            "host": "10.16.0.2",
            "port": "443"
          },
          ...
        ]
      }

      Where:

      • name and location are the values you specified in your initial request to create an instance.
      • host is the internal IP address of the cluster's load balancer. Apigee will connect to this when fulfilling API runtime requests.
      • port is the load balancer's port on which Apigee connects.

      If there are no runtime instances available (and Apigee has not yet created a new one for you), Apigee responds with an error.

For more details about creating a runtime instance, including additional context and troubleshooting information, see Step 5: Create a runtime instance.

Step 6: Create an environment

To create an environment and attach it to the runtime on the command line:

  1. Create a new environment with the Environments API by using the following command:
    curl -i -H "$AUTH" -X POST \
      -H "Content-Type:application/json" \
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/environments" \
      -d '{"name":"test"}'

    Where -d contains the data payload that defines your new environment's name.

    Apigee creates the new environment. This operation can take several minutes to complete.

  2. Before you continue, check that Apigee finished creating the new environment by sending a GET request to the Environments API:
    curl -i -H "$AUTH" \
      -H "Content-Type:application/json" \
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/environments"

    Apigee responds with a list of available environments:

    [
      "test"
    ]
  3. Attach the new environment to the runtime instance, as the following example shows:
    curl -i -H "$AUTH" -X POST -H "content-type:application/json"
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/instances/$REGION/attachments"
      -d '{ "environment":"test"}'

    As with the previous command, this command also triggers a long-running operation. Check that the environment has been attached to the runtime before continuing. If it has, you should get a 409 error, as the following example shows:

    {
      "error": {
        "code": 409,
        "message": "the attachment '61a36502-4242-4e6d-4242-0ab2e22c0ffc' already exists for instance
          'us-west1' and environment 'test'",
        "status": "ALREADY_EXISTS"
      }
    }

    Only after you're sure the new environment has been created and attached to the runtime, you can create a new environment group.

  4. Create a new environment group, as the following example shows:
    curl -i -H "$AUTH" -X POST \
      -H "Content-Type:application/json" \
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/envgroups" \
      -d '{ \
        "name": "test-group", \
        "hostnames":["sales.example.com"] \
      }'
  5. Wait for the operation to complete. You can check the status of your new group using a request like the following:
    curl -i -H "$AUTH" -H "Content-Type:application/json"
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/environments/test"
  6. Attach the new environment to the new environment group, as the following example shows:
    curl -i -H "$AUTH" \
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/envgroups/test-group/attachments" \
      -X POST -H "content-type:application/json" \
      -d '{ \
        "environment":"test" \
      }'

    This triggers a long-running operation that can take several minutes to complete. We recommend that you wait a minute or two before continuing to the next step. In the meantime you can check the status of the operation as before.

  7. (Optional) Send a PATCH request to update the host alias, as the following example shows:
    curl -i -H "$AUTH" -X PATCH -H "content-type:application/merge-patch+json" \
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/envgroups/test-group?updateMask=hostnames" \
      -d '{ \
        "hostnames":["test-group-new.sandbox.apigeeks.net"] \
      }'

    This triggers a long-running operation that can take several minutes to complete. We recommend that you wait a minute or two before continuing to the next step. In the meantime you can check the status of the operation as before.

For more details about creating Apigee environments, including additional context and troubleshooting information, see Step 6: Create an environment.

Step 7: Configure routing

Decide whether to allow external access or internal only access:

Access Type Difficulty Level Description of the configuration and deployment process
Internal (recommended) MEDIUM

Allow only internal access to your API.

You download the Hello World proxy from GitHub and then deploy it to your runtime instance. You must then create a new VM inside the network and connect to it. From the new VM, you can send a request to the API proxy.

External HIGH

Allow external access to your API.

You download the Hello World proxy from GitHub and then deploy it to your runtime instance. You can then send a request to the API proxy from your administration machine or any network-enabled machine.

Given the complexity of this process, Apigee recommends that only advanced users carry this out.

Each of these approaches is presented on a tab in the instructions below.

Internal

There are no tasks to perform for this step if you are using the command line to set up an API proxy for internal access only. You can skip to Step 8: Deploy a sample proxy, where you will send a request to your API proxy.

External

This section describes how to configure routing when you're using the command line and you want to allow external access to your API proxy. You must do this before you can send a request from an external client to your Apigee runtime instance.

Difficulty Level: HIGH

This section describes how to set up an Envoy proxy, configure external load balancing, and modify the firewall.

The general process is as follows:

Step 7a: Enable Private Google Access
Step 7b: Set up environment variables
Step 7c: Launch the Envoy proxy
Step 7d: Create firewall rules
Step 7e: Upload credentials
Step 7f: Create a global Load Balancer

Each of these steps is described in the sections that follow.

Step 7a: Enable Private Google Access

Enable Private Google Access for your VPC as described in Enabling Private Google Access.

Step 7b: Set up environment variables

The instructions in this section use environment variables to refer to repeatedly used strings. Apigee strongly recommends that you set these before continuing.

To set up your environment variables, issue the following commands:

MIG_NAME=apigee-envoy-us-west1
VPC_NAME=default
VPC_SUBNET=default
REGION=us-west1        # The same region as your Apigee runtime instance
APIGEE_ENDPOINT=runtime_instance_hostname

You'll use these variables several times during the remaining processes.

Step 7c: Launch the Envoy proxy

You must create and configure an Envoy proxy. You do this by creating an instance template and group. You configure autoscaling for the group and define a port to which external requests can be made.

To launch the Envoy proxy:

  1. Create an instance template by executing the following command:
    gcloud compute instance-templates create $MIG_NAME \
      --project $PROJECT_ID --region $REGION --network $VPC_NAME \
      --subnet $VPC_SUBNET \
      --tags=https-server,apigee-envoy-proxy,gke-apigee-proxy \
      --machine-type n1-standard-1 --image-family centos-7 \
      --image-project centos-cloud --boot-disk-size 20GB \
      --metadata ENDPOINT=$apigee_endpoint,startup-script-url=gs://apigee-5g-saas/apigee-envoy-proxy-release/latest/conf/startup-script-envoy.sh

    As you can see from this command, machines are of type "n1-standard-1". They run CentOS 7 and have 20GB of disk.

  2. Create a managed instance group by executing the following command:
    gcloud compute instance-groups managed create $MIG_NAME \
      --project $PROJECT_ID --base-instance-name apigee-envoy \
      --size 2 --template $MIG_NAME --region $REGION
  3. Configure autoscaling for the group by executing the following command:
    gcloud compute instance-groups managed set-autoscaling $MIG_NAME \
      --project $PROJECT_ID --region $REGION --max-num-replicas 20 \
      --target-cpu-utilization 0.75 --cool-down-period 90
  4. Defined a named port by executing the following command:
    gcloud compute instance-groups managed set-named-ports $MIG_NAME \
      --project $PROJECT_ID --region $REGION --named-ports https:443

Step 7d: Create firewall rules

You must assign an IP address to the Load Balancer and then create rules that allow the Load Balancer to access Envoy.

To create firewall rules:

  1. Reserve an IP address for the Load Balancer:
    gcloud compute addresses create lb-ipv4-vip-1 \
      --project $PROJECT_ID --ip-version=IPV4 --global
  2. Get a reserved IP address by executing the following command:
    gcloud compute addresses describe lb-ipv4-vip-1
      --project $PROJECT_ID --format="get(address)" --global
  3. Create a firewall rule that lets the Load Balancer access Envoy by using the following command:
    gcloud compute firewall-rules create k8s-allow-lb-to-apigee-envoy \
      --description "Allow incoming from GLB on TCP port 443 to Apigee Proxy" \
      --project $PROJECT_ID --network $VPC_NAME --allow=tcp:443 \
      --source-ranges=130.211.0.0/22,35.191.0.0/16 --target-tags=gke-apigee-proxy

    Note that the IP ranges 130.211.0.0/22 and 35.191.0.0/16 are the source IP ranges for Google Load Balancing. This firewall rule allows Google Cloud Load Balancing to make health check requests to the Envoy proxy.

Step 7e: Upload credentials

To upload your credentials to Cloud:

  1. Generate a certificate and key. You can use a self-signed certificate if you want. Your certificate and key can be self-signed, although Apigee does not recommend this in a production environment.
  2. Upload your SSL server certificate and key to GCP by using the gcloud compute ssl-certificates create command:
    gcloud compute ssl-certificates create apigee-ssl-cert \
      --project $PROJECT_ID --certificate=/path/to/tls_certificate \
      --private-key=/path/to/tls_key

Step 7f: Create a global Load Balancer

To create a global Load Balancer:

  1. Create a health check:
    gcloud compute health-checks create https hc-apigee-envoy-443 \
      --project $PROJECT_ID --port 443 --global \
      --request-path /healthz/ingress

    You'll use this health check to ensure that the backend service is running.

  2. Create a backend service called "apigee-envoy-backend" with the following command:
    gcloud compute backend-services create apigee-envoy-backend \
      --project $PROJECT_ID --protocol HTTPS --health-checks hc-apigee-envoy-443
      --port-name https --timeout 60s --connection-draining-timeout 300s --global
  3. Add the Envoy instance group to your backend service with the following command:
    gcloud compute backend-services add-backend apigee-envoy-backend \
      --project $PROJECT_ID --instance-group $MIG_NAME \
      --instance-group-region $REGION \
      --balancing-mode UTILIZATION --max-utilization 0.8 --global
  4. Create a Load Balancing URL map with the following command:
    gcloud compute url-maps create apigee-envoy-proxy-map \
      --project $PROJECT_ID --default-service apigee-envoy-backend
  5. Create a Load Balancing target HTTPS proxy with the following command:
    gcloud compute target-https-proxies create apigee-envoy-https-proxy \
      --project $PROJECT_ID --url-map apigee-envoy-proxy-map \
      --ssl-certificates apigee-ssl-cert
  6. Create a global forwarding rule with the following command:
    gcloud compute forwarding-rules create apigee-envoy-https-lb-rule \
      --project $PROJECT_ID --address lb-ipv4-vip-1 --global \
      --target-https-proxy apigee-envoy-https-proxy --ports 443

You're done! You're now ready to call your API proxy.

For details about configuring routing, including additional context and troubleshooting information, see Step 7: Configure routing.

Step 8: Deploy a sample proxy

To deploy the example proxy using the command line:

  1. Download the Hello World sample proxy from GitHub.

    An API proxy in this form is known as a proxy bundle.

  2. Upload the API proxy bundle to the runtime using the API proxies API, as the following example shows:
    curl -i -H "$AUTH"
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/apis?name=PROXY_BASE_PATH&action=import"
      -X POST -F 'file=@/path/to/file/httpbin_rev1_2020_02_02.zip'
      -H "Content-Type:multipart/form-data"
  3. Deploy the API Proxy to the "test" environment as the following example shows:
    curl -i -H "$AUTH" -X POST
      "https://apigee.googleapis.com/v1/organizations/$PROJECT_ID/environments/test/apis/PROXY_BASE_PATH/revisions/1/deployments"

    You can use $PROXY_BASEPATH for PROXY_BASE_PATH.

  4. If you allowed external access to the API proxy:

    Send a request to the API proxy from any network-enabled machine by executing the following command:

    curl -i -H "Host: test-group.sandbox.apigeeks.net" \
      https://PUBLIC_LOAD_BALANCER_IP/helloworld

    Optionally add the -k option to skip authentication in a testing scenario such as this.

  5. If you allowed internal access only to the API proxy, see Accessing a privately deployed API proxy.

For more information about deploying proxies, including additional troubleshooting information, see Step 8: Deploy a sample proxy.