Administering Transfer for on-premises agents

This document describes how to download, install, and administer Transfer service for on-premises data agents on your servers.

Installing and running the on-premises agent

Important: Users that create an Transfer for on-premises job can choose any source directory that is accessible by the agent, regardless of the data access restrictions in place in your data center. To prevent unauthorized access to data, select users that are granted access to the transfer project and that are permitted to run the user agent in alignment with your organization's security and access policies.

To install and run the on-premises agent:

1. Go to the Transfer service for on-premises data Web Console page in the Google Cloud Console.

   Go to the Transfer service for on-premises data Page

2. Click Connection Settings.

3. Click the Install Agent button.

   Installation instructions for the agent are displayed. Follow the instructions displayed to install and run the agent.

   Note: For more information about the agent's command-line options, see Agent command-line options

   You can install more than one agent for each machine. For more information about determining how many agents to run, see Sizing agents.

Confirming agents are connected

After you install your on-premises agents, you can verify that they're connected to your Transfer service for on-premises data job.

To confirm that your agents are connected:

1. Go to the Transfer service for on-premises data Web Console page in the Google Cloud Console.

   Go to the Transfer service for on-premises data Page

2. Click Connection Settings.

   Connected agents are displayed.

   Note: If you installed agents before submitting your first transfer job, this page may not display your agents.

Troubleshooting agent connection

If you don't see any of your agents displayed in Transfer service for on-premises data Web Console within ten minutes after installing an agent, then do the following to confirm that you can connect to Cloud Storage APIs and Pub/Sub APIs:

1. For Cloud Storage APIs, run the following command from the same machine as the transfer agent to test moving a file into your Cloud Storage bucket: gsutil cp test.txt gs://my-bucket where my-bucket is the name of your Cloud Storage bucket. If the transfer works, the test is successful.
2. For Pub/Sub APIs, run the following command from the same machine as the transfer agent to confirm that you can find existing Pub/Sub topics: gcloud pubsub topics list --project=project-id where project-id is the Google Cloud project name. If a list of Pub/Sub topics is displayed, the test is successful.

Monitoring agents

You can monitor agents from either the Google Cloud Console, or use Cloud Monitoring alerts to notify you based on agent behavior.

Monitoring agents from the Google Cloud Console

To monitor agents from the Google Cloud Console:

1. Go to the Transfer service for on-premises data Connection Settings page in the Google Cloud Console.

   Go to the Transfer service for on-premises data Connection Settings Page

   All Transfer service for on-premises data transfer agents and their current connection status are displayed.

Using Cloud Monitoring to monitor agents

To use Cloud Monitoring, you create a Monitoring alert on either of the following Google Cloud metrics:

Metric name	What it describes	Suggested uses
storagetransfer.googleapis.com/agent/transferred_bytes_count	Measures how quickly a specific agent is moving data (across all jobs that it services) at a point in time.	Alert for dips in performance.
storagetransfer.googleapis.com/agent/connected	A boolean that is True for each agent that Google Cloud received a recent heartbeat message from.	Alert for failing agents Failing below a number of agents that you consider necessary for reasonable performance Signal an issue with agent machines

Stopping an agent

To stop an agent:

1. Find your agent's Docker container ID:

   1. Go to the Transfer service for on-premises data Web Console page in the Google Cloud Console.

      Go to the Transfer service for on-premises data Page

   2. Click Connection Settings.

      Connected agents are displayed. The Agent ID field lists each agent's ID in the format:

      agent-prefix:hostname:container-ID.

   3. Note the Docker container ID.

2. Log into the server.

3. Run sudo docker stop container-id in the agent's terminal window, where

   container-id is the Docker container ID you previously noted.

   Note: For more information on the docker stop command, see docker stop