Before you create a file system transfer job, you'll need to enable required APIs and install Docker.
If you receive errors while performing first-time setup, confirm that the user you logged in with has permissions required to perform the set up steps. In many cases, these permissions are not available to all users, and you may need to contact a project administrator for assistance.
To use Storage Transfer Service for file system transfers, you need:
A POSIX-compliant source and/or destination.
TCP ports 80 (HTTP) and 443 (HTTPS) open for outbound connections.
The transfer agent machine requirements are listed below:
To use other Linux operating systems, see Docker Enterprise.
At least 8 GB of memory per container and at least four CPUs to be most effective.
Scaling restrictions on jobs and agents
Storage Transfer Service has the following scale restrictions on transfer jobs and agents:
- Fewer than one billion files per job
- 100 agents or fewer per transfer project
- Bandwidth cap must be over 1MBps
Install Docker Community Edition on a physical or virtual Linux machine:
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo systemctl enable docker
If you encounter an installation error, see Troubleshooting.
Allow Docker to authenticate
To allow the Docker container to authenticate with your
credentials, create a Docker volume containing a file with your application
default credentials by running the following command:
sudo docker run -ti --name gcloud-config google/cloud-sdk gcloud auth application-default login