This page describes how to prepare for, install, and configure your Transfer Appliance.
Prepare the destination Cloud Storage bucket
To store your data in Cloud Storage, you must prepare a bucket. Buckets are the basic containers that hold your data within Cloud Storage.
We'll use two service accounts to move your data from Transfer Appliance to the destination Cloud Storage bucket that you prepare. Service accounts are special accounts that are used by an application, not a person, to do work. In this case, the service accounts allow Transfer Appliance to use Cloud Storage resources on your behalf to copy data from the appliance to your Cloud Storage bucket. You'll grant these accounts the necessary roles to copy data from the appliance to your Cloud Storage bucket.
To prepare the destination Cloud Storage bucket, follow these steps:
In an email titled Google Transfer Appliance Prepare Destination Bucket, Transfer Appliance Team provides you the following service accounts:
A session service account that is tied to this particular transfer. It looks like the following example:
SESSION_ID@transfer-appliance-zimbru.iam.gserviceaccount.com
In this example,
SESSION_ID
is the session ID for this particular transfer.A static service account that is tied to the Transfer service for on-premises data service, which we use to transfer data from the appliance to your Cloud Storage bucket:
cloud-ingest-dcp@cloud-ingest-prod.iam.gserviceaccount.com
Note the service accounts for the next steps.
The service accounts allow Transfer Appliance to manipulate Google Cloud resources on your behalf, namely to copy data from the appliance to Cloud Storage. You'll grant these accounts the necessary roles to copy data from the appliance to your Cloud Storage bucket.
Cloud Storage buckets are tied to Google Cloud projects. To create a Cloud Storage bucket, you'll choose a project to place the bucket in. Run the following command to list the available project IDs:
gcloud projects list
If you don't have a Cloud Storage bucket, do the following to create one:
Google Cloud Console
Open the Cloud Storage browser in the Google Cloud Console.
Click Create bucket to open the bucket creation form.
Enter your bucket information and click Continue to complete each step:
Specify a Name, subject to the bucket naming requirements.
Select a Default storage class for the bucket. The default storage class is assigned by default to all objects uploaded to the bucket. Next, select a Location for the bucket data.
Select an Access control model to determine how you control access to the bucket's objects.
Optionally, you can add bucket labels, set a retention policy, and choose an encryption method.
Click Done.
Command line
Use the
gsutil mb
command:gsutil mb -b on -l LOCATION -p PROJECT_ID gs://BUCKET_NAME
In this example:
LOCATION
: Your desired Cloud Storage bucket location.PROJECT_ID
: The project ID to create your bucket under.BUCKET_NAME
: The name of the bucket you're creating, subject to bucket naming requirements.
Do the following to grant the Transfer Appliance service accounts permission to use your bucket.
Google Cloud Console
- Open the Cloud Storage browser in the Google Cloud Console.
Open the Cloud Storage browser Click the Bucket overflow menu (
) associated with the bucket to which you are granting member a role.
Choose Edit bucket permissions.
Click the + Add members button.
In the New members field, enter the following identities:
The session service account. It looks like the following example:
SESSION_ID@transfer-appliance-zimbru.iam.gserviceaccount.com
In this example,
SESSION_ID
is the session ID for this particular transfer.The static service account. It looks like the following example:
cloud-ingest-dcp@cloud-ingest-prod.iam.gserviceaccount.com
From the Select a role drop-down menu, select the Storage Admin role.
The roles you select appear in the pane with a short description of the permissions they grant.
Click Save.
Command line
Use the
gsutil iam ch
command:gsutil iam ch \ serviceAccount:SESSION_ID@transfer-appliance-zimbru.iam.gserviceaccount.com:roles/storage.admin \ serviceAccount:cloud-ingest-dcp@cloud-ingest-prod.iam.gserviceaccount.com:roles/storage.admin \ gs://BUCKET_NAME
In this example:
SESSION_ID
: The session ID for this particular transfer.BUCKET_NAME
: The name of the bucket you're creating.
- Open the Cloud Storage browser in the Google Cloud Console.
Prepare the Cloud KMS key
Transfer Appliance secures your data on the appliance by encrypting the data before you ship the appliance back to us. A Cloud Key Management Service (Cloud KMS) public key is used to encrypt your data on Transfer Appliance, and a private key is used to decrypt your data. We'll use the session service account from Prepare the destination Cloud Storage bucket to upload the data from the appliance to your Cloud Storage bucket.
To enable us to upload data from the appliance to your Cloud Storage bucket, you must prepare a Cloud KMS asymmetric decryption key and add the session service account to the key. The session service account, once granted to the asymmetric decryption key, uses the key to decrypt your data and copy it to Cloud Storage.
To prepare Cloud KMS keys, do the following:
If you don't have a Cloud Key Management Service key ring, do the following to create one:
Google Cloud Console
Go to the Cryptographic Keys page in the Cloud Console.
Click Create key ring.
In the Key ring name field, enter the desired name for your key ring.
From the Key ring location dropdown, select a location like
"us-east1"
.Click Create.
Command line
gcloud kms keyrings create KEYRING --location=LOCATION --project=PROJECT_ID
In this example:
LOCATION
: The Cloud Key Management Service location for the key ring. For example,global
.KEYRING
: The key ring's name.PROJECT_ID
: The Google Cloud project ID that your storage bucket is under.
Create an asymmetric decryption key by doing the following:
Google Cloud Console
Go to the Cryptographic Keys page in the Cloud Console.
Click the name of the key ring that you want to create a key for.
Click Create key.
In the What type of key do you want to create? section, choose Generated key.
In the Key name field, enter the name for your key.
Click the Protection level dropdown and select Software.
Click the Purpose dropdown and select Asymmetric decrypt.
Click the Algorithm dropdown and select 4096 bit RSA - OAEP Padding - SHA256 Digest
Click Create.
Command line
Run the following command to create an asymmetric decryption key:
gcloud kms keys create KEY --keyring=KEYRING \ --location=LOCATION --purpose=asymmetric-encryption \ --default-algorithm=rsa-decrypt-oaep-4096-sha256 \ --project=PROJECT_ID
In this example:
KEY
: The name of the Cloud Key Management Service key. For example,ta-key
.KEYRING
: The key ring's name.LOCATION
: The Cloud Key Management Service location for the key ring. For example,global
.PROJECT_ID
: The Google Cloud project ID that your storage bucket is under.
Add the session service account as a member to the asymmetric key by doing the following:
Google Cloud Console
Go to the Cryptographic Keys page in Google Cloud Console.
Click on the key ring that contains your asymmetric key.
Select the checkbox for the asymmetric key.
In the Info panel, click Add member.
Add members is displayed.
In the New members field, enter the session service account provided by Transfer Appliance Team. It looks like the following example:
SESSION_ID@transfer-appliance-zimbru.iam.gserviceaccount.com
In this example,
SESSION_ID
is the session ID for this particular transfer.In the Select a role field, add the Cloud KMS CrytoKey Public Key Viewer role.
Click Add another role.
In the Select a role field, add the Cloud KMS CryptoKey Decrypter role.
Click Save.
Command line
Run the following commands to grant the session service account the
roles/cloudkms.cryptoKeyDecrypter
role:gcloud kms keys add-iam-policy-binding KEY \ --keyring=KEYRING --location=LOCATION \ --member=serviceAccount:SESSION_ID@transfer-appliance-zimbru.iam.gserviceaccount.com \ --role=roles/cloudkms.cryptoKeyDecrypter
In this example:
KEY
: The name of the Cloud Key Management Service key. For example,ta-key
.KEYRING
: The key ring's name.LOCATION
: The Cloud Key Management Service location for the key ring. For example,global
.PROJECT_ID
: The Google Cloud project ID that your storage bucket is under.SESSION_ID
: The session ID for this particular transfer.
Run the following commands to grant the session service account the
roles/cloudkms.publicKeyViewer
role:gcloud kms keys add-iam-policy-binding KEY \ --keyring=KEYRING --location=LOCATION \ --member=serviceAccount:SESSION_ID@transfer-appliance-zimbru.iam.gserviceaccount.com \ --role=roles/cloudkms.publicKeyViewer
In this example:
KEY
: The name of the Cloud Key Management Service key. For example,ta-key
.KEYRING
: The key ring's name.LOCATION
: The Cloud Key Management Service location for the key ring. For example,global
.PROJECT_ID
: The Google Cloud project ID that your storage bucket is under.SESSION_ID
: The session ID for this particular transfer.
Provide Transfer Appliance Team with bucket configuration data
To enable us to secure your data on the appliance, provide us with the encryption key version resource name used to encrypt your data on the appliance before you ship it to us. We also need the Cloud Storage bucket name to copy your data to.
To provide the Transfer Appliance Team the required information about your Cloud Storage bucket, do the following:
Obtain your asymmetric key's path by doing the following:
Google Cloud Console
Go to the Cryptographic Keys page in the Google Cloud Console.
Click on the key ring that contains your asymmetric decryption key.
Click on the asymmetric decryption key's name.
Select the key version that you want, and click More more_vert.
Click Copy Resource Name.
An example of the key format is:
projects/PROJECT_ID/locations/LOCATION/keyRings/KEYRING/cryptoKeys/KEY/cryptoKeyVersions/VERSION_NUMBER
In this example:
PROJECT_ID
: The Google Cloud project ID that your storage bucket is under.LOCATION
: The Cloud Key Management Service location for the key ring.KEYRING
: The key ring's name.KEY
: The name of the Cloud Key Management Service key.VERSION_NUMBER
: The key's version number.
The Transfer Appliance Team requires the entire key path, including the version number, so they can apply the correct key to your data.
Command line
Run the following command to list your asymmetric key's full path, including its version number:
gcloud kms keys versions list --keyring=KEYRING \ --key=KEY --location=LOCATION \ --project=PROJECT_ID
In this example:
KEYRING
: The name of your key ring.KEY
: The name of your asymmetric key.LOCATION
: The Google Cloud location of the key ring.PROJECT_ID
: The Google Cloud project ID that your storage bucket is under.
The following sample response resembles the output that is returned:
NAME STATE projects/PROJECT_ID/locations/LOCATION/keyRings/KEYRING/cryptoKeys/KEY/cryptoKeyVersions/VERSION_NUMBER ENABLED
In this example:
PROJECT_ID
: The Google Cloud project ID that your storage bucket is under.LOCATION
: The Cloud Key Management Service location for the key ring.KEYRING
: The key ring's name.KEY
: The name of the Cloud Key Management Service key.VERSION_NUMBER
: The key's version number.
The Transfer Appliance Team requires the string under
NAME
that ends in/cryptoKeyVersions/VERSION_NUMBER
, whereVERSION_NUMBER
is your key's version number.Complete the form in the email titled Google Transfer Appliance Access Credentials sent by the Transfer Appliance Team for the following information:
- Your asymmetric key's path, including its version number.
- The Bucket Name for the bucket you want data copied to.
- Optional: Enter an Object prefix. Without an object prefix, objects are transferred
to Cloud Storage with the source's path, not including the root path, before
the file name on the filesystem. For example, if you have the following files:
/source_root_path/file1.txt
/source_root_path/dirA/file2.txt
/source_root_path/dirA/dirB/file3.txt
file1.txt
dirA/file2.txt
dirA/dirB/file3.txt
/
character of the destination bucket name and before any path names that the object was transferred from, not including the source's root path. This can help you distinguish between objects transferred from other transfer jobs. The following table demonstrates several examples of object prefixes and their resulting object names in Cloud Storage, if the source object's path is/source_root_path/sub_folder_name/object_name
:Prefix Destination object name None /destination_bucket/sub_folder_name/object_name
prefix/
/destination_bucket/prefix/sub_folder_name/object_name
Configure IP network ports
Work with your network administrator to configure your network's IP ports for Transfer Appliance.
Verify the package contents
Before connecting Transfer Appliance, we strongly recommend that you verify that the package arrived intact, with all required cables and equipment. We'll ensure you have everything necessary for a successful data transfer.
After receiving Transfer Appliance, verify the that package contents are intact by doing the following:
Confirm that the shipping case and tamper-evident seals are intact.
Contact Transfer Appliance Team if any of the following items are missing:
Item Description NEMA 5-15p to C13 power cable C14 to C13 power cable, for connections to the power distribution unit (PDU) Category 6 (Cat6) network cable QSFP+ Twinax copper network cable QSFP+ to 4xSFP+ network cable USB-to-serial adapter cable – use only if instructed by Transfer Appliance Team Pin-to-socket serial adapter Wire tamper-evident tag Tie-on tag Shipping-label pouch Transfer Appliance
Validating the appliance
Before connecting the appliance to your network, you must run a small application on your laptop to validate that the appliance hasn't been tampered with during shipping.
The Transfer Appliance Attestation Application supports the following 64-bit operating systems:
- Linux kernel 2.6.23 or later
- Microsoft Windows Server 2012 or later
- Microsoft Windows 10
- Apple macOS 10.11 or later
Do the following to validate the appliance:
Find a location to place the appliance. Acceptable locations include the following:
- The floor
- A desk
Connect the following cables:
- The supplied Cat6 cable to the left RJ45 (management) port on the appliance, and to an open RJ45 port on your laptop or workstation.
- The supplied power cable to the power socket on the appliance, and to a power socket on a power distribution unit (PDU).
Power on Transfer Appliance.
Use
ping
to verify that you can contact the appliance through its management port:ping 169.254.20.1
The following sample response resembles the output that is returned:
PING 169.254.20.1 (169.254.20.1) 56(84) bytes of data. 64 bytes from 169.254.20.1: icmp_seq=1 ttl=64 time=0.060 ms 64 bytes from 169.254.20.1: icmp_seq=2 ttl=64 time=0.039 ms 64 bytes from 169.254.20.1: icmp_seq=3 ttl=64 time=0.039 ms ^C --- 169.254.20.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2045ms rtt min/avg/max/mdev = 0.039/0.046/0.060/0.009 ms
Do the following to download the Transfer Appliance Attestation Application to your laptop:
Microsoft Windows
Download the Transfer Appliance Attestation Application to your laptop.
Open a Command Prompt, and navigate to the location where you downloaded the Transfer Appliance Attestation Application.
Run the following command on your laptop to validate the appliance:
taattestator_x86_64-windows.exe
Linux
On your laptop, open a terminal app.
Run the following command on your laptop to download the Transfer Appliance Attestation Application:
wget https://storage.googleapis.com/transferappliance/attestator/taattestator_x86_64-linux
Run the following commands on your laptop to validate the appliance:
chmod 0777 taattestator_x86_64-linux ./taattestator_x86_64-linux
Apple macOS
Open the Terminal app.
Run the following command on your laptop to download the Transfer Appliance Attestation Application:
curl -O https://storage.googleapis.com/transferappliance/attestator/taattestator_x86_64-darwin
Run the following commands on your laptop to validate the appliance:
chmod 0777 taattestator_x86_64-darwin ./taattestator_x86_64-darwin
Complete the form in the email titled Google Transfer Appliance Delivered and Validation Steps. Enter the Transfer Appliance Attestation Passcode returned by the appliance in the form.
If the device was not tampered with during shipping, Transfer Appliance Team provides the appliance login credentials.
If the device was tampered with during shipping, Transfer Appliance Team provides you with return shipping instructions, and arranges for another appliance to ship to you.
Connecting to Transfer Appliance
You need the login details for the appliance to connect to it. The Transfer Appliance Team provides the appliance login details in exchange for a Transfer Appliance Attestation Passcode that indicates the appliance wasn't tampered with during shipping.
After you have the login details, follow these steps to connect the appliance to your network:
Connect one end of a Cat6 cable to the right network port on the appliance, highlighted in the image below, and the other end to an open RJ45 port that's connected to your network.
- Connect to the appliance:
Windows
- Open Putty.
- For Connection type, select SSH.
-
In the Host Name field, enter:
169.254.20.1
- For Connection type, verify that SSH is selected.
- Click Open.
- Enter the username provided by Transfer Appliance Team when prompted.
- Enter the password provided by Transfer Appliance Team when prompted.
Linux
-
Run the following command:
ssh USERNAME@169.254.20.1
Replacing
USERNAME
with the username provided by Transfer Appliance Team. - Enter the Transfer Appliance password when prompted.
Mac
-
Run the following command:
ssh USERNAME@169.254.20.1
Replacing
USERNAME
with the username provided by Transfer Appliance Team. - Enter the Transfer Appliance password when prompted.
Configure appliance software
Before you transfer data to Transfer Appliance you must verify the status of the appliance, and configure the software on the appliance to work on your network.
Do the following to configure the appliance's software:
Verify the status of the appliance by running the following command from your laptop or workstation connected to the appliance:
ta status
The following sample response resembles the output that is returned:
You are ready to configure the appliance. Next steps: » Configure the appliance using one of the following commands: > ta config --data_port=RJ45 --ip=dhcp > ta config --data_port=QSFP --ip=dhcp » To set a static IP address and netmask, use the '--ip' flag. For example: > ta config --data_port=RJ45 --ip=192.168.0.100/24 > ta config --data_port=QSFP --ip=192.168.0.100/24
Configure the appliance. You can configure the appliance to use DHCP to obtain an IP address automatically, or you can configure the appliance to use a static IP address.
Run the following command to configure the appliance:
DHCP
ta config --data-port=PORT --ip=dhcp
Replace
PORT
with the data port you are using on the appliance, eitherRJ45
orQSFP
.Static IP
ta config --data-port=PORT --ip=IP_ADDRESS/NETMASK
Replace the following:
PORT
: The data port you are using on the appliance, eitherRJ45
orQSFP
.IP_ADDRESS
: the IP address for the appliance, ordhcp
for automatic IP address configuration.NETMASK
: The netmask in CIDR notation. Not required if you are using DHCP.
The following sample response resembles the output that is returned:
Configuring, encrypting, and mounting data partition... Verified partition settings for "/mnt/ta_metadata". Partition key generated and encrypted. Verifying partition settings and mounting data partition. This may take several minutes to finish... Mounted partition "/mnt/ta_data". Verified partition settings for "/mnt/ta_data". Data partition is mounted. Network is configured. Configured NFS share "/mnt/ta_data". NFS share "/mnt/ta_data" is configured. Appliance has been successfully configured. You can begin copying data. Use these commands to mount the NFS share from your client: > sudo mkdir /mnt/data > sudo mount 192.168.0.100:/mnt/ta_data /mnt/data
Copying data
To copy data, you mount the appliance to your data source and use your preferred copy utility to transfer data from your data source to Transfer Appliance.
Transfer Appliance supports the following methods to copy data to the appliance:
- Using SCP or SFTP for Microsoft Windows, Linux, and Apple macOS.
- using NFS share For Linux and Apple macOS.
Using SCP or SFTP to copy data
To copy data to the appliance using SCP or SFTP, follow these steps:
Microsoft Windows
Download an SCP or SFTP client that supports UTF-8 filenames, such as WinSCP.
Using the SCP tool, connect to the appliance using the following settings:
- File protocol: SFTP
- Host name: The IP address for the appliance.
- Port number: 22
- Username: The appliance username provided by Transfer Appliance Team.
- Password: The appliance password provided by Transfer Appliance Team.
Linux
Run the following command:
scp PATH_TO_FILES USERNAME@IP_ADDRESS:/mnt/ta_data
Replace the following:
PATH_TO_FILES
: The path to the files you are copying.USERNAME
: The appliance username provided by Transfer Appliance Team.IP_ADDRESS
: The IP address for the appliance.
When prompted, enter the appliance password provided by Transfer Appliance Team.
Apple macOS
Run the following command:
scp PATH_TO_FILES USERNAME@IP_ADDRESS:/mnt/ta_data
Replace the following:
PATH_TO_FILES
: The path to the files you are copying.USERNAME
: The appliance username provided by Transfer Appliance Team.IP_ADDRESS
: The IP address for the appliance.
When prompted, enter the appliance password provided by Transfer Appliance Team.
Using NFS share to copy data
To copy data to the appliance using NFS share, follow these steps:
Run the following commands on your data source to mount Transfer Appliance:
Linux
sudo mkdir /mnt/data
sudo mount -o vers=4 IP_ADDRESS:/mnt/ta_data /mnt/data
Replace
IP_ADDRESS
with the IP address for the appliance.
Apple macOS
cd ~
mkdir ta_data
sudo mount -t nfs -o vers=4,resvport,nolocks IP_ADDRESS:/mnt/ta_data ~/ta_data
Replace
IP_ADDRESS
with the IP address for the appliance.
Use a copy utility of your choice to copy data to the appliance.
Copying data is a long-running operation. On Apple macOS or Linux, we recommend using
tmux
orscreen
to ensure the copy process survives logout or network disconnects.
Monitoring copy status
While you're transferring data, you can monitor the progress of your transfer.
To monitor the progress of Transfer Appliance, do the following:
connect to the appliance and run
ta status
.The following sample response resembles the output that is returned:
You are ready to copy data to the appliance. Next steps: » Use these commands to mount the NFS share from your client: > sudo mkdir /mnt/data > sudo mount 192.168.0.100:/mnt/ta_data /mnt/data » When done copying, finalize the appliance with this command: > ta finalize
To view additional information, run the following command:
ta status --verbose
The following sample response resembles the output that is returned:
Checking the state of the appliance: » The encrypted partition key is present. » The data partition is mounted and shared. You are ready to copy data to the appliance. Data partition: » Mount path: /mnt/ta_data » Used space: 2.6M » Available space: 919M » Used inodes: 11 » Available inodes (required to create new files): 64k Next steps: » Use these commands to mount the NFS share from your client: > sudo mkdir /mnt/data > sudo mount 192.168.0.100:/mnt/ta_data /mnt/data » When done copying, finalize the appliance with this command: > ta finalize
Finalizing copied data
Finalizing prepares the appliance for shipping to Google by removing the decryption key, rendering the data inaccessible until it reaches Google. When the decryption key is removed, you won't be able to copy additional data to the appliance without deleting all previously copied data.
To finalize the copied data, follow these steps:
Run the command
ta finalize
and note the passcode returned in the output.The following sample response resembles the output that is returned:
Finalizing prepares the appliance for shipping by removing the decryption key, making the data inaccessible until it reaches Google. Once finalized, there is no way to access your data or add more data without deleting everything on the drive. Are you ready to finalize the appliance and ship it back to Google? (y/n) y Umounted data partition and disabled automount. Finalize step is complete. This passcode must be used to get a return shipping label: NNNN If you forget your passcode, run 'ta status' to display it again. TA:~$ ta status You are ready to ship the appliance back to Google. Next steps: » Use this passcode when requesting a shipping label: NNNN » Contact the appliance team at data-support@google.com.
Enter the passcode returned by the
ta finalize
command in the form included in the email titled Google Transfer Appliance Return Instructions sent by the Transfer Appliance Team.To view the passcode again, run
ta status
.The Transfer Appliance Team responds with the shipping label for the Transfer Appliance for the given passcode.
Once you receive the shipping label, turn off the appliance and disconnect all cables to it.
Packaging and returning the appliance
After transferring and finalizing your data to Transfer Appliance, you prepare the appliance for shipping. This includes printing and applying a shipping label, gathering the appliance's cables, packing the appliance, and contacting your shipper to return the appliance.
You use the same shipping case that Transfer Appliance arrived in
Do the following to return the appliance:
Print the shipping label.
Place the following in the shipping case's cable compartment:
Item Description NEMA 5-15p to C13 power cable C14 to C13 power cable, for connections to the power distribution unit (PDU) Category 6 (Cat6) network cable QSFP+ Twinax copper network cable QSFP+ to 4xSFP+ network cable USB-to-serial adapter cable – use only if instructed by Transfer Appliance Team Pin-to-socket serial adapter Place the Transfer Appliance in the shipping case.
Close the lid to the shipping case and secure the latches.
Secure the shipping case with a wire tamper-evident tag by following these steps:
Insert the wire tamper-evident tag through the shipping case's D-rings.
Insert the end of the wire into the lock.
Draw the tamper-evident tag through the tag lock, until the D-rings are secured.
Affix a tie-on tag to the shipping-case handle.
Affix a shipping-label pouch to the tie-on tag.
Insert the shipping return label in the pouch. Ensure that the return address and barcodes are visible.
If necessary, contact your shipper to schedule a pick-up.
Validating data in the destination Cloud Storage bucket
After we receive your appliance, we'll transfer the data from the appliance to your Cloud Storage destination bucket. When we complete copying your data to your destination Cloud Storage bucket, we send you an email titled Google Transfer Appliance Your Data Transfer Has Been Completed. After you receive our email we recommend that you validate the data we transferred from the appliance to your Cloud Storage bucket.
Do the following to validate your data:
List the objects in your bucket. If you provided an object prefix when providing bucket configuration details, the objects are displayed after the prefix.
Verify that the data you transferred to the appliance is listed in your bucket.
If you have questions, contact data-support@google.com.