Import data from other cloud providers

Migration Center lets you import performance and usage data from your infrastructure hosted on other cloud providers, such as Amazon Web Services (AWS) and Azure.

To import data from other cloud providers, perform the following steps:

  1. Download the collection script.
  2. Run the script in your cloud infrastructure and generate the output tables.
  3. Upload the output tables to Migration Center.

Best practices to run the collection script

When you run the collection script, consider the following:

  • Run the collection script by using an instance of AWS CloudShell (when collecting data from AWS) or Azure Cloud Shell (when collecting data from Azure). Using these shells avoids the need to generate, distribute, and maintain authentication keys to run the collection script in other runtime environments.
  • Run the collection script by impersonating an account that has minimal, read-only permissions. Using read-only permissions avoids any accidental modifications to your AWS or Azure resources.
  • The scripts are open source, so if your organization has specific regulatory requirements, review the code to understand exactly what type of data is collected.

Limitations

  • The maximum size for a file upload is 100 Mb.

Download the script

To download the scripts, select one of the following links, depending on your cloud provider:

Run the script

To run the script, follow the instructions in the README file of the repository. The script generates a ZIP file with the tables of collected data. Extract the files from the ZIP file.

The ZIP file contains the following files that you upload to Migration Center:

  • vmInfo.csv
  • perfInfo.csv
  • diskInfo.csv
  • tagInfo.csv

It might also contain other files that you cannot upload to Migration Center, for example the resources.json file. Uploading JSON files to Migration Center results in an error.

Upload the collection script results

To upload the files, follow these steps:

Console

  1. Go to the Data import page.

    Go to Data import

  2. Click Add data > Upload files.

  3. In the Set up file upload section, enter the name of the file upload.

  4. From the File format list, select AWS exports or Azure exports.

  5. Click Select files to upload, then select all the files that you want to upload.

  6. To confirm and start uploading the files, click Upload files.

  7. If the files are validated correctly, to create the assets from the files, click Import data, then click Confirm.

API

  1. Create an import job.

    POST https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs?importJobId=IMPORT_JOB_ID

{
  "asset_source": "projects/PROJECT_ID/locations/REGION/sources/SOURCE_ID"
}

    Replace the following:

    • PROJECT_ID: ID of the project to create the import job.
    • REGION: Region to create the import job.
    • IMPORT_JOB_ID: ID of the new import job.
    • SOURCE_ID: ID of an existing source to be associated with the import job.

  2. Optional: To confirm that the import job was correctly created, get the import job.

    GET https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs/IMPORT_JOB_ID

  3. Create a import data file attached to the import job.

    POST https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs/IMPORT_JOB_ID/importDataFiles?import_data_file_id=DATA_FILE_ID

{
  "format": "IMPORT_JOB_FORMAT"
}

    Replace the following:

    • DATA_FILE_ID: ID of the new data file
    • IMPORT_JOB_FORMAT: Either IMPORT_JOB_FORMAT_EXPORTED_AWS_CSV or IMPORT_JOB_FORMAT_EXPORTED_AZURE_CSV

  4. Get the data file.

    GET https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs/IMPORT_JOB_ID/importDataFiles/DATA_FILE_ID

  5. Copy the URL from the signedUrl field from the response.

  6. Upload a file to the copied URL.

    PUT -H 'Content-Type: application/octet-stream' --upload-file UPLOAD_FILE_PATH 'COPIED_URL'

    Replace the following:

    • UPLOAD_FILE_PATH: The local path of the file that is uploaded.
    • COPIED_URL: The copied signed URL from the previews steps.

  7. Optional: Repeat steps 3-6 to create more data files under the same import job.

  8. Optional: Get all data files of an import job.

    GET https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs/IMPORT_JOB_ID/importDataFiles

  9. Validate the import job.

    POST https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs/IMPORT_JOB_ID:validate

  10. Get the import job and view the validation report. If the state is READY, it is possible to continue to the next steps. Otherwise, you need to fix the job or specific files, this might require deleting files or uploading new ones.

    GET https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs/IMPORT_JOB_ID

  11. Run the import job.

    POST https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs/IMPORT_JOB_ID:run

  12. Get the import job and view the execution report. If the state is COMPLETED, the job was executed successfully. Otherwise, if the job failed and in a terminal state, the execution report includes the errors. Create a new import job and apply the required changes.

    GET https://migrationcenter.googleapis.com/v1alpha1/projects/PROJECT_ID/locations/REGION/importJobs/IMPORT_JOB_ID

If you experience problems with your file upload, see how to troubleshoot common error messages.

Review the upload

After you upload your files, Migration Center validates them to check if they are formally correct. If the validation is successful, Migration Center then processes the data to create new assets, or update existing ones. You can check the status of your upload jobs from the Google Cloud console.

To review the status of your upload job, follow these steps:

  1. Go to the Data import page, then click File uploads.
  2. From the list of file imports, select the upload you want to review.
  3. On the file upload page, under Data import progress, review the Status for your uploaded files.

You can see your files in one of the following statuses.

List of statuses

  • Completed. The import of your file was successful.
  • Ready. Your file passed validation, and is ready to be imported.
  • Pending. Your file is waiting for another file to finish processing.
  • Running. Your file is processing.
  • Validating. Your file is in the validation stage.
  • Failed validation. Your file contains some errors. Fix the errors then try to upload your file again.
  • Failed. Your file couldn't be imported.

Review error details

If you encounter errors after importing your files, you can review the error message directly in the Google Cloud console.

To review the errors of your upload job, follow these steps:

  1. Go to the Data import page, then click File uploads.
  2. Click the import file that shows a warning or error. The import file details page shows the complete list of errors.

On the import file details page, you can review the assets and data fields that cause the errors, and view a description of the error. Errors in the file import job might prevent Migration Center from creating new assets or update existing ones with the new changes. To fix the error, edit your files and create a new file import job to upload them again.

For more information about the possible error messages, see Troubleshoot file import errors.

What's next