The migration procedure uses the gcloud CLI to configure and
actuate the migration steps. If required, install the gcloud CLI by
following instructions on the Install the gcloud CLI page.
Source-specific configuration
MongoDB on Compute Engine
A MongoDB database that runs on Compute Engine VMs in a self-managed
cluster isn't normally exposed to the public internet. The migration
procedure will use
Private Service Connect in Datastream
to connect the Datastream pipeline to your source database.
Note the Configuration ID parameter of the created configuration. You
will use it in later stages to set up required environment variables.
Azure Cosmos DB
Make sure that
Azure CLI
is installed on your computer.
Amazon DocumentDB
Amazon DocumentDB clusters are not directly accessible from outside the Amazon
VPC network. To connect to a DocumentDB cluster you will have to provision
an EC2 instance within the Amazon VPC network, and use that instance as a
Bastion Host for an SSH tunnel.
To establish a connection to the DocumentDB cluster you will have to obtain
the private SSH key for the EC2 instance as well as the appropriate
Certificate Bundle for the region in which your cluster is deployed. Visit the
Certificate bundles by AWS Region
resource page to download the appropriate bundle in PEM format.
Note that the Certificate Bundle contains multiple certificates. You have to
extract a single certificate to set up the a Datastream connection. It is
recommended that you validate the DocumentDB connectivity manually to ensure
that you have both a valid SSH key and a valid DocumentDB certificate from
the regional bundle. The DocumentDB developer guide provides commandline examples for establish a direct connection from outside
the VPC.
Configure a destination Firestore with MongoDB compatibility database
Make sure that your project has a Firestore with MongoDB
compatibility database where you will migrate data
from your source Mongo database. For more information about creating a
database, see
Create and manage databases.
For the purpose of this migration, we recommend to
create a username and password
for the Firestore with MongoDB compatibility database to
use with the SCRAM-SHA-256 authentication protocol. This username can be
safely deleted after the migration completes, or you may choose to keep
using these credentials to connect your Mongo clients to your new
Firestore with MongoDB compatibility database.
You will use the name of this Firestore with MongoDB
compatibility database and the user credentials in later steps.
Adjust the oplog window size of the source database
We recommend to adjust the oplog window of your source Mongo database to 3 days
worth of total write traffic to this database. If the rate of write traffic
exceeds the rate at which Datastream can consume changes from
your database, this adjustment will prevent data loss.
The value might need to be adjusted further, depending on the pattern of the
traffic and the volume of peak traffic. For example, if a week's worth of
traffic is written in a short period of time, then Datastream
might not capture the changes from the oplog fast enough before changes fall
out of the oplog window. In this case you might need to resize the oplog window
to 7 days worth of total write traffic.
Create a Cloud Storage bucket
Create a new Cloud Storage bucket in the following way:
Choose a Cloud Storage region where the
migration pipeline will run. We recommend to use the region where your
destination Firestore with MongoDB compatibility database
is located.
Choose the name for this bucket. Example: mongo-migration-bucket. You will
use this name in later steps.
Create a new Cloud Storage bucket with the chosen name and in the
chosen region by following instructions provided on the
Create a bucket page in
the Cloud Storage documentation.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[],[],null,["# Configure resources for migration\n=================================\n\nThis page describes the preparation part of the\n[migration process](/firestore/mongodb-compatibility/docs/migrate-data)\nwhere you configure resources for the migration.\n\nAt this stage, you do the following:\n\n1. Install command-line tools required for running commands at later stages of the migration process.\n2. Configure Firestore with MongoDB compatibility database.\n3. (Optional) Adjust the operation log (oplog) settings on your MongoDB compatible source.\n4. Create a Cloud Storage bucket for intermediate data storage.\n\nConfigure IAM permissions\n-------------------------\n\nYour account requires appropriate IAM roles in all\nservices used in the migration process:\n\n- [Datastream roles](/iam/docs/roles-permissions/datastream)\n- [Dataflow roles](/dataflow/docs/concepts/access-control)\n- [Cloud Storage roles](/storage/docs/access-control/iam-roles)\n- [Datastore roles](/datastore/docs/access/iam)\n\nInstall command-line tools\n--------------------------\n\nThe migration procedure uses the gcloud CLI to configure and\nactuate the migration steps. If required, install the gcloud CLI by\nfollowing instructions on the [Install the gcloud CLI](/sdk/docs/install) page.\n\nSource-specific configuration\n-----------------------------\n\n### MongoDB on Compute Engine\n\nA MongoDB database that runs on Compute Engine VMs in a self-managed\ncluster isn't normally exposed to the public internet. The migration\nprocedure will use\n[Private Service Connect in Datastream](/datastream/docs/psc-interfaces)\nto connect the Datastream pipeline to your source database.\n\nDo the following:\n\n1. Configure Datastream Private Connectivity by following\n instructions outlined in\n [Create a private connectivity configuration](/datastream/docs/create-a-private-connectivity-configuration#create-the-configuration).\n\n2. Note the `Configuration ID` parameter of the created configuration. You\n will use it in later stages to set up required environment variables.\n\n### Azure Cosmos DB\n\nMake sure that\n[Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest)\nis installed on your computer.\n\n### Amazon DocumentDB\n\nAmazon DocumentDB clusters are not directly accessible from outside the Amazon\nVPC network. To connect to a DocumentDB cluster you will have to provision\nan EC2 instance within the Amazon VPC network, and use that instance as a\n[Bastion Host](https://en.wikipedia.org/wiki/Bastion_host) for an SSH tunnel.\n\nVisit the [Connecting to an Amazon DocumentDB cluster from outside an Amazon VPC](https://docs.aws.amazon.com/documentdb/latest/developerguide/connect-from-outside-a-vpc.html) developer guide for instructions on setting\nup an EC2 instance for external connectivity to DocumentDB.\n\nTo establish a connection to the DocumentDB cluster you will have to obtain\nthe private SSH key for the EC2 instance as well as the appropriate\nCertificate Bundle for the region in which your cluster is deployed. Visit the\n[Certificate bundles by AWS Region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html#UsingWithRDS.SSL.CertificatesAllRegions)\nresource page to download the appropriate bundle in PEM format.\n\nNote that the Certificate Bundle contains multiple certificates. You have to\nextract a single certificate to set up the a Datastream connection. It is\nrecommended that you validate the DocumentDB connectivity manually to ensure\nthat you have both a valid SSH key and a valid DocumentDB certificate from\nthe regional bundle. The DocumentDB [developer guide](https://docs.aws.amazon.com/documentdb/latest/developerguide/connect-from-outside-a-vpc.html) provides commandline examples for establish a direct connection from outside\nthe VPC.\n\nConfigure a destination Firestore with MongoDB compatibility database\n---------------------------------------------------------------------\n\n1. Make sure that your project has a Firestore with MongoDB\n compatibility database where you will migrate data\n from your source Mongo database. For more information about creating a\n database, see\n [Create and manage databases](/firestore/mongodb-compatibility/docs/create-databases).\n\n2. For the purpose of this migration, we recommend to\n [create a username and password](/firestore/mongodb-compatibility/docs/connect#scram)\n for the Firestore with MongoDB compatibility database to\n use with the SCRAM-SHA-256 authentication protocol. This username can be\n safely deleted after the migration completes, or you may choose to keep\n using these credentials to connect your Mongo clients to your new\n Firestore with MongoDB compatibility database.\n\nYou will use the name of this Firestore with MongoDB\ncompatibility database and the user credentials in later steps.\n\nAdjust the oplog window size of the source database\n---------------------------------------------------\n\nWe recommend to adjust the oplog window of your source Mongo database to 3 days\nworth of total write traffic to this database. If the rate of write traffic\nexceeds the rate at which Datastream can consume changes from\nyour database, this adjustment will prevent data loss.\n\nThe value might need to be adjusted further, depending on the pattern of the\ntraffic and the volume of peak traffic. For example, if a week's worth of\ntraffic is written in a short period of time, then Datastream\nmight not capture the changes from the oplog fast enough before changes fall\nout of the oplog window. In this case you might need to resize the oplog window\nto 7 days worth of total write traffic.\n\nCreate a Cloud Storage bucket\n-----------------------------\n\nCreate a new Cloud Storage bucket in the following way:\n\n1. Choose a [Cloud Storage region](/storage/docs/locations) where the\n migration pipeline will run. We recommend to use the region where your\n destination Firestore with MongoDB compatibility database\n is located.\n\n2. Choose the name for this bucket. Example: `mongo-migration-bucket`. You will\n use this name in later steps.\n\n3. Create a new Cloud Storage bucket with the chosen name and in the\n chosen region by following instructions provided on the\n [Create a bucket](/storage/docs/creating-buckets#create-bucket) page in\n the Cloud Storage documentation.\n\nWhat's next\n-----------\n\nProceed to\n[Configure environment variables](/firestore/mongodb-compatibility/docs/migrate-configure-env-vars)."]]
