Train and test models designed to detect money laundering
Learn how to do basic operations in Anti Money Laundering AI by using command-line tools on your development machine or in the Google Cloud console.
In this guide, you provide sample bank transaction data in the form of BigQuery tables as input to AML AI. The API outputs BigQuery tables that include backtest results and prediction results. The results are used to analyze an example party that is laundering money by structuring funds.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
-
Create or select a Google Cloud project.
-
Create a Google Cloud project:
gcloud projects create PROJECT_ID
Replace
PROJECT_ID
with a name for the Google Cloud project you are creating. -
Select the Google Cloud project that you created:
gcloud config set project PROJECT_ID
Replace
PROJECT_ID
with your Google Cloud project name.
-
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the required APIs:
gcloud services enable financialservices.googleapis.com
bigquery.googleapis.com cloudkms.googleapis.com bigquerydatatransfer.googleapis.com -
Create local authentication credentials for your Google Account:
gcloud auth application-default login
-
Grant roles to your Google Account. Run the following command once for each of the following IAM roles:
roles/financialservices.admin
gcloud projects add-iam-policy-binding PROJECT_ID --member="user:EMAIL_ADDRESS" --role=ROLE
- Replace
PROJECT_ID
with your project ID. - Replace
EMAIL_ADDRESS
with your email address. - Replace
ROLE
with each individual role.
- Replace
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
-
Create or select a Google Cloud project.
-
Create a Google Cloud project:
gcloud projects create PROJECT_ID
Replace
PROJECT_ID
with a name for the Google Cloud project you are creating. -
Select the Google Cloud project that you created:
gcloud config set project PROJECT_ID
Replace
PROJECT_ID
with your Google Cloud project name.
-
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the required APIs:
gcloud services enable financialservices.googleapis.com
bigquery.googleapis.com cloudkms.googleapis.com bigquerydatatransfer.googleapis.com -
Create local authentication credentials for your Google Account:
gcloud auth application-default login
-
Grant roles to your Google Account. Run the following command once for each of the following IAM roles:
roles/financialservices.admin
gcloud projects add-iam-policy-binding PROJECT_ID --member="user:EMAIL_ADDRESS" --role=ROLE
- Replace
PROJECT_ID
with your project ID. - Replace
EMAIL_ADDRESS
with your email address. - Replace
ROLE
with each individual role.
- Replace
- The API requests in this guide use the same Google Cloud project and location
and hard-coded resource IDs to make the guide easier to complete. The resource
IDs follow the pattern
my-
resource-type (for example,my-key-ring
andmy-model
).Make sure the following replacements are defined for this guide:
PROJECT_ID
: your Google Cloud project ID listed in the IAM SettingsPROJECT_NUMBER
: the project number associated with PROJECT_ID. You can find the project number on the IAM Settings page.LOCATION
: the location of the API resources; use one of the supported regionsShow locationsus-central1
us-east1
asia-south1
europe-west1
europe-west2
europe-west4
northamerica-northeast1
southamerica-east1
Create an instance
This section describes how to create an instance. The AML AI instance sits at the root of all other AML AI resources. Each instance requires a single associated customer-managed encryption key (CMEK) which is used to encrypt any data created by AML AI.
Create a key ring
To create a key ring, use the
projects.locations.keyRings.create
method.
REST
To send your request, choose one of these options:
curl
Execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d "" \
"https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/keyRings?key_ring_id=my-key-ring"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-Uri "https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/keyRings?key_ring_id=my-key-ring" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring", "createTime": CREATE_TIME }
gcloud
Execute the following command:
Linux, macOS, or Cloud Shell
gcloud kms keyrings create my-key-ring \ --location LOCATION
Windows (PowerShell)
gcloud kms keyrings create my-key-ring ` --location LOCATION
Windows (cmd.exe)
gcloud kms keyrings create my-key-ring ^ --location LOCATION
$
Create a key
To create a key, use the
projects.locations.keyRings.cryptoKeys
method.
REST
Request JSON body:
{ "purpose": "ENCRYPT_DECRYPT" }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "purpose": "ENCRYPT_DECRYPT" } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys?crypto_key_id=my-key"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "purpose": "ENCRYPT_DECRYPT" } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://cloudkms.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys?crypto_key_id=my-key" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key", "primary": { "name": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key/cryptoKeyVersions/1", "state": "ENABLED", "createTime": CREATE_TIME, "protectionLevel": "SOFTWARE", "algorithm": "GOOGLE_SYMMETRIC_ENCRYPTION", "generateTime": GENERATE_TIME }, "purpose": "ENCRYPT_DECRYPT", "createTime": CREATE_TIME, "versionTemplate": { "protectionLevel": "SOFTWARE", "algorithm": "GOOGLE_SYMMETRIC_ENCRYPTION" }, "destroyScheduledDuration": "86400s" }
gcloud
Before using any of the command data below, make the following replacements:
LOCATION
: the location of the key ring; use one of the supported regionsShow locationsus-central1
us-east1
asia-south1
europe-west1
europe-west2
europe-west4
northamerica-northeast1
southamerica-east1
Execute the following command:
Linux, macOS, or Cloud Shell
gcloud kms keys create my-key \ --keyring my-key-ring \ --location LOCATION \ --purpose "encryption"
Windows (PowerShell)
gcloud kms keys create my-key ` --keyring my-key-ring ` --location LOCATION ` --purpose "encryption"
Windows (cmd.exe)
gcloud kms keys create my-key ^ --keyring my-key-ring ^ --location LOCATION ^ --purpose "encryption"
$
Create the instance using the API
To create an instance, use the
projects.locations.instances.create
method.
Request JSON body:
{ "kmsKey": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key" }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "kmsKey": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key" } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?instance_id=my-instance"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "kmsKey": "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key" } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?instance_id=my-instance" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance", "verb": "create", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
If successful, the response body contains a long-running operation that contains an ID which can be used to retrieve the ongoing status of the asynchronous operation. Copy the returned OPERATION_ID to use in the next section.
Check for the result
Use the
projects.locations.operations.get
method to check if the instance has been created. If the response contains
"done": false
, repeat the command until the response contains "done": true
.
Operations in this guide can take a few minutes to several hours to complete. You must wait until an operation completes before moving forward in this guide because the API uses the output of some methods as input to other methods.
Before using any of the request data, make the following replacements:
OPERATION_ID
: the identifier for the operation
To send your request, choose one of these options:
curl
Execute the following command:
curl -X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "endTime": END_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance", "verb": "create", "requestedCancellation": false, "apiVersion": "v1" }, "done": true, "response": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.Instance", "name": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance", "createTime": CREATE_TIME, "updateTime": UPDATE_TIME, "kmsKey": "projects/KMS_PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key", "state": "ACTIVE" } }
Grant access to the CMEK key
The API automatically creates a service account in your project. The service account needs access to the CMEK key so it can use the key to encrypt and decrypt the underlying data. Grant access to the key.
gcloud kms keys add-iam-policy-binding "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring/cryptoKeys/my-key" \
--keyring "projects/PROJECT_ID/locations/LOCATION/keyRings/my-key-ring" \
--location "LOCATION" \
--member "serviceAccount:service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com" \
--role="roles/cloudkms.cryptoKeyEncrypterDecrypter" \
--project="PROJECT_ID"
Create BigQuery datasets
This section describes how to create input and output BigQuery datasets, and then copy sample banking data into the input dataset.
Create an output dataset
Run the following command to create a dataset to be used to send the AML pipeline outputs to.
bq mk \
--location=LOCATION \
--project_id=PROJECT_ID \
my_bq_output_dataset
Create an input dataset
Run the following command to create a dataset to copy the sample banking tables into.
bq mk \
--location=LOCATION \
--project_id=PROJECT_ID \
my_bq_input_dataset
Copy the sample dataset
Sample banking data is provided as a BigQuery public dataset. Key features of this dataset include the following:
- 100,000 parties
- A core time range from January 1, 2020 to January 1, 2023 and an additional 24 months of lookback data
- 300 negative and 20 positive risk cases per month
- Risk cases with the following attributes:
- Half of the positive risk cases are for structuring activity which
occurred in the two months preceding the
AML_PROCESS_START
event - The other half covers parties with the highest amount of received money
in the two months preceding the
AML_PROCESS_START
event - Negative cases are randomly generated
- A 0.1% chance for the risk case to be generated in the opposite state (for example, a random party that is positive, or a party that has structuring activity or the highest income and is reported negative)
- Half of the positive risk cases are for structuring activity which
occurred in the two months preceding the
Run the following command to copy the sample banking data into the input dataset you created.
bq mk --transfer_config \ --project_id="PROJECT_ID" \ --data_source=cross_region_copy \ --target_dataset="my_bq_input_dataset" \ --display_name="Copy the AML sample dataset." \ --schedule=None \ --params='{ "source_project_id":"bigquery-public-data", "source_dataset_id":"aml_ai_input_dataset", "overwrite_destination_table":"true" }'
Open BigQuery in the Google Cloud console.
Find and expand the input dataset in the Explorer pane. After a few minutes, you should see tables in the input dataset. You can also check the status of the transfer by selecting Data transfers from the BigQuery navigation menu. The AML schema is defined in the AML input data model.
Grant access to the BigQuery datasets
The API automatically creates a service account in your project. The service account needs access to the BigQuery input and output datasets.
- Install
jq
on your development machine. If you cannot installjq
on your development machine, you can use Cloud Shell or one of the other methods for granting access to a resource found in the BigQuery documentation. - Run the following commands to grant read access to the input dataset and its tables.
# Request the current access permissions on the BigQuery dataset and store them in a temp file.
bq show --format=prettyjson "PROJECT_ID:my_bq_input_dataset" | jq '.access+=[{"role":"READER","userByEmail":"service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com" }]'> /tmp/mydataset.json
# Update the BigQuery dataset access permissions using the temp file.
bq update --source /tmp/mydataset.json "PROJECT_ID:my_bq_input_dataset"
# Grant the API read access to the BigQuery table if the table is provided.
for table in party_registration party account_party_link transaction risk_case_event party_supplementary_data
do
[ -n table ] && bq add-iam-policy-binding \
--member="serviceAccount:service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com" --role="roles/bigquery.dataViewer" \
PROJECT_ID:my_bq_input_dataset.${table}
done
Run the following commands to grant write access to the output dataset.
# Request the current access permissions on the BigQuery dataset and store them in a temp file.
bq show --format=prettyjson "PROJECT_ID:my_bq_output_dataset" | jq '.access+=[{"role":"roles/bigquery.dataEditor","userByEmail":"service-PROJECT_NUMBER@gcp-sa-financialservices.iam.gserviceaccount.com" }]'> /tmp/mydataset.json
# Update the BigQuery dataset access permissions using the temp file.
bq update --source /tmp/mydataset.json "PROJECT_ID:my_bq_output_dataset"
Create an AML AI dataset
Create an AML AI dataset to specify the input BigQuery dataset tables and the time range to use.
To create a dataset, use the
projects.locations.instances.datasets.create
method.
Request JSON body:
{ "tableSpecs": { "party": "bq://PROJECT_ID.my_bq_input_dataset.party", "account_party_link": "bq://PROJECT_ID.my_bq_input_dataset.account_party_link", "transaction": "bq://PROJECT_ID.my_bq_input_dataset.transaction", "risk_case_event": "bq://PROJECT_ID.my_bq_input_dataset.risk_case_event", "party_supplementary_data": "bq://PROJECT_ID.my_bq_input_dataset.party_supplementary_data" }, "dateRange": { "startTime": "2020-01-01T00:00:0.00Z", "endTime": "2023-01-01T00:00:0.00Z" }, "timeZone": { "id": "UTC" } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "tableSpecs": { "party": "bq://PROJECT_ID.my_bq_input_dataset.party", "account_party_link": "bq://PROJECT_ID.my_bq_input_dataset.account_party_link", "transaction": "bq://PROJECT_ID.my_bq_input_dataset.transaction", "risk_case_event": "bq://PROJECT_ID.my_bq_input_dataset.risk_case_event", "party_supplementary_data": "bq://PROJECT_ID.my_bq_input_dataset.party_supplementary_data" }, "dateRange": { "startTime": "2020-01-01T00:00:0.00Z", "endTime": "2023-01-01T00:00:0.00Z" }, "timeZone": { "id": "UTC" } } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets?dataset_id=my-dataset"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "tableSpecs": { "party": "bq://PROJECT_ID.my_bq_input_dataset.party", "account_party_link": "bq://PROJECT_ID.my_bq_input_dataset.account_party_link", "transaction": "bq://PROJECT_ID.my_bq_input_dataset.transaction", "risk_case_event": "bq://PROJECT_ID.my_bq_input_dataset.risk_case_event", "party_supplementary_data": "bq://PROJECT_ID.my_bq_input_dataset.party_supplementary_data" }, "dateRange": { "startTime": "2020-01-01T00:00:0.00Z", "endTime": "2023-01-01T00:00:0.00Z" }, "timeZone": { "id": "UTC" } } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets?dataset_id=my-dataset" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "verb": "create", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
You can check for the result of the operation using the new operation ID. (You can do this for the remaining API requests used in this guide.)
Create an engine config
Create an AML AI engine config to automatically tune hyperparameters based on a given engine version. Engine versions are released periodically and correspond to different model logic (for example, targeting a retail line of business versus a commercial one).
To create an engine config, use the
projects.locations.instances.engineConfigs.create
method.
Request JSON body:
{ "engineVersion": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineVersions/aml-commercial.default.v004.000.202312-000", "tuning": { "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2021-07-01T00:00:00Z", }, "performanceTarget": { "partyInvestigationsPerPeriodHint": "30" } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "engineVersion": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineVersions/aml-commercial.default.v004.000.202312-000", "tuning": { "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2021-07-01T00:00:00Z", }, "performanceTarget": { "partyInvestigationsPerPeriodHint": "30" } } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs?engine_config_id=my-engine-config"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "engineVersion": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineVersions/aml-commercial.default.v004.000.202312-000", "tuning": { "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2021-07-01T00:00:00Z", }, "performanceTarget": { "partyInvestigationsPerPeriodHint": "30" } } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs?engine_config_id=my-engine-config" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config", "verb": "create", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Create a model
Create an AML AI model to start the AML training pipeline.
To create a model, use the
projects.locations.instances.models.create
method.
Request JSON body:
{ "engineConfig": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config", "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2021-07-01T00:00:00Z" }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "engineConfig": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config", "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2021-07-01T00:00:00Z" } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models?model_id=my-model"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "engineConfig": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config", "primaryDataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2021-07-01T00:00:00Z" } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models?model_id=my-model" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model", "verb": "create", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Create a backtest result
Backtest prediction uses the trained model on existing historical data. Create a backtest result on the last three months in the dataset; these months were not used in training.
To create a backtest result, use the
projects.locations.instances.backtestResults.create
method.
Request JSON body:
{ "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model", "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2023-01-01T00:00:00Z", "backtestPeriods": 12, "performanceTarget": { "partyInvestigationsPerPeriodHint": "150" } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model", "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2023-01-01T00:00:00Z", "backtestPeriods": 12, "performanceTarget": { "partyInvestigationsPerPeriodHint": "150" } } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults?backtest_result_id=my-backtest-results"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model", "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2023-01-01T00:00:00Z", "backtestPeriods": 12, "performanceTarget": { "partyInvestigationsPerPeriodHint": "150" } } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults?backtest_result_id=my-backtest-results" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results", "verb": "create", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Export backtest results metadata
To export metadata from the backtest result, use the
projects.locations.instances.backtestResults.exportMetadata
method.
Request JSON body:
{ "structuredMetadataDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_backtest_results_metadata", "writeDisposition": "WRITE_TRUNCATE" } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "structuredMetadataDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_backtest_results_metadata", "writeDisposition": "WRITE_TRUNCATE" } } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results:exportMetadata"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "structuredMetadataDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_backtest_results_metadata", "writeDisposition": "WRITE_TRUNCATE" } } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results:exportMetadata" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results", "verb": "exportMetadata", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Open BigQuery in the Google Cloud console.
Find and expand the output dataset in the Explorer pane.
Select the table and click Preview.
Find the row with the name ObservedRecallValues.
Assume that your capacity for investigations is 120 per month. Find the recall value object with
"partyInvestigationsPerPeriod": "120"
`. For the following sample values, if you limit investigations to parties with risk scores greater than 0.53, then you can expect to investigate 120 new parties each month. Over the backtesting period, the year 2022, you would identify 86% of cases that the previous system identified (and possibly others, which were not identified by the old system).{ "recallValues": [ ... { "partyInvestigationsPerPeriod": "105", "recallValue": 0.8142077, "scoreThreshold": 0.6071321 }, { "partyInvestigationsPerPeriod": "120", "recallValue": 0.863388, "scoreThreshold": 0.5339603 }, { "partyInvestigationsPerPeriod": "135", "recallValue": 0.89071035, "scoreThreshold": 0.4739899 }, ... ] }
Import registered parties
Before creating prediction results, you need to import registered parties (that is, customers in the dataset).
To import registered parties, use the
projects.locations.instances.importRegisteredParties
method.
Request JSON body:
{ "partyTables": [ "bq://PROJECT_ID.my_bq_input_dataset.party_registration" ], "mode": "REPLACE", "lineOfBusiness": "COMMERCIAL" }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "partyTables": [ "bq://PROJECT_ID.my_bq_input_dataset.party_registration" ], "mode": "REPLACE", "lineOfBusiness": "COMMERCIAL" } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance:importRegisteredParties"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "partyTables": [ "bq://PROJECT_ID.my_bq_input_dataset.party_registration" ], "mode": "REPLACE", "lineOfBusiness": "COMMERCIAL" } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance:importRegisteredParties" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance", "verb": "importRegisteredParties", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
When the operation completes, you should see that 10,000 parties were registered.
Create a prediction result
Create a prediction result on the last 12 months in the dataset; these months were not used by training.
To create a prediction result, use the
projects.locations.instances.predictionResults.create
method.
Request JSON body:
{ "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model", "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2023-01-01T00:00:00Z", "predictionPeriods": "12", "outputs": { "predictionDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results", "writeDisposition": "WRITE_TRUNCATE" }, "explainabilityDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results_explainability", "writeDisposition": "WRITE_TRUNCATE" } } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
cat > request.json << 'EOF' { "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model", "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2023-01-01T00:00:00Z", "predictionPeriods": "12", "outputs": { "predictionDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results", "writeDisposition": "WRITE_TRUNCATE" }, "explainabilityDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results_explainability", "writeDisposition": "WRITE_TRUNCATE" } } } EOF
Then execute the following command to send your REST request:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults?prediction_result_id=my-prediction-results"
PowerShell
Save the request body in a file named request.json
.
Run the following command in the terminal to create or overwrite
this file in the current directory:
@' { "model": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model", "dataset": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "endTime": "2023-01-01T00:00:00Z", "predictionPeriods": "12", "outputs": { "predictionDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results", "writeDisposition": "WRITE_TRUNCATE" }, "explainabilityDestination": { "tableUri": "bq://PROJECT_ID.my_bq_output_dataset.my_prediction_results_explainability", "writeDisposition": "WRITE_TRUNCATE" } } } '@ | Out-File -FilePath request.json -Encoding utf8
Then execute the following command to send your REST request:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults?prediction_result_id=my-prediction-results" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults/my-prediction-results", "verb": "create", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Analyze a single structuring case in the Google Cloud console
Open BigQuery in the Google Cloud console. Make sure SQL workspace is selected.
The BigQuery page has three main sections:
- The BigQuery navigation menu
- The Explorer pane
- The details pane
In the details pane, click Compose a New Query to open the query editor.
Copy the following SQL statement into the editor and click Run.
SELECT * FROM `PROJECT_ID.my_bq_input_dataset.transaction` WHERE account_id = '1E60OAUNKP84WDKB' AND DATE_TRUNC(book_time, MONTH) = "2022-08-01" ORDER by book_time
This statement checks account ID
1E60OAUNKP84WDKB
in August 2022. This account is linked to party IDEGS4NJD38JZ8NTL8
. You can find the party ID for a given account ID by using the AccountPartyLink table.The transaction data shows a series of round transactions targeted at a single account, which looks suspicious.
Copy the following SQL statement into the editor and click Run.
SELECT * FROM `PROJECT_ID.my_bq_input_dataset.risk_case_event` WHERE party_id = 'EGS4NJD38JZ8NTL8'
This statement shows that there was a risk case leading to the exit of this party. The risk case started two months after the suspicious activity.
Copy the following SQL statement into the editor and click Run.
SELECT * FROM `PROJECT_ID.my_bq_output_dataset.my_prediction_results` WHERE party_id = 'EGS4NJD38JZ8NTL8' ORDER BY risk_period_end_time
By checking the prediction results, you can see that the party's risk score jumps from nearly zero (note the exponent value) to high values in the months following the suspicious activity. Your results may vary from the results shown.
The risk score is not a probability. A risk score should always be evaluated relative to other risk scores. For example, a seemly small value can be considered positive in cases where the other risk scores are lower.
Copy the following SQL statement into the editor and click Run.
SELECT * FROM `PROJECT_ID.my_bq_output_dataset.my_prediction_results_explainability` WHERE party_id = 'EGS4NJD38JZ8NTL8' AND risk_period_end_time = '2022-10-01'
By checking the explainability results, you can see that the correct feature families score the highest values.
Clean up
To avoid incurring charges to your Google Cloud account for the resources used on this page, delete the Google Cloud project with the resources.
Delete the prediction result
To delete a prediction result, use the
projects.locations.instances.predictionResults.delete
method.
To send your request, choose one of these options:
curl
Execute the following command:
curl -X DELETE \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults/my-prediction-results"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method DELETE `
-Headers $headers `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults/my-prediction-results" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/predictionResults/my-prediction-results", "verb": "delete", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Delete the backtest result
To delete a backtest result, use the
projects.locations.instances.backtestResults.delete
method.
To send your request, choose one of these options:
curl
Execute the following command:
curl -X DELETE \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method DELETE `
-Headers $headers `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/backtestResults/my-backtest-results", "verb": "delete", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Delete the model
To delete a model, use the
projects.locations.instances.models.delete
method.
To send your request, choose one of these options:
curl
Execute the following command:
curl -X DELETE \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method DELETE `
-Headers $headers `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/models/my-model", "verb": "delete", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Delete the engine config
To delete an engine config, use the
projects.locations.instances.engineConfigs.delete
method.
To send your request, choose one of these options:
curl
Execute the following command:
curl -X DELETE \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method DELETE `
-Headers $headers `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/engineConfigs/my-engine-config", "verb": "delete", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Delete the dataset
To delete a dataset, use the
projects.locations.instances.datasets.delete
method.
To send your request, choose one of these options:
curl
Execute the following command:
curl -X DELETE \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method DELETE `
-Headers $headers `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance/datasets/my-dataset", "verb": "delete", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Delete the instance
To delete an instance, use the
projects.locations.instances.delete
method.
To send your request, choose one of these options:
curl
Execute the following command:
curl -X DELETE \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance"
PowerShell
Execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method DELETE `
-Headers $headers `
-Uri "https://financialservices.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/my-instance" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_ID/locations/LOCATION/operations/OPERATION_ID", "metadata": { "@type": "type.googleapis.com/google.cloud.financialservices.v1.OperationMetadata", "createTime": CREATE_TIME, "target": "projects/PROJECT_ID/locations/LOCATION/instances/my-instance", "verb": "delete", "requestedCancellation": false, "apiVersion": "v1" }, "done": false }
Delete the BigQuery datasets
bq rm -r -f -d PROJECT_ID:my_bq_input_dataset
bq rm -r -f -d PROJECT_ID:my_bq_output_dataset
What's next
- Read about key concepts in the overview
- View the reference documentation