Redacting sensitive data from images

Sensitive Data Protection can redact sensitive text from an image. Using infoType detectors and Cloud Vision, Sensitive Data Protection inspects an image for text, detects sensitive data within the text, and then returns an image with any matching sensitive data obscured by an opaque rectangle.

For example, consider the following "before" and "after" images. The original image is an example of a typical image file generated from a scan of a paper document. In this example, Sensitive Data Protection has been configured to redact US Social Security numbers, email addresses, and telephone numbers using rectangles of different colors, depending on the content.

Before and after redacted image (click to enlarge).

1. Scanned image before image redaction
2. Scanned image after image redaction

Limitations and considerations

Consider the following points when redacting content from images.

Supported file types

Sensitive Data Protection can redact sensitive data from many image types, including JPEG, BMP, and PNG. For more information, see Supported file types.

Content redaction is not supported for SVG, PDF, XLSX, PPTX, or DOCX files.

Finding limits

When redacting data from images, finding limits don't apply. They can cause unexpected or potentially inconsistent results, where only some data is redacted. If you include FindingLimits in your request, Sensitive Data Protection generates an error.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the DLP API.

   Enable the API

5. Create a service account:

   1. In the Google Cloud console, go to the Create service account page.

      Go to Create service account
   2. Select your project.

   3. In the Service account name field, enter a name. The Google Cloud console fills in the Service account ID field based on this name.

      In the Service account description field, enter a description. For example, Service account for quickstart.

   4. Click Create and continue.

   5. Grant the Project > Owner role to the service account.

      To grant the role, find the Select a role list, then select Project > Owner.

      Note: The Role field affects which resources the service account can access in your project. You can revoke these roles or grant additional roles later. In production environments, do not grant the Owner, Editor, or Viewer roles. Instead, grant a predefined role or custom role that meets your needs.
   6. Click Continue.

   7. Click Done to finish creating the service account.

      Do not close your browser window. You will use it in the next step.

6. Create a service account key:

   1. In the Google Cloud console, click the email address for the service account that you created.
   2. Click Keys.
   3. Click Add key, and then click Create new key.
   4. Click Create. A JSON key file is downloaded to your computer.
   5. Click Close.

7. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.

   Example: Linux or macOS

   export GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"

   Replace KEY_PATH with the path of the JSON file that contains your service account key.

   For example:

   export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/service-account-file.json"

   Example: Windows

   For PowerShell:

   $env:GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"

   Replace KEY_PATH with the path of the JSON file that contains your service account key.

   For example:

   $env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\service-account-file.json"

   For command prompt:

   set GOOGLE_APPLICATION_CREDENTIALS=KEY_PATH

   Replace KEY_PATH with the path of the JSON file that contains your service account key.

8. Install the Google Cloud CLI.

9. To initialize the gcloud CLI, run the following command:

   gcloud init

10. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

    Go to project selector

11. Make sure that billing is enabled for your Google Cloud project.

12. Enable the DLP API.

    Enable the API

13. Create a service account:

    1. In the Google Cloud console, go to the Create service account page.

       Go to Create service account
    2. Select your project.

    3. In the Service account name field, enter a name. The Google Cloud console fills in the Service account ID field based on this name.

       In the Service account description field, enter a description. For example, Service account for quickstart.

    4. Click Create and continue.

    5. Grant the Project > Owner role to the service account.

       To grant the role, find the Select a role list, then select Project > Owner.

       Note: The Role field affects which resources the service account can access in your project. You can revoke these roles or grant additional roles later. In production environments, do not grant the Owner, Editor, or Viewer roles. Instead, grant a predefined role or custom role that meets your needs.
    6. Click Continue.

    7. Click Done to finish creating the service account.

       Do not close your browser window. You will use it in the next step.

14. Create a service account key:

    1. In the Google Cloud console, click the email address for the service account that you created.
    2. Click Keys.
    3. Click Add key, and then click Create new key.
    4. Click Create. A JSON key file is downloaded to your computer.
    5. Click Close.

15. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.

    Example: Linux or macOS

    export GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"

    Replace KEY_PATH with the path of the JSON file that contains your service account key.

    For example:

    export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/service-account-file.json"

    Example: Windows

    For PowerShell:

    $env:GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"

    Replace KEY_PATH with the path of the JSON file that contains your service account key.

    For example:

    $env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\service-account-file.json"

    For command prompt:

    set GOOGLE_APPLICATION_CREDENTIALS=KEY_PATH

    Replace KEY_PATH with the path of the JSON file that contains your service account key.

16. Install the Google Cloud CLI.

17. To initialize the gcloud CLI, run the following command:

    gcloud init

Note: You can do these tasks with your Google Cloud user account, or you can use a service account. In either case, make sure the account you use has a role with the serviceusage.services.use permission, such as Owner (roles/owner), Editor (roles/editor), or a custom role.

Redacting all default infoTypes from an image

To redact sensitive data from an image, submit the image to the DLP API's image.redact method. Unless you specify specific information types (infoTypes) to search for, Sensitive Data Protection searches for the most common infoTypes.

To redact default infoTypes from an image, do the following:

1. Encode the image as a base64 string.

   If you plan to use one of the Sensitive Data Protection client libraries to perform this task, skip this step.

2. Submit a request to the image.redact method.

   If you want to redact default infoTypes, the request requires only the base64-encoded image.

For example, consider the following image. This is an example of a typical image file generated from a scan of a paper document.

Original unredacted image (click to enlarge).

To redact the default infoTypes from this image, send the following request to the DLP API's image.redact method:

Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, see Sensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)

Java

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.ByteContentItem;
import com.google.privacy.dlp.v2.ByteContentItem.BytesType;
import com.google.privacy.dlp.v2.LocationName;
import com.google.privacy.dlp.v2.RedactImageRequest;
import com.google.privacy.dlp.v2.RedactImageResponse;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;

class RedactImageFileAllInfoTypes {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "my-project-id";
    String inputPath = "src/test/resources/sensitive-data-image.jpeg";
    String outputPath = "sensitive-data-image-redacted.jpeg";
    redactImageFileAllInfoTypes(projectId, inputPath, outputPath);
  }

  static void redactImageFileAllInfoTypes(String projectId, String inputPath, String outputPath)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlp = DlpServiceClient.create()) {
      // Specify the content to be redacted.
      ByteString fileBytes = ByteString.readFrom(new FileInputStream(inputPath));
      ByteContentItem byteItem =
          ByteContentItem.newBuilder().setType(BytesType.IMAGE_JPEG).setData(fileBytes).build();

      // Construct the Redact request to be sent by the client.
      // Do not specify the type of info to redact.
      RedactImageRequest request =
          RedactImageRequest.newBuilder()
              .setParent(LocationName.of(projectId, "global").toString())
              .setByteItem(byteItem)
              .build();

      // Use the client to send the API request.
      RedactImageResponse response = dlp.redactImage(request);

      // Parse the response and process results.
      FileOutputStream redacted = new FileOutputStream(outputPath);
      redacted.write(response.getRedactedImage().toByteArray());
      redacted.close();
      System.out.println("Redacted image written to " + outputPath);
    }
  }
}

REST

Important: Before sending this request to the REST API, use a base64 utility to encode the image into ASCII text data, and then replace the [BASE64-ENCODED-IMAGE] placeholder with the base64-encoded text. Most development environments contain a native base64 utility. To encode an image file:

• Linux: base64 input.jpg > output.txt
• macOS: base64 -i input.jpg -o output.txt
• Windows: C:> Base64.exe -e input.jpg > output.txt
• PowerShell: [Convert]::ToBase64String([IO.File]::ReadAllBytes("./input.jpg")) > output.txt

{
  "byteItem": {
    "data": "[BASE64-ENCODED-IMAGE]",
    "type": "IMAGE_JPEG"
  }
}

Sensitive Data Protection returns the following:

    {
      "redactedImage": "[BASE64-ENCODED-IMAGE]"
    }

Decode the base64-encoded image.

The resulting image appears as follows:

Redacted image, all infoTypes (click to enlarge).

Note that in addition to masking the handwritten Social Security number, the email address, and the phone number, Sensitive Data Protection also redacted the year. The next section demonstrates how to redact only certain infoTypes.

Tip: Along with the resulting image, you can get a report of pixel coordinate and length values that indicate the areas the redaction boxes were drawn on. This is the same information that image inspection returns. To enable this functionality, set includeFindings to true in your request to image.redact.

Redacting specific infoTypes from an image

If you want to redact only certain sensitive data from an image, specify their corresponding built-in infoTypes.

Note: You can also redact data matching your custom infoType detectors.

To redact specific infoTypes from an image, submit a request to the DLP API's image.redact method. The request must include the following: * The image. * One or more infoType detectors.

Consider the original image from the previous section. To redact only US Social Security numbers, email addresses, and telephone numbers, send the following JSON to the DLP API's image.redact method:

Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, see Sensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)

C#

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

using Google.Api.Gax.ResourceNames;
using Google.Cloud.Dlp.V2;
using Google.Protobuf;
using System;
using System.IO;

public class RedactImageWithListedInfotypes
{
    public static RedactImageResponse Redact(string projectId, string originalImagePath, string redactedImagePath)
    {
        var request = new RedactImageRequest
        {
            Parent = new LocationName(projectId, "global").ToString(),
            InspectConfig = new InspectConfig
            {
                MinLikelihood = Likelihood.Likely,
                Limits = new InspectConfig.Types.FindingLimits() { MaxFindingsPerItem = 5 },
                IncludeQuote = true,
                InfoTypes =
                    {
                        new InfoType { Name = "PHONE_NUMBER" },
                        new InfoType { Name = "EMAIL_ADDRESS" }
                    }
            },
            ByteItem = new ByteContentItem
            {
                Type = ByteContentItem.Types.BytesType.ImagePng,
                Data = ByteString.FromStream(new FileStream(originalImagePath, FileMode.Open))
            },
        };

        var client = DlpServiceClient.Create();
        var response = client.RedactImage(request);

        Console.WriteLine($"Extracted text: {response.ExtractedText}");

        // Writes redacted image into file
        response.RedactedImage.WriteTo(new FileStream(redactedImagePath, FileMode.Create, FileAccess.Write));

        return response;
    }
}

Go

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import (
	"context"
	"fmt"
	"io"
	"io/ioutil"

	dlp "cloud.google.com/go/dlp/apiv2"
	"cloud.google.com/go/dlp/apiv2/dlppb"
)

// redactImage blacks out the identified portions of the input image (with type bytesType)
// and stores the result in outputPath.
func redactImage(w io.Writer, projectID string, infoTypeNames []string, bytesType dlppb.ByteContentItem_BytesType, inputPath, outputPath string) error {
	// projectID := "my-project-id"
	// infoTypeNames := []string{"US_SOCIAL_SECURITY_NUMBER"}
	// bytesType := dlppb.ByteContentItem_IMAGE_PNG
	// inputPath := /tmp/input
	// outputPath := /tmp/output

	ctx := context.Background()

	client, err := dlp.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("dlp.NewClient: %w", err)
	}
	defer client.Close()

	// Convert the info type strings to a list of InfoTypes.
	var infoTypes []*dlppb.InfoType
	for _, it := range infoTypeNames {
		infoTypes = append(infoTypes, &dlppb.InfoType{Name: it})
	}

	// Convert the info type strings to a list of types to redact in the image.
	var redactInfoTypes []*dlppb.RedactImageRequest_ImageRedactionConfig
	for _, it := range infoTypeNames {
		redactInfoTypes = append(redactInfoTypes, &dlppb.RedactImageRequest_ImageRedactionConfig{
			Target: &dlppb.RedactImageRequest_ImageRedactionConfig_InfoType{
				InfoType: &dlppb.InfoType{Name: it},
			},
		})
	}

	// Read the input file.
	b, err := ioutil.ReadFile(inputPath)
	if err != nil {
		return fmt.Errorf("ioutil.ReadFile: %w", err)
	}

	// Create a configured request.
	req := &dlppb.RedactImageRequest{
		Parent: fmt.Sprintf("projects/%s/locations/global", projectID),
		InspectConfig: &dlppb.InspectConfig{
			InfoTypes:     infoTypes,
			MinLikelihood: dlppb.Likelihood_POSSIBLE,
		},
		// The item to analyze.
		ByteItem: &dlppb.ByteContentItem{
			Type: bytesType,
			Data: b,
		},
		ImageRedactionConfigs: redactInfoTypes,
	}
	// Send the request.
	resp, err := client.RedactImage(ctx, req)
	if err != nil {
		return fmt.Errorf("RedactImage: %w", err)
	}
	// Write the output file.
	if err := ioutil.WriteFile(outputPath, resp.GetRedactedImage(), 0644); err != nil {
		return fmt.Errorf("ioutil.WriteFile: %w", err)
	}
	fmt.Fprintf(w, "Wrote output to %s", outputPath)
	return nil
}

Java

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.ByteContentItem;
import com.google.privacy.dlp.v2.ByteContentItem.BytesType;
import com.google.privacy.dlp.v2.InfoType;
import com.google.privacy.dlp.v2.InspectConfig;
import com.google.privacy.dlp.v2.LocationName;
import com.google.privacy.dlp.v2.RedactImageRequest;
import com.google.privacy.dlp.v2.RedactImageRequest.ImageRedactionConfig;
import com.google.privacy.dlp.v2.RedactImageResponse;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;

class RedactImageFileListedInfoTypes {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "my-project-id";
    String inputPath = "src/test/resources/sensitive-data-image.jpeg";
    String outputPath = "sensitive-data-image-redacted.jpeg";
    redactImageFileListedInfoTypes(projectId, inputPath, outputPath);
  }

  static void redactImageFileListedInfoTypes(String projectId, String inputPath, String outputPath)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlp = DlpServiceClient.create()) {
      // Specify the content to be redacted.
      ByteString fileBytes = ByteString.readFrom(new FileInputStream(inputPath));
      ByteContentItem byteItem =
          ByteContentItem.newBuilder().setType(BytesType.IMAGE_JPEG).setData(fileBytes).build();

      // Specify the types of info necessary to redact.
      List<InfoType> infoTypes = new ArrayList<>();
      // See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info types
      for (String typeName :
          new String[] {"US_SOCIAL_SECURITY_NUMBER", "EMAIL_ADDRESS", "PHONE_NUMBER"}) {
        infoTypes.add(InfoType.newBuilder().setName(typeName).build());
      }
      InspectConfig inspectConfig = InspectConfig.newBuilder().addAllInfoTypes(infoTypes).build();

      // Prepare redaction configs.
      List<ImageRedactionConfig> imageRedactionConfigs =
          infoTypes.stream()
              .map(infoType -> ImageRedactionConfig.newBuilder().setInfoType(infoType).build())
              .collect(Collectors.toList());

      // Construct the Redact request to be sent by the client.
      RedactImageRequest request =
          RedactImageRequest.newBuilder()
              .setParent(LocationName.of(projectId, "global").toString())
              .setByteItem(byteItem)
              .addAllImageRedactionConfigs(imageRedactionConfigs)
              .setInspectConfig(inspectConfig)
              .build();

      // Use the client to send the API request.
      RedactImageResponse response = dlp.redactImage(request);

      // Parse the response and process results.
      FileOutputStream redacted = new FileOutputStream(outputPath);
      redacted.write(response.getRedactedImage().toByteArray());
      redacted.close();
      System.out.println("Redacted image written to " + outputPath);
    }
  }
}

Node.js

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

// Imports the Google Cloud Data Loss Prevention library
const DLP = require('@google-cloud/dlp');

// Imports required Node.js libraries
const mime = require('mime');
const fs = require('fs');

// Instantiates a client
const dlp = new DLP.DlpServiceClient();

// The project ID to run the API call under
// const projectId = 'my-project';

// The path to a local file to inspect. Can be a JPG or PNG image file.
// const filepath = 'path/to/image.png';

// The minimum likelihood required before redacting a match
// const minLikelihood = 'LIKELIHOOD_UNSPECIFIED';

// The infoTypes of information to redact
// const infoTypes = [{ name: 'EMAIL_ADDRESS' }, { name: 'PHONE_NUMBER' }];

// The local path to save the resulting image to.
// const outputPath = 'result.png';
async function redactImage() {
  const imageRedactionConfigs = infoTypes.map(infoType => {
    return {infoType: infoType};
  });

  // Load image
  const fileTypeConstant =
    ['image/jpeg', 'image/bmp', 'image/png', 'image/svg'].indexOf(
      mime.getType(filepath)
    ) + 1;
  const fileBytes = Buffer.from(fs.readFileSync(filepath)).toString('base64');

  // Construct image redaction request
  const request = {
    parent: `projects/${projectId}/locations/global`,
    byteItem: {
      type: fileTypeConstant,
      data: fileBytes,
    },
    inspectConfig: {
      minLikelihood: minLikelihood,
      infoTypes: infoTypes,
    },
    imageRedactionConfigs: imageRedactionConfigs,
  };

  // Run image redaction request
  const [response] = await dlp.redactImage(request);
  const image = response.redactedImage;
  fs.writeFileSync(outputPath, image);
  console.log(`Saved image redaction results to path: ${outputPath}`);
}
redactImage();

PHP

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

use Google\Cloud\Dlp\V2\DlpServiceClient;
use Google\Cloud\Dlp\V2\InfoType;
use Google\Cloud\Dlp\V2\InspectConfig;
use Google\Cloud\Dlp\V2\RedactImageRequest\ImageRedactionConfig;
use Google\Cloud\Dlp\V2\Likelihood;
use Google\Cloud\Dlp\V2\ByteContentItem;

/**
 * Redact sensitive data from an image.
 *
 * @param string $callingProjectId    The project ID to run the API call under
 * @param string $imagePath           The local filepath of the image to inspect
 * @param string $outputPath          The local filepath to save the resulting image to
 */
function redact_image(
    string $callingProjectId,
    string $imagePath,
    string $outputPath
): void {
    // Instantiate a client.
    $dlp = new DlpServiceClient();

    // The infoTypes of information to match
    $phoneNumberInfoType = (new InfoType())
        ->setName('PHONE_NUMBER');
    $infoTypes = [$phoneNumberInfoType];

    // The minimum likelihood required before returning a match
    $minLikelihood = likelihood::LIKELIHOOD_UNSPECIFIED;

    // Whether to include the matching string in the response
    $includeQuote = true;

    // Create the configuration object
    $inspectConfig = (new InspectConfig())
        ->setMinLikelihood($minLikelihood)
        ->setInfoTypes($infoTypes);

    // Read image file into a buffer
    $imageRef = fopen($imagePath, 'rb');
    $imageBytes = fread($imageRef, filesize($imagePath));
    fclose($imageRef);

    // Get the image's content type
    $typeConstant = (int) array_search(
        mime_content_type($imagePath),
        [false, 'image/jpeg', 'image/bmp', 'image/png', 'image/svg']
    );

    // Create the byte-storing object
    $byteContent = (new ByteContentItem())
        ->setType($typeConstant)
        ->setData($imageBytes);

    // Create the image redaction config objects
    $imageRedactionConfigs = [];
    foreach ($infoTypes as $infoType) {
        $config = (new ImageRedactionConfig())
            ->setInfoType($infoType);
        $imageRedactionConfigs[] = $config;
    }

    $parent = "projects/$callingProjectId/locations/global";

    // Run request
    $response = $dlp->redactImage([
        'parent' => $parent,
        'inspectConfig' => $inspectConfig,
        'byteItem' => $byteContent,
        'imageRedactionConfigs' => $imageRedactionConfigs
    ]);

    // Save result to file
    file_put_contents($outputPath, $response->getRedactedImage());

    // Print completion message
    print('Redacted image saved to ' . $outputPath . PHP_EOL);
}

Python

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import mimetypes

import google.cloud.dlp


def redact_image(
    project: str,
    filename: str,
    output_filename: str,
    info_types: List[str],
    min_likelihood: str = None,
    mime_type: str = None,
) -> None:
    """Uses the Data Loss Prevention API to redact protected data in an image.
    Args:
        project: The Google Cloud project id to use as a parent resource.
        filename: The path to the file to inspect.
        output_filename: The path to which the redacted image will be written.
        info_types: A list of strings representing info types to look for.
            A full list of info type categories can be fetched from the API.
        min_likelihood: A string representing the minimum likelihood threshold
            that constitutes a match. One of: 'LIKELIHOOD_UNSPECIFIED',
            'VERY_UNLIKELY', 'UNLIKELY', 'POSSIBLE', 'LIKELY', 'VERY_LIKELY'.
        mime_type: The MIME type of the file. If not specified, the type is
            inferred via the Python standard library's mimetypes module.
    Returns:
        None; the response from the API is printed to the terminal.
    """

    # Instantiate a client.
    dlp = google.cloud.dlp_v2.DlpServiceClient()

    # Prepare info_types by converting the list of strings into a list of
    # dictionaries (protos are also accepted).
    info_types = [{"name": info_type} for info_type in info_types]

    # Prepare image_redaction_configs, a list of dictionaries. Each dictionary
    # contains an info_type and optionally the color used for the replacement.
    # The color is omitted in this sample, so the default (black) will be used.
    image_redaction_configs = []

    if info_types is not None:
        for info_type in info_types:
            image_redaction_configs.append({"info_type": info_type})

    # Construct the configuration dictionary. Keys which are None may
    # optionally be omitted entirely.
    inspect_config = {
        "min_likelihood": min_likelihood,
        "info_types": info_types,
    }

    # If mime_type is not specified, guess it from the filename.
    if mime_type is None:
        mime_guess = mimetypes.MimeTypes().guess_type(filename)
        mime_type = mime_guess[0] or "application/octet-stream"

    # Select the content type index from the list of supported types.
    supported_content_types = {
        None: 0,  # "Unspecified"
        "image/jpeg": 1,
        "image/bmp": 2,
        "image/png": 3,
        "image/svg": 4,
        "text/plain": 5,
    }
    content_type_index = supported_content_types.get(mime_type, 0)

    # Construct the byte_item, containing the file's byte data.
    with open(filename, mode="rb") as f:
        byte_item = {"type_": content_type_index, "data": f.read()}

    # Convert the project id into a full resource id.
    parent = f"projects/{project}"

    # Call the API.
    response = dlp.redact_image(
        request={
            "parent": parent,
            "inspect_config": inspect_config,
            "image_redaction_configs": image_redaction_configs,
            "byte_item": byte_item,
        }
    )

    # Write out the results.
    with open(output_filename, mode="wb") as f:
        f.write(response.redacted_image)
    print(
        "Wrote {byte_count} to {filename}".format(
            byte_count=len(response.redacted_image), filename=output_filename
        )
    )

REST

Important: Before sending this request, use a base64 utility to encode the image into ASCII text data, and then replace the [BASE64-ENCODED-IMAGE] placeholder with the base64-encoded text. Most development environments contain a native base64 utility. To encode an image file:

• Linux: base64 input.jpg > output.txt
• macOS: base64 -i input.jpg -o output.txt
• Windows: C:> Base64.exe -e input.jpg > output.txt
• PowerShell: [Convert]::ToBase64String([IO.File]::ReadAllBytes("./input.jpg")) > output.txt

    {
      "byteItem": {
        "data": "[BASE64-ENCODED-IMAGE]",
        "type": "IMAGE_JPEG"
      },
      "imageRedactionConfigs": [
        {
          "infoType": {
            "name": "US_SOCIAL_SECURITY_NUMBER"
          }
        },
        {
          "infoType": {
            "name": "EMAIL_ADDRESS"
          }
        },
        {
          "infoType": {
            "name": "PHONE_NUMBER"
          }
        }
      ]
    }

Sensitive Data Protection returns the following:

    {
      "redactedImage": "[BASE64-ENCODED-IMAGE]"
    }

Decode the base64-encoded image.

The resulting image appears as follows:

Redacted image, three infoTypes (click to enlarge).

You can color code redacted information by infoType when you want to tell at a glance what's been redacted. See the following section for more information.

Redacting infoTypes from an image with color coding

To color code redacted information by infoType, you pair infoType detectors with RGB color space values.

To color-code infoTypes redacted from an image, submit a request to the DLP API's image.redact method. The request must include the following: * The image. * One or more infoType detectors, each of which is assigned a color using RGB color space values.

Note: Color space values are expressed in the three chromaticities of red, green, and blue. Each value is between 0 and 1, inclusive. If you're used to expressing RGB chromaticity values between 0 and 255, inclusive, you'll need to divide each one by 255 before including it in the call to image.redact. For example, a purple RGB value of (77,26,153) would be approximately (0.3,0.1,0.6).

Consider the original image from the first section. To redact US Social Security numbers with a purple box, email addresses with a green box, and telephone numbers with an orange box, send the following JSON to the DLP API's image.redact method:

Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, see Sensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)

Java

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.ByteContentItem;
import com.google.privacy.dlp.v2.ByteContentItem.BytesType;
import com.google.privacy.dlp.v2.Color;
import com.google.privacy.dlp.v2.InfoType;
import com.google.privacy.dlp.v2.InspectConfig;
import com.google.privacy.dlp.v2.LocationName;
import com.google.privacy.dlp.v2.RedactImageRequest;
import com.google.privacy.dlp.v2.RedactImageRequest.ImageRedactionConfig;
import com.google.privacy.dlp.v2.RedactImageResponse;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

class RedactImageFileColoredInfoTypes {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "my-project-id";
    String inputPath = "src/test/resources/test.png";
    String outputPath = "redacted.png";
    redactImageFileColoredInfoTypes(projectId, inputPath, outputPath);
  }

  static void redactImageFileColoredInfoTypes(String projectId, String inputPath, String outputPath)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlp = DlpServiceClient.create()) {
      // Specify the content to be redacted.
      ByteString fileBytes = ByteString.readFrom(new FileInputStream(inputPath));
      ByteContentItem byteItem =
          ByteContentItem.newBuilder().setType(BytesType.IMAGE_JPEG).setData(fileBytes).build();

      // Define types of info to redact associate each one with a different color.
      // See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info types
      ImageRedactionConfig ssnRedactionConfig =
          ImageRedactionConfig.newBuilder()
              .setInfoType(InfoType.newBuilder().setName("US_SOCIAL_SECURITY_NUMBER").build())
              .setRedactionColor(Color.newBuilder().setRed(.3f).setGreen(.1f).setBlue(.6f).build())
              .build();
      ImageRedactionConfig emailRedactionConfig =
          ImageRedactionConfig.newBuilder()
              .setInfoType(InfoType.newBuilder().setName("EMAIL_ADDRESS").build())
              .setRedactionColor(Color.newBuilder().setRed(.5f).setGreen(.5f).setBlue(1).build())
              .build();
      ImageRedactionConfig phoneRedactionConfig =
          ImageRedactionConfig.newBuilder()
              .setInfoType(InfoType.newBuilder().setName("PHONE_NUMBER").build())
              .setRedactionColor(Color.newBuilder().setRed(1).setGreen(0).setBlue(.6f).build())
              .build();

      // Create collection of all redact configurations.
      List<ImageRedactionConfig> imageRedactionConfigs =
          Arrays.asList(ssnRedactionConfig, emailRedactionConfig, phoneRedactionConfig);

      // List types of info to search for.
      InspectConfig config =
          InspectConfig.newBuilder()
              .addAllInfoTypes(
                  imageRedactionConfigs.stream()
                      .map(ImageRedactionConfig::getInfoType)
                      .collect(Collectors.toList()))
              .build();

      // Construct the Redact request to be sent by the client.
      RedactImageRequest request =
          RedactImageRequest.newBuilder()
              .setParent(LocationName.of(projectId, "global").toString())
              .setByteItem(byteItem)
              .addAllImageRedactionConfigs(imageRedactionConfigs)
              .setInspectConfig(config)
              .build();

      // Use the client to send the API request.
      RedactImageResponse response = dlp.redactImage(request);

      // Parse the response and process results.
      FileOutputStream redacted = new FileOutputStream(outputPath);
      redacted.write(response.getRedactedImage().toByteArray());
      redacted.close();
      System.out.println("Redacted image written to " + outputPath);
    }
  }
}

REST

Important: Before sending this request, use a base64 utility to encode the image into ASCII text data, and then replace the [BASE64-ENCODED-IMAGE] placeholder with the base64-encoded text. Most development environments contain a native base64 utility. To encode an image file:

• Linux: base64 input.jpg > output.txt
• macOS: base64 -i input.jpg -o output.txt
• Windows: C:> Base64.exe -e input.jpg > output.txt
• PowerShell: [Convert]::ToBase64String([IO.File]::ReadAllBytes("./input.jpg")) > output.txt

    {
      "byteItem": {
        "data": "[BASE64-ENCODED-IMAGE]",
        "type": "IMAGE_JPEG"
      },
      "imageRedactionConfigs": [
        {
          "infoType": {
            "name": "US_SOCIAL_SECURITY_NUMBER"
          },
          "redactionColor": {
            "red": 0.3,
            "green": 0.1,
            "blue": 0.6
          }
        },
        {
          "infoType": {
            "name": "EMAIL_ADDRESS"
          },
          "redactionColor": {
            "red": 0.5,
            "blue": 0.5,
            "green": 1
          }
        },
        {
          "infoType": {
            "name": "PHONE_NUMBER"
          },
          "redactionColor": {
            "red": 1,
            "blue": 0,
            "green": 0.6
          }
        }
      ]
    }

Sensitive Data Protection returns the following:

    {
      "redactedImage": "[BASE64-ENCODED-IMAGE]"
    }

Decode the base64-encoded image.

The resulting image appears as follows:

Redacted image, three infoTypes color-coded (click to enlarge).

Redacting all text from an image

Sensitive Data Protection also contains an option to redact all detected text in an image.

To redact all text from an image, submit a request to the DLP API's image.redact method. The request must include the following: * The image. * The redactAllText option set to true.

Consider the original image from the first section. To redact all text, send the following JSON to the DLP API's image.redact method:

Important: The code on this page requires that you first set up a Sensitive Data Protection client. For more information about installing and creating a Sensitive Data Protection client, see Sensitive Data Protection client libraries. (Sending JSON to Sensitive Data Protection REST endpoints does not require a client library.)

Python

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import google.cloud.dlp  # noqa: E402, F811


def redact_image_all_text(
    project: str,
    filename: str,
    output_filename: str,
) -> None:
    """Uses the Data Loss Prevention API to redact all text in an image.

    Args:
        project: The Google Cloud project id to use as a parent resource.
        filename: The path to the file to inspect.
        output_filename: The path to which the redacted image will be written.

    Returns:
        None; the response from the API is printed to the terminal.
    """

    # Instantiate a client.
    dlp = google.cloud.dlp_v2.DlpServiceClient()

    # Construct the image_redaction_configs, indicating to DLP that all text in
    # the input image should be redacted.
    image_redaction_configs = [{"redact_all_text": True}]

    # Construct the byte_item, containing the file's byte data.
    with open(filename, mode="rb") as f:
        byte_item = {"type_": google.cloud.dlp_v2.FileType.IMAGE, "data": f.read()}

    # Convert the project id into a full resource id.
    parent = f"projects/{project}"

    # Call the API.
    response = dlp.redact_image(
        request={
            "parent": parent,
            "image_redaction_configs": image_redaction_configs,
            "byte_item": byte_item,
        }
    )

    # Write out the results.
    with open(output_filename, mode="wb") as f:
        f.write(response.redacted_image)

    print(
        "Wrote {byte_count} to {filename}".format(
            byte_count=len(response.redacted_image), filename=output_filename
        )
    )

Java

To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.

To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.ByteContentItem;
import com.google.privacy.dlp.v2.ByteContentItem.BytesType;
import com.google.privacy.dlp.v2.LocationName;
import com.google.privacy.dlp.v2.RedactImageRequest;
import com.google.privacy.dlp.v2.RedactImageRequest.ImageRedactionConfig;
import com.google.privacy.dlp.v2.RedactImageResponse;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;

class RedactImageFileAllText {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "my-project-id";
    String inputPath = "src/test/resources/sensitive-data-image.jpeg";
    String outputPath = "sensitive-data-image-redacted.jpeg";
    redactImageFileAllText(projectId, inputPath, outputPath);
  }

  static void redactImageFileAllText(String projectId, String inputPath, String outputPath)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlp = DlpServiceClient.create()) {
      // Specify the content to be redacted.
      ByteString fileBytes = ByteString.readFrom(new FileInputStream(inputPath));
      ByteContentItem byteItem =
          ByteContentItem.newBuilder().setType(BytesType.IMAGE_JPEG).setData(fileBytes).build();

      // Enable redaction of all text.
      ImageRedactionConfig imageRedactionConfig =
          ImageRedactionConfig.newBuilder().setRedactAllText(true).build();

      // Construct the Redact request to be sent by the client.
      // Do not specify the type of info to redact.
      RedactImageRequest request =
          RedactImageRequest.newBuilder()
              .setParent(LocationName.of(projectId, "global").toString())
              .setByteItem(byteItem)
              .addImageRedactionConfigs(imageRedactionConfig)
              .build();

      // Use the client to send the API request.
      RedactImageResponse response = dlp.redactImage(request);

      // Parse the response and process results.
      FileOutputStream redacted = new FileOutputStream(outputPath);
      redacted.write(response.getRedactedImage().toByteArray());
      redacted.close();
      System.out.println("Redacted image written to " + outputPath);
    }
  }
}

REST

Important: Before sending this request, use a base64 utility to encode the image into ASCII text data, and then replace the [BASE64-ENCODED-IMAGE] placeholder with the base64-encoded text. Most development environments contain a native base64 utility. To encode an image file:

• Linux: base64 input.jpg > output.txt
• macOS: base64 -i input.jpg -o output.txt
• Windows: C:> Base64.exe -e input.jpg > output.txt
• PowerShell: [Convert]::ToBase64String([IO.File]::ReadAllBytes("./input.jpg")) > output.txt

    {
      "byteItem": {
        "data": "[BASE64-ENCODED-IMAGE]",
        "type": "IMAGE_JPEG"
      },
      "imageRedactionConfigs": [
        {
          "redactAllText": true
        }
      ]
    }

Sensitive Data Protection returns the following:

    {
      "redactedImage": "[BASE64-ENCODED-IMAGE]"
    }

Decode the base64-encoded image.

The API returns the same image(s) you gave it, but any text identified as containing sensitive information according to your criteria has been redacted.

The resulting image appears as follows:

Redacted image, all text (click to enlarge).

Try it out

You can try out each of the examples on this page yourself—or experiment with your own images—in the APIs Explorer on the reference page for image.redact:

Go to APIs Explorer

Tip: For experiments with non-sensitive materials, it's sufficient to uncheck Google OAuth 2.0 and enter projects/testff in the parent field.

What's next

• Learn more about image inspection and redaction.
• Work through the Redacting Sensitive Data with Sensitive Data Protection codelab.
• Learn how to inspect images for sensitive data.
• Learn more about creating a de-identified copy of data in storage.