Redacting sensitive data from images

Cloud Data Loss Prevention (DLP) can redact sensitive text from an image. Sensitive data such as personally identifiable information (PII) is detected by the API, which then obscures it using an opaque rectangle.

For example, consider the following image:

Image before redaction

After submitting the image to Cloud DLP using default settings, the image appears as follows:

Image after redaction

The API’s image.redact method takes the following as arguments:

  • At least one Base64-encoded image. Cloud DLP currently supports the following image types: IMAGE_JPEG, IMAGE_BMP and IMAGE_PNG.
  • You must include an inspectConfig argument to specify detection configuration information (InspectConfig) such as what infoTypes to look for.
  • You can also include an imageRedactionConfigs[] argument, which specifies redaction configuration information (ImageRedactionConfig) such as what color to use to obscure redacted text or whether to redact only text of certain infoTypes or all text.

The API returns the same image(s) you gave it, in the same format, but any text identified as containing sensitive information according to your criteria has been redacted.

REST example

The following JSON includes a base64-encoded image and sets several parameters for using the image redaction functionality of Cloud DLP. The image before redaction is shown here:

Image before redaction

The image after redaction is shown here:

See the JSON quickstart for more information about using Cloud DLP with JSON.

JSON Input:

POST https://dlp.googleapis.com/v2/projects/[PROJECT_ID]/image:redact?key={YOUR_API_KEY}

{
  "byteItem":{
    "data":"/9j/4AAQSkZJRgABAQAASABIAAD/4QBARXhpZgAATU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAAqACAAQAAAABAAAAe6ADAAQAAAABAAAADAAAAAD/7QA4UGhvdG9zaG9wIDMuMAA4QklNBAQAAAAAAAA4QklNBCUAAAAAABDUHYzZjwCyBOmACZjs+EJ+/8AAEQgADAB7AwEiAAIRAQMRAf/EAB8AAAEFAQEBAQEBAAAAAAAAAAABAgMEBQYHCAkKC//EALUQAAIBAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQygZGhCCNCscEVUtHwJDNicoIJChYXGBkaJSYnKCkqNDU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6g4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh4uPk5ebn6Onq8fLz9PX29/j5+v/EAB8BAAMBAQEBAQEBAQEAAAAAAAABAgMEBQYHCAkKC//EALURAAIBAgQEAwQHBQQEAAECdwABAgMRBAUhMQYSQVEHYXETIjKBCBRCkaGxwQkjM1LwFWJy0QoWJDThJfEXGBkaJicoKSo1Njc4OTpDREVGR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoKDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uLj5OXm5+jp6vLz9PX29/j5+v/bAEMAHBwcHBwcMBwcMEQwMDBEXERERERcdFxcXFxcdIx0dHR0dHSMjIyMjIyMjKioqKioqMTExMTE3Nzc3Nzc3Nzc3P/bAEMBIiQkODQ4YDQ0YOacgJzm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubm5v/dAAQACP/aAAwDAQACEQMRAD8A2UZ5JJGLlQjY2jGMAA85Heojcs4G35cMQcd/k3DqBVwwxM/mMilvUjmhYYV+6ij6AUAVZppFtk8s4kcAA4z2yTikW5LsHGQGWPj0LMQavbVGCAOBge1NEcY6KPXp75/nzQBWN03lLJsz5h+UAknoTzgcdKklkIVQuQW/PsP5mpPJhwRsXBOTwOTQ8SOoU8Y6UARQSNIh3H6Hj/8AVUQeUhV3nDvhWwM7dufTHUenSrYjULsxkHrwOab5EITywi7TzjAxQBTS4kdkL7gu1M4xgljjnPOPpVmR3dCIdwYEdsHGecbhjpUvlxkhtoyvQ46U5lVxtcAj0NAFMyMfKkDt5ZA5wvJJGM8fyp6O6yuJS3cgYBGM9sDNT+XHkHaPl6cdPpShEUllABPU+tAFSeZ8ZjJUBGbkYyVxxyOnNOHmNcYWRsAZZflwM9B0z79asPGkmN6hscjIzTtq88D5uvvQBRV5ipGW+V8HgFgMZ7DHX9KtQO0kCSN1ZQTS+TDt2bF29cY4qTpwKAP/2Q==",
    "type":"IMAGE_JPEG"
  },
  "imageRedactionConfigs":[
    {
      "infoType":{
        "name":"PHONE_NUMBER"
      },
      "redactionColor":{
        "blue":0.1,
        "green":0.1,
        "red":0.8
      }
    },
    {
      "infoType":{
        "name":"PERSON_NAME"
      },
      "redactionColor":{
        "blue":0.1,
        "green":0.8,
        "red":0.1
      }
    }
  ],
  "inspectConfig":{
    "infoTypes":[
      {
        "name":"PHONE_NUMBER"
      },
      {
        "name":"PERSON_NAME"
      }
    ]
  }
}

JSON Output:

{
 "redactedImage": "/9j/4AAQSkZJRgABAgAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAAMAHsDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD1C0mu9T1PWJpdSubWOxu/KFrD5fliNY43+femRuyfoD1qjL4nubwRNb/6MI7p4ZBFz5v+hm4T/WRp7fl+XRzaTpdxqYvZ9LspbsDHnvCC/wCdWLbQtItx/o+lWMP/AFzt0X+lAHNa1rWo23hHTfsFwY9Wvoo44pvI83/lnveTy/wNMtPFEt5cpeKs0cNza6aRDj/VSTXMkb/4fhXXG2ghYMkEYNvFti+X7g9B+VVF0+xQfLY24/e7uIx1z53/AKM+b60AYj+LbhtFtL3+zPM/tB8WsMU0kkh/du/z7Y/k/wBX2z39KvapqU0VvbRRmaKS4HGP9YOUj/8AQ5RV4aHpG0x/2XZbLh/OmHkLh39TxyaXVdMtZ9PRGQhYOY8HpQB5p471K41DwPbtO/mAX6bJPl5Hlyf3fl/KvL69g+J9vHb+CLDy1UF7xGchFBY+W/JwBXj9ebiv4h9tkX+5r1YUUUVzHshRRRQAUUUUAFFFFAH/2Q=="
}

Code examples

Following is sample code in several languages that demonstrates how to use Cloud DLP to redact sensitive text from an image.

Java

import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.ByteContentItem;
import com.google.privacy.dlp.v2.ByteContentItem.BytesType;
import com.google.privacy.dlp.v2.InfoType;
import com.google.privacy.dlp.v2.InspectConfig;
import com.google.privacy.dlp.v2.Likelihood;
import com.google.privacy.dlp.v2.ProjectName;
import com.google.privacy.dlp.v2.RedactImageRequest;
import com.google.privacy.dlp.v2.RedactImageResponse;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.ArrayList;
import java.util.List;

class RedactImageFile {

  static void redactImageFile(String projectId, String filePath) {
    // String projectId = "my-project-id";
    // String filePath = "path/to/image.png";

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlp = DlpServiceClient.create()) {
      // Specify the project used for request.
      ProjectName project = ProjectName.of(projectId);

      // Specify the content to be inspected.
      ByteString fileBytes = ByteString.readFrom(new FileInputStream(filePath));
      ByteContentItem byteItem =
          ByteContentItem.newBuilder().setType(BytesType.IMAGE).setData(fileBytes).build();

      // Specify the type of info and likelihood necessary to redact.
      List<InfoType> infoTypes = new ArrayList<>();
      // See https://cloud.google.com/dlp/docs/infotypes-reference for complete list of info types
      for (String typeName : new String[] {"PHONE_NUMBER", "EMAIL_ADDRESS", "CREDIT_CARD_NUMBER"}) {
        infoTypes.add(InfoType.newBuilder().setName(typeName).build());
      }
      InspectConfig config =
          InspectConfig.newBuilder()
              .addAllInfoTypes(infoTypes)
              .setMinLikelihood(Likelihood.LIKELY)
              .build();

      // Construct the Redact request to be sent by the client.
      RedactImageRequest request =
          RedactImageRequest.newBuilder()
              .setParent(project.toString())
              .setByteItem(byteItem)
              .setInspectConfig(config)
              .build();

      // Use the client to send the API request.
      RedactImageResponse response = dlp.redactImage(request);

      // Parse the response and process results.
      String outputPath = "redacted.png";
      FileOutputStream redacted = new FileOutputStream(outputPath);
      redacted.write(response.getRedactedImage().toByteArray());
      redacted.close();
      System.out.println("Redacted image written to " + outputPath);

    } catch (Exception e) {
      System.out.println("Error during inspectFile: \n" + e.toString());
    }
  }
}

Node.js

// Imports the Google Cloud Data Loss Prevention library
const DLP = require('@google-cloud/dlp');

// Imports required Node.js libraries
const mime = require('mime');
const fs = require('fs');

// Instantiates a client
const dlp = new DLP.DlpServiceClient();

// The project ID to run the API call under
// const callingProjectId = process.env.GCLOUD_PROJECT;

// The path to a local file to inspect. Can be a JPG or PNG image file.
// const filepath = 'path/to/image.png';

// The minimum likelihood required before redacting a match
// const minLikelihood = 'LIKELIHOOD_UNSPECIFIED';

// The infoTypes of information to redact
// const infoTypes = [{ name: 'EMAIL_ADDRESS' }, { name: 'PHONE_NUMBER' }];

// The local path to save the resulting image to.
// const outputPath = 'result.png';

const imageRedactionConfigs = infoTypes.map(infoType => {
  return {infoType: infoType};
});

// Load image
const fileTypeConstant =
  ['image/jpeg', 'image/bmp', 'image/png', 'image/svg'].indexOf(
    mime.getType(filepath)
  ) + 1;
const fileBytes = Buffer.from(fs.readFileSync(filepath)).toString('base64');

// Construct image redaction request
const request = {
  parent: dlp.projectPath(callingProjectId),
  byteItem: {
    type: fileTypeConstant,
    data: fileBytes,
  },
  inspectConfig: {
    minLikelihood: minLikelihood,
    infoTypes: infoTypes,
  },
  imageRedactionConfigs: imageRedactionConfigs,
};

// Run image redaction request
try {
  const [response] = await dlp.redactImage(request);
  const image = response.redactedImage;
  fs.writeFileSync(outputPath, image);
  console.log(`Saved image redaction results to path: ${outputPath}`);
} catch (err) {
  console.log(`Error in redactImage: ${err.message || err}`);
}

Python

import mimetypes

def redact_image(project, filename, output_filename,
                 info_types, min_likelihood=None, mime_type=None):
    """Uses the Data Loss Prevention API to redact protected data in an image.
    Args:
        project: The Google Cloud project id to use as a parent resource.
        filename: The path to the file to inspect.
        output_filename: The path to which the redacted image will be written.
        info_types: A list of strings representing info types to look for.
            A full list of info type categories can be fetched from the API.
        min_likelihood: A string representing the minimum likelihood threshold
            that constitutes a match. One of: 'LIKELIHOOD_UNSPECIFIED',
            'VERY_UNLIKELY', 'UNLIKELY', 'POSSIBLE', 'LIKELY', 'VERY_LIKELY'.
        mime_type: The MIME type of the file. If not specified, the type is
            inferred via the Python standard library's mimetypes module.
    Returns:
        None; the response from the API is printed to the terminal.
    """
    # Import the client library
    import google.cloud.dlp

    # Instantiate a client.
    dlp = google.cloud.dlp.DlpServiceClient()

    # Prepare info_types by converting the list of strings into a list of
    # dictionaries (protos are also accepted).
    info_types = [{'name': info_type} for info_type in info_types]

    # Prepare image_redaction_configs, a list of dictionaries. Each dictionary
    # contains an info_type and optionally the color used for the replacement.
    # The color is omitted in this sample, so the default (black) will be used.
    image_redaction_configs = []

    if info_types is not None:
        for info_type in info_types:
            image_redaction_configs.append({'info_type': info_type})

    # Construct the configuration dictionary. Keys which are None may
    # optionally be omitted entirely.
    inspect_config = {
        'min_likelihood': min_likelihood,
        'info_types': info_types,
    }

    # If mime_type is not specified, guess it from the filename.
    if mime_type is None:
        mime_guess = mimetypes.MimeTypes().guess_type(filename)
        mime_type = mime_guess[0] or 'application/octet-stream'

    # Select the content type index from the list of supported types.
    supported_content_types = {
        None: 0,  # "Unspecified"
        'image/jpeg': 1,
        'image/bmp': 2,
        'image/png': 3,
        'image/svg': 4,
        'text/plain': 5,
    }
    content_type_index = supported_content_types.get(mime_type, 0)

    # Construct the byte_item, containing the file's byte data.
    with open(filename, mode='rb') as f:
        byte_item = {'type': content_type_index, 'data': f.read()}

    # Convert the project id into a full resource id.
    parent = dlp.project_path(project)

    # Call the API.
    response = dlp.redact_image(
        parent, inspect_config=inspect_config,
        image_redaction_configs=image_redaction_configs,
        byte_item=byte_item)

    # Write out the results.
    with open(output_filename, mode='wb') as f:
        f.write(response.redacted_image)
    print("Wrote {byte_count} to {filename}".format(
        byte_count=len(response.redacted_image), filename=output_filename))

Go

// redactImage blacks out the identified portions of the input image (with type bytesType)
// and stores the result in outputPath.
func redactImage(w io.Writer, client *dlp.Client, project string, minLikelihood dlppb.Likelihood, infoTypes []string, bytesType dlppb.ByteContentItem_BytesType, inputPath, outputPath string) {
	// Convert the info type strings to a list of InfoTypes.
	var i []*dlppb.InfoType
	for _, it := range infoTypes {
		i = append(i, &dlppb.InfoType{Name: it})
	}

	// Convert the info type strings to a list of types to redact in the image.
	var ir []*dlppb.RedactImageRequest_ImageRedactionConfig
	for _, it := range infoTypes {
		ir = append(ir, &dlppb.RedactImageRequest_ImageRedactionConfig{
			Target: &dlppb.RedactImageRequest_ImageRedactionConfig_InfoType{
				InfoType: &dlppb.InfoType{Name: it},
			},
		})
	}

	// Read the input file.
	b, err := ioutil.ReadFile(inputPath)
	if err != nil {
		log.Fatalf("error reading file: %v", err)
	}

	// Create a configured request.
	req := &dlppb.RedactImageRequest{
		Parent: "projects/" + project,
		InspectConfig: &dlppb.InspectConfig{
			InfoTypes:     i,
			MinLikelihood: minLikelihood,
		},
		// The item to analyze.
		ByteItem: &dlppb.ByteContentItem{
			Type: bytesType,
			Data: b,
		},
		ImageRedactionConfigs: ir,
	}
	// Send the request.
	resp, err := client.RedactImage(context.Background(), req)
	if err != nil {
		log.Fatal(err)
	}
	// Write the output file.
	if err := ioutil.WriteFile(outputPath, resp.GetRedactedImage(), 0644); err != nil {
		log.Fatal(err)
	}
	fmt.Fprintf(w, "Wrote output to %s", outputPath)
}

PHP

/**
 * Redact sensitive data from an image.
 */
use Google\Cloud\Dlp\V2\DlpServiceClient;
use Google\Cloud\Dlp\V2\InfoType;
use Google\Cloud\Dlp\V2\InspectConfig;
use Google\Cloud\Dlp\V2\RedactImageRequest\ImageRedactionConfig;
use Google\Cloud\Dlp\V2\Likelihood;
use Google\Cloud\Dlp\V2\ByteContentItem;

/** Uncomment and populate these variables in your code */
// $callingProjectId = 'The project ID to run the API call under';
// $imagePath = 'The local filepath of the image to inspect';
// $outputPath = 'The local filepath to save the resulting image to';

// Instantiate a client.
$dlp = new DlpServiceClient();

// The infoTypes of information to match
$phoneNumberInfoType = (new InfoType())
    ->setName('PHONE_NUMBER');
$infoTypes = [$phoneNumberInfoType];

// The minimum likelihood required before returning a match
$minLikelihood = likelihood::LIKELIHOOD_UNSPECIFIED;

// Whether to include the matching string in the response
$includeQuote = true;

// Create the configuration object
$inspectConfig = (new InspectConfig())
    ->setMinLikelihood($minLikelihood)
    ->setInfoTypes($infoTypes);

// Read image file into a buffer
$imageRef = fopen($imagePath, 'rb');
$imageBytes = fread($imageRef, filesize($imagePath));
fclose($imageRef);

// Get the image's content type
$typeConstant = (int) array_search(
    mime_content_type($imagePath),
    [false, 'image/jpeg', 'image/bmp', 'image/png', 'image/svg']
);

// Create the byte-storing object
$byteContent = (new ByteContentItem())
    ->setType($typeConstant)
    ->setData($imageBytes);

// Create the image redaction config objects
$imageRedactionConfigs = [];
foreach ($infoTypes as $infoType) {
    $config = (new ImageRedactionConfig())
        ->setInfoType($infoType);
    $imageRedactionConfigs[] = $config;
}

$parent = $dlp->projectName($callingProjectId);

// Run request
$response = $dlp->redactImage($parent, [
    'inspectConfig' => $inspectConfig,
    'byteItem' => $byteContent,
    'imageRedactionConfigs' => $imageRedactionConfigs
]);

// Save result to file
file_put_contents($outputPath, $response->getRedactedImage());

// Print completion message
print('Redacted image saved to ' . $outputPath . PHP_EOL);

C#

        public static object RedactFromImage(string projectId, string originalImagePath, string redactedImagePath)
        {
            var request = new RedactImageRequest
            {
                ParentAsProjectName = new ProjectName(projectId),
                InspectConfig = new InspectConfig
                {
                    MinLikelihood = Likelihood.Likely,
                    Limits = new InspectConfig.Types.FindingLimits() { MaxFindingsPerItem = 5 },
                    IncludeQuote = true,
                    InfoTypes =
                    {
                        new InfoType { Name = "PHONE_NUMBER" },
                        new InfoType { Name = "EMAIL_ADDRESS" }
                    }
                },
                ByteItem = new ByteContentItem
                {
                    Type = ByteContentItem.Types.BytesType.ImagePng,
                    Data = ByteString.FromStream(new FileStream(originalImagePath, FileMode.Open))
                },
            };

            DlpServiceClient client = DlpServiceClient.Create();
            var response = client.RedactImage(request);

            Console.WriteLine($"Extracted text: {response.ExtractedText}");

            // Writes redacted image into file
            response.RedactedImage.WriteTo(new FileStream(redactedImagePath, FileMode.Create, FileAccess.Write));

            return 0;
        }
    }
}

Was this page helpful? Let us know how we did:

Send feedback about...

This page
Documentation feedback
Cloud Data Loss Prevention
Product feedback