Optical Character Recognition (OCR) Tutorial

Learn how to perform optical character recognition (OCR) on Google Cloud Platform. This tutorial demonstrates how to upload image files to Google Cloud Storage, extract text from the images using the Google Cloud Vision API, translate the text using the Google Cloud Translation API, and save your translations back to Cloud Storage. Google Cloud Pub/Sub is used to queue various tasks and trigger the right Cloud Functions to carry them out.

Objectives

  • Write and deploy several Background Cloud Functions.
  • Upload images to Cloud Storage.
  • Extract, translate and save text contained in uploaded images.

Costs

This tutorial uses billable components of Cloud Platform, including:

  • Google Cloud Functions
  • Google Cloud Pub/Sub
  • Google Cloud Storage
  • Google Cloud Translation API
  • Google Cloud Vision API

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Cloud Platform users might be eligible for a free trial.

Before you begin

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. Select or create a GCP project.

    Go to the Manage resources page

  3. Make sure that billing is enabled for your project.

    Learn how to enable billing

  4. Enable the Cloud Functions, Cloud Pub/Sub, Cloud Storage, Cloud Translation, and Cloud Vision APIs.

    Enable the APIs

  5. Update and install gcloud components:

    Node.js 6

    gcloud components update

    Python (Beta)

    gcloud components update &&
    gcloud components install beta
  6. Prepare your development environment.

Visualizing the flow of data

The flow of data in the OCR tutorial application involves several steps:

  1. An image is uploaded to Cloud Storage with text in any language (text that appears in the image itself).
  2. A Cloud Function is triggered, which uses the Vision API to extract the text, and queues the text to be translated into the configured translation languages.
  3. For each queued translation, a Cloud Function is triggered which uses the Translation API to translate the text and queue it to be saved to Cloud Storage.
  4. For each translated text, a Cloud Function is triggered which saves the translated text to Cloud Storage.

It may help to visualize the steps:

Preparing the application

  1. Create a Cloud Storage bucket to upload your images, where YOUR_IMAGE_BUCKET_NAME is a globally unique bucket name:

    gsutil mb gs://YOUR_IMAGE_BUCKET_NAME
    

  2. Create a Cloud Storage bucket to save the translations, where YOUR_TEXT_BUCKET_NAME is a globally unique bucket name:

    gsutil mb gs://YOUR_TEXT_BUCKET_NAME
    

  3. Clone the sample app repository to your local machine:

    Node.js 6

    git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Python (Beta)

    git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

  4. Change to the directory that contains the Cloud Functions sample code:

    Node.js 6

    cd nodejs-docs-samples/functions/ocr/app/

    Python (Beta)

    cd python-docs-samples/functions/ocr/app/

  5. Configure the app:

    Node.js

    Using the config.default.json file as a template, create a config.json file in the app directory with the following contents:
    {
    "RESULT_TOPIC": "YOUR_RESULT_TOPIC_NAME",
    "RESULT_BUCKET": "YOUR_TEXT_BUCKET_NAME",
    "TRANSLATE_TOPIC": "YOUR_TRANSLATE_TOPIC_NAME",
    "TRANSLATE": true,
    "TO_LANG": ["en", "fr", "es", "ja", "ru"]
    }
    • Replace YOUR_RESULT_TOPIC_NAME with a topic name to be used for saving results.
    • Replace YOUR_TEXT_BUCKET_NAME with a bucket name used for saving results.
    • Replace YOUR_TRANSLATE_TOPIC_NAME with a topic name to be used for translating results.

    Python (Beta)

    Edit the config.json file in the app directory to have the following contents:
    {
    "RESULT_TOPIC": "YOUR_RESULT_TOPIC_NAME",
    "RESULT_BUCKET": "YOUR_TEXT_BUCKET_NAME",
    "TRANSLATE_TOPIC": "YOUR_TRANSLATE_TOPIC_NAME",
    "TRANSLATE": true,
    "TO_LANG": ["en", "fr", "es", "ja", "ru"]
    }
    • Replace YOUR_RESULT_TOPIC_NAME with a topic name to be used for saving results.
    • Replace YOUR_TEXT_BUCKET_NAME with a bucket name used for saving results.
    • Replace YOUR_TRANSLATE_TOPIC_NAME with a topic name to be used for translating results.

Understanding the code

Importing dependencies

The application must import several dependencies in order to communicate with Google Cloud Platform services:

Node.js 6

const config = require('./config.json');

// Get a reference to the Pub/Sub component
const pubsub = require('@google-cloud/pubsub')();
// Get a reference to the Cloud Storage component
const storage = require('@google-cloud/storage')();
// Get a reference to the Cloud Vision API component
const vision = require('@google-cloud/vision')();
// Get a reference to the Translate API component
const translate = require('@google-cloud/translate')();

const Buffer = require('safe-buffer').Buffer;

Python (Beta)

import base64
import json
import os

from google.cloud import pubsub_v1
from google.cloud import storage
from google.cloud import translate
from google.cloud import vision

vision_client = vision.ImageAnnotatorClient()
translate_client = translate.Client()
publisher = pubsub_v1.PublisherClient()
storage_client = storage.Client()

project_id = os.environ['GCP_PROJECT']

with open('config.json') as f:
    data = f.read()
config = json.loads(data)

Processing images

The following function reads an uploaded image file from Cloud Storage and calls a function to detect whether the image contains text:

Node.js 6

/**
 * This function is exported by index.js, and is executed when
 * a file is uploaded to the Cloud Storage bucket you created
 * for uploading images.
 *
 * @param {object} event The Cloud Functions event.
 * @param {object} event.data A Google Cloud Storage File object.
 */
exports.processImage = (event) => {
  let file = event.data;

  return Promise.resolve()
    .then(() => {
      if (file.resourceState === 'not_exists') {
        // This was a deletion event, we don't want to process this
        return;
      }

      if (!file.bucket) {
        throw new Error('Bucket not provided. Make sure you have a "bucket" property in your request');
      }
      if (!file.name) {
        throw new Error('Filename not provided. Make sure you have a "name" property in your request');
      }

      return detectText(file.bucket, file.name);
    })
    .then(() => {
      console.log(`File ${file.name} processed.`);
    });
};

Python (Beta)

def process_image(file, context):
    """Cloud Function triggered by Cloud Storage when a file is changed.
    Args:
        file (dict): Metadata of the changed file, provided by the triggering
                                 Cloud Storage event.
        context (google.cloud.functions.Context): Metadata of triggering event.
    Returns:
        None; the output is written to stdout and Stackdriver Logging
    """
    bucket = validate_message(file, 'bucket')
    name = validate_message(file, 'name')

    detect_text(bucket, name)

    print('File {} processed.'.format(file['name']))

The following function extracts text from the image using the Cloud Vision API and queues the text for translation:

Node.js 6

/**
 * Detects the text in an image using the Google Vision API.
 *
 * @param {string} bucketName Cloud Storage bucket name.
 * @param {string} filename Cloud Storage file name.
 * @returns {Promise}
 */
function detectText (bucketName, filename) {
  let text;

  console.log(`Looking for text in image ${filename}`);
  return vision.textDetection({ source: { imageUri: `gs://${bucketName}/${filename}` } })
    .then(([detections]) => {
      const annotation = detections.textAnnotations[0];
      text = annotation ? annotation.description : '';
      console.log(`Extracted text from image (${text.length} chars)`);
      return translate.detect(text);
    })
    .then(([detection]) => {
      if (Array.isArray(detection)) {
        detection = detection[0];
      }
      console.log(`Detected language "${detection.language}" for ${filename}`);

      // Submit a message to the bus for each language we're going to translate to
      const tasks = config.TO_LANG.map((lang) => {
        let topicName = config.TRANSLATE_TOPIC;
        if (detection.language === lang) {
          topicName = config.RESULT_TOPIC;
        }
        const messageData = {
          text: text,
          filename: filename,
          lang: lang,
          from: detection.language
        };

        return publishResult(topicName, messageData);
      });

      return Promise.all(tasks);
    });
}

Python (Beta)

def detect_text(bucket, filename):
    print('Looking for text in image {}'.format(filename))

    futures = []

    text_detection_response = vision_client.text_detection({
        'source': {'image_uri': 'gs://{}/{}'.format(bucket, filename)}
    })
    annotations = text_detection_response.text_annotations
    if len(annotations) > 0:
        text = annotations[0].description
    else:
        text = ''
    print('Extracted text {} from image ({} chars).'.format(text, len(text)))

    detect_language_response = translate_client.detect_language(text)
    src_lang = detect_language_response['language']
    print('Detected language {} for text {}.'.format(src_lang, text))

    # Submit a message to the bus for each target language
    for target_lang in config.get('TO_LANG', []):
        topic_name = config['TRANSLATE_TOPIC']
        if src_lang == target_lang:
            topic_name = config['RESULT_TOPIC']
        message = {
            'text': text,
            'filename': filename,
            'target_lang': target_lang,
            'src_lang': src_lang
        }
        message_data = json.dumps(message).encode('utf-8')
        topic_path = publisher.topic_path(project_id, topic_name)
        future = publisher.publish(topic_path, data=message_data)
        futures.append(future)
    for future in futures:
        future.result()

Translating text

The following function translates the extracted text and queues the translated text to be saved back to Cloud Storage:

Node.js 6

/**
 * This function is exported by index.js, and is executed when
 * a message is published to the Cloud Pub/Sub topic specified
 * by the TRANSLATE_TOPIC value in the config.json file. The
 * function translates text using the Google Translate API.
 *
 * @param {object} event The Cloud Functions event.
 * @param {object} event.data The Cloud Pub/Sub Message object.
 * @param {string} event.data.data The "data" property of the Cloud Pub/Sub
 * Message. This property will be a base64-encoded string that you must decode.
 */
exports.translateText = (event) => {
  const pubsubMessage = event.data;
  const jsonStr = Buffer.from(pubsubMessage.data, 'base64').toString();
  const payload = JSON.parse(jsonStr);

  return Promise.resolve()
    .then(() => {
      if (!payload.text) {
        throw new Error('Text not provided. Make sure you have a "text" property in your request');
      }
      if (!payload.filename) {
        throw new Error('Filename not provided. Make sure you have a "filename" property in your request');
      }
      if (!payload.lang) {
        throw new Error('Language not provided. Make sure you have a "lang" property in your request');
      }

      const options = {
        from: payload.from,
        to: payload.lang
      };

      console.log(`Translating text into ${payload.lang}`);
      return translate.translate(payload.text, options);
    })
    .then(([translation]) => {
      const messageData = {
        text: translation,
        filename: payload.filename,
        lang: payload.lang
      };

      return publishResult(config.RESULT_TOPIC, messageData);
    })
    .then(() => {
      console.log(`Text translated to ${payload.lang}`);
    });
};

Python (Beta)

def translate_text(event, context):
    if event.get('data'):
        message_data = base64.b64decode(event['data']).decode('utf-8')
        message = json.loads(message_data)
    else:
        raise ValueError('Data sector is missing in the Pub/Sub message.')

    text = validate_message(message, 'text')
    filename = validate_message(message, 'filename')
    target_lang = validate_message(message, 'target_lang')
    src_lang = validate_message(message, 'src_lang')

    print('Translating text into {}.'.format(target_lang))
    translated_text = translate_client.translate(text,
                                                 target_language=target_lang,
                                                 source_language=src_lang)
    topic_name = config['RESULT_TOPIC']
    message = {
        'text': translated_text['translatedText'],
        'filename': filename,
        'lang': target_lang,
    }
    message_data = json.dumps(message).encode('utf-8')
    topic_path = publisher.topic_path(project_id, topic_name)
    future = publisher.publish(topic_path, data=message_data)
    future.result()

Saving the translations

Finally, the following function receives the translated text and saves it back to Cloud Storage:

Node.js 6

/**
 * This function is exported by index.js, and is executed when
 * a message is published to the Cloud Pub/Sub topic specified
 * by the RESULT_TOPIC value in the config.json file. The
 * function saves the data packet to a file in GCS.
 *
 * @param {object} event The Cloud Functions event.
 * @param {object} event.data The Cloud Pub/Sub Message object.
 * @param {string} event.data.data The "data" property of the Cloud Pub/Sub
 * Message. This property will be a base64-encoded string that you must decode.
 */
exports.saveResult = (event) => {
  const pubsubMessage = event.data;
  const jsonStr = Buffer.from(pubsubMessage.data, 'base64').toString();
  const payload = JSON.parse(jsonStr);

  return Promise.resolve()
    .then(() => {
      if (!payload.text) {
        throw new Error('Text not provided. Make sure you have a "text" property in your request');
      }
      if (!payload.filename) {
        throw new Error('Filename not provided. Make sure you have a "filename" property in your request');
      }
      if (!payload.lang) {
        throw new Error('Language not provided. Make sure you have a "lang" property in your request');
      }

      console.log(`Received request to save file ${payload.filename}`);

      const bucketName = config.RESULT_BUCKET;
      const filename = renameImageForSave(payload.filename, payload.lang);
      const file = storage.bucket(bucketName).file(filename);

      console.log(`Saving result to ${filename} in bucket ${bucketName}`);

      return file.save(payload.text);
    })
    .then(() => {
      console.log(`File saved.`);
    });
};

Python (Beta)

def save_result(event, context):
    if event.get('data'):
        message_data = base64.b64decode(event['data']).decode('utf-8')
        message = json.loads(message_data)
    else:
        raise ValueError('Data sector is missing in the Pub/Sub message.')

    text = validate_message(message, 'text')
    filename = validate_message(message, 'filename')
    lang = validate_message(message, 'lang')

    print('Received request to save file {}.'.format(filename))

    bucket_name = config['RESULT_BUCKET']
    result_filename = '{}_{}.txt'.format(filename, lang)
    bucket = storage_client.get_bucket(bucket_name)
    blob = bucket.blob(result_filename)

    print('Saving result to {} in bucket {}.'.format(result_filename,
                                                     bucket_name))

    blob.upload_from_string(text)

    print('File saved.')

Deploying the functions

This section describes how to deploy your functions.

  1. To deploy the image processing function with a Cloud Storage trigger, run the following command in the app directory:

    Node.js 6

    gcloud functions deploy ocr-extract --trigger-bucket YOUR_IMAGE_BUCKET_NAME --entry-point processImage

    Python (Beta)

    gcloud functions deploy ocr-extract --runtime python37 --trigger-bucket YOUR_IMAGE_BUCKET_NAME --entry-point process_image

    where YOUR_IMAGE_BUCKET_NAME is the name of your Cloud Storage bucket where you will be uploading images.

  2. To deploy the text translation function with a Cloud Pub/Sub trigger, run the following command in the app directory:

    Node.js 6

    gcloud functions deploy ocr-translate --trigger-topic YOUR_TRANSLATE_TOPIC_NAME --entry-point translateText

    Python (Beta)

    gcloud functions deploy ocr-translate  --runtime python37 --trigger-topic YOUR_TRANSLATE_TOPIC_NAME --entry-point translate_text

    where YOUR_TRANSLATE_TOPIC_NAME is the name of your Cloud Pub/Sub topic with which translations will be triggered.

  3. To deploy the function that saves results to Cloud Storage with a Cloud Pub/Sub trigger, run the following command in the app directory:

    Node.js 6

    gcloud functions deploy ocr-save --trigger-topic YOUR_RESULT_TOPIC_NAME --entry-point saveResult

    Python (Beta)

    gcloud functions deploy ocr-save --runtime python37 --trigger-topic YOUR_RESULT_TOPIC_NAME --entry-point save_result

    where YOUR_RESULT_TOPIC_NAME is the name of your Cloud Pub/Sub topic with which saving of results will be triggered.

Uploading an image

  1. Upload an image to your image Cloud Storage bucket:

    gsutil cp PATH_TO_IMAGE gs://YOUR_IMAGE_BUCKET_NAME
    

    where

    • PATH_TO_IMAGE is a path to an image file (that contains text) on your local system.
    • YOUR_IMAGE_BUCKET_NAME is the name of the bucket where you are uploading images.

    You can download one of the images from the sample project.

  2. Watch the logs to be sure the executions have completed:

    gcloud functions logs read --limit 100
    

  3. You can view the saved translations in the Cloud Storage bucket specified by the RESULT_BUCKET value in your configuration file.

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

Deleting the project

The easiest way to eliminate billing is to delete the project you created for the tutorial.

To delete the project:

  1. In the GCP Console, go to the Projects page.

    Go to the Projects page

  2. In the project list, select the project you want to delete and click Delete project. After selecting the checkbox next to the project name, click
      Delete project
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Deleting the Cloud Functions

Deleting Cloud Functions does not remove any resources stored in Cloud Storage.

To delete a Cloud Function, run the following command:

gcloud functions delete NAME_OF_FUNCTION

where NAME_OF_FUNCTION is the name of the function to delete.

You can also delete Cloud Functions from the Google Cloud Platform Console.

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Functions Documentation