Detect Handwriting (OCR)

Optical Character Recognition (OCR)

The Vision API can detect and extract text from images:

  • DOCUMENT_TEXT_DETECTION extracts text from an image; the response is optimized for dense text and documents. The JSON includes page, block, paragraph, word, and break information.

One specific use of DOCUMENT_TEXT_DETECTION is to detect handwriting in an image.

Code samples

For samples in a number of programming languages, see:

Document text detection requests

Set up your GCP project and authentication

Detect Handwriting

You can detect handwriting on an image by passing the image's filepath or public URL to Cloud Vision API.

curl command

To make a handwriting detection request using curl from the Linux or MacOS command line, make a POST request to the https://vision.googleapis.com/v1/images:annotate endpoint and specify DOCUMENT_TEXT_DETECTION as the value of features.type, as shown in the following example:

curl -X POST \
     -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
     -H "Content-Type: application/json; charset=utf-8" \
     --data "{
      'requests': [
        {
          'image': {
            'source': {
              'imageUri': 'gs://vision-api-handwriting-ocr-bucket/handwriting_image.png'
            }
          },
          'features': [
            {
              'type': 'DOCUMENT_TEXT_DETECTION'
            }
          ]
        }
      ]
    }" "https://vision.googleapis.com/v1/images:annotate"

Images can be passed in one of three ways: as a base64-encoded string (shown above); as a Google Cloud Storage URI; or as a web URI. See Making requests for more information.

See the AnnotateImageRequest reference documentation for more information on configuring the request body.

GCLOUD COMMAND

To perform entity analysis, use the gcloud ml vision detect-document command as shown in the following example:

gcloud ml vision detect-document "gs://vision-api-handwriting-ocr-bucket/handwriting_image.png"

PowerShell

To make a handwriting detection request using Windows PowerShell, make a POST request to the https://vision.googleapis.com/v1/images:annotate endpoint and specify DOCUMENT_TEXT_DETECTION as the value of features.type, as shown in the following example:

$cred = gcloud auth application-default print-access-token
$headers = @{ Authorization = "Bearer $cred" }

Invoke-WebRequest `
  -Method Post `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -Body "{
      'requests': [
        {
          'image': {
            'source': {
              'imageUri': 'gs://vision-api-handwriting-ocr-bucket/handwriting_image.png'
            }
          },
          'features': [
            {
              'type': 'DOCUMENT_TEXT_DETECTION'
            }
          ]
        }
      ]
    }" `
  -Uri "https://vision.googleapis.com/v1/images:annotate" | Select-Object -Expand Content

Images can be passed in one of three ways: as a base64-encoded string; as a Google Cloud Storage URI; or as a publicly-accessible HTTPS or HTTP URL. See Making requests for more information.

See the AnnotateImageRequest reference documentation for more information on configuring the request body.

Document text detection response

A DOCUMENT_TEXT_DETECTION response includes additional layout information, such as page, block, paragraph, word, and break information, along with confidence scores for each. (The sample below uses a simplified example; the response from a dense document is too long to display on this page.)

{
  "responses": [
    {
      "textAnnotations": [
        {
          "locale": "en",
          "description": "Google Cloud\nPlatform\n",
          "boundingPoly": {
            "vertices": [
              {
                "x": 312,
                "y": 95
              },
              {
                "x": 694,
                "y": 95
              },
              {
                "x": 694,
                "y": 253
              },
              {
                "x": 312,
                "y": 253
              }
            ]
          }
        },
        {
          "description": "Google",
          "boundingPoly": {
            "vertices": [
              {
                "x": 312,
                "y": 103
              },
              {
                "x": 530,
                "y": 99
              },
              {
                "x": 531,
                "y": 172
              },
              {
                "x": 313,
                "y": 176
              }
            ]
          }
        },
        {
          "description": "Cloud",
          "boundingPoly": {
            "vertices": [
              {
                "x": 602,
                "y": 97
              },
              {
                "x": 693,
                "y": 95
              },
              {
                "x": 694,
                "y": 168
              },
              {
                "x": 603,
                "y": 170
              }
            ]
          }
        },
        {
          "description": "Platform",
          "boundingPoly": {
            "vertices": [
              {
                "x": 429,
                "y": 190
              },
              {
                "x": 582,
                "y": 190
              },
              {
                "x": 582,
                "y": 253
              },
              {
                "x": 429,
                "y": 253
              }
            ]
          }
        }
      ],
      "fullTextAnnotation": {
        "pages": [
          {
            "property": {
              "detectedLanguages": [
                {
                  "languageCode": "en",
                  "confidence": 1
                }
              ]
            },
            "width": 850,
            "height": 550,
            "blocks": [
              {
                "boundingBox": {
                  "vertices": [
                    {
                      "x": 312,
                      "y": 103
                    },
                    {
                      "x": 693,
                      "y": 95
                    },
                    {
                      "x": 696,
                      "y": 250
                    },
                    {
                      "x": 315,
                      "y": 258
                    }
                  ]
                },
                "paragraphs": [
                  {
                    "boundingBox": {
                      "vertices": [
                        {
                          "x": 312,
                          "y": 103
                        },
                        {
                          "x": 693,
                          "y": 95
                        },
                        {
                          "x": 694,
                          "y": 168
                        },
                        {
                          "x": 313,
                          "y": 176
                        }
                      ]
                    },
                    "words": [
                      {
                        "property": {
                          "detectedLanguages": [
                            {
                              "languageCode": "en"
                            }
                          ]
                        },
                        "boundingBox": {
                          "vertices": [
                            {
                              "x": 312,
                              "y": 103
                            },
                            {
                              "x": 530,
                              "y": 99
                            },
                            {
                              "x": 531,
                              "y": 172
                            },
                            {
                              "x": 313,
                              "y": 176
                            }
                          ]
                        },
                        "symbols": [
                          {
                            "property": {
                              "detectedLanguages": [
                                {
                                  "languageCode": "en"
                                }
                              ]
                            },
                            "boundingBox": {
                              "vertices": [
                                {
                                  "x": 312,
                                  "y": 104
                                },
                                {
                                  "x": 371,
                                  "y": 103
                                },
                                {
                                  "x": 372,
                                  "y": 175
                                },
                                {
                                  "x": 313,
                                  "y": 176
                                }
                              ]
                            },
                            "text": "G",
                            "confidence": 0.99
                          },
                          ...
                        ],
                        "confidence": 0.99
                      },
                      {
                        "property": {
                          "detectedLanguages": [
                            {
                              "languageCode": "en"
                            }
                          ]
                        },
                        "boundingBox": {
                          "vertices": [
                            {
                              "x": 602,
                              "y": 97
                            },
                            {
                              "x": 693,
                              "y": 95
                            },
                            {
                              "x": 694,
                              "y": 168
                            },
                            {
                              "x": 603,
                              "y": 170
                            }
                          ]
                        },
                        "symbols": [
                          {
                            "property": {
                              "detectedLanguages": [
                                {
                                  "languageCode": "en"
                                }
                              ]
                            },
                            "boundingBox": {
                              "vertices": [
                                {
                                  "x": 602,
                                  "y": 98
                                },
                                {
                                  "x": 620,
                                  "y": 98
                                },
                                {
                                  "x": 621,
                                  "y": 170
                                },
                                {
                                  "x": 603,
                                  "y": 170
                                }
                              ]
                            },
                            "text": "C",
                            "confidence": 0.99
                          },
                          ...
                        ],
                        "confidence": 0.99
                      }
                    ],
                    "confidence": 0.99
                  },
                  {
                    "boundingBox": {
                      "vertices": [
                        {
                          "x": 429,
                          "y": 190
                        },
                        {
                          "x": 582,
                          "y": 190
                        },
                        {
                          "x": 582,
                          "y": 253
                        },
                        {
                          "x": 429,
                          "y": 253
                        }
                      ]
                    },
                    "words": [
                      {
                        "property": {
                          "detectedLanguages": [
                            {
                              "languageCode": "en"
                            }
                          ]
                        },
                        "boundingBox": {
                          "vertices": [
                            {
                              "x": 429,
                              "y": 190
                            },
                            {
                              "x": 582,
                              "y": 190
                            },
                            {
                              "x": 582,
                              "y": 253
                            },
                            {
                              "x": 429,
                              "y": 253
                            }
                          ]
                        },
                        "symbols": [
                          {
                            "property": {
                              "detectedLanguages": [
                                {
                                  "languageCode": "en"
                                }
                              ]
                            },
                            "boundingBox": {
                              "vertices": [
                                {
                                  "x": 429,
                                  "y": 190
                                },
                                {
                                  "x": 449,
                                  "y": 190
                                },
                                {
                                  "x": 449,
                                  "y": 253
                                },
                                {
                                  "x": 429,
                                  "y": 253
                                }
                              ]
                            },
                            "text": "P",
                            "confidence": 0.92
                          },
                          ...
                        ],
                        "confidence": 0.97
                      }
                    ],
                    "confidence": 0.97
                  }
                ],
                "blockType": "TEXT",
                "confidence": 0.98
              }
            ]
          }
        ],
        "text": "Google Cloud\nPlatform\n"
      }
    }
  ]
}

Specifying the language (optional)

Both types of OCR requests support one or more languageHints that specify the language of any text in the image. However, in most cases, an empty value yields the best results since it enables automatic language detection. For languages based on the Latin alphabet, setting languageHints is not needed. In rare cases, when the language of the text in the image is known, setting a hint will help get better results (although it will be a significant hindrance if the hint is wrong). Text detection returns an error if one or more of the specified languages is not one of the supported languages.

Try it

Try text detection and document text detection below. You can use the image specified already (gs://vision-api-handwriting-ocr-bucket/handwriting_image.png) by clicking Execute, or you can specify your own image in its place.

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Vision API
Need help? Visit our support page.