Chirp: Universal speech model

Chirp is the next generation of Google's speech-to-text models. Representing the culmination of years of research, the first version of Chirp is now available for Speech-to-Text. We intend to improve and expand Chirp to more languages and domains. For details, see our paper, Google USM.

We trained Chirp models with a different architecture than our current speech models. A single model unifies data from multiple languages. However, users still specify the language in which the model should recognize speech. Chirp does not support some of the Google Speech features that other models have. See below for a complete list.

Model identifiers

Chirp is available in the Speech-to-Text API v2. You can leverage it like any other model.

The model identifier for Chirp is: chirp.

You can specify this model while creating a recognizer or inline in synchronous or batch recognition requests.

Available API methods

Chirp processes speech in much larger chunks than other models do. This means it might not be suitable for true, real-time use. Chirp is available through the following API methods:

• v2 Speech.Recognize (good for short audio < 1 min)
• v2 Speech.BatchRecognize (good for long audio 1 min to 8 hrs)

Chirp is not available on the following API methods:

• v2 Speech.StreamingRecognize
• v1 Speech.StreamingRecognize
• v1 Speech.Recognize
• v1 Speech.LongRunningRecognize
• v1p1beta1 Speech.StreamingRecognize
• v1p1beta1 Speech.Recognize
• v1p1beta1 Speech.LongRunningRecognize

Regions

Chirp is available in the following regions:

• us-central1
• europe-west4
• asia-southeast1

See the languages page for more information.

Languages

You can see the supported languages in the full language list.

Feature support and limitations

Chirp does not currently support many of the STT API features. See below for specific restrictions.

• Confidence scores: The API returns a value, but it isn't truly a confidence score.
• Speech adaptation: No adaptation features supported.
• Diarization: Automatic diarization isn't supported.
• Forced normalization: Not supported.
• Word level confidence: Not supported.
• Language detection: Not supported.

Chirp does support the following features:

• Automatic punctuation: The punctuation is predicted by the model. It can be disabled.
• Word timings: Optionally returned.
• Language-agnostic audio transcription: The model automatically infers the spoken language in your audio file and adds it to the results.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Speech-to-Text APIs.

   Enable the APIs

5. Make sure that you have the following role or roles on the project: Cloud Speech Administrator

   Check for the roles

   1. In the Google Cloud console, go to the IAM page.

      Go to IAM
   2. Select the project.

   3. In the Principal column, find the row that has your email address.

      If your email address isn't in that column, then you do not have any roles.

   4. In the Role column for the row with your email address, check whether the list of roles includes the required roles.

   Grant the roles

   1. In the Google Cloud console, go to the IAM page.

      Go to IAM
   2. Select the project.
   3. Click Grant access.
   4. In the New principals field, enter your email address.
   5. In the Select a role list, select a role.
   6. To grant additional roles, click Add another role and add each additional role.
   7. Click Save.
6. Install the Google Cloud CLI.

7. To initialize the gcloud CLI, run the following command:

   gcloud init

   Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.

8. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

9. Make sure that billing is enabled for your Google Cloud project.

10. Enable the Speech-to-Text APIs.

    Enable the APIs

11. Make sure that you have the following role or roles on the project: Cloud Speech Administrator

    Check for the roles

    1. In the Google Cloud console, go to the IAM page.

       Go to IAM
    2. Select the project.

    3. In the Principal column, find the row that has your email address.

       If your email address isn't in that column, then you do not have any roles.

    4. In the Role column for the row with your email address, check whether the list of roles includes the required roles.

    Grant the roles

    1. In the Google Cloud console, go to the IAM page.

       Go to IAM
    2. Select the project.
    3. Click Grant access.
    4. In the New principals field, enter your email address.
    5. In the Select a role list, select a role.
    6. To grant additional roles, click Add another role and add each additional role.
    7. Click Save.
12. Install the Google Cloud CLI.

13. To initialize the gcloud CLI, run the following command:

    gcloud init

    Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.

Client libraries can use Application Default Credentials to easily authenticate with Google APIs and send requests to those APIs. With Application Default Credentials, you can test your application locally and deploy it without changing the underlying code. For more information, see Authenticate for using client libraries.

14. Create local authentication credentials for your Google Account:

    gcloud auth application-default login

Also ensure you have installed the client library.

Perform synchronous speech recognition with Chirp

Here is an example of performing synchronous speech recognition on a local audio file using Chirp:

Python

from google.api_core.client_options import ClientOptions
from google.cloud.speech_v2 import SpeechClient
from google.cloud.speech_v2.types import cloud_speech


def transcribe_chirp(
    project_id: str,
    audio_file: str,
) -> cloud_speech.RecognizeResponse:
    """Transcribe an audio file using Chirp."""
    # Instantiates a client
    client = SpeechClient(
        client_options=ClientOptions(
            api_endpoint="us-central1-speech.googleapis.com",
        )
    )

    # Reads a file as bytes
    with open(audio_file, "rb") as f:
        content = f.read()

    config = cloud_speech.RecognitionConfig(
        auto_decoding_config=cloud_speech.AutoDetectDecodingConfig(),
        language_codes=["en-US"],
        model="chirp",
    )

    request = cloud_speech.RecognizeRequest(
        recognizer=f"projects/{project_id}/locations/us-central1/recognizers/_",
        config=config,
        content=content,
    )

    # Transcribes the audio into text
    response = client.recognize(request=request)

    for result in response.results:
        print(f"Transcript: {result.alternatives[0].transcript}")

    return response

Make a request with language-agnostic transcription enabled

The following code samples demonstrate how to make a request with language-agnostic transcription enabled.

Python

from google.api_core.client_options import ClientOptions
from google.cloud.speech_v2 import SpeechClient
from google.cloud.speech_v2.types import cloud_speech


def transcribe_chirp_auto_detect_language(
    project_id: str,
    audio_file: str,
    region: str = "us-central1",
) -> cloud_speech.RecognizeResponse:
    """Transcribe an audio file and auto-detect spoken language using Chirp.

    Please see https://cloud.google.com/speech-to-text/v2/docs/encoding for more
    information on which audio encodings are supported.
    """
    # Instantiates a client
    client = SpeechClient(
        client_options=ClientOptions(
            api_endpoint=f"{region}-speech.googleapis.com",
        )
    )

    # Reads a file as bytes
    with open(audio_file, "rb") as f:
        content = f.read()

    config = cloud_speech.RecognitionConfig(
        auto_decoding_config=cloud_speech.AutoDetectDecodingConfig(),
        language_codes=["auto"],  # Set language code to auto to detect language.
        model="chirp",
    )

    request = cloud_speech.RecognizeRequest(
        recognizer=f"projects/{project_id}/locations/{region}/recognizers/_",
        config=config,
        content=content,
    )

    # Transcribes the audio into text
    response = client.recognize(request=request)

    for result in response.results:
        print(f"Transcript: {result.alternatives[0].transcript}")
        print(f"Detected Language: {result.language_code}")

    return response

Getting started with Chirp in the Google Cloud console

1. Ensure you have signed up for a Google Cloud account and created a project.
2. Go to Speech in Google Cloud console.
3. Enable the API if it's not already enabled.

4. Create an STT Recognizer that uses Chirp. a. Go to the Recognizers tab and click Create.

   

   b. From the Create Recognizer page, enter the necessary fields for Chirp.

   

   i. Name your recognizer.

   ii. Select chirp as the Model.

   iii. Select the language you want to use. You must use one recognizer per language that you plan to test.

   iv. Do not select any other features.

5. Make sure that you have an STT UI Workspace. If you do not have one already, you need to create a workspace. a. Visit the transcriptions page, and click New Transcription.

   b. Open the Workspace dropdown and click New Workspace to create a workspace for transcription.

   c. From the Create a new workspace navigation sidebar, click Browse.

   d. Click to create a new bucket.

   e. Enter a name for your bucket and click Continue.

   f. Click Create to create your Cloud Storage bucket.

   g. Once the bucket is created, click Select to select your bucket for use.

   h. Click Create to finish creating your workspace for the speech-to-text UI.

6. Perform a transcription on your actual audio.

   

   a. From the New Transcription page, select your audio file through either upload (Local upload) or specifying an existing Cloud Storage file (Cloud storage). Note: The UI tries to assess your audio file parameters automatically.

   b. Click Continue to move to the Transcription options.

   

   c. Select the Spoken language that you plan to use for recognition with Chirp from your previously created recognizer.

   d. In the model dropdown, select Chirp - Universal Speech Model.

   e. In the Recognizer dropdown, select your newly created recognizer.

   f. Click Submit to run your first recognition request using Chirp.

7. View your Chirp transcription result. a. From the Transcriptions page, click the name of the transcription to view its result.

   b. In the Transcription details page, view your transcription result, and optionally playback the audio in the browser.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

1. Optional: Revoke the authentication credentials that you created, and delete the local credential file.

   gcloud auth application-default revoke

2. Optional: Revoke credentials from the gcloud CLI.

   gcloud auth revoke

Console

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects can help you avoid exceeding project quota limits.

In the Google Cloud console, go to the Manage resources page.

Go to Manage resources

In the project list, select the project that you want to delete, and then click Delete.

In the dialog, type the project ID, and then click Shut down to delete the project.

gcloud

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects can help you avoid exceeding project quota limits.

Delete a Google Cloud project:

gcloud projects delete PROJECT_ID

What's next

• Practice transcribing short audio files.
• Learn how to transcribe streaming audio.
• Learn how to transcribe long audio files.
• For best performance, accuracy, and other tips, see the best practices documentation.