添加字词时间戳

本页面介绍了如何获取由 Speech-to-Text 转录的音频的时间偏移值。

Speech-to-Text 可以在识别请求的响应文本中加上时间偏移(时间戳)值。时间偏移值能显示从提供的音频中识别出的所说字词的开始时间和结束时间。时间偏移值表示从音频开头起已经过的时间长度,以 100 毫秒为增量。

在分析较长的音频文件时,您可能需要在识别出的文字中搜索特定字词并在原始音频中对其进行定位(跳转),这种情况下时间偏移特别有用。Speech-to-Text 的时间偏移支持以下所有语音识别方法:speech:recognizespeech:longrunningrecognize流式

Speech-to-Text 仅会为识别响应中提供的第一个备选项加上时间偏移值。

如需在请求结果中加入时间偏移,请在请求配置中将 enableWordTimeOffsets 参数设置为 true

协议

如需了解完整的详细信息,请参阅 speech:longrunningrecognize API 端点。

如需执行同步语音识别,请发出 POST 请求并提供相应的请求正文。以下示例展示了一个使用 curl 发出的 POST 请求。该示例使用通过 Google Cloud Cloud SDK 为项目设置的服务帐号的访问令牌。如需了解有关安装 Cloud SDK、使用服务帐号设置项目以及获取访问令牌的说明,请参阅快速入门

    curl -X POST \
         -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
         -H "Content-Type: application/json; charset=utf-8" \
         --data "{
      'config': {
        'language_code': 'en-US',
        'enableWordTimeOffsets': true
      },
      'audio':{
        'uri':'gs://gcs-test-data/vr.flac'
      }
    }" "https://speech.googleapis.com/v1/speech:longrunningrecognize"
    

如需详细了解配置请求正文的信息,请参阅 RecognitionConfigRecognitionAudio 参考文档。

如果请求成功,服务器将返回一个 200 OK HTTP 状态代码以及 JSON 格式的响应。

    {
      "name": "7612202767953098924"
    }
    

其中,name 是为请求创建的长时间运行的操作的名称。

处理 vr.flac 文件大约需要 30 秒才能完成。如需检索该操作的结果,请向 https://speech.googleapis.com/v1/operations/ 端点发出 GET 请求。请将 your-operation-name 替换为通过 longrunningrecognize 请求收到的 name

    curl -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
         -H "Content-Type: application/json; charset=utf-8"
         "https://speech.googleapis.com/v1/operations/your-operation-name"
    

如果请求成功,服务器将返回一个 200 OK HTTP 状态代码以及 JSON 格式的响应。

    {
      "name": "7612202767953098924",
      "metadata": {
        "@type": "type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeMetadata",
        "progressPercent": 100,
        "startTime": "2017-07-20T16:36:55.033650Z",
        "lastUpdateTime": "2017-07-20T16:37:17.158630Z"
      },
      "done": true,
      "response": {
        "@type": "type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeResponse",
        "results": [
          {
            "alternatives": [
              {
                "transcript": "okay so what am I doing here...(etc)...",
                "confidence": 0.96596134,
                "words": [
                  {
                    "startTime": "1.400s",
                    "endTime": "1.800s",
                    "word": "okay"
                  },
                  {
                    "startTime": "1.800s",
                    "endTime": "2.300s",
                    "word": "so"
                  },
                  {
                    "startTime": "2.300s",
                    "endTime": "2.400s",
                    "word": "what"
                  },
                  {
                    "startTime": "2.400s",
                    "endTime": "2.600s",
                    "word": "am"
                  },
                  {
                    "startTime": "2.600s",
                    "endTime": "2.600s",
                    "word": "I"
                  },
                  {
                    "startTime": "2.600s",
                    "endTime": "2.700s",
                    "word": "doing"
                  },
                  {
                    "startTime": "2.700s",
                    "endTime": "3s",
                    "word": "here"
                  },
                  {
                    "startTime": "3s",
                    "endTime": "3.300s",
                    "word": "why"
                  },
                  {
                    "startTime": "3.300s",
                    "endTime": "3.400s",
                    "word": "am"
                  },
                  {
                    "startTime": "3.400s",
                    "endTime": "3.500s",
                    "word": "I"
                  },
                  {
                    "startTime": "3.500s",
                    "endTime": "3.500s",
                    "word": "here"
                  },
                  ...
                ]
              }
            ]
          },
          {
            "alternatives": [
              {
                "transcript": "so so what am I doing here...(etc)...",
                "confidence": 0.9642093,
              }
            ]
          }
        ]
      }
    }
    

如果该操作尚未完成,则可以通过反复发出 GET 请求来轮询此端点,直到相应响应的 done 属性为 true 为止。

gcloud 命令

如需查看完整的详细信息,请参阅 recognize-long-running 命令。

如需执行异步语音识别,请使用 gcloud 命令行工具,并提供本地文件的路径或 Google Cloud Storage 网址。包含 --include-word-time-offsets 标志。

    gcloud ml speech recognize-long-running \
        'gs://cloud-samples-tests/speech/brooklyn.flac' \
        --language-code='en-US' --include-word-time-offsets --async
    

如果请求成功,则服务器以 JSON 格式返回长音频转录操作的 ID。

    {
      "name": OPERATION_ID
    }
    

然后,您可以通过运行以下命令来获取该操作的信息。

    gcloud ml speech operations describe OPERATION_ID
    

您也可以通过运行以下命令来轮询该操作,直到它完成为止。

    gcloud ml speech operations wait OPERATION_ID
    

该操作完成后,它会以 JSON 格式返回音频。

    {
      "@type": "type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeResponse",
      "results": [
        {
          "alternatives": [
            {
              "confidence": 0.9840146,
              "transcript": "how old is the Brooklyn Bridge",
              "words": [
                {
                  "endTime": "0.300s",
                  "startTime": "0s",
                  "word": "how"
                },
                {
                  "endTime": "0.600s",
                  "startTime": "0.300s",
                  "word": "old"
                },
                {
                  "endTime": "0.800s",
                  "startTime": "0.600s",
                  "word": "is"
                },
                {
                  "endTime": "0.900s",
                  "startTime": "0.800s",
                  "word": "the"
                },
                {
                  "endTime": "1.100s",
                  "startTime": "0.900s",
                  "word": "Brooklyn"
                },
                {
                  "endTime": "1.500s",
                  "startTime": "1.100s",
                  "word": "Bridge"
                }
              ]
            }
          ]
        }
      ]
    }
    

C#

static object AsyncRecognizeGcsWords(string storageUri)
    {
        var speech = SpeechClient.Create();
        var longOperation = speech.LongRunningRecognize(new RecognitionConfig()
        {
            Encoding = RecognitionConfig.Types.AudioEncoding.Linear16,
            SampleRateHertz = 16000,
            LanguageCode = "en",
            EnableWordTimeOffsets = true,
        }, RecognitionAudio.FromStorageUri(storageUri));
        longOperation = longOperation.PollUntilCompleted();
        var response = longOperation.Result;
        foreach (var result in response.Results)
        {
            foreach (var alternative in result.Alternatives)
            {
                Console.WriteLine($"Transcript: { alternative.Transcript}");
                Console.WriteLine("Word details:");
                Console.WriteLine($" Word count:{alternative.Words.Count}");
                foreach (var item in alternative.Words)
                {
                    Console.WriteLine($"  {item.Word}");
                    Console.WriteLine($"    WordStartTime: {item.StartTime}");
                    Console.WriteLine($"    WordEndTime: {item.EndTime}");
                }
            }
        }
        return 0;
    }

Go


    func asyncWords(client *speech.Client, out io.Writer, gcsURI string) error {
    	ctx := context.Background()

    	// Send the contents of the audio file with the encoding and
    	// and sample rate information to be transcripted.
    	req := &speechpb.LongRunningRecognizeRequest{
    		Config: &speechpb.RecognitionConfig{
    			Encoding:              speechpb.RecognitionConfig_LINEAR16,
    			SampleRateHertz:       16000,
    			LanguageCode:          "en-US",
    			EnableWordTimeOffsets: true,
    		},
    		Audio: &speechpb.RecognitionAudio{
    			AudioSource: &speechpb.RecognitionAudio_Uri{Uri: gcsURI},
    		},
    	}

    	op, err := client.LongRunningRecognize(ctx, req)
    	if err != nil {
    		return err
    	}
    	resp, err := op.Wait(ctx)
    	if err != nil {
    		return err
    	}

    	// Print the results.
    	for _, result := range resp.Results {
    		for _, alt := range result.Alternatives {
    			fmt.Fprintf(out, "\"%v\" (confidence=%3f)\n", alt.Transcript, alt.Confidence)
    			for _, w := range alt.Words {
    				fmt.Fprintf(out,
    					"Word: \"%v\" (startTime=%3f, endTime=%3f)\n",
    					w.Word,
    					float64(w.StartTime.Seconds)+float64(w.StartTime.Nanos)*1e-9,
    					float64(w.EndTime.Seconds)+float64(w.EndTime.Nanos)*1e-9,
    				)
    			}
    		}
    	}
    	return nil
    }
    

Java

/**
     * Performs non-blocking speech recognition on remote FLAC file and prints the transcription as
     * well as word time offsets.
     *
     * @param gcsUri the path to the remote LINEAR16 audio file to transcribe.
     */
    public static void asyncRecognizeWords(String gcsUri) throws Exception {
      // Instantiates a client with GOOGLE_APPLICATION_CREDENTIALS
      try (SpeechClient speech = SpeechClient.create()) {

        // Configure remote file request for FLAC
        RecognitionConfig config =
            RecognitionConfig.newBuilder()
                .setEncoding(AudioEncoding.FLAC)
                .setLanguageCode("en-US")
                .setSampleRateHertz(16000)
                .setEnableWordTimeOffsets(true)
                .build();
        RecognitionAudio audio = RecognitionAudio.newBuilder().setUri(gcsUri).build();

        // Use non-blocking call for getting file transcription
        OperationFuture<LongRunningRecognizeResponse, LongRunningRecognizeMetadata> response =
            speech.longRunningRecognizeAsync(config, audio);
        while (!response.isDone()) {
          System.out.println("Waiting for response...");
          Thread.sleep(10000);
        }

        List<SpeechRecognitionResult> results = response.get().getResultsList();

        for (SpeechRecognitionResult result : results) {
          // There can be several alternative transcripts for a given chunk of speech. Just use the
          // first (most likely) one here.
          SpeechRecognitionAlternative alternative = result.getAlternativesList().get(0);
          System.out.printf("Transcription: %s\n", alternative.getTranscript());
          for (WordInfo wordInfo : alternative.getWordsList()) {
            System.out.println(wordInfo.getWord());
            System.out.printf(
                "\t%s.%s sec - %s.%s sec\n",
                wordInfo.getStartTime().getSeconds(),
                wordInfo.getStartTime().getNanos() / 100000000,
                wordInfo.getEndTime().getSeconds(),
                wordInfo.getEndTime().getNanos() / 100000000);
          }
        }
      }
    }

Node.js

// Imports the Google Cloud client library
    const speech = require('@google-cloud/speech');

    // Creates a client
    const client = new speech.SpeechClient();

    /**
     * TODO(developer): Uncomment the following lines before running the sample.
     */
    // const gcsUri = 'gs://my-bucket/audio.raw';
    // const encoding = 'Encoding of the audio file, e.g. LINEAR16';
    // const sampleRateHertz = 16000;
    // const languageCode = 'BCP-47 language code, e.g. en-US';

    const config = {
      enableWordTimeOffsets: true,
      encoding: encoding,
      sampleRateHertz: sampleRateHertz,
      languageCode: languageCode,
    };

    const audio = {
      uri: gcsUri,
    };

    const request = {
      config: config,
      audio: audio,
    };

    // Detects speech in the audio file. This creates a recognition job that you
    // can wait for now, or get its result later.
    const [operation] = await client.longRunningRecognize(request);

    // Get a Promise representation of the final result of the job
    const [response] = await operation.promise();
    response.results.forEach(result => {
      console.log(`Transcription: ${result.alternatives[0].transcript}`);
      result.alternatives[0].words.forEach(wordInfo => {
        // NOTE: If you have a time offset exceeding 2^32 seconds, use the
        // wordInfo.{x}Time.seconds.high to calculate seconds.
        const startSecs =
          `${wordInfo.startTime.seconds}` +
          '.' +
          wordInfo.startTime.nanos / 100000000;
        const endSecs =
          `${wordInfo.endTime.seconds}` +
          '.' +
          wordInfo.endTime.nanos / 100000000;
        console.log(`Word: ${wordInfo.word}`);
        console.log(`\t ${startSecs} secs - ${endSecs} secs`);
      });
    });

PHP

use Google\Cloud\Speech\V1\SpeechClient;
    use Google\Cloud\Speech\V1\RecognitionAudio;
    use Google\Cloud\Speech\V1\RecognitionConfig;
    use Google\Cloud\Speech\V1\RecognitionConfig\AudioEncoding;

    /** Uncomment and populate these variables in your code */
    // $audioFile = 'path to an audio file';

    // change these variables if necessary
    $encoding = AudioEncoding::LINEAR16;
    $sampleRateHertz = 32000;
    $languageCode = 'en-US';

    if (!extension_loaded('grpc')) {
        throw new \Exception('Install the grpc extension (pecl install grpc)');
    }

    // When true, time offsets for every word will be included in the response.
    $enableWordTimeOffsets = true;

    // get contents of a file into a string
    $content = file_get_contents($audioFile);

    // set string as audio content
    $audio = (new RecognitionAudio())
        ->setContent($content);

    // set config
    $config = (new RecognitionConfig())
        ->setEncoding($encoding)
        ->setSampleRateHertz($sampleRateHertz)
        ->setLanguageCode($languageCode)
        ->setEnableWordTimeOffsets($enableWordTimeOffsets);

    // create the speech client
    $client = new SpeechClient();

    // create the asyncronous recognize operation
    $operation = $client->longRunningRecognize($config, $audio);
    $operation->pollUntilComplete();

    if ($operation->operationSucceeded()) {
        $response = $operation->getResult();

        // each result is for a consecutive portion of the audio. iterate
        // through them to get the transcripts for the entire audio file.
        foreach ($response->getResults() as $result) {
            $alternatives = $result->getAlternatives();
            $mostLikely = $alternatives[0];
            $transcript = $mostLikely->getTranscript();
            $confidence = $mostLikely->getConfidence();
            printf('Transcript: %s' . PHP_EOL, $transcript);
            printf('Confidence: %s' . PHP_EOL, $confidence);
            foreach ($mostLikely->getWords() as $wordInfo) {
                $startTime = $wordInfo->getStartTime();
                $endTime = $wordInfo->getEndTime();
                printf('  Word: %s (start: %s, end: %s)' . PHP_EOL,
                    $wordInfo->getWord(),
                    $startTime->serializeToJsonString(),
                    $endTime->serializeToJsonString());
            }
        }
    } else {
        print_r($operation->getError());
    }

    $client->close();

Python

from google.cloud import speech_v1

    def sample_long_running_recognize(storage_uri):
        """
        Print start and end time of each word spoken in audio file from Cloud Storage

        Args:
          storage_uri URI for audio file in Cloud Storage, e.g. gs://[BUCKET]/[FILE]
        """

        client = speech_v1.SpeechClient()

        # storage_uri = 'gs://cloud-samples-data/speech/brooklyn_bridge.flac'

        # When enabled, the first result returned by the API will include a list
        # of words and the start and end time offsets (timestamps) for those words.
        enable_word_time_offsets = True

        # The language of the supplied audio
        language_code = "en-US"
        config = {
            "enable_word_time_offsets": enable_word_time_offsets,
            "language_code": language_code,
        }
        audio = {"uri": storage_uri}

        operation = client.long_running_recognize(config, audio)

        print(u"Waiting for operation to complete...")
        response = operation.result()

        # The first result includes start and end time word offsets
        result = response.results[0]
        # First alternative is the most probable result
        alternative = result.alternatives[0]
        print(u"Transcript: {}".format(alternative.transcript))
        # Print the start and end time of each word
        for word in alternative.words:
            print(u"Word: {}".format(word.word))
            print(
                u"Start time: {} seconds {} nanos".format(
                    word.start_time.seconds, word.start_time.nanos
                )
            )
            print(
                u"End time: {} seconds {} nanos".format(
                    word.end_time.seconds, word.end_time.nanos
                )
            )

    

Ruby

# storage_path = "Path to file in Cloud Storage, eg. gs://bucket/audio.raw"

    require "google/cloud/speech"

    speech = Google::Cloud::Speech.new

    config = { encoding:                 :LINEAR16,
               sample_rate_hertz:        16_000,
               language_code:            "en-US",
               enable_word_time_offsets: true }
    audio  = { uri: storage_path }

    operation = speech.long_running_recognize config, audio

    puts "Operation started"

    operation.wait_until_done!

    raise operation.results.message if operation.error?

    results = operation.response.results

    alternatives = results.first.alternatives
    alternatives.each do |alternative|
      puts "Transcription: #{alternative.transcript}"

      alternative.words.each do |word|
        start_time = word.start_time.seconds + word.start_time.nanos / 1_000_000_000.0
        end_time   = word.end_time.seconds + word.end_time.nanos / 1_000_000_000.0

        puts "Word: #{word.word} #{start_time} #{end_time}"
      end
    end