Creating jobs in batch mode

This page shows you how to create Transcoder API jobs in batch mode. Use batch mode to submit many jobs for which the processing latency isn't as critical as jobs submitted in the default interactive mode. For more information on job modes, see the overview.

You can specify the job processing priority for a job in this mode. Jobs submitted in batch mode with the same priority value are put in a first-in, first out queue and have a job state of PENDING while they wait to be run.

Set up your Google Cloud project and authentication

If you have not created a Google Cloud project and credentials, see Before you begin.

Set the job mode

To start a job in batch mode, set the job mode to PROCESSING_MODE_BATCH.

The following samples use the default job preset.

REST

Before using any of the request data, make the following replacements:

  • PROJECT_ID: Your Google Cloud project ID listed in the IAM Settings.
  • LOCATION: The location where your job will run. Use one of the supported regions.
    Show locations
    • us-central1
    • us-west1
    • us-west2
    • us-east1
    • us-east4
    • southamerica-east1
    • northamerica-northeast1
    • asia-east1
    • asia-northeast1
    • asia-northeast3
    • asia-south1
    • asia-southeast1
    • australia-southeast1
    • europe-west1
    • europe-west2
    • europe-west4
  • STORAGE_BUCKET_NAME: The name of the Cloud Storage bucket you created.
  • STORAGE_INPUT_VIDEO: The name of the video in your Cloud Storage bucket that you are transcoding, such as my-vid.mp4. This field should take into account any folders that you created in the bucket (for example, input/my-vid.mp4).
  • STORAGE_OUTPUT_FOLDER: The Cloud Storage folder name where you want to save the encoded video outputs.

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION/jobs/JOB_ID",
  "config": {
    "inputs": [
      {
        "key": "input0",
        "uri": "gs://STORAGE_BUCKET_NAME/STORAGE_INPUT_VIDEO"
      }
    ],
    "editList": [
      {
        "key": "atom0",
        "inputs": [
          "input0"
        ],
        "startTimeOffset": "0s"
      }
    ],
    "elementaryStreams": [
      {
        "videoStream": {
          "h264": {
            "widthPixels": 640,
            "heightPixels": 360,
            "frameRate": 30,
            "bitrateBps": 550000,
            "pixelFormat": "yuv420p",
            "rateControlMode": "vbr",
            "crfLevel": 21,
            "gopDuration": "3s",
            "vbvSizeBits": 550000,
            "vbvFullnessBits": 495000,
            "entropyCoder": "cabac",
            "bFrameCount": 3,
            "aqStrength": 1,
            "profile": "high",
            "preset": "veryfast"
          }
        },
        "key": "video-stream0"
      },
      {
        "videoStream": {
          "h264": {
            "widthPixels": 1280,
            "heightPixels": 720,
            "frameRate": 30,
            "bitrateBps": 2500000,
            "pixelFormat": "yuv420p",
            "rateControlMode": "vbr",
            "crfLevel": 21,
            "gopDuration": "3s",
            "vbvSizeBits": 2500000,
            "vbvFullnessBits": 2250000,
            "entropyCoder": "cabac",
            "bFrameCount": 3,
            "aqStrength": 1,
            "profile": "high",
            "preset": "veryfast"
          }
        },
        "key": "video-stream1"
      },
      {
        "audioStream": {
          "codec": "aac",
          "bitrateBps": 64000,
          "channelCount": 2,
          "channelLayout": [
            "fl",
            "fr"
          ],
          "sampleRateHertz": 48000
        },
        "key": "audio-stream0"
      }
    ],
    "muxStreams": [
      {
        "key": "sd",
        "fileName": "sd.mp4",
        "container": "mp4",
        "elementaryStreams": [
          "video-stream0",
          "audio-stream0"
        ]
      },
      {
        "key": "hd",
        "fileName": "hd.mp4",
        "container": "mp4",
        "elementaryStreams": [
          "video-stream1",
          "audio-stream0"
        ]
      },
      {
        "key": "media-sd",
        "fileName": "media-sd.ts",
        "container": "ts",
        "elementaryStreams": [
          "video-stream0",
          "audio-stream0"
        ]
      },
      {
        "key": "media-hd",
        "fileName": "media-hd.ts",
        "container": "ts",
        "elementaryStreams": [
          "video-stream1",
          "audio-stream0"
        ]
      },
      {
        "key": "video-only-sd",
        "fileName": "video-only-sd.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "video-stream0"
        ]
      },
      {
        "key": "video-only-hd",
        "fileName": "video-only-hd.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "video-stream1"
        ]
      },
      {
        "key": "audio-only",
        "fileName": "audio-only.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "audio-stream0"
        ]
      }
    ],
    "manifests": [
      {
        "fileName": "manifest.m3u8",
        "type": "HLS",
        "muxStreams": [
          "media-sd",
          "media-hd"
        ]
      },
      {
        "fileName": "manifest.mpd",
        "type": "DASH",
        "muxStreams": [
          "video-only-sd",
          "video-only-hd",
          "audio-only"
        ]
      }
    ],
    "output": {
      "uri": "gs://STORAGE_BUCKET_NAME/STORAGE_OUTPUT_FOLDER/"
    }
  },
  "state": "PENDING",
  "createTime": CREATE_TIME,
  "ttlAfterCompletionDays": 30
}

gcloud

Before using any of the command data below, make the following replacements:

  • LOCATION: The location where your job will run. Use one of the supported regions.
    Show locations
    • us-central1
    • us-west1
    • us-west2
    • us-east1
    • us-east4
    • southamerica-east1
    • northamerica-northeast1
    • asia-east1
    • asia-northeast1
    • asia-northeast3
    • asia-south1
    • asia-southeast1
    • australia-southeast1
    • europe-west1
    • europe-west2
    • europe-west4
  • STORAGE_BUCKET_NAME: The name of the Cloud Storage bucket you created.
  • STORAGE_INPUT_VIDEO: The name of the video in your Cloud Storage bucket that you are transcoding, such as my-vid.mp4. This field should take into account any folders that you created in the bucket (for example, input/my-vid.mp4).
  • STORAGE_OUTPUT_FOLDER: The Cloud Storage folder name where you want to save the encoded video outputs.

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud transcoder jobs create \
  --input-uri="gs://STORAGE_BUCKET_NAME/STORAGE_INPUT_VIDEO" \
  --location=LOCATION \
  --output-uri="gs://STORAGE_BUCKET_NAME/STORAGE_OUTPUT_FOLDER/" \
  --template-id="preset/web-hd" \
  --batch-mode-priority=10

Windows (PowerShell)

gcloud transcoder jobs create `
  --input-uri="gs://STORAGE_BUCKET_NAME/STORAGE_INPUT_VIDEO" `
  --location=LOCATION `
  --output-uri="gs://STORAGE_BUCKET_NAME/STORAGE_OUTPUT_FOLDER/" `
  --template-id="preset/web-hd" `
  --batch-mode-priority=10

Windows (cmd.exe)

gcloud transcoder jobs create ^
  --input-uri="gs://STORAGE_BUCKET_NAME/STORAGE_INPUT_VIDEO" ^
  --location=LOCATION ^
  --output-uri="gs://STORAGE_BUCKET_NAME/STORAGE_OUTPUT_FOLDER/" ^
  --template-id="preset/web-hd" ^
  --batch-mode-priority=10

You should receive a response similar to the following:

Response

{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION/jobs/JOB_ID",
  "config": {
    "inputs": [
      {
        "key": "input0",
        "uri": "gs://STORAGE_BUCKET_NAME/STORAGE_INPUT_VIDEO"
      }
    ],
    "editList": [
      {
        "key": "atom0",
        "inputs": [
          "input0"
        ],
        "startTimeOffset": "0s"
      }
    ],
    "elementaryStreams": [
      {
        "videoStream": {
          "h264": {
            "widthPixels": 640,
            "heightPixels": 360,
            "frameRate": 30,
            "bitrateBps": 550000,
            "pixelFormat": "yuv420p",
            "rateControlMode": "vbr",
            "crfLevel": 21,
            "gopDuration": "3s",
            "vbvSizeBits": 550000,
            "vbvFullnessBits": 495000,
            "entropyCoder": "cabac",
            "bFrameCount": 3,
            "aqStrength": 1,
            "profile": "high",
            "preset": "veryfast"
          }
        },
        "key": "video-stream0"
      },
      {
        "videoStream": {
          "h264": {
            "widthPixels": 1280,
            "heightPixels": 720,
            "frameRate": 30,
            "bitrateBps": 2500000,
            "pixelFormat": "yuv420p",
            "rateControlMode": "vbr",
            "crfLevel": 21,
            "gopDuration": "3s",
            "vbvSizeBits": 2500000,
            "vbvFullnessBits": 2250000,
            "entropyCoder": "cabac",
            "bFrameCount": 3,
            "aqStrength": 1,
            "profile": "high",
            "preset": "veryfast"
          }
        },
        "key": "video-stream1"
      },
      {
        "audioStream": {
          "codec": "aac",
          "bitrateBps": 64000,
          "channelCount": 2,
          "channelLayout": [
            "fl",
            "fr"
          ],
          "sampleRateHertz": 48000
        },
        "key": "audio-stream0"
      }
    ],
    "muxStreams": [
      {
        "key": "sd",
        "fileName": "sd.mp4",
        "container": "mp4",
        "elementaryStreams": [
          "video-stream0",
          "audio-stream0"
        ]
      },
      {
        "key": "hd",
        "fileName": "hd.mp4",
        "container": "mp4",
        "elementaryStreams": [
          "video-stream1",
          "audio-stream0"
        ]
      },
      {
        "key": "media-sd",
        "fileName": "media-sd.ts",
        "container": "ts",
        "elementaryStreams": [
          "video-stream0",
          "audio-stream0"
        ]
      },
      {
        "key": "media-hd",
        "fileName": "media-hd.ts",
        "container": "ts",
        "elementaryStreams": [
          "video-stream1",
          "audio-stream0"
        ]
      },
      {
        "key": "video-only-sd",
        "fileName": "video-only-sd.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "video-stream0"
        ]
      },
      {
        "key": "video-only-hd",
        "fileName": "video-only-hd.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "video-stream1"
        ]
      },
      {
        "key": "audio-only",
        "fileName": "audio-only.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "audio-stream0"
        ]
      }
    ],
    "manifests": [
      {
        "fileName": "manifest.m3u8",
        "type": "HLS",
        "muxStreams": [
          "media-sd",
          "media-hd"
        ]
      },
      {
        "fileName": "manifest.mpd",
        "type": "DASH",
        "muxStreams": [
          "video-only-sd",
          "video-only-hd",
          "audio-only"
        ]
      }
    ],
    "output": {
      "uri": "gs://STORAGE_BUCKET_NAME/STORAGE_OUTPUT_FOLDER/"
    }
  },
  "state": "PENDING",
  "createTime": CREATE_TIME,
  "ttlAfterCompletionDays": 30
}

C#

Before trying this sample, follow the C# setup instructions in the Transcoder API quickstart using client libraries. For more information, see the Transcoder API C# API reference documentation.

To authenticate to Transcoder API, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.


using Google.Api.Gax.ResourceNames;
using Google.Cloud.Video.Transcoder.V1;

public class CreateJobFromPresetBatchModeSample
{
    public Job CreateJobFromPresetBatchMode(
        string projectId, string location, string inputUri, string outputUri, string preset)
    {
        // Create the client.
        TranscoderServiceClient client = TranscoderServiceClient.Create();

        // Build the parent location name.
        LocationName parent = new LocationName(projectId, location);

        // Build the job.
        Job newJob = new Job
        {
            InputUri = inputUri,
            OutputUri = outputUri,
            TemplateId = preset,
            Mode = Job.Types.ProcessingMode.Batch,
            BatchModePriority = 10
        };

        // Call the API.
        Job job = client.CreateJob(parent, newJob);

        // Return the result.
        return job;
    }
}

Go

Before trying this sample, follow the Go setup instructions in the Transcoder API quickstart using client libraries. For more information, see the Transcoder API Go API reference documentation.

To authenticate to Transcoder API, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

import (
	"context"
	"fmt"
	"io"

	transcoder "cloud.google.com/go/video/transcoder/apiv1"
	"cloud.google.com/go/video/transcoder/apiv1/transcoderpb"
)

// createJobFromPresetBatchMode creates a batch mode job based on a given preset
// template. See
// https://cloud.google.com/transcoder/docs/how-to/create-batch-jobs
// for more information.
func createJobFromPresetBatchMode(w io.Writer, projectID string, location string, inputURI string, outputURI string) error {
	// projectID := "my-project-id"
	// location := "us-central1"
	// inputURI := "gs://my-bucket/my-video-file"
	// outputURI := "gs://my-bucket/my-output-folder/"
	preset := "preset/web-hd"
	ctx := context.Background()
	client, err := transcoder.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("NewClient: %w", err)
	}
	defer client.Close()

	req := &transcoderpb.CreateJobRequest{
		Parent: fmt.Sprintf("projects/%s/locations/%s", projectID, location),
		Job: &transcoderpb.Job{
			InputUri:  inputURI,
			OutputUri: outputURI,
			JobConfig: &transcoderpb.Job_TemplateId{
				TemplateId: preset,
			},
			Mode:              transcoderpb.Job_PROCESSING_MODE_BATCH,
			BatchModePriority: 10,
		},
	}
	// Creates the job, Jobs take a variable amount of time to run.
	// You can query for the job state.
	response, err := client.CreateJob(ctx, req)
	if err != nil {
		return fmt.Errorf("createJobFromPresetBatchMode: %w", err)
	}

	fmt.Fprintf(w, "Job: %v", response.GetName())
	return nil
}

Java

Before trying this sample, follow the Java setup instructions in the Transcoder API quickstart using client libraries. For more information, see the Transcoder API Java API reference documentation.

To authenticate to Transcoder API, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.


import com.google.cloud.video.transcoder.v1.CreateJobRequest;
import com.google.cloud.video.transcoder.v1.Job;
import com.google.cloud.video.transcoder.v1.Job.ProcessingMode;
import com.google.cloud.video.transcoder.v1.LocationName;
import com.google.cloud.video.transcoder.v1.TranscoderServiceClient;
import java.io.IOException;

public class CreateJobFromPresetBatchMode {

  public static void main(String[] args) throws Exception {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "my-project-id";
    String location = "us-central1";
    String inputUri = "gs://my-bucket/my-video-file";
    String outputUri = "gs://my-bucket/my-output-folder/";
    // See https://cloud.google.com/transcoder/docs/concepts/overview#job_template
    // for information on this preset.
    String preset = "preset/web-hd";

    createJobFromPresetBatchMode(projectId, location, inputUri, outputUri, preset);
  }

  // Creates a job from a preset in batch mode.
  public static void createJobFromPresetBatchMode(
      String projectId, String location, String inputUri, String outputUri, String preset)
      throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests.
    try (TranscoderServiceClient transcoderServiceClient = TranscoderServiceClient.create()) {

      CreateJobRequest createJobRequest =
          CreateJobRequest.newBuilder()
              .setJob(
                  Job.newBuilder()
                      .setInputUri(inputUri)
                      .setOutputUri(outputUri)
                      .setTemplateId(preset)
                      .setMode(ProcessingMode.PROCESSING_MODE_BATCH)
                      .setBatchModePriority(10)
                      .build())
              .setParent(LocationName.of(projectId, location).toString())
              .build();

      // Send the job creation request and process the response.
      Job job = transcoderServiceClient.createJob(createJobRequest);
      System.out.println("Job: " + job.getName());
    }
  }
}

Node.js

Before trying this sample, follow the Node.js setup instructions in the Transcoder API quickstart using client libraries. For more information, see the Transcoder API Node.js API reference documentation.

To authenticate to Transcoder API, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

/**
 * TODO(developer): Uncomment these variables before running the sample.
 */
// projectId = 'my-project-id';
// location = 'us-central1';
// inputUri = 'gs://my-bucket/my-video-file';
// outputUri = 'gs://my-bucket/my-output-folder/';
// preset = 'preset/web-hd';

// Imports the Transcoder library
const {TranscoderServiceClient} =
  require('@google-cloud/video-transcoder').v1;

// Instantiates a client
const transcoderServiceClient = new TranscoderServiceClient();

async function createJobFromPresetBatchMode() {
  // Construct request
  const request = {
    parent: transcoderServiceClient.locationPath(projectId, location),
    job: {
      inputUri: inputUri,
      outputUri: outputUri,
      templateId: preset,
      mode: 'PROCESSING_MODE_BATCH',
      batchModePriority: 10,
    },
  };

  // Run request
  const [response] = await transcoderServiceClient.createJob(request);
  console.log(`Job: ${response.name}`);
}

createJobFromPresetBatchMode();

PHP

Before trying this sample, follow the PHP setup instructions in the Transcoder API quickstart using client libraries. For more information, see the Transcoder API PHP API reference documentation.

To authenticate to Transcoder API, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

use Google\Cloud\Video\Transcoder\V1\Client\TranscoderServiceClient;
use Google\Cloud\Video\Transcoder\V1\CreateJobRequest;
use Google\Cloud\Video\Transcoder\V1\Job;

/**
 * Creates a job in batch mode based on a job preset.
 *
 * @param string $projectId The ID of your Google Cloud Platform project.
 * @param string $location The location of the job.
 * @param string $inputUri Uri of the video in the Cloud Storage bucket.
 * @param string $outputUri Uri of the video output folder in the Cloud Storage bucket.
 * @param string $preset The preset template (for example, "preset/web-hd").
 */
function create_job_from_preset_batch_mode($projectId, $location, $inputUri, $outputUri, $preset)
{
    // Instantiate a client.
    $transcoderServiceClient = new TranscoderServiceClient();

    $formattedParent = $transcoderServiceClient->locationName($projectId, $location);
    $job = new Job();
    $job->setInputUri($inputUri);
    $job->setOutputUri($outputUri);
    $job->setTemplateId($preset);
    $job->setMode(Job\ProcessingMode::PROCESSING_MODE_BATCH);
    $job->setBatchModePriority(10);
    $request = (new CreateJobRequest())
        ->setParent($formattedParent)
        ->setJob($job);

    $response = $transcoderServiceClient->createJob($request);

    // Print job name.
    printf('Job: %s' . PHP_EOL, $response->getName());
}

Python

Before trying this sample, follow the Python setup instructions in the Transcoder API quickstart using client libraries. For more information, see the Transcoder API Python API reference documentation.

To authenticate to Transcoder API, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.


import argparse

from google.cloud.video import transcoder_v1
from google.cloud.video.transcoder_v1.services.transcoder_service import (
    TranscoderServiceClient,
)


def create_job_from_preset_batch_mode(
    project_id: str,
    location: str,
    input_uri: str,
    output_uri: str,
    preset: str,
) -> transcoder_v1.types.resources.Job:
    """Creates a job in batch mode based on a job preset.

    Args:
        project_id: The GCP project ID.
        location: The location to start the job in.
        input_uri: Uri of the video in the Cloud Storage bucket.
        output_uri: Uri of the video output folder in the Cloud Storage bucket.
        preset: The preset template (for example, 'preset/web-hd').

    Returns:
        The job resource.
    """

    client = TranscoderServiceClient()

    parent = f"projects/{project_id}/locations/{location}"
    job = transcoder_v1.types.Job()
    job.input_uri = input_uri
    job.output_uri = output_uri
    job.template_id = preset
    job.mode = transcoder_v1.types.Job.ProcessingMode.PROCESSING_MODE_BATCH
    job.batch_mode_priority = 10

    response = client.create_job(parent=parent, job=job)
    print(f"Job: {response.name}")
    return response

Ruby

Before trying this sample, follow the Ruby setup instructions in the Transcoder API quickstart using client libraries. For more information, see the Transcoder API Ruby API reference documentation.

To authenticate to Transcoder API, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

# project_id  = "YOUR-GOOGLE-CLOUD-PROJECT"  # (e.g. "my-project")
# location    = "YOUR-JOB-LOCATION"  # (e.g. "us-central1")
# input_uri   = "YOUR-GCS-INPUT-VIDEO"  # (e.g. "gs://my-bucket/my-video-file")
# output_uri  = "YOUR-GCS-OUTPUT-FOLDER/"  # (e.g. "gs://my-bucket/my-output-folder/")
# preset      = "YOUR-JOB-PRESET"  # (e.g. "preset/web-hd")

# Require the Transcoder client library.
require "google/cloud/video/transcoder"

# Create a Transcoder client.
client = Google::Cloud::Video::Transcoder.transcoder_service

# Build the resource name of the parent.
parent = client.location_path project: project_id, location: location

# Set the job fields.
new_job = {
  input_uri: input_uri,
  output_uri: output_uri,
  template_id: preset,
  mode: Google::Cloud::Video::Transcoder::V1::Job::ProcessingMode::PROCESSING_MODE_BATCH,
  batch_mode_priority: 10
}

job = client.create_job parent: parent, job: new_job

# Print the job name.
puts "Job: #{job.name}"