google-cloud-dataflow overview (0.5.0)

Version 0.5.0

• 0.59.0 (latest)
• 0.58.0
• 0.57.0
• 0.56.0
• 0.55.0
• 0.53.0
• 0.52.0
• 0.51.0
• 0.50.0
• 0.49.0
• 0.48.0
• 0.47.0
• 0.46.0
• 0.45.0
• 0.44.0
• 0.43.0
• 0.41.0
• 0.40.0
• 0.39.0
• 0.38.0
• 0.37.0
• 0.36.0
• 0.35.0
• 0.34.0
• 0.33.0
• 0.32.0
• 0.31.0
• 0.28.0
• 0.27.0
• 0.26.0
• 0.25.0
• 0.24.0
• 0.23.0
• 0.22.0
• 0.21.0
• 0.20.0
• 0.19.0
• 0.18.0
• 0.17.0
• 0.16.0
• 0.15.0
• 0.13.0
• 0.12.0
• 0.11.0
• 0.10.0
• 0.9.0
• 0.8.0
• 0.5.0

com.google.dataflow.v1beta3

A client to Dataflow API

The interfaces provided are listed below, along with usage samples.

JobsV1Beta3Client

Service Description: Provides a method to create and modify Google Cloud Dataflow jobs. A Job is a multi-stage computation graph run by the Cloud Dataflow service.

Sample for JobsV1Beta3Client:

 try (JobsV1Beta3Client jobsV1Beta3Client = JobsV1Beta3Client.create()) {
   CreateJobRequest request =
       CreateJobRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setJob(Job.newBuilder().build())
           .setView(JobView.forNumber(0))
           .setReplaceJobId("replaceJobId441554788")
           .setLocation("location1901043637")
           .build();
   Job response = jobsV1Beta3Client.createJob(request);
 }
 

MessagesV1Beta3Client

Service Description: The Dataflow Messages API is used for monitoring the progress of Dataflow jobs.

Sample for MessagesV1Beta3Client:

 try (MessagesV1Beta3Client messagesV1Beta3Client = MessagesV1Beta3Client.create()) {
   ListJobMessagesRequest request =
       ListJobMessagesRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setJobId("jobId101296568")
           .setMinimumImportance(JobMessageImportance.forNumber(0))
           .setPageSize(883849137)
           .setPageToken("pageToken873572522")
           .setStartTime(Timestamp.newBuilder().build())
           .setEndTime(Timestamp.newBuilder().build())
           .setLocation("location1901043637")
           .build();
   for (JobMessage element : messagesV1Beta3Client.listJobMessages(request).iterateAll()) {
     // doThingsWith(element);
   }
 }
 

MetricsV1Beta3Client

Service Description: The Dataflow Metrics API lets you monitor the progress of Dataflow jobs.

Sample for MetricsV1Beta3Client:

 try (MetricsV1Beta3Client metricsV1Beta3Client = MetricsV1Beta3Client.create()) {
   GetJobMetricsRequest request =
       GetJobMetricsRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setJobId("jobId101296568")
           .setStartTime(Timestamp.newBuilder().build())
           .setLocation("location1901043637")
           .build();
   JobMetrics response = metricsV1Beta3Client.getJobMetrics(request);
 }
 

SnapshotsV1Beta3Client

Service Description: Provides methods to manage snapshots of Google Cloud Dataflow jobs.

Sample for SnapshotsV1Beta3Client:

 try (SnapshotsV1Beta3Client snapshotsV1Beta3Client = SnapshotsV1Beta3Client.create()) {
   GetSnapshotRequest request =
       GetSnapshotRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setSnapshotId("snapshotId-1113817601")
           .setLocation("location1901043637")
           .build();
   Snapshot response = snapshotsV1Beta3Client.getSnapshot(request);
 }
 

TemplatesServiceClient

Service Description: Provides a method to create Cloud Dataflow jobs from templates.

Sample for TemplatesServiceClient:

 try (TemplatesServiceClient templatesServiceClient = TemplatesServiceClient.create()) {
   CreateJobFromTemplateRequest request =
       CreateJobFromTemplateRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setJobName("jobName-1438096408")
           .putAllParameters(new HashMap<String, String>())
           .setEnvironment(RuntimeEnvironment.newBuilder().build())
           .setLocation("location1901043637")
           .build();
   Job response = templatesServiceClient.createJobFromTemplate(request);
 }
 

FlexTemplatesServiceClient

Service Description: Provides a service for Flex templates. This feature is not ready yet.

Sample for FlexTemplatesServiceClient:

 try (FlexTemplatesServiceClient flexTemplatesServiceClient =
     FlexTemplatesServiceClient.create()) {
   LaunchFlexTemplateRequest request =
       LaunchFlexTemplateRequest.newBuilder()
           .setProjectId("projectId-894832108")
           .setLaunchParameter(LaunchFlexTemplateParameter.newBuilder().build())
           .setLocation("location1901043637")
           .setValidateOnly(true)
           .build();
   LaunchFlexTemplateResponse response = flexTemplatesServiceClient.launchFlexTemplate(request);
 }
 

com.google.dataflow.v1beta3.stub