TestPipeline (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.testing

Class TestPipeline


  • public class TestPipeline
extends Pipeline
    A creator of test pipelines that can be used inside of tests that can be configured to run locally or against the live service.

    It is recommended to tag hand-selected tests for this purpose using the @RunnableOnService category annotation, and use a category aware test executor such as JUnit. Each test run against the service will spin up and tear down VMs.

    In order to run tests on Google Cloud Dataflow, the following conditions must be met:

    • System property runIntegrationTestOnService must be set to true.
    • System property dataflowOptions must contain a JSON delimited list of pipeline options with all options that the Google Cloud Dataflow service requires. For example: 
       [
     "--project=mygcpproject",
     "--stagingLocation=gs://mygcsbucket/path"
     ]
      Note that the set of pipeline options required can also be test specific.
    • Jars containing the SDK and test classes must be available on the classpath.

    Use DataflowAssert for tests, as it integrates with this test harness in both direct and remote execution modes. For example: 

    
 Pipeline p = TestPipeline.create();
 PCollection<Integer> output = ...

 DataflowAssert.that(output)
     .containsInAnyOrder(1, 2, 3, 4);
 p.run();