ShuffledSequenceFileInputFormat (Cloud Bigtable HBase Client for Java 1.12.0 API)

com.google.cloud.bigtable.mapreduce

Class ShuffledSequenceFileInputFormat<K,V>

• java.lang.Object
• • org.apache.hadoop.mapreduce.InputFormat<K,V>
  • • org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>
    • • org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat<K,V>
      • • com.google.cloud.bigtable.mapreduce.ShuffledSequenceFileInputFormat<K,V>

• ---

  
  

  public class ShuffledSequenceFileInputFormat<K,V>
  extends org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat<K,V>

  An InputFormat randomizes the order of SequenceFile splits. This is needed for the Bigtable import job to spread the load across all of the Bigtable tablet servers.

  Version:

  $Id: $Id

  Author:

  igorbernstein2

• • Nested Class Summary

    • Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

      org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter

  • Field Summary

    • Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

      DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

  • Constructor Summary

    Constructors 

    Constructor and Description
    ShuffledSequenceFileInputFormat()

  • Method Summary

    All Methods Instance Methods Concrete Methods 

    Modifier and Type	Method and Description
    List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext job)

    • Methods inherited from class org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat

      createRecordReader, getFormatMinSplitSize, listStatus

    • Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

      addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, isSplitable, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

    • Methods inherited from class java.lang.Object

      clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

• • Constructor Detail

    • ShuffledSequenceFileInputFormat

      public ShuffledSequenceFileInputFormat()

  • Method Detail

    • getSplits

      public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job)
                                                             throws IOException

      Overrides:

      getSplits in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>

      Throws:

      IOException