ShuffledSequenceFileInputFormat (Cloud Bigtable HBase Client for Java 0.9.7.1 API)

com.google.cloud.bigtable.mapreduce

Class ShuffledSequenceFileInputFormat<K,V>

  • java.lang.Object
    • org.apache.hadoop.mapreduce.InputFormat<K,V>
      • org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>
        • org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat<K,V>
          • com.google.cloud.bigtable.mapreduce.ShuffledSequenceFileInputFormat<K,V>


  • public class ShuffledSequenceFileInputFormat<K,V>
    extends org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat<K,V>
    An InputFormat randomizes the order of SequenceFile splits. This is needed for the Bigtable import job to spread the load across all of the Bigtable tablet servers.
    Version:
    $Id: $Id
    Author:
    igorbernstein2
    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

        org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter
    • Field Summary

      • Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

        DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method and Description
      List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job) 
      • Methods inherited from class org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat

        createRecordReader, getFormatMinSplitSize, listStatus
      • Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

        addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, isSplitable, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize
    • Constructor Detail

      • ShuffledSequenceFileInputFormat

        public ShuffledSequenceFileInputFormat()
    • Method Detail

      • getSplits

        public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job)
                                                               throws IOException
        Overrides:
        getSplits in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>
        Throws:
        IOException


Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Cloud Bigtable Documentation