Import (Cloud Bigtable HBase Client for Java 1.12.0 API)

com.google.cloud.bigtable.mapreduce

Class Import

  • java.lang.Object
    • org.apache.hadoop.conf.Configured
      • com.google.cloud.bigtable.mapreduce.Import
  • All Implemented Interfaces:
    org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool


    @InterfaceAudience.Public
     @InterfaceStability.Stable
    public class Import
    extends org.apache.hadoop.conf.Configured
    implements org.apache.hadoop.util.Tool
    Import data written by Export.
    Version:
    $Id: $Id
    Author:
    sduskis
    • Field Detail

      • CF_RENAME_PROP

        public static final String CF_RENAME_PROP
        Constant CF_RENAME_PROP="HBASE_IMPORTER_RENAME_CFS"
        See Also:
        Constant Field Values
      • BULK_OUTPUT_CONF_KEY

        public static final String BULK_OUTPUT_CONF_KEY
        Constant BULK_OUTPUT_CONF_KEY="import.bulk.output"
        See Also:
        Constant Field Values
      • FILTER_CLASS_CONF_KEY

        public static final String FILTER_CLASS_CONF_KEY
        Constant FILTER_CLASS_CONF_KEY="import.filter.class"
        See Also:
        Constant Field Values
      • FILTER_ARGS_CONF_KEY

        public static final String FILTER_ARGS_CONF_KEY
        Constant FILTER_ARGS_CONF_KEY="import.filter.args"
        See Also:
        Constant Field Values
    • Constructor Detail

      • Import

        public Import()
    • Method Detail

      • instantiateFilter

        public static Filter instantiateFilter(org.apache.hadoop.conf.Configuration conf)
        Create a Filter to apply to all incoming keys (KeyValues) to optionally not include in the job output
        Parameters:
        conf - Configuration from which to load the filter
        Returns:
        the filter to use for the task, or null if no filter to should be used
        Throws:
        IllegalArgumentException - if the filter is misconfigured
      • filterKv

        public static Cell filterKv(Filter filter,
                                    Cell kv)
                             throws IOException
        Attempt to filter out the keyvalue
        Parameters:
        kv - KeyValue on which to apply the filter
        filter - a Filter object.
        Returns:
        null if the key should not be written, otherwise returns the original KeyValue
        Throws:
        IOException - if any.
      • configureCfRenaming

        public static void configureCfRenaming(org.apache.hadoop.conf.Configuration conf,
                                               Map<String,String> renameMap)
        Sets a configuration property with key CF_RENAME_PROP in conf that tells the mapper how to rename column families.

        Alternately, instead of calling this function, you could set the configuration key CF_RENAME_PROP yourself. The value should look like

        srcCf1:destCf1,srcCf2:destCf2,....
        . This would have the same effect on the mapper behavior.

        Parameters:
        conf - the Configuration in which the CF_RENAME_PROP key will be set
        renameMap - a mapping from source CF names to destination CF names
      • addFilterAndArguments

        public static void addFilterAndArguments(org.apache.hadoop.conf.Configuration conf,
                                                 Class<? extends Filter> clazz,
                                                 List<String> filterArgs)
        Add a Filter to be instantiated on import
        Parameters:
        conf - Configuration to update (will be passed to the job)
        clazz - Filter subclass to instantiate on the server.
        filterArgs - List of arguments to pass to the filter on instantiation
      • createSubmittableJob

        public static org.apache.hadoop.mapreduce.Job createSubmittableJob(org.apache.hadoop.conf.Configuration conf,
                                                                           String[] args)
                                                                    throws IOException
        Sets up the actual job.
        Parameters:
        conf - The current configuration.
        args - The command line parameters.
        Returns:
        The newly created job.
        Throws:
        IOException - When setting up the job fails.
      • run

        public int run(String[] args)
                throws Exception
        Specified by:
        run in interface org.apache.hadoop.util.Tool
        Throws:
        Exception
      • main

        public static void main(String[] args)
                         throws Exception
        Main entry point.
        Parameters:
        args - The command line parameters.
        Throws:
        Exception - When running the job fails.