com.google.cloud.bigtable.mapreduce
Class Import
- java.lang.Object
-
- org.apache.hadoop.conf.Configured
-
- com.google.cloud.bigtable.mapreduce.Import
-
- All Implemented Interfaces:
- org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
@InterfaceAudience.Public @InterfaceStability.Stable public class Import extends org.apache.hadoop.conf.Configured implements org.apache.hadoop.util.Tool
Import data written byExport
.- Version:
- $Id: $Id
- Author:
- sduskis
-
-
Nested Class Summary
Nested Classes Modifier and Type Class and Description static class
Import.Importer
Write table content out to files.static class
Import.KeyValueImporter
A mapper that just writes out KeyValues.
-
Field Summary
Fields Modifier and Type Field and Description static String
BULK_OUTPUT_CONF_KEY
ConstantBULK_OUTPUT_CONF_KEY="import.bulk.output"
static String
CF_RENAME_PROP
ConstantCF_RENAME_PROP="HBASE_IMPORTER_RENAME_CFS"
static String
FILTER_ARGS_CONF_KEY
ConstantFILTER_ARGS_CONF_KEY="import.filter.args"
static String
FILTER_CLASS_CONF_KEY
ConstantFILTER_CLASS_CONF_KEY="import.filter.class"
static String
TABLE_NAME
ConstantTABLE_NAME="import.table.name"
-
Constructor Summary
Constructors Constructor and Description Import()
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method and Description static void
addFilterAndArguments(org.apache.hadoop.conf.Configuration conf, Class<? extends Filter> clazz, List<String> filterArgs)
Add a Filter to be instantiated on importstatic void
configureCfRenaming(org.apache.hadoop.conf.Configuration conf, Map<String,String> renameMap)
Sets a configuration property with keyCF_RENAME_PROP
in conf that tells the mapper how to rename column families.static org.apache.hadoop.mapreduce.Job
createSubmittableJob(org.apache.hadoop.conf.Configuration conf, String[] args)
Sets up the actual job.static Cell
filterKv(Filter filter, Cell kv)
Attempt to filter out the keyvaluestatic Filter
instantiateFilter(org.apache.hadoop.conf.Configuration conf)
static void
main(String[] args)
Main entry point.int
run(String[] args)
-
-
-
Field Detail
-
CF_RENAME_PROP
public static final String CF_RENAME_PROP
ConstantCF_RENAME_PROP="HBASE_IMPORTER_RENAME_CFS"
- See Also:
- Constant Field Values
-
BULK_OUTPUT_CONF_KEY
public static final String BULK_OUTPUT_CONF_KEY
ConstantBULK_OUTPUT_CONF_KEY="import.bulk.output"
- See Also:
- Constant Field Values
-
FILTER_CLASS_CONF_KEY
public static final String FILTER_CLASS_CONF_KEY
ConstantFILTER_CLASS_CONF_KEY="import.filter.class"
- See Also:
- Constant Field Values
-
FILTER_ARGS_CONF_KEY
public static final String FILTER_ARGS_CONF_KEY
ConstantFILTER_ARGS_CONF_KEY="import.filter.args"
- See Also:
- Constant Field Values
-
TABLE_NAME
public static final String TABLE_NAME
ConstantTABLE_NAME="import.table.name"
- See Also:
- Constant Field Values
-
-
Method Detail
-
instantiateFilter
public static Filter instantiateFilter(org.apache.hadoop.conf.Configuration conf)
Create aFilter
to apply to all incoming keys (KeyValues
) to optionally not include in the job output- Parameters:
conf
-Configuration
from which to load the filter- Returns:
- the filter to use for the task, or null if no filter to should be used
- Throws:
IllegalArgumentException
- if the filter is misconfigured
-
filterKv
public static Cell filterKv(Filter filter, Cell kv) throws IOException
Attempt to filter out the keyvalue- Parameters:
kv
-KeyValue
on which to apply the filterfilter
- aFilter
object.- Returns:
- null if the key should not be written, otherwise returns the original
KeyValue
- Throws:
IOException
- if any.
-
configureCfRenaming
public static void configureCfRenaming(org.apache.hadoop.conf.Configuration conf, Map<String,String> renameMap)
Sets a configuration property with keyCF_RENAME_PROP
in conf that tells the mapper how to rename column families.Alternately, instead of calling this function, you could set the configuration key
CF_RENAME_PROP
yourself. The value should look like . This would have the same effect on the mapper behavior.srcCf1:destCf1,srcCf2:destCf2,....
- Parameters:
conf
- the Configuration in which theCF_RENAME_PROP
key will be setrenameMap
- a mapping from source CF names to destination CF names
-
addFilterAndArguments
public static void addFilterAndArguments(org.apache.hadoop.conf.Configuration conf, Class<? extends Filter> clazz, List<String> filterArgs)
Add a Filter to be instantiated on import- Parameters:
conf
- Configuration to update (will be passed to the job)clazz
-Filter
subclass to instantiate on the server.filterArgs
- List of arguments to pass to the filter on instantiation
-
createSubmittableJob
public static org.apache.hadoop.mapreduce.Job createSubmittableJob(org.apache.hadoop.conf.Configuration conf, String[] args) throws IOException
Sets up the actual job.- Parameters:
conf
- The current configuration.args
- The command line parameters.- Returns:
- The newly created job.
- Throws:
IOException
- When setting up the job fails.
-
run
public int run(String[] args) throws Exception
- Specified by:
run
in interfaceorg.apache.hadoop.util.Tool
- Throws:
Exception
-
-