pig 导出导入

1. 安装pig

下载 pig-0.17.0.tar.gz
解压到~/

~# mv pig-0.17.0 pig
~# vi ~/.bashrc
-----------------------------------------------------------
export PIG_HOME=/home/jeffse/soft/pig
export PIG_CLASSPATH=$HADOOP_HOME/etc/hadoop
export PATH=$PATH:$PIG_HOME/bin:$PIG_HOME/conf
-----------------------------------------------------------
~# vim /home/hadoop/pig/conf/pig.properties
------------------------------------------------------------------
pig.logfile=/home/hadoop/pig/logs
------------------------------------------------------------------
~# source ~/.bashrc
~# pig --help
------------------------------------------------------------
显示内容表示已经安装完成
------------------------------------------------------------

2. 编写pig 导出脚本

REGISTER /home/jeffse/soft/phoenix/phoenix-4.14.3-HBase-1.4-client.jar;
rows = load 'hbase://query/SELECT * FROM POINT_DATA' USING org.apache.phoenix.pig.PhoenixHBaseLoader('hadoop1:2181');
STORE rows INTO '/home/jeffse/POINT_DATA.csv' USING PigStorage(',');

3. 将 phoenix 中 phoenix-4.14.3-HBase-1.4-pig.jar 放入 hbase/lib 下面

4. 执行导出命令

~# pig -x local export.pig

可以看到 /home/jeffse/ 下面已经导出 一个文件夹 POINT_DATA.csv

5. 编写导入脚本

REGISTER /data/app/phoenix/*.jar;
REGISTER /data/app/pig/*.jar;
A = load 'hdfs://mycluster/POINT_DATA.csv' USING PigStorage(',') as (ID:int,DATA_VALUE:chararray,POINT_ID:int,YMD:int,HMS:int,FLAG:int,STATUS:int,type:int,data_type:int,SYSINFO_CODE:chararray,PROC_MS:int,date:long);
STORE A into 'hbase://POINT_DATA2' using org.apache.phoenix.pig.PhoenixHBaseStorage('host194,host147,host120:2181','-batchSize 100');

安装导入环境的pig 已经设置环境变量

6. 执行导入操作

~# pig -x mapreduce import.pig