美文网首页
hive 永久性添加 json serde jar

hive 永久性添加 json serde jar

作者: cyangssrs | 来源:发表于2019-07-15 11:51 被阅读0次

hive 添加jar

  1. 你可以在每个session中手动去添加某个jar
add jar /home/hive/xxxxx.jar

2 . 你也可以在为所有session永久性添加某个jar

  • 在你的hiveserver host 上 先创建一个存放jar的目录,例如:
mkdir /var/lib/hive
  • 将这个目录添加配置到你的 hive-site.xml里
<property>
  <name>hive.aux.jars.path</name>
  <value>/var/lib/hive</value>
</property>

重启 hiveserver2

  1. 另外,你也可以分别配置单独的jar,例如
<property>
  <name>hive.aux.jars.path </name>
  <value>file:///home/amal/hive/udf1.jar,file:///usr/lib/hive/lib/hive-hbase-handler.jar</value>
</property>
  1. 如果你使用的cloudera CDH ,那么你可以直接在 hiveserver2 的 configuration 里面直接找到这个配置项


    image.png

    配置之后,重启就行了

hive json serde jar

Row Format Description
JSON
ROW FORMAT SERDE
'org.apache.hive.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE
Stored as plain text file in JSON format.

The JsonSerDe for JSON files is available in Hive 0.12 and later.

In some distributions, a reference to hive-hcatalog-core.jar is required.

ADD JAR /usr/lib/hive-hcatalog/lib/hive-hcatalog-core.jar;<br><br>CREATE TABLE my_table(a string, b ``bigint``, ...)

ROW FORMAT SERDE ``'org.apache.hive.hcatalog.data.JsonSerDe'

STORED ``AS TEXTFILE;

The JsonSerDe was moved to Hive from HCatalog and before it was in hive-contrib project. It was added to the Hive distribution by HIVE-4895.
An Amazon SerDe is available at s3://elasticmapreduce/samples/hive-ads/libs/jsonserde.jar for releases prior to 0.12.0.

The JsonSerDe for JSON files is available in Hive 0.12 and later.

Starting in Hive 3.0.0, JsonSerDe is added to Hive Serde as "org.apache.hadoop.hive.serde2.JsonSerDe" (HIVE-19211).

CREATE TABLE my_table(a string, b ``bigint``, ...)
ROW FORMAT SERDE ``'org.apache.hadoop.hive.serde2.JsonSerDe' STORED ``AS TEXTFILE;

Or STORED AS JSONFILE is supported starting in Hive 4.0.0 (HIVE-19899), so you can create table as follows:

CREATE TABLE my_table(a string, b ``bigint``, ...) STORED AS JSONFILE;

参考文献:https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-RowFormats&SerDe

相关文章

网友评论

      本文标题:hive 永久性添加 json serde jar

      本文链接:https://www.haomeiwen.com/subject/bkogkctx.html