我正在尝试从oozie启动一个shell作业,在那里我将文件从hdfsMap到hbase
shell脚本可以从命令行工作,但不能从oozie工作
shell脚本:
HADOOP_CLASSPATH=`/usr/bin/hbase classpath` /usr/bin/hadoop jar /usr/lib/hbase/hbase-server.jar importtsv -Dimporttsv.separator=, -Dimporttsv.columns=ORIGINAL:FIELD1,ORIGINAL:FIELD2,ORIGINAL:FIELD3 HBASE_Table /user/U0H8048/file
错误是:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/cloudera/opt/cloudera/parcels/CDH-5.0.2- 1.cdh5.0.2.p0.13/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/cloudera/yarn/nm/filecache/2282/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/cloudera/yarn/nm/filecache/2288/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:54)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.FileNotFoundException: File does not exist: hdfs://salcls0226.com:8020/cloudera/opt/cloudera/parcels/CDH-5.0.2-1.cdh5.0.2.p0.13/lib/hbase/lib/hbase-server-0.96.1.1-cdh5.0.2.jar
at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
...
有人知道可能的解决办法吗?
如何指定 hbase-server-0.96.1.1-cdh5.0.2.jar
文件位于本地路径而不在hdfs上?
谢谢
1条答案
按热度按时间pgx2nnw81#
我解决了这个问题。
它需要添加行
oozie.use.system.libpath=true
当工作流启动时,由oozie读取job.properties文件。您必须将所有需要的文件添加到hdfs路径:/user/oozie/share/lib中。通常主库文件已经存在。