oozie运行示例错误：illegalargumentexception:错误的fs:hdfs://**/user/ubuntu/share/lib，应输入：文件：///

ctehm74n  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(266)

最近，我对oozie进行了调查，并努力建立一个本地的oozie系统。在反复阅读官方网页后，我终于成功了。但当我试着用 examples/ 方向，我总是出错：

  1. 2016-07-21 21:55:17,936  WARN ActionStartXCommand:523 - SERVER[namenode] USER[ubuntu] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000001-160720220441466-oozie-ubun-W] ACTION[0000001-160720220441466-oozie-ubun-W@mr-node] Error starting action [mr-node]. ErrorType [ERROR], ErrorCode [IllegalArgumentException], Message [IllegalArgumentException: Wrong FS: hdfs://166.111.81.254:9000/user/ubuntu/share/lib, expected: file:///]
  2. org.apache.oozie.action.ActionExecutorException: IllegalArgumentException: Wrong FS: hdfs://166.111.81.254:9000/user/ubuntu/share/lib, expected: file:///
  3.     at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:445)
  4.     at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1132)
  5.     at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1286)
  6.     at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:250)
  7.     at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:64)
  8.     at org.apache.oozie.command.XCommand.call(XCommand.java:286)
  9.     at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:321)
  10.     at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:250)
  11.     at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
  12.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  13.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  14.     at java.lang.Thread.run(Thread.java:745)
  15. Caused by: java.lang.IllegalArgumentException: Wrong FS: hdfs://166.111.81.254:9000/user/ubuntu/share/lib, expected: file:///
  16.     at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
  17.     at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:80)
  18.     at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:372)
  19.     at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1485)
  20.     at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1525)
  21.     at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:570)
  22.     at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1485)
  23.     at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1525)
  24.     at org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:687)
  25.     at org.apache.oozie.service.ShareLibService.updateShareLib(ShareLibService.java:551)
  26.     at org.apache.oozie.service.ShareLibService.getShareLibJars(ShareLibService.java:346)
  27.     at org.apache.oozie.service.ShareLibService.getSystemLibJars(ShareLibService.java:412)
  28.     at org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAction(JavaActionExecutor.java:721)
  29.     at org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActionExecutor.java:818)
  30.     at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:809)
  31.     at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1037)
  32.     ... 10 more
  33. 2016-07-21 21:55:17,939  WARN ActionStartXCommand:523 - SERVER[namenode] USER[ubuntu] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000001-160720220441466-oozie-ubun-W] ACTION[0000001-160720220441466-oozie-ubun-W@mr-node] Setting Action Status to [DONE]

这个错误困扰了我好几天，我一直在努力解决它。如果你有什么建议，请教我~非常感谢！
我的配置如下：
hadoop软件
core-site.xml文件

  1. <configuration>
  2.   <property>
  3.     <name>fs.defaultFS</name>
  4.     <value>hdfs://166.111.81.254:9000</value>
  5.   </property>
  7.   <property>
  8.     <name>hadoop.proxyuser.ubuntu.hosts</name>
  9.     <value>*</value>
  10.   </property>
  11.   <property>
  12.     <name>hadoop.proxyuser.ubuntu.groups</name>
  13.     <value>*</value>
  14.   </property>
  15. </configuration>

yarn-site.xml文件

  1. <configuration>
  3. <!-- Site specific YARN configuration properties -->
  4.   <property>
  5.     <name>yarn.nodemanager.aux-services</name>
  6.     <value>mapreduce_shuffle</value>
  7.   </property>
  8.   <property>
  9.     <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  10.     <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  11.   </property>
  12.   <property>
  13.     <name>yarn.resourcemanager.hostname</name>
  14.     <value>166.111.81.254</value>
  15.   </property>
  16. </configuration>

mapred-site.xml文件

  1. <configuration>
  2.   <property>
  3.     <name>mapreduce.jobtracker.address</name>
  4.     <value>166.111.81.254:54311</value>
  5.   </property>
  6.   <property>
  7.     <name>mapreduce.framework.name</name>
  8.     <value>yarn</value>
  9.   </property>
  10. </configuration>

hdfs-site.xml文件

  1. <configuration>
  2.   <property>
  3.     <name>dfs.replication</name>
  4.     <value>2</value>
  5.   </property>
  6.   <property>
  7.     <name>dfs.namenode.name.dir</name>
  8.     <value>file:///usr/local/hadoop-2.6.4/hadoop_data/hdfs/namenode</value>
  9.   </property>
  10.   <property>
  11.     <name>dfs.permissions.enabled</name>
  12.     <value>false</value>
  13.   </property>
  14.   <property>
  15.     <name>dfs.webhdfs.enabled</name>
  16.     <value>true</value>
  17.   </property>
  18. </configuration>

奥齐
oozie-site.xml文件

  1. <configuration>
  2.   <property>
  3.       <name>oozie.service.HadoopAccessorService.root.configurations</name>
  4.       <value>*=/usr/local/hadoop-2.6.4/etc/hadoop/</value>
  5.       <description>
  6.           Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
  7.           the Hadoop service (JobTracker, HDFS). The wildcard '*' configuration is
  8.           used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
  9.           the relevant Hadoop *-site.xml files. If the path is relative is looked within
  10.           the Oozie configuration directory; though the path can be absolute (i.e. to point
  11.           to Hadoop client conf/ directories in the local filesystem.
  12.       </description>
  13.   </property>
  14.   <property>
  15.     <name>oozie.service.WorkflowAppService.system.libpath</name>
  16.     <value>hdfs://166.111.81.254:9000/user/${user.name}/share/lib</value>
  17.     <description>
  18.         System library path to use for workflow applications.
  19.         This path is added to workflow application if their job properties sets
  20.         the property 'oozie.use.system.libpath' to true.
  21.     </description>
  22. </property>
  23. <property>
  24.     <name>oozie.service.ProxyUserService.proxyuser.ubuntu.hosts</name>
  25.     <value>*</value>
  26. </property>
  27. <property>
  28.     <name>oozie.service.ProxyUserService.proxyuser.ubuntu.groups</name>
  29.     <value>*</value>
  30. </property>
hadoopoozie

暂无答案！

目前还没有任何答案，快来回答吧！

我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com