java—如何在执行作业时将oracledriver添加到hadoop类路径？

mctunoxg 于 2021-06-03 发布在 Hadoop

关注(0)|答案(2)|浏览(296)

我使用hadoop处理数据库中的大量数据。我使用oracle的jdbc驱动程序连接到oracledb并进行处理。但是当我试图通过 bin/hadoop 对于打包的jar文件，它显示找不到oracledriver类。我该怎么解决这个问题？ $ bin/hadoop jar ~/hadoop1.jar name.hadoop.Hadoop1 ~/output ```
Exception in thread "main" java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:575)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1051)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1043)
at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323)
at name.shahalpk.poc.hadoop.Hadoop1.main(Hadoop1.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 20 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: oracle.jdbc.driver.OracleDriver
at org.apache.hadoop.mapred.lib.db.DBInputFormat.configure(DBInputFormat.java:271)
... 25 more
Caused by: java.lang.ClassNotFoundException: oracle.jdbc.driver.OracleDriver
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:186)
at org.apache.hadoop.mapred.lib.db.DBConfiguration.getConnection(DBConfiguration.java:123)
at org.apache.hadoop.mapred.lib.db.DBInputFormat.configure(DBInputFormat.java:266)
... 25 more

Java hadoop hdfs mapreduce

来源：https://stackoverflow.com/questions/13930072/how-to-add-oracledriver-to-hadoop-classpath-while-executing-the-job

2条答案

按热度按时间

nnvyjq4y1#

将ojdbc5.jar添加到类路径：

${JRE_HOME}\jre\lib\ext

注：

${JRE_HOME} means JRE(Java Runtime Environment) Installed Directory; Like below
${JRE_HOME}=C:\Program Files\Java\jre6\

赞(0）回复(0）举报 2021-06-04

6rqinv9w2#

你可以用 -libjars 通用选项更容易实现这一点，它还处理jar到集群节点的分发：

$ bin/hadoop jar ~/hadoop1.jar name.hadoop.Hadoop1 -libjars ojdbc5.jar ~/output

这是假设你的主课( name.hadoop.Hadoop1 )正在使用toolrunner.run（）方法启动作业：

public class Hadoop1 extends Configured implements Tool {
  public static void main(String args[]) throws Exception {
    ToolRunner.run(new Hadoop1(), args);
  }

  public int run(String args[]) {
    JobConf job = new JobConf(getConf());

    // rest of your job init code...

    RunningJob rj = JobClient.runJob(job);
    rj.waitForCompletion();
    return rj.isSuccessful() ? 0 : 1;
  }
}

（手工输入代码，对任何输入错误或编译错误表示歉意）

赞(0）回复(0）举报 2021-06-04

我来回答

java—如何在执行作业时将oracledriver添加到hadoop类路径？

2条答案

相关问题

热门标签

最新问答