线程main中的异常:classnotfoundexception

vddsk6oq  于 2021-06-04  发布在  Hadoop
关注(0)|答案(1)|浏览(401)

我在学校群里运行hadoop。我在线程主线程中得到异常,类未找到异常。

  1. Exception in thread "main" java.lang.ClassNotFoundException: movielens.MovieLensDriver
  2. at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
  3. at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
  4. at java.security.AccessController.doPrivileged(Native Method)
  5. at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
  6. at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
  7. at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
  8. at java.lang.Class.forName0(Native Method)
  9. at java.lang.Class.forName(Class.java:264)
  10. at org.apache.hadoop.util.RunJar.main(RunJar.java:153)

但是我知道我必须在命令中使用完整的包名,我也做了同样的事情。下面是我使用的命令

  1. hadoop jar movielens.jar movielens.MovieLensDriver input output

下面是我的驱动程序类的代码。

  1. package movielens;
  2. import java.io.IOException;
  3. import org.apache.hadoop.fs.Path;
  4. import org.apache.hadoop.io.IntWritable;
  5. import org.apache.hadoop.io.Text;
  6. import org.apache.hadoop.mapred.FileInputFormat;
  7. import org.apache.hadoop.mapred.FileOutputFormat;
  8. import org.apache.hadoop.mapred.JobConf;
  9. import org.apache.hadoop.mapred.KeyValueTextInputFormat;
  10. import org.apache.hadoop.mapred.jobcontrol.Job;
  11. import org.apache.hadoop.mapred.jobcontrol.JobControl;
  12. public class MovieLensDriver {
  13. public static class JobRunner implements Runnable {
  14. private JobControl control;
  15. public JobRunner(JobControl _control) {
  16. this.control = _control;
  17. }
  18. public void run() {
  19. this.control.run();
  20. }
  21. }
  22. public static void handleRun(JobControl control)
  23. throws InterruptedException {
  24. JobRunner runner = new JobRunner(control);
  25. Thread t = new Thread(runner);
  26. t.start();
  27. while (!control.allFinished()) {
  28. System.out.println("Still running...");
  29. Thread.sleep(5000);
  30. }
  31. }
  32. public static void main(String args[]) throws IOException,
  33. InterruptedException {
  34. System.out.println("Program started");
  35. if (args.length != 2) {
  36. System.err
  37. .println("Usage: MovieLensDriver <input path> <output path>");
  38. System.exit(-1);
  39. }
  40. JobConf conf1 = new JobConf(movielens.MovieLensDriver.class);
  41. conf1.setMapperClass(MoviePairsMapper.class);
  42. conf1.setReducerClass(MoviePairsReducer.class);
  43. conf1.setJarByClass(MovieLensDriver.class);
  44. FileInputFormat.addInputPath(conf1, new Path(args[0]));
  45. FileOutputFormat.setOutputPath(conf1, new Path("temp"));
  46. conf1.setMapOutputKeyClass(Text.class);
  47. conf1.setMapOutputValueClass(Text.class);
  48. conf1.setOutputKeyClass(Text.class);
  49. conf1.setOutputValueClass(IntWritable.class);
  50. JobConf conf2 = new JobConf(MovieLensDriver.class);
  51. conf2.setMapperClass(MoviePairsCoOccurMapper.class);
  52. conf2.setReducerClass(MoviePairsCoOccurReducer.class);
  53. conf2.setJarByClass(MovieLensDriver.class);
  54. FileInputFormat.addInputPath(conf2, new Path("temp"));
  55. FileOutputFormat.setOutputPath(conf2, new Path(args[1]));
  56. conf2.setInputFormat(KeyValueTextInputFormat.class);
  57. conf2.setMapOutputKeyClass(Text.class);
  58. conf2.setMapOutputValueClass(IntWritable.class);
  59. conf2.setOutputKeyClass(Text.class);
  60. conf2.setOutputValueClass(IntWritable.class);
  61. Job job1 = new Job(conf1);
  62. Job job2 = new Job(conf2);
  63. JobControl jobControl = new JobControl("jobControl");
  64. jobControl.addJob(job1);
  65. jobControl.addJob(job2);
  66. job2.addDependingJob(job1);
  67. handleRun(jobControl);
  68. System.out.println("Program complete.");
  69. System.exit(0);
  70. }
  71. }

这是一个令人沮丧的错误搜索过去3个小时,任何帮助表示感谢。

rqcrx0a6

rqcrx0a61#

您可以尝试“libjar”选项,它将获取jar并将其放置在分布式缓存中。这使得jar可用于作业的所有任务尝试。请注意,libjars参数采用逗号分隔的列表,而不是冒号或分号分隔的列表。
export LIBJARS=/path/jars1,/path/jars2,/path/movielens.jar hadoop jar movielens.jar movielens.MovieLensDriver -libjars ${LIBJARS} input output

相关问题