在apachezeppelin上运行pig查询

2ul0zpep  于 2021-07-15  发布在  Pig
关注(0)|答案(0)|浏览(396)

我在apachezeppelin中运行以下pig查询

  1. %pig.query
  2. A = load '/Pig_data' using PigStorage(',') as(ExamName,ExamId,BITSID, StudentName,Issue_Type,Time);
  3. B = group A by Issue_Type;
  4. C = FOREACH B GENERATE group as Issue_Type, COUNT($1);

但它给了我以下的错误

  1. org.apache.zeppelin.interpreter.InterpreterException: java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/conf/YarnConfiguration
  2. at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
  3. at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:836)
  4. at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:744)
  5. at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
  6. at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:132)
  7. at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
  8. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  9. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  10. at java.lang.Thread.run(Thread.java:748)
  11. Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/conf/YarnConfiguration
  12. at org.apache.pig.PigServer.<init>(PigServer.java:249)
  13. at org.apache.pig.PigServer.<init>(PigServer.java:220)
  14. at org.apache.pig.PigServer.<init>(PigServer.java:193)
  15. at org.apache.pig.PigServer.<init>(PigServer.java:185)
  16. at org.apache.zeppelin.pig.PigInterpreter.open(PigInterpreter.java:64)
  17. at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
  18. ... 8 more
  19. Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.conf.YarnConfiguration
  20. at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
  21. at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  22. at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
  23. at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  24. ... 14 more

我已经检查过hadoop类路径和yarn类路径已经设置好了

  1. [hadoop@localhost ~]$ hadoop classpath
  2. /home/hadoop/hadoop/etc/hadoop:/home/hadoop/hadoop/share/hadoop/common/lib/*:/home/hadoop/hadoop/share/hadoop/common/*:/home/hadoop/hadoop/share/hadoop/hdfs:/home/hadoop/hadoop/share/hadoop/hdfs/lib/*:/home/hadoop/hadoop/share/hadoop/hdfs/*:/home/hadoop/hadoop/share/hadoop/mapreduce/lib/*:/home/hadoop/hadoop/share/hadoop/mapreduce/*:/home/hadoop/hadoop/share/hadoop/yarn:/home/hadoop/hadoop/share/hadoop/yarn/lib/*:/home/hadoop/hadoop/share/hadoop/yarn/*
  3. [hadoop@localhost ~]$ yarn classpath
  4. /home/hadoop/hadoop/etc/hadoop:/home/hadoop/hadoop/share/hadoop/common/lib/*:/home/hadoop/hadoop/share/hadoop/common/*:/home/hadoop/hadoop/share/hadoop/hdfs:/home/hadoop/hadoop/share/hadoop/hdfs/lib/*:/home/hadoop/hadoop/share/hadoop/hdfs/*:/home/hadoop/hadoop/share/hadoop/mapreduce/lib/*:/home/hadoop/hadoop/share/hadoop/mapreduce/*:/home/hadoop/hadoop/share/hadoop/yarn:/home/hadoop/hadoop/share/hadoop/yarn/lib/*:/home/hadoop/hadoop/share/hadoop/yarn/*

也设置在齐柏林飞艇-env.sh

  1. export USE_HADOOP=True
  2. export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop

请帮助我哪里出了问题。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题