如何在eclipse中使用java运行嵌入式pig程序?

ffscu2ro  于 2021-05-30  发布在  Hadoop
关注(0)|答案(1)|浏览(367)

我是hadoop新手。我正在使用pig0.14.0和hadoop1.2.1。我已经成功地从gruntshell和pig批处理脚本在本地和map reduce模式下运行了pig。现在我正在尝试从java中嵌入的pig运行pig。在eclipse中编译代码时,出现以下错误:
线程“main”java.lang.noclassdeffounderror中出现异常:org/apache/commons/logging/logfactory,位于org.apache.pig.impl.util.propertiesutil.(propertiesutil)。java:34)在org.apache.pig.pigserver.(pigserver。java:202)测试主(测试。java:6)原因:java.lang.classnotfoundexception:org.apache.commons.logging.logfactory atjava.net.urlclassloader$1.run(urlclassloader。java:366)在java.net.urlclassloader$1.run(urlclassloader。java:355)位于java.net.urlclassloader.findclass(urlclassloader)的java.security.accesscontroller.doprivileged(本机方法)。java:354)在java.lang.classloader.loadclass(classloader。java:425)在sun.misc.launcher$appclassloader.loadclass(launcher。java:308)在java.lang.classloader.loadclass(classloader。java:358) ... 3个以上

import org.apache.pig.ExecType;
import org.apache.pig.PigServer;
public class test {
    public static void main(String[] args) {
        try {
            PigServer pigServer = new PigServer(ExecType.MAPREDUCE);
            runQuery(pigServer);
        }catch(Exception e) {
            e.printStackTrace();
        }
    }
    public static void runQuery(PigServer pigServer) {
        try {
            pigServer.registerQuery("input1 = LOAD '/mydata/wct.txt' as (line:chararray);");
            pigServer.registerQuery("words = foreach input1 generate FLATTEN(TOKENIZE(line)) as word;");
            pigServer.registerQuery("word_groups = group words by word;");
            pigServer.registerQuery("word_count = foreach word_groups generate group, COUNT(words);");
            pigServer.registerQuery("ordered_word_count = order word_count by group desc;");
            pigServer.registerQuery("store ordered_word_count into '/mydata/wct';");
        } catch(Exception e) {
            e.printStackTrace();
        }
    }
}

请帮帮我。提前谢谢。

laawzig2

laawzig21#

请添加公共依赖项:

<dependency>
      <groupId>commons-logging</groupId>
      <artifactId>commons-logging</artifactId>
      <version>1.1.1</version>
  </dependency>

相关问题