执行时,以下代码会弹出一个错误消息error 2017:internal error creating job configuration。在Pig身上。
data = LOAD 'info.txt' USING PigStorage();
name_col_one = FOREACH data GENERATE $0 AS timeStamp, $1 AS one, $2 AS two, $3 AS info, $4 AS four, $5 AS five, $6 AS six, $7 AS seven, $8 AS eight, $9 AS nine, $10 AS ten, $11 AS eleven;
process_col_one = FOREACH name_col_one GENERATE FLATTEN(STRSPLIT(timeStamp,'\\s+',2)) AS (time:chararray, date:chararray), one, two;
new_timestamp = FOREACH process_col_one GENERATE CONCAT(date,CONCAT(' ',time)), one, two;
sys_info = FOREACH name_col_one GENERATE info;
split_ = FOREACH sys_info GENERATE REPLACE(info, '\\[', '') AS new_split;
split_again = FOREACH split_ GENERATE REPLACE(new_split, ']', '\t') AS final_split;
others = FOREACH name_col_one GENERATE four, five, six, seven, eight, nine, ten, eleven;
r1 = RANK new_timestamp;
r2 = RANK split_again;
r3 = RANK others;
final = JOIN r1 BY rank_new_timestamp, r2 BY rank_split_again;
DUMP final;
info.txt中的示例数据
23:58:19 02/23/2015良好1042559519[linux][baseline][lrtp2nosqlprod1][filesystem][tmp]文件系统/tmp\use%=1%9:5603 0 1
23:58:15 02/23/2015良好1042559519[linux][baseline][lrtp2nosqlprod1][filesystem][/boot]文件系统/boot\use%=37%3:5603 0 37
23:58:15 02/23/2015良好1042559537[linux][baseline][lrtp2nosqlprod1][process][srmclient][sisexclude]运行3:5599运行true无数据1 0
23:58:15 02/23/2015良好1042559537[linux][baseline][lrtp2nosqlprod1][process][oswatcher][sisexclude]正在运行,2个进程4:5599正在运行真无数据2 0 0
relations new\u timestamp将时间戳从输入数据中反转,split\u再次删除$3中的方括号并用'\t'分隔。
Pig Stack Trace
---------------
ERROR 2017: Internal error creating job configuration.
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias final
at org.apache.pig.PigServer.openIterator(PigServer.java:880)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:541)
at org.apache.pig.Main.main(Main.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias final
at org.apache.pig.PigServer.storeEx(PigServer.java:982)
at org.apache.pig.PigServer.store(PigServer.java:942)
at org.apache.pig.PigServer.openIterator(PigServer.java:855)
... 12 more
Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: ERROR 2017: Internal error creating job configuration.
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:873)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:298)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:190)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
at org.apache.pig.PigServer.storeEx(PigServer.java:978)
... 14 more
Caused by: java.lang.NullPointerException
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:817)
... 19 more
================================================================================
欢迎任何帮助。提前谢谢。
1条答案
按热度按时间f0brbegy1#
这个问题以前有报道过(https://issues.apache.org/jira/browse/pig-3469)并且已经修复了,也许可以尝试使用最新版本的pig。
有时可以通过指定输入数据文件的路径来解决此问题,例如“/home/user/doc/info.txt”