失败：parseexception行1:94不匹配的输入“hdfs”在分区位置的“location”附近需要stringliteral

7dl7o3gd  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(606)

java代码：

String cmd0 = "hive  -e \"use "
                        + hiveuser
                        + ";set hive.exec.compress.output=true;set mapred.output.compression.codec=com.hadoop.compression.lzo.LzopCodec;set mapreduce.job.queuename="
                        + queue + ";alter table " + " resident_tmp "
                        + " add if not exists partition (weekday ='"
                        + "weekday=20170807" + "') location " + location + "\"";
    ComandUtil.exec(cmd0);

在hadoop集群中执行它可以是成功的，但是使用JavaAPI commandutil.exec（..）执行它是失败的。

failure log:
    17/12/13 01:18:52 INFO main.Run_WLOD: excute hive sentence ：hive  -e "use vendorsjgt;set hive.exec.compress.output=true;set mapred.output.compression.codec=com.hadoop.compression.lzo.LzopCodec;set mapreduce.job.queuename=ven5;alter table  resident_tmp  add if not exists partition (weekday ='weekday=20170807') location "hdfs://gzresearch/user/vendorsjgt/zcy/wlod/tmp/resident/weekday=20170807""
    17/12/13 01:18:55 WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce does not exist
    17/12/13 01:18:55 WARN conf.HiveConf: HiveConf of name hive.server2.custom.authentication.file does not exist
    17/12/13 01:18:55 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
    17/12/13 01:18:55 WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist
    17/12/13 01:18:55 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation does not exist
    17/12/13 01:18:55 WARN conf.HiveConf: HiveConf of name hive.semantic.analyzer.factory.impl does not exist
    17/12/13 01:18:55 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask does not exist

    Logging initialized using configuration in file:/software/servers/apache-hive-0.14.0-bin/conf/hive-log4j.properties
    OK
    Time taken: 1.654 seconds
    MismatchedTokenException(26!=313)
            at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
            at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
            at org.apache.hadoop.hive.ql.parse.HiveParser.partitionLocation(HiveParser.java:11426)
            at org.apache.hadoop.hive.ql.parse.HiveParser.alterStatementSuffixAddPartitionsElement(HiveParser.java:10973)
            at org.apache.hadoop.hive.ql.parse.HiveParser.alterStatementSuffixAddPartitions(HiveParser.java:10800)
            at org.apache.hadoop.hive.ql.parse.HiveParser.alterTableStatementSuffix(HiveParser.java:7983)
            at org.apache.hadoop.hive.ql.parse.HiveParser.alterStatement(HiveParser.java:7401)
            at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2685)
            at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1650)
            at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1109)
            at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:202)
            at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
            at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:396)
            at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
            at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
            at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
            at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
            at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
            at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
            at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
            at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:708)
            at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
            at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
            at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    FAILED: ParseException line 1:94 mismatched input 'hdfs' expecting StringLiteral near 'location' in partition location
    Exception in thread "main" java.lang.RuntimeException: org.apache.commons.exec.ExecuteException: Process exited with an error: 64 (Exit value: 64)
            at com.cn.cennavi.util.ComandUtil.exec(ComandUtil.java:46)
            at cn.com.cennavi.main.Run_WLOD.main(Run_WLOD.java:177)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
            at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 64 (Exit value: 64)
            at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
            at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
            at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153)
            at com.cn.cennavi.util.ComandUtil.exec(ComandUtil.java:43)
            ... 7 more

命令语句：

hive  -e "use vendorsjgt;set hive.exec.compress.output=true;set mapred.output.compression.codec=com.hadoop.compression.lzo.LzopCodec;set mapreduce.job.queuename=ven5;alter table  resident_tmp  add if not exists partition (weekday ='weekday=20170807') location "hdfs://gzresearch/user/vendorsjgt/zcy/wlod/tmp/resident/weekday=20170807""

在hadoop cluster（）中执行它可能是成功的，但是在java代码中使用它并在comandutil.exec（…）excute failure中使用它。。
在hadoop cluster（）中执行：

[vendorsjgt@GZA05-104-BigData-164000005 ~]$ hive  -e "use vendorsjgt;set hive.exec.compress.output=true;set mapred.output.compression.codec=com.hadoop.compression.lzo.LzopCodec;set mapreduce.job.queuename=ven5;alter table  resident_tmp  add if not exists partition (weekday ='weekday=20170807') location 'hdfs://gzresearch/user/vendorsjgt/zcy/wlod/tmp/resident/weekday=20170807'"
17/12/13 00:55:40 WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce does not exist
17/12/13 00:55:40 WARN conf.HiveConf: HiveConf of name hive.server2.custom.authentication.file does not exist
17/12/13 00:55:40 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
17/12/13 00:55:40 WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist
17/12/13 00:55:40 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation does not exist
17/12/13 00:55:40 WARN conf.HiveConf: HiveConf of name hive.semantic.analyzer.factory.impl does not exist
17/12/13 00:55:40 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask does not exist

Logging initialized using configuration in file:/software/servers/apache-hive-0.14.0-bin/conf/hive-log4j.properties
OK
Time taken: 1.663 seconds
OK
Time taken: 0.26 seconds

我的项目和我的目标只是处理一个关于hql的问题..并使用JavaAPI apache.commons.exec来完成。。。我尝试了许多现有的解决方案，比如“hive create table query上的mismatchedtokenexception”，但都没有成功。

JavahadoopmysqlHive

1条答案

按热度按时间
yhxst69z

yhxst69z1#

在hadoop集群中执行它可以获得成功
你把你的话弄混了。你在候机楼查的

hive -e "some query ... location 'hdfs://location'"

你用java运行了这个

hive -e "some query ... location "hdfs://location
                                ^^ Query failed here

配置单元cli已弃用。使用 beeline .
beeline使用jdbc，您也应该使用它

赞(0）分享  回复(0）举报  2021-05-29
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com