puthiveql处理器以tez作为执行引擎有时会失败

mv1qrgav  于 2021-05-27  发布在  Hadoop
关注(0)|答案(0)|浏览(403)

中的错误 nifi-app.log :

>     ERROR [Timer-Driven Process Thread-3] o.apache.nifi.processors.hive.PutHiveQL
> PutHiveQL[id=1a203b03-0311-192f-b7a1-9959aea10e41] Failed to update
> Hive for
> StandardFlowFileRecord[uuid=e2f0dc1f-9a86-40c7-a37a-17926367fc19,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1581674370847-15,
> container=default, section=15], offset=8465,
> length=381],offset=0,name=eacbb330-2aa1-44f2-96e2-74675fc91263_toto_20190809150423.csv,size=381]
> due to java.sql.SQLException: Error while processing statement:
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask; it is 
> possible that retrying the operation will succeed, so routing to retry:
> java.sql.SQLException: Error while processing statement: FAILED:
> Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask
>     java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask
>             at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401)
>             at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266)
>             at org.apache.hive.jdbc.HivePreparedStatement.execute(HivePreparedStatement.java:101)
>             at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
>             at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
>             at org.apache.nifi.processors.hive.PutHiveQL.lambda$null$3(PutHiveQL.java:254)
>             at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127)
>             at org.apache.nifi.processors.hive.PutHiveQL.lambda$new$4(PutHiveQL.java:226)
>             at org.apache.nifi.processor.util.pattern.Put.putFlowFiles(Put.java:59)
>             at org.apache.nifi.processor.util.pattern.Put.onTrigger(Put.java:102)
>             at org.apache.nifi.processors.hive.PutHiveQL.lambda$onTrigger$6(PutHiveQL.java:295)
>             at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
>             at org.apache.nifi.processor.util.pattern.RollbackOnFailure.onTrigger(RollbackOnFailure.java:184)
>             at org.apache.nifi.processors.hive.PutHiveQL.onTrigger(PutHiveQL.java:295)
>             at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
>             at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
>             at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
>             at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
>             at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>             at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>             at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>             at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>             at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>             at java.lang.Thread.run(Thread.java:748)

此错误在nifi中发生,但在使用hivecli时也会发生。简单的请求永远不会发生这种情况,复杂的查询有时也会发生这种情况。比如1个查询超过5个,这对我来说是随机的。
流程:

处理器配置:

配置单元连接池配置:

查询: insert into table Db_X.Table_Y VALUES ("string","string","string""string"); jdbc的网址是从ambari,我没有问题时,连接到直线与。
当使用nifi或hivecli时,kerberos配置良好。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题