尝试在配置单元中创建外部表时出错

35g0bw71  于 2021-05-27  发布在  Hadoop
关注(0)|答案(1)|浏览(587)

我正试图用hadoop的hive创建一个外部表,但不知怎么的失败了。这些是我尝试运行查询时遇到的错误。

  1. 02:23:29.516 [HiveServer2-Background-Pool: Thread-39] ERROR hive.ql.exec.DDLTask - org.apache.hadoop.hive.ql.metadata.HiveException: Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
  2. at org.apache.hadoop.hive.ql.exec.DDLTask.validateSerDe(DDLTask.java:3858)
  3. at org.apache.hadoop.hive.ql.plan.CreateTableDesc.toTable(CreateTableDesc.java:700)
  4. at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3960)
  5. at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:333)
  6. at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
  7. at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
  8. at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)
  9. at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)
  10. at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)
  11. at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
  12. at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077)
  13. at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235)
  14. at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90)
  15. at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299)
  16. at java.security.AccessController.doPrivileged(Native Method)
  17. at javax.security.auth.Subject.doAs(Subject.java:422)
  18. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
  19. at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312)
  20. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  21. at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  22. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  23. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  24. at java.lang.Thread.run(Thread.java:748)
  25. Caused by: java.lang.ClassNotFoundException: Class org.openx.data.jsonserde.JsonSerDe not found
  26. at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2329)
  27. at org.apache.hadoop.hive.ql.exec.DDLTask.validateSerDe(DDLTask.java:3852)
  28. ... 22 more

我该怎么解决?

wn9m85ua

wn9m85ua1#

例外情况是
java.lang.classnotfoundexception:找不到类org.openx.data.jsonserde.jsonserde
安装jsonserde(从http://www.congiu.net/hive-json-serde/ 并将它们放入hive/lib),请阅读以下说明:hivejson serde
也可以尝试在配置单元会话中添加jar,而不是将jar放入配置单元/lib:

  1. ADD JAR ADD JAR /usr/lib/hive/lib/json-serde-1.3.8-jar-with-dependencies.jar;
  2. ADD JAR ADD JAR /usr/lib/hive/lib/json-udf-1.3.8-jar-with-dependencies.jar;

或者,您也可以尝试本机配置单元jsonserde: org.apache.hive.hcatalog.data.JsonSerDe -只需更改表ddl serde中的类名。应该已经安装了。请在此处阅读有关差异的更多详细信息:https://docs.aws.amazon.com/athena/latest/ug/json-serde.html

相关问题