我使用sparksql加载带有lzo压缩的配置单元表。误差如下:
原因:java.lang.classnotfoundexception:com.hadoop.mapred.deprecatedlzotextinputformat at at java.net.urlclassloader.findclass(urlclassloader)。java:381)在java.lang.classloader.loadclass(classloader。java:424)在java.lang.classloader.loadclass(classloader。java:357)位于java.lang.class.forname0(本机方法)java.lang.class.forname(类。java:348)在org.apache.spark.util.utils$.classforname(utils。scala:239)位于org.apache.spark.sql.hive.client.hiveclientimpl$.org$apache$spark$sql$hive$client$hiveclientimpl$$toinputformat(hiveclientimpl)。scala:892)在
我所做的:
set
spark.driver.extraClassPath /usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.3.1.4.0-315.jar
spark.driver.extraLibraryPath /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64
暂无答案!
目前还没有任何答案,快来回答吧!