我正在使用ubuntu-12.04,hadoop-1.0.2,hive-0.10.0从hive中读取大约100万条记录的数据,我在下面得到了一个查询错误
select * from raw_pos limit 10000;
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
所以我在$hadoop\u home/lib文件夹中安装了snappy for hadoop,它生成libsnappy.a、libsnappy.la、libsnappy.so、libsnappy.so.1、libsnappy.so.1.1.4文件
另外,在$hadoop\u home/lib/中添加hadoop-lzo-0.4.3.jar,并在cor site.xml、mapred-site.xml中做如下更改
核心站点。xml:-
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:54310</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/apache/hadoop-1.0.4/hadoop_temp/</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>io.compression.codecs</name>
<value>
org.apache.hadoop.io.compress.SnappyCodec
</value>
mapred-site.xml:-
<property>
<name>mapred.job.tracker</name>
<value>hdfs://localhost:54311</value>
</property>
<property>
<name>mapred.compress.map.output</name>
<value>true</value>
</property>
<property>
<name>mapred.map.output.compression.codec</name>
<value>org.apache.hadoop.io.compress.SnappyCodec</value>
</property>
但是当我开始做Hive的时候 show databases
,给出error:-
Failed with exception java.io.IOException:java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
1条答案
按热度按时间slmsl1lt1#
将core-site.xml修改为以下内容,看看是否有帮助: