处理减速机时bulkload失败,出现以下错误。我们正在m5集群上运行mapreduce,试图更新m7表。
java.io.IOException: bulkLoaderClose() on '/home/test/account122' failed
with error: Function not implemented (38).
at com.mapr.fs.Inode.checkError(Inode.java:1611)
at com.mapr.fs.Inode.checkError(Inode.java:1583)
at com.mapr.fs.Inode.bulkLoaderClose(Inode.java:1278)
at com.mapr.fs.MapRHTable.bulkLoaderClose(MapRHTable.java:119)
at com.mapr.fs.hbase.BulkLoadRecordWriter.close(BulkLoadRecordWriter.java:160)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:621)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:458)
at org.apache.hadoop.mapred.Child$4.run(Child.java:278)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566) at org.apache.hadoop.mapred.Child.main(Child.java:267)
hbase版本为0.98.12-mapr-1506。
mapr db表已启用批量加载。减速机处理所有的数据,在批量加载时,它会失败。
请帮忙。
3条答案
按热度按时间bq3bfh9z1#
默认情况下,maprddb表不支持批量加载。错误函数未实现(38)表示不支持此功能。
polhcujo2#
我测试了下面的数据文件和导入。如果这在您的集群上有效,请尝试。
1) 将以下数据添加到群集中的mfs。将其替换为路径。/mapr/demo.mapr.com/home/datafile.csv:2014,1,1,32014-01-01,aa,n338aa,1,jfk,lax,0914,14.001238,13.00,0.00,,385.00359.002475.00,,,,,,,,2014,1,1,2,42014-01-02,aa,n338aa,1,jfk,lax,0857,-3.001226,1.00,0.00,,385.00340.002475.00,,,,,,
导出cf=“cf1”
2) maprcli table delete-path/home/test/account122 3)maprcli table create-path/home/test/account122 3)maprcli table cf create-path/home/test/account122-cfname$cf
运行导入作业
4) java 语-cp
hbase classpath
org.apache.hadoop.hbase.mapreduce.importtsv -dimporttsv.separator=,-dimporttsv.columns=$cf:year,$cf:qtr,$cf:month,$cf:dom,$cf:dow,hbase\u row\u key,$cf:carrier,$cf:tailnum,$cf:flightnumber,$cf:origin,$cf:dest,$cf:deptime,$cf:deptime,$cf:arrtime,$cf:arrdelime,$cf:cncl,$cf:cnclcode,$cf:elaptime,$cf:airtime,$cf:distance,$cf:car,$cf:weatherdelay,$cf:nasdelay,$cf:securitydelay,$cf:aircraftdelay,$cf:dummy/home/test/account122/mapr/demo.mapr.com/home/datafile.csvmtb9vblg3#
你使用什么工具进行批量加载?您可以使用importtsv批量加载到mapr db表中。