我已经做了haddop的升级和降级操作,然后旧数据就不能打开了。你能帮我检查一下这个问题吗
https://hadoop.apache.org/docs/r2.10.1/hadoop-project-dist/hadoop-hdfs/hdfsrollingupgrade.html#namenode_-滚动升级
[root@master current]# hdfs dfs -cat /test/test.csv
21/03/02 14:42:17 WARN hdfs.BlockReaderFactory: I/O error constructing remote block reader.
java.io.IOException: Got error, status message opReadBlock BP-1289313299-192.168.1.26-1533200460191:blk_1073762237_21423
received exception java.io.IOException: BlockId 1073762237 is not valid.,
for OP_READ_BLOCK, self=/10.10.202.26:47930, remote=/10.10.202.26:50010,
for file /test/test.csv, for pool BP-1289313299-192.168.1.26-1533200460191 block 1073762237_21423
at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140)
at org.apache.hadoop.hdfs.RemoteBlockReader2.checkSuccess(RemoteBlockReader2.java:456)
at org.apache.hadoop.hdfs.RemoteBlockReader2.newBlockReader(RemoteBlockReader2.java:424)
at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReader(BlockReaderFactory.java:818)
at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:69
...
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1289313299-192.168.1.26-1533200460191:blk_1073762237_21423 file=/test/test.csv
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:983)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:642)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:882)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:934)
at java.io.DataInputStrea
-rw-r--r-- 1 root supergroup 359 2018-08-21 11:19 /test/test.csv
[root@master current]# hdfs fsck /test/
Connecting to namenode via http://master:50070/fsck?ugi=root&path=%2Ftest
FSCK started by root (auth:SIMPLE) from /10.10.202.26 for path /test at Tue Mar 02 15:05:23 CST 2021
...................................................Status: HEALTHY
Total size: 6720241197 B
Total dirs: 17
Total files: 51
Total symlinks: 0
Total blocks (validated): 85 (avg. block size 79061661 B)
Minimally replicated blocks: 85 (100.0 %)
Over-replicated blocks: 0
文件下载失败,请在此处输入图像说明
1条答案
按热度按时间yb3bgrhw1#
确保时间正确并在所有服务器上同步。确保datanode文件在linux文件系统上具有正确的权限。
尝试:
在某些情况下,更改hdfs-site.xml文件:
帮助解决了问题。