读取hadoop namenode中的编辑日志时出错

szqfcxe2  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(1035)

我使用的是3节点hdfs群集,它在过去几个月运行良好,但从几天开始,我经常在其中一个namenode的日志中发现以下异常,它是活动节点,但由于此错误hdfs将自身故障转移到辅助namenode,虽然事情很好,但我想解决这个问题,请suggest:-

  1. org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpGetFailedException: Fetch of http://nn1.cluster.com:8480/getJournal?jid=ha-cluster&segmentTxId=827873&storageInfo=-63%3A1513620929%3A0%3ACID-c43f63a9-b064-4dc6-b175-378bf0db96e3 failed with status code 500
  2. Response message:
  3. No realm
  4. at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:468)
  5. at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog$1.run(EditLogFileInputStream.java:456)
  6. at java.security.AccessController.doPrivileged(Native Method)
  7. at javax.security.auth.Subject.doAs(Subject.java:422)
  8. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
  9. at org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:448)
  10. at org.apache.hadoop.security.SecurityUtil.doAsCurrentUser(SecurityUtil.java:442)
  11. at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream$URLLog.getInputStream(EditLogFileInputStream.java:455)
  12. at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.init(EditLogFileInputStream.java:141)
  13. at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOpImpl(EditLogFileInputStream.java:192)
  14. at org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream.nextOp(EditLogFileInputStream.java:250)
  15. at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
  16. at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
  17. at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
  18. at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
  19. at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
  20. at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
  21. at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
  22. at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
  23. at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
  24. at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
  25. at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:818)
  26. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:232)
  27. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:331)
  28. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$200(EditLogTailer.java:284)
  29. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:301)
  30. at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:415)
  31. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:297)
  32. 2016-10-14 01:38:32,375 ERROR org.apache.hadoop.hdfs.server.namenode.EditLogInputStream: Got error reading edit log input stream http://nn1.cluster.com:8480/getJournal?jid=ha-cluster&segmentTxId=827873&storageInfo=-63%3A1513620929%3A0%3ACID-c43f63a9-b064-4dc6-b175-378bf0db96e3; failing over to edit log http://nn2.cluster.com:8480/getJournal?jid=ha-cluster&segmentTxId=827873&storageInfo=-63%3A1513620929%3A0%3ACID-c43f63a9-b064-4dc6-b175-378bf0db96e3
  33. org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream$PrematureEOFException: got premature end-of-file at txid 827872; expected file to go up to 827874
  34. at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:194)
  35. at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
  36. at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.skipUntil(EditLogInputStream.java:151)
  37. at org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream.nextOp(RedundantEditLogInputStream.java:178)
  38. at org.apache.hadoop.hdfs.server.namenode.EditLogInputStream.readOp(EditLogInputStream.java:85)
  39. at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:190)
  40. at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
  41. at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
  42. at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:818)
  43. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:232)
  44. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:331)
  45. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$200(EditLogTailer.java:284)
  46. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:301)
  47. at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:415)
  48. at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:297)
mbjcgjjk

mbjcgjjk1#

您的namenode似乎已损坏。尝试使用以下命令恢复它

  1. hadoop namenode -recover

相关问题