hadoop3.2:找不到logger的appender(org.apache.hadoop.mapreduce.v2.app.mrappmaster)

aoyhnmkz  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(404)

我有一个本地hadoop3.2安装：1个主程序+1个工作程序都在我的笔记本电脑上运行。这是一个实验设置，用于在提交到实际集群之前进行快速测试。
一切都很健康：

  1. $ jps
  2. 22326 NodeManager
  3. 21641 DataNode
  4. 25530 Jps
  5. 22042 ResourceManager
  6. 21803 SecondaryNameNode
  7. 21517 NameNode
  9. $ hdfs fsck /         
  10. Connecting to namenode via http://master:9870/fsck?ugi=david&path=%2F
  11. FSCK started by david (auth:SIMPLE) from /127.0.0.1 for path / at Wed Sep 04 13:54:59 CEST 2019
  13. Status: HEALTHY
  14.  Number of data-nodes:  1
  15.  Number of racks:       1
  16.  Total dirs:            1
  17.  Total symlinks:        0
  19. Replicated Blocks:
  20.  Total size:    0 B
  21.  Total files:   0
  22.  Total blocks (validated):  0
  23.  Minimally replicated blocks:   0
  24.  Over-replicated blocks:    0
  25.  Under-replicated blocks:   0
  26.  Mis-replicated blocks:     0
  27.  Default replication factor:    1
  28.  Average block replication: 0.0
  29.  Missing blocks:        0
  30.  Corrupt blocks:        0
  31.  Missing replicas:      0
  33. Erasure Coded Block Groups:
  34.  Total size:    0 B
  35.  Total files:   0
  36.  Total block groups (validated):    0
  37.  Minimally erasure-coded block groups:  0
  38.  Over-erasure-coded block groups:   0
  39.  Under-erasure-coded block groups:  0
  40.  Unsatisfactory placement block groups: 0
  41.  Average block group size:  0.0
  42.  Missing block groups:      0
  43.  Corrupt block groups:      0
  44.  Missing internal blocks:   0
  45. FSCK ended at Wed Sep 04 13:54:59 CEST 2019 in 0 milliseconds
  47. The filesystem under path '/' is HEALTHY

运行提供的pi示例时，出现以下错误：

  1. $ yarn jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.0.jar pi 16 1000
  2. Number of Maps  = 16
  3. Samples per Map = 1000
  4. Wrote input for Map #0
  5. Wrote input for Map #1
  6. Wrote input for Map #2
  7. Wrote input for Map #3
  8. Wrote input for Map #4
  9. Wrote input for Map #5
  10. Wrote input for Map #6
  11. Wrote input for Map #7
  12. Wrote input for Map #8
  13. Wrote input for Map #9
  14. Wrote input for Map #10
  15. Wrote input for Map #11
  16. Wrote input for Map #12
  17. Wrote input for Map #13
  18. Wrote input for Map #14
  19. Wrote input for Map #15
  20. Starting Job
  21. 2019-09-04 13:55:47,665 INFO client.RMProxy: Connecting to ResourceManager at master/0.0.0.0:8032
  22. 2019-09-04 13:55:47,887 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/david/.staging/job_1567598091808_0001
  23. 2019-09-04 13:55:48,020 INFO input.FileInputFormat: Total input files to process : 16
  24. 2019-09-04 13:55:48,450 INFO mapreduce.JobSubmitter: number of splits:16
  25. 2019-09-04 13:55:48,508 INFO Configuration.deprecation: yarn.resourcemanager.system-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publisher.enabled
  26. 2019-09-04 13:55:49,000 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1567598091808_0001
  27. 2019-09-04 13:55:49,003 INFO mapreduce.JobSubmitter: Executing with tokens: []
  28. 2019-09-04 13:55:49,164 INFO conf.Configuration: resource-types.xml not found
  29. 2019-09-04 13:55:49,164 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
  30. 2019-09-04 13:55:49,375 INFO impl.YarnClientImpl: Submitted application application_1567598091808_0001
  31. 2019-09-04 13:55:49,411 INFO mapreduce.Job: The url to track the job: http://cyclimse:8088/proxy/application_1567598091808_0001/
  32. 2019-09-04 13:55:49,412 INFO mapreduce.Job: Running job: job_1567598091808_0001
  33. 2019-09-04 13:55:55,477 INFO mapreduce.Job: Job job_1567598091808_0001 running in uber mode : false
  34. 2019-09-04 13:55:55,480 INFO mapreduce.Job:  map 0% reduce 0%
  35. 2019-09-04 13:55:55,509 INFO mapreduce.Job: Job job_1567598091808_0001 failed with state FAILED due to: Application application_1567598091808_0001 failed 2 times due to AM Container for appattempt_1567598091808_0001_000002 exited with  exitCode: 1
  36. Failing this attempt.Diagnostics: [2019-09-04 13:55:54.458]Exception from container-launch.
  37. Container id: container_1567598091808_0001_02_000001
  38. Exit code: 1
  40. [2019-09-04 13:55:54.464]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
  41. Last 4096 bytes of prelaunch.err :
  42. Last 4096 bytes of stderr :
  43. log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
  44. log4j:WARN Please initialize the log4j system properly.
  45. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  47. [2019-09-04 13:55:54.465]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
  48. Last 4096 bytes of prelaunch.err :
  49. Last 4096 bytes of stderr :
  50. log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
  51. log4j:WARN Please initialize the log4j system properly.
  52. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  54. For more detailed output, check the application tracking page: http://cyclimse:8088/cluster/app/application_1567598091808_0001 Then click on links to logs of each attempt.
  55. . Failing the application.
  56. 2019-09-04 13:55:55,546 INFO mapreduce.Job: Counters: 0
  57. Job job_1567598091808_0001 failed!

似乎log4j的配置有问题： No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster). . 但是它使用的是默认配置( $HADOOP_CONF_DIR/log4j.properties ).
执行之后，hdfs状态如下所示：

  1. $ hdfs fsck /        
  2. Connecting to namenode via http://master:9870/fsck?ugi=david&path=%2F
  3. FSCK started by david (auth:SIMPLE) from /127.0.0.1 for path / at Wed Sep 04 14:01:43 CEST 2019
  5. /tmp/hadoop-yarn/staging/david/.staging/job_1567598091808_0001/job.jar:  Under replicated BP-24234081-0.0.0.0-1567598050928:blk_1073741841_1017. Target Replicas is 10 but found 1 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s).
  7. /tmp/hadoop-yarn/staging/david/.staging/job_1567598091808_0001/job.split:  Under replicated BP-24234081-0.0.0.0-1567598050928:blk_1073741842_1018. Target Replicas is 10 but found 1 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s).
  9. Status: HEALTHY
  10.  Number of data-nodes:  1
  11.  Number of racks:       1
  12.  Total dirs:            11
  13.  Total symlinks:        0
  15. Replicated Blocks:
  16.  Total size:    510411 B
  17.  Total files:   20
  18.  Total blocks (validated):  20 (avg. block size 25520 B)
  19.  Minimally replicated blocks:   20 (100.0 %)
  20.  Over-replicated blocks:    0 (0.0 %)
  21.  Under-replicated blocks:   2 (10.0 %)
  22.  Mis-replicated blocks:     0 (0.0 %)
  23.  Default replication factor:    1
  24.  Average block replication: 1.0
  25.  Missing blocks:        0
  26.  Corrupt blocks:        0
  27.  Missing replicas:      18 (47.36842 %)
  29. Erasure Coded Block Groups:
  30.  Total size:    0 B
  31.  Total files:   0
  32.  Total block groups (validated):    0
  33.  Minimally erasure-coded block groups:  0
  34.  Over-erasure-coded block groups:   0
  35.  Under-erasure-coded block groups:  0
  36.  Unsatisfactory placement block groups: 0
  37.  Average block group size:  0.0
  38.  Missing block groups:      0
  39.  Corrupt block groups:      0
  40.  Missing internal blocks:   0
  41. FSCK ended at Wed Sep 04 14:01:43 CEST 2019 in 5 milliseconds
  43. The filesystem under path '/' is HEALTHY

因为我在网上找不到任何解决办法，所以我来了：）。

hadoophdfsyarnhadoop3Log4j

暂无答案！

目前还没有任何答案，快来回答吧！

我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com