执行map reduce程序时出错

bq3bfh9z  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(472)

我使用hadoop3.1.0在ubuntu上运行mapreduce wordcount程序,但它总是得到这些信息。
我以前见过有人问过类似的问题,但那是行不通的。
我想知道我应该修改哪个文件,或者我遗漏了什么。
我的java程序就在这里。 master@kevin-VirtualBox:~/MapReduceTutorial$ $HADOOP_HOME/bin/hadoop jar ProductSalePerCountry.jar /inputMapReduce /mapreduce_output_sales ```
$HADOOP_HOME/bin/hadoop jar ProductSalePerCountry.jar /inputMapReduce /mapreduce_output_sales
2018-05-20 00:58:37,856 INFO client.RMProxy: Connecting to ResourceManager at kevin-VirtualBox/127.0.1.1:8032
2018-05-20 00:58:38,511 INFO client.RMProxy: Connecting to ResourceManager at kevin-VirtualBox/127.0.1.1:8032
2018-05-20 00:58:38,980 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2018-05-20 00:58:39,058 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/master/.staging/job_1526748071526_0004
2018-05-20 00:58:39,579 INFO mapred.FileInputFormat: Total input files to process : 1
2018-05-20 00:58:39,773 INFO mapreduce.JobSubmitter: number of splits:2
2018-05-20 00:58:39,926 INFO Configuration.deprecation: yarn.resourcemanager.system-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publisher.enabled
2018-05-20 00:58:40,251 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1526748071526_0004
2018-05-20 00:58:40,254 INFO mapreduce.JobSubmitter: Executing with tokens: []
2018-05-20 00:58:40,742 INFO conf.Configuration: resource-types.xml not found
2018-05-20 00:58:40,744 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2018-05-20 00:58:40,930 INFO impl.YarnClientImpl: Submitted application application_1526748071526_0004
2018-05-20 00:58:41,095 INFO mapreduce.Job: The url to track the job: http://kevin-VirtualBox:8088/proxy/application_1526748071526_0004/
2018-05-20 00:58:41,097 INFO mapreduce.Job: Running job: job_1526748071526_0004

core-site.xml文件
sbtkgmzw

sbtkgmzw1#

谢谢@cricket\u007我的问题是我不给Yarn记忆
在yarn-site.xml中设置可以使用的最大内存

<name>yarn.nodemanager.resource.memory-mb</name>
<value>40960</value>

指定要分配的ram的最小单位

<name>yarn.scheduler.minimum-allocation-mb</name>
 <value>2048</value>

相关问题