我正在尝试计算mr代码中所有任务的Map、洗牌和减少的总次数。我需要帮助检索每个mapreduce作业的信息。有人可以发布任何代码片段来进行计算吗?
zphenhs41#
您需要使用如下所示的jobclientapi:但是api有一些特殊之处。试试看,让我知道我会帮你的。
JobClient client = null; Configuration configuration = new Configuration(); configuration.set("mapred.job.tracker", jobTrackerURL); client = new JobClient(new JobConf(configuration)); while (true) { List<JobStatus> jobEntries = getTrackerEntries(jobName, client); for (JobStatus jobStatus : jobEntries) { JobID jobId = jobStatus.getJobID(); String trackerJobName = client.getJob(jobId) .getJobName(); TaskReport[] mapReports = client .getMapTaskReports(jobId); TaskReport[] reduceReports = client .getReduceTaskReports(jobId); client.getJob(jobId).getJobStatus().getStartTime(); int jobMapper = mapReports.length; mapNumber = +jobMapper; int jobReducers = reduceReports.length; reduceNumber = +jobReducers; } }
1条答案
按热度按时间zphenhs41#
您需要使用如下所示的jobclientapi:但是api有一些特殊之处。试试看,让我知道我会帮你的。