hadoop作业未终止

66bbxpm5  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(285)

我的工作在Map缩小步骤之前有一些例外,但工作不会被杀死。如何配置hadoop,使作业在异常后被终止?
正在调用主类
心跳心跳
主类调用已完成
oozie启动程序结束
标准日志

org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Io exception: Unknown host specified )
    at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:82)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:577)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:792)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:815)
    at com.seven.crcs.export.dao.ReportDAOImpl.recreateReportEntity(ReportDAOImpl.java:151)
    at com.seven.crcs.export.dao.ReportDAOImpl.saveActiveUserCount(ReportDAOImpl.java:93)
    at com.seven.crcs.export.ReportJdbcExporter.saveActiveUserCount(ReportJdbcExporter.java:55)
    at com.seven.dataprocessor.oc.jobs.reports.export.day.ExportDailyUserReducer.exportUserCounts(ExportDailyUserReducer.java:32)
    at com.seven.dataprocessor.oc.jobs.reports.export.ExportActiveUser
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Io exception: Unknown host specified )

2013-02-28 06:06:46,487 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201302270945_0181_r_000000_0, Status : FAILED
2013-02-28 06:07:00,600 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201302270945_0181_r_000000_1, Status : FAILED
2013-02-28 06:07:16,650 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201302270945_0181_r_000000_2, Status : FAILED
2013-02-28 06:07:31,731 INFO org.apache.hadoop.mapred.JobClient: Job complete: job_201302270945_0181

但乔布斯成功了

kd3sttzy

kd3sttzy1#

您的作业实际上已终止,但仅在尝试Map任务3次失败后,如任务ID所示:
尝试\u 201302270945 \u 0181 \u r\u0000000
尝试\u 201302270945 \u 0181 \u r\u000000 \u 1
尝试\u 201302270945 \u 0181 \u r\u000000 \u 2
您可以通过设置参数来限制每个任务的最大尝试次数 mapred.map.max.attempts1 或者使用jobconf#setmaxmapattempts(int)jobconf#setmaxmapattempts。
这将导致map任务在第一个异常时失败,从而更快地终止作业。

相关问题