我们计划通过springxd执行sqoop作业,并按照springxd文档中的建议进行了配置。
下面是servers.xml文件中的配置
spring:
hadoop:
fsUri: hdfs://nnservice
resourceManagerHost: server-m01.mydomain.com
resourceManagerPort: 8032
resourceManagerSchedulerAddress: ${spring.hadoop.resourceManagerHost}:8030
jobHistoryAddress: server-m02.mydomain.com:10020
security:
authMethod: kerberos
userPrincipal: springxd@mydomain.COM
userKeytab: /home/springxd/springxd.keytab
namenodePrincipal: nn/_HOST@mydomain.COM
rmManagerPrincipal: rm/_HOST@mydomain.COM
config:
mapreduce.framework.name: yarn
dfs.nameservices: nnservice
dfs.ha.namenodes.nnservice: nn1, nn2
dfs.namenode.rpc-address.nnservice.nn1: server-m01.mydomain.com:8020
dfs.namenode.rpc-address.nnservice.nn2: server-m02.mydomain.com:8020
dfs.client.failover.proxy.provider.nnservice: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.automatic-failover.enabled: True
mapreduce.application.framework.path: '/hdp/apps/2.4.2.0-258/mapreduce/mapreduce.tar.gz#mr-framework'
mapreduce.application.classpath: '$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:/usr/hdp/2.4.2.0-258/hadoop/lib/hadoop-lzo-0.6.0.2.4.2.0-258.jar:/etc/hadoop/conf/secure'
yarnApplicationClasspath: '$HADOOP_CONF_DIR,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*'
sqoop作业因以下错误而终止
2016-10-28T10:08:35-0400 1.3.1.RELEASE INFO main mapreduce.Job - Job job_1477527264038_0051 failed with state FAILED due to: Application application_1477527264038_0051 failed 2 times due to AM Container for appattempt_1477527264038_0051_000002 exited with exitCode: 1
你知道我们是否缺少任何配置吗??
暂无答案!
目前还没有任何答案,快来回答吧!