无法在Yarn上启动flink 1.11.2会话

sczxawaw  于 2021-06-21  发布在  Flink
关注(0)|答案(0)|浏览(421)

我想开始关于Yarn的flink 1.11.2课程。我已经设置了环境变量 HADOOP_CLASSPATH , HADOOP_CONF_DIR 但是当我发射的时候 ./bin/yarn-session.sh 我有一个错误:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/someuser/flink-1.11.2/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2020-11-26 15:56:35,249 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf
2020-11-26 15:56:35,253 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: env.java.home key, /usr/java/latest
2020-11-26 15:56:35,253 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: jobmanager.rpc.address, localhost
2020-11-26 15:56:35,253 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: jobmanager.rpc.port, 6123
2020-11-26 15:56:35,253 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: jobmanager.memory.process.size, 1600m
2020-11-26 15:56:35,253 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: taskmanager.memory.process.size, 1728m
2020-11-26 15:56:35,253 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: taskmanager.numberOfTaskSlots, 1
2020-11-26 15:56:35,254 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: parallelism.default, 1
2020-11-26 15:56:35,254 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: state.checkpoints.dir, hdfs://some/user/path/.flink11Checkpoint
2020-11-26 15:56:35,254 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: jobmanager.execution.failover-strategy, region
2020-11-26 15:56:35,254 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: rest.port, 8081
2020-11-26 15:56:35,255 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: security.kerberos.login.use-ticket-cache, true
2020-11-26 15:56:35,255 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: security.kerberos.login.keytab, /home/someuser/some.keytab
2020-11-26 15:56:35,255 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: security.kerberos.login.principal, someuser@cluster.com
2020-11-26 15:56:35,255 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: security.kerberos.login.contexts, Client,KafkaClient
2020-11-26 15:56:35,255 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: yarn.tags, streaming
2020-11-26 15:56:35,256 INFO  org.apache.flink.configuration.GlobalConfiguration           [] - Loading configuration property: env.java.opts, "-server"
2020-11-26 15:56:35,314 INFO  org.apache.flink.yarn.cli.FlinkYarnSessionCli                [] - Found Yarn properties file under /tmp/.yarn-properties-someuser.
2020-11-26 15:56:35,812 INFO  org.apache.hadoop.security.UserGroupInformation              [] - Login successful for user localisation@NETXLR8.FR using keytab file /home/someuser/some.keytab
2020-11-26 15:56:35,812 INFO  org.apache.flink.runtime.security.modules.HadoopModule       [] - Hadoop user set to someuser@cluster.com (auth:KERBEROS)
2020-11-26 15:56:35,813 INFO  org.apache.flink.runtime.security.modules.HadoopModule       [] - Kerberos security is enabled and credentials are valid.
2020-11-26 15:56:35,820 INFO  org.apache.flink.runtime.security.modules.JaasModule         [] - Jaas file will be created as /tmp/jaas-7703404462212921557.conf.
2020-11-26 15:56:35,855 WARN  org.apache.flink.yarn.configuration.YarnLogConfigUtil        [] - The configuration directory ('/home/someuser/flink-1.11.2/conf') already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file.
2020-11-26 15:56:36,332 WARN  org.apache.hadoop.security.UserGroupInformation              [] - PriviledgedActionException as:someuser@cluster.com (auth:KERBEROS) cause:org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
2020-11-26 15:56:36,333 ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli                [] - Error while running the Flink session.
org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
        at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:382) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:514) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:751) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_131]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_131]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924) ~[hadoop-common-2.6.0-cdh5.15.1.jar:?]
        at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:751) [flink-dist_2.11-1.11.2.jar:1.11.2]
Caused by: java.lang.ArithmeticException: / by zero
        at org.apache.flink.yarn.YarnClusterDescriptor.logIfComponentMemNotIntegerMultipleOfYarnMinAllocation(YarnClusterDescriptor.java:604) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
        at org.apache.flink.yarn.YarnClusterDescriptor.validateClusterResources(YarnClusterDescriptor.java:559) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
        at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:506) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
        at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:375) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
        ... 7 more

------------------------------------------------------------
 The program finished with the following exception:

org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
        at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:382)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:514)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:751)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
        at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:751)
Caused by: java.lang.ArithmeticException: / by zero
        at org.apache.flink.yarn.YarnClusterDescriptor.logIfComponentMemNotIntegerMultipleOfYarnMinAllocation(YarnClusterDescriptor.java:604)
        at org.apache.flink.yarn.YarnClusterDescriptor.validateClusterResources(YarnClusterDescriptor.java:559)
        at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:506)
        at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:375)
        ... 7 more

hadoop版本:hadoop 2.6.0-cdh5.15.1
我们已经在同一个集群上运行了Flink1.7.0。
我应该改变什么才能正常运行?
是否可以在同一个Yarn簇上运行两个版本的flink?
更新:
我可以开始Flink的故事 1.9.3 以及 1.10.2 在同一个集群上但不是 1.11.2 我还是不知道该换什么来运行它。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题