在同一台机器上运行两个apache flink集群

laawzig2  于 2021-06-21  发布在  Flink
关注(0)|答案(1)|浏览(662)

我想在一台机器上,在不同的unix用户下运行两个ApacheFlink示例。
我已经为它设置了不同的端口 jobmanager.rpc.port 以及 rest.port 但是,当尝试通过 start-cluster.sh 它抱怨经常使用的地址:

  1. 2019-04-04 13:48:08,218 INFO akka.remote.Remoting - Starting remoting
  2. 2019-04-04 13:48:08,226 INFO akka.remote.Remoting - Remoting started; listening on addresses :[akka.tcp://flink-metrics@localhost:43345]
  3. 2019-04-04 13:48:08,227 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Actor system started at akka.tcp://flink-metrics@localhost:43345
  4. 2019-04-04 13:48:08,233 INFO org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore - Initializing FileArchivedExecutionGraphStore: Storage directory /tmp/executionGraphStore-dd135030-d700-4b52-8bda-4bcb7e0a7b35, expiration time 3600000, maximum
  5. cache size 52428800 bytes.
  6. 2019-04-04 13:48:08,269 INFO org.apache.flink.runtime.blob.TransientBlobCache - Created BLOB cache storage directory /tmp/blobStore-2f3c12d2-ae4f-4185-ae41-a4d845d5a3e8
  7. 2019-04-04 13:48:08,288 WARN org.apache.flink.configuration.Configuration - Config uses deprecated configuration key 'jobmanager.rpc.address' instead of proper key 'rest.address'
  8. 2019-04-04 13:48:08,289 WARN org.apache.flink.configuration.Configuration - Config uses deprecated configuration key 'web.address' instead of proper key 'rest.bind-address'
  9. 2019-04-04 13:48:08,290 WARN org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Upload directory /tmp/flink-web-30ad01a5-24d0-47d0-b73e-6c9e0a2fab30/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
  10. 2019-04-04 13:48:08,291 INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Created directory /tmp/flink-web-30ad01a5-24d0-47d0-b73e-6c9e0a2fab30/flink-web-upload for file uploads.
  11. 2019-04-04 13:48:08,295 INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Starting rest endpoint.
  12. 2019-04-04 13:48:08,786 INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined location of main cluster component log file: /home/ideventer/system/flink/flink-1.7.2/log/flink-ideventer-standalonesession-1-devsrv-java-1.log
  13. 2019-04-04 13:48:08,786 INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined location of main cluster component stdout file: /home/ideventer/system/flink/flink-1.7.2/log/flink-ideventer-standalonesession-1-devsrv-java-1.out
  14. 2019-04-04 13:48:08,949 INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shutting down rest endpoint.
  15. 2019-04-04 13:48:08,954 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Shutting StandaloneSessionClusterEntrypoint down with application status FAILED. Diagnostics org.apache.flink.util.FlinkException: Could not create the DispatcherResourceManagerComponent.
  16. at org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResourceManagerComponentFactory.java:242)
  17. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:215)
  18. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:163)
  19. at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
  20. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:162)
  21. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:517)
  22. at org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.main(StandaloneSessionClusterEntrypoint.java:65)
  23. Caused by: java.net.BindException: Address already in use
  24. at sun.nio.ch.Net.bind0(Native Method)
  25. at sun.nio.ch.Net.bind(Net.java:433)
  26. at sun.nio.ch.Net.bind(Net.java:425)
  27. at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
  28. at org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128)
  29. at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:558)
  30. at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1358)
  31. at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
  32. at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
  33. at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:1019)
  34. at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel.bind(AbstractChannel.java:254)
  35. at org.apache.flink.shaded.netty4.io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:366)
  36. at org.apache.flink.shaded.netty4.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
  37. at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
  38. at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
  39. at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
  40. at java.lang.Thread.run(Thread.java:745)
  41. .
  42. 2019-04-04 13:48:08,958 INFO org.apache.flink.runtime.blob.BlobServer - Stopped BLOB server at 0.0.0.0:60277
  43. 2019-04-04 13:48:08,959 INFO org.apache.flink.runtime.blob.TransientBlobCache - Shutting down BLOB cache
  44. 2019-04-04 13:48:08,970 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping Akka RPC service.
  45. 2019-04-04 13:48:08,973 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon.
  46. 2019-04-04 13:48:08,973 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon.
  47. 2019-04-04 13:48:08,975 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports.
  48. 2019-04-04 13:48:08,975 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports.
  49. 2019-04-04 13:48:08,997 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting shut down.
  50. 2019-04-04 13:48:09,010 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting shut down.
  51. 2019-04-04 13:48:09,028 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped Akka RPC service.
  52. 2019-04-04 13:48:09,032 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could not start cluster entrypoint StandaloneSessionClusterEntrypoint.
  53. org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint StandaloneSessionClusterEntrypoint.
  54. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:181)
  55. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:517)
  56. at org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.main(StandaloneSessionClusterEntrypoint.java:65)
  57. Caused by: org.apache.flink.util.FlinkException: Could not create the DispatcherResourceManagerComponent.
  58. at org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResourceManagerComponentFactory.java:242)
  59. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:215)
  60. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:163)
  61. at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
  62. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:162)
  63. ... 2 more
  64. Caused by: java.net.BindException: Address already in use
  65. at sun.nio.ch.Net.bind0(Native Method)
  66. at sun.nio.ch.Net.bind(Net.java:433)
  67. at sun.nio.ch.Net.bind(Net.java:425)
  68. at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
  69. at org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128)
  70. at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:558)
  71. at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1358)
  72. at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
  73. at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
  74. at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:1019)
  75. at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel.bind(AbstractChannel.java:254)
  76. at org.apache.flink.shaded.netty4.io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:366)
  77. at org.apache.flink.shaded.netty4.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
  78. at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
  79. at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
  80. at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
  81. at java.lang.Thread.run(Thread.java:745)

为了启动第二个示例,还必须设置哪些其他选项?

dddzy1tm

dddzy1tm1#

我自己也没试过,但仔细看看https://ci.apache.org/projects/flink/flink-docs-release-1.7/ops/config.html 在搜索“port”时,我发现有几个地方会说“建议设置一系列端口以避免冲突”。
首先看一下taskmanager.rpc.port和metrics.internal.query-service.port。可查询状态和blob服务器端口也可能是一个因素——不确定。

相关问题