使用docker-compose的Flink连接被拒绝

blmhpbnm 于 2023-10-14 发布在 Apache

关注(0)|答案(2)|浏览(220)

我有下面的docker-compose文件，它是来自docker apache flink站点的docker-compose的副本。唯一的区别是我使用的是Mac M1版本。

version: "2.2"
services:
  jobmanager:
    image: arm64v8/flink:alpine
    ports:
      - "8081:8081"
    command: standalone-job --job-classname com.job.ClassName [--job-id <job id>] [--fromSavepoint /path/to/savepoint [--allowNonRestoredState]] [job arguments]
    volumes:
      - ~/sg_flink/artifacts:/opt/flink/usrlib
    networks:
      - flink-network
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager
        parallelism.default: 2

  taskmanager:
    image: arm64v8/flink:alpine
    depends_on:
      - jobmanager
    command: taskmanager
    scale: 1
    volumes:
      - ~/sg_flink/artifacts:/opt/flink/usrlib
    networks:
      - flink-network
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager
        taskmanager.numberOfTaskSlots: 2
        parallelism.default: 2
networks:
  flink-network:

错误是连接被拒绝

taskmanager_1  | 2021-11-03 17:43:02,724 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor            - Could not resolve ResourceManager address akka.tcp://flink@9cf35ea13c8b:6123/user/resourcemanager, retrying in 10000 ms: Could not connect to rpc endpoint under address akka.tcp://flink@9cf35ea13c8b:6123/user/resourcemanager..
taskmanager_1  | 2021-11-03 17:43:12,753 WARN  akka.remote.transport.netty.NettyTransport                    - Remote connection to [null] failed with java.net.ConnectException: Connection refused: 9cf35ea13c8b/172.20.0.3:6123
taskmanager_1  | 2021-11-03 17:43:12,756 WARN  akka.remote.ReliableDeliverySupervisor                        - Association with remote system [akka.tcp://flink@9cf35ea13c8b:6123] has failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@9cf35ea13c8b:6123]] Caused by: [Connection refused: 9cf35ea13c8b/172.20.0.3:6123]
taskmanager_1  | 2021-11-03 17:43:12,758 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor            - Could not resolve ResourceManager address akka.tcp://flink@9cf35ea13c8b:6123/user/resourcemanager, retrying in 10000 ms: Could not connect to rpc endpoint under address akka.tcp://flink@9cf35ea13c8b:6123/user/resourcemanager..

docker ps输出如下所示

CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                              NAMES
4416f88f60c2   arm64v8/flink:alpine   "/docker-entrypoint.…"   44 seconds ago   Up 44 seconds   6123/tcp, 8081/tcp                 sg_flink_taskmanager_1
c211940acf41   arm64v8/flink:alpine   "/docker-entrypoint.…"   45 seconds ago   Up 44 seconds   6123/tcp, 0.0.0.0:8081->8081/tcp   sg_flink_jobmanager_1

apache-flink

来源：https://stackoverflow.com/questions/69829539/connection-refused-for-flink-with-docker-compose

2条答案

按热度按时间

rta7y2nd1#

我在同样的情况下遇到过这个问题，我用docker-compse链接解决了它。

taskmanager:
  links:
    - jobmanager

赞(0）回复(0）举报 2023-10-14

nszi6y052#

对不起，我不能看到谁拥有一个地址172.20.0.3什么帮助了我在同样的情况：
1.检查你的taskmanager可以连接到'jobmaster：6123'（我使用flink：1.17-scala_2.12，所以你可能不需要安装netcat）：

docker exec -it sg_flink_taskmanager_1 /bin/bash
apt-get update
apt-get install -y netcat
nc -vz jobmaster 6123

如果你得到'Connection refused'，请检查你的容器是否在同一个网络中（或者你的主机名实际上是taskmanager和jobmanager）。
如果你得到'连接到jobmanager（这里是jobmanager IP地址）6123端口[tcp/*]成功！“进入下一步。
1.你可以在taskmanager contaner中的flink-conf.yaml或者taskmanager contaner的日志中检查你的默认flink参数：

在我的例子中，属性文件/opt/flink/conf/flink-conf.yaml有：

jobmanager.rpc.address: taskmanager (Look! this is by default and wrong for this utilize! It must be jobmanager)
 taskmanager.numberOfTaskSlots: 1(Look! it is by default, it must be 2)
 parallelism.default: 1(Look! it is by default, it must be 2)

taskmanager contaner日志与上述错误值相同：

INFO  [] - Loading configuration property: jobmanager.rpc.address, taskmanager
 INFO  [] - Loading configuration property: parallelism.default, 1
 INFO  [] - Loading configuration property: taskmanager.numberOfTaskSlots, 1

1.所以我看到|由于某种原因没有工作，为了确保我尝试了这条线：

- "FLINK_PROPERTIES=jobmanager.rpc.address: jobmanager\ntaskmanager.numberOfTaskSlots: 2\nparallelism.default: 2\njobmanager.rpc.port: 6123"

在docker-compose文件中，它提交了我的env。

赞(0）回复(0）举报 2023-10-14

我来回答

使用docker-compose的Flink连接被拒绝

2条答案

相关问题

热门标签

最新问答