MacOS M1上的Docker Kafka配置时卡住的问题

4dc9hkyq  于 6个月前  发布在  Apache
关注(0)|答案(5)|浏览(72)

我使用的是macOS M1 Big Sur 11.2.3,但是我的Kafka不能很好地运行,并且不能创建/列出主题。我不知道这是否是操作系统的原因,但是Kafka的日志只有这样:
docker-compose logs
list the topics logs
下面是我Docker编写器:

services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    hostname: zookeeper
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
    networks:
      - kafka_net
  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    ports:
      - 9092:9092
    expose:
      - 29092
    depends_on:
      - zookeeper
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_LISTENERS: INSIDE://0.0.0.0:29092,OUTSIDE://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:29092,OUTSIDE://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 1
    restart: always
    networks:
      - kafka_net
networks:
  kafka_net:
    driver: bridge

字符串
我认为Kafka还没有运行,所以我不能列出/创建一个主题.你们有什么想法吗?我已经搜索关于这一点的可能性,但我仍然没有解决问题.谢谢

uinbv5nw

uinbv5nw1#

只需将平台添加到docker-compose.yml文件中:
平台:Linux/amd 64

services:
  zookeeper:
    image: wurstmeister/zookeeper
    platform: linux/amd64
    container_name: zookeeper
    hostname: zookeeper
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
    networks:
      - kafka_net
    ...

字符串
如果以上选项没有帮助,那么您应该尝试以下操作:

version: '3'

services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
  
  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181

bd1hkmkf

bd1hkmkf2#

docker-compose.yaml文件没问题,你只需要在Kafka容器中执行以下命令:

  1. kafka-topics.sh--zookeeper zookeeper:2181 --主题测试--创建--分区3 --复制因子1
  2. kafka-topics.sh--zookeeper zookeeper:2181 --list
    顺便说一句:您可以使用以下命令进入Kafka容器:docker exec -it CONTAINER_ID bash
3qpi33ja

3qpi33ja3#

让我提供我花了一整天创建的docker-compose文件。
下面的docker-compose文件为zookeeper、Kafka和kafka-manager CMAK构建容器

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    restart: always
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    restart: always
    ports:
      - "9092:9092"
      - "1099:1099"
    environment:
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_HOST_NAME: kafka
      JMX_PORT: 1099
  kafka_manager:
    image: hlebalbau/kafka-manager:stable
    restart: always
    ports:
      - "9000:9000"
    environment:
      ZK_HOSTS: zookeeper:2181
    command: -Dpidfile.path=/dev/null

字符串
第一个月
然后,为了通过CMAK管理群集,您需要执行以下操作

docker exec -it zookeeper bash
./bin/zkCli.sh
create /kafka-manager/mutex ""
create /kafka-manager/mutex/locks ""
create /kafka-manager/mutex/leases ""

wgmfuz8q

wgmfuz8q4#

上面提到的docker-compose.yml在我的Mac M1-Pro上也可以工作。最新的Kafka也不需要zookeeper。这也包含了Kafka-UI容器。

version: '3'

networks:
  app-tier:
    driver: bridge

services:
  kafka:
    image: 'bitnami/kafka:latest'
    container_name: kafka
    hostname: kafka
    ports:
      - '9092:9092'
    networks:
      - app-tier
    environment:
      KAFKA_CFG_NODE_ID: 0
      KAFKA_CFG_PROCESS_ROLES: controller,broker
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 0@kafka:29093
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LISTENERS: 'PLAINTEXT://kafka:29092,CONTROLLER://kafka:29093,PLAINTEXT_HOST://0.0.0.0:9092'
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092

  kafka-ui:
    image: provectuslabs/kafka-ui:latest
    container_name: kafka-ui
    ports:
      - "8090:8080"
    depends_on:
      - kafka
    networks:
      - app-tier
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAP_SERVERS: kafka:29092

字符串
图片来源:Kafka-UI


的数据
友情链接:https://hub.docker.com/r/bitnami/kafka

kninwzqo

kninwzqo5#

对于在mac m1上的docker中启动Kafka,我做了以下事情:
1.克隆Kafka repo并在我的macbook pro m1上构建(./gradlew clean releaseTarGz
1.创建docker-compose项目目录与next结构

  • 根目录
  • kafka_m1
  • Kafka(unziped Kafka distr)
  • Dockerfile
  • docker-compose.yml

1.配置server.properties文件(root目录/kafka_m1/kafka/config/server.properties),主要参数:

zookeeper.connect=zookeeper:2181
   listeners=PLAINTEXT://:9092
   advertised.listeners=PLAINTEXT://127.0.0.1:9092
   listener.security.protocol.map=PLAINTEXT:PLAINTEXT

字符串

  1. Kafka的Dockerfileopenjdk:15.0.2-jdk是针对m1构建的):
FROM openjdk:15.0.2-jdk
    
    WORKDIR /
    COPY . /
    
    EXPOSE 9092
    EXPOSE 8092
    EXPOSE 9092
    EXPOSE 10092
    EXPOSE 11092
    EXPOSE 12092
    
    ENTRYPOINT [ "/kafka/bin/kafka-server-start.sh", "/kafka/config/server.properties" ]


1.docker-compose.yml(我选择zookeeper:3.7.0,因为它有arm build):

version: "2"
  services:
  zookeeper:
    image: zookeeper:3.7.0
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
  kafka:
    build: ./kafka_m1
    ports:
      - "127.0.0.1:9092:9092"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock


1.使用kafka_m1镜像构建启动docker-compose。
在docker中运行本地Kafka的快速回执

相关问题