我得到的错误是:- 20/11/02 13:34:51 WARN NetworkClient: [Consumer clientId=consumer-spark-kafka-source-366ac503-c5a4-4338-869c-84786983aab3--188679505-driver-0-1, groupId=spark-kafka-source-366ac503-c5a4-4338-869c-84786983aab3--188679505-driver-0] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
我的docker-compose.yml文件
version: "3"
services:
zookeeper:
image: wurstmeister/zookeeper
container_name: zookeeper
ports:
- 2181:2181
kafka:
image: wurstmeister/kafka
container_name: kafka
ports:
- 9092:9092
environment:
KAFKA_LISTENERS: PLAINTEXT://:9092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_CREATE_TOPICS: "kafkatutorial:1:1"
ALLOW_PLAINTEXT_LISTENER: "yes"
SSL: localhost:9092
expose:
- 9092
spark:
build:
dockerfile: DockerFileSpark
context: .
environment:
- SPARK_MODE=master
- SPARK_RPC_AUTHENTICATION_ENABLED=no
- SPARK_RPC_ENCRYPTION_ENABLED=no
- SPARK_LOCAL_STORAGE_ENCRYPTION_ENABLED=no
- SPARK_SSL_ENABLED=no
ports:
- '8080:8080'
links:
- kafka
depends_on:
- zookeeper
- kafka
1条答案
按热度按时间8qgya5xd1#
SSL: localhost:9092
不是kafka容器的有效环境变量;您需要在两个侦听器变量中设置ssl,并向映像中添加ssl证书不清楚你的Spark代码在哪里运行,但是在一个容器里,
localhost
无法连接到外部服务。拆下links
在您的yaml中并连接到kafka:9092
在代码中