我的上下文是我正在尝试创建一个docker compose,它将启动几个容器来运行elk+filebeats和3个kafka容器。整个docker compose及其yml配置文件可以在github docker compose中找到
简单地说,这是我的docker compose部件下载和启动我的kafka容器之一:
kafka1:
image: wurstmeister/kafka
command: [start-kafka.sh]
depends_on:
- zoo1
- zoo2
- zoo3
links:
- zoo1
- zoo2
- zoo3
ports:
- "9092:9092"
environment:
KAFKA_LISTENERS: PLAINTEXT://:9092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
KAFKA_BROKER_ID: 1
KAFKA_ADVERTISED_PORT: 9092
KAFKA_LOG_RETENTION_HOURS: "168"
KAFKA_LOG_RETENTION_BYTES: "100000000"
KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
KAFKA_CREATE_TOPICS: "log:3:3"
KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
我想用Kafka调查Kafka
C:\Dockers\megalog-try-1>docker exec -it megalog-try-1_kafka1_1 bash
bash-4.4# kafka-topics.sh --list --bootstrap-server localhost:9092
__consumer_offsets
log
bash-4.4# kafkacat -b megalog-try-1_kafka1_1:9092 L
bash: kafkacat: command not found
bash-4.4# apt-get install kafkacat
bash: apt-get: command not found
bash-4.4# exit
exit
编辑
安装kafkacat后的日志
/ # kafkacat -C -b localhost:9092 -t log -o beginning -f '\nKey (%K bytes): %k\t\nValue (%S bytes): %s\n\Partition: %p\t
Offset: %o\n--\n'
/bin/sh: kafkacat: not found
/ # apk --no-cache add ca-certificates wget
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/1) Installing wget (1.20.3-r0)
Executing busybox-1.29.3-r10.trigger
OK: 359 MiB in 76 packages
/ # wget --quiet --output-document=/etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub
/ # wget https://github.com/sgerrand/alpine-pkg-kafkacat/releases/download/1.5.0-r0/kafkacat-1.5.0-r0.apk
--2020-02-05 22:56:46-- https://github.com/sgerrand/alpine-pkg-kafkacat/releases/download/1.5.0-r0/kafkacat-1.5.0-r0.apk
Resolving github.com... 18.228.52.138
Connecting to github.com|18.228.52.138|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/120518176/95d10080-eee3-11e9-8e83-c0ba853d6714?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200206%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200206T131952Z&X-Amz-Expires=300&X-Amz-Signature=19d4778753313a7a535c3f8c10645e13851d482f49035903de83ba93c087a247&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dkafkacat-1.5.0-r0.apk&response-content-type=application%2Fvnd.android.package-archive [following]
--2020-02-05 22:56:46-- https://github-production-release-asset-2e65be.s3.amazonaws.com/120518176/95d10080-eee3-11e9-8e83-c0ba853d6714?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200206%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200206T131952Z&X-Amz-Expires=300&X-Amz-Signature=19d4778753313a7a535c3f8c10645e13851d482f49035903de83ba93c087a247&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dkafkacat-1.5.0-r0.apk&response-content-type=application%2Fvnd.android.package-archive
Resolving github-production-release-asset-2e65be.s3.amazonaws.com... 52.216.236.203
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com|52.216.236.203|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22602 (22K) [application/vnd.android.package-archive]
Saving to: 'kafkacat-1.5.0-r0.apk'
kafkacat-1.5.0-r0.apk 100%[=================================================>] 22.07K --.-KB/s in 0.1s
2020-02-05 22:56:48 (167 KB/s) - 'kafkacat-1.5.0-r0.apk' saved [22602/22602]
/ # apk add --no-cache kafkacat-1.5.0-r0.apk
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/6) Installing lz4-libs (1.8.3-r2)
(2/6) Installing db (5.3.28-r1)
(3/6) Installing libsasl (2.1.27-r2)
(4/6) Installing librdkafka (0.11.6-r1)
(5/6) Installing yajl (2.1.0-r0)
(6/6) Installing kafkacat (1.5.0-r0)
Executing busybox-1.29.3-r10.trigger
OK: 361 MiB in 82 packages
/ # kafkacat -C -b localhost:9092 -t log -o beginning -f '\nKey (%K bytes): %k\t\nValue (%S bytes): %s\n\Partition: %p\t
Offset: %o\n--\n'
Error relocating /usr/bin/kafkacat: rd_kafka_fatal_error: symbol not found
/ # kafkacat -b megalog-try-1_kafka1_1:9092 L
Error relocating /usr/bin/kafkacat: rd_kafka_fatal_error: symbol not found
/ # kafka-topics.sh --list --bootstrap-server localhost:9092
__consumer_offsets
log
/ #
根据cricket的建议编辑
docker-compose.yml(仅编辑行)
...
image: confluentinc/cp-kafka:latest
...
image: confluentinc/cp-zookeeper:latest
...
相关日志
Kafka:
===> Configuring ...
advertised.port is deprecated. Please use KAFKA_ADVERTISED_LISTENERS instead.
Zookeeper:
ZOOKEEPER_CLIENT_PORT is required.
Command [/usr/local/bin/dub ensure ZOOKEEPER_CLIENT_PORT] FAILED !
2条答案
按热度按时间sirbozc51#
您必须使用apk在这个kafka图像容器上安装任何包。
此软件包的当前安装方法是使用wget或curl将其拉入并使用apk安装本地文件:
ippsafx72#
请改用Kafka听众
只需删除端口变量。你已经有这个了
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
需要zookeeper\客户端\端口添加它
ZOOKEEPER_CLIENT_PORT: 2181
你可以混合和匹配容器。。。我只是建议您在现有设置中使用现有的kafkacat容器,它可以连接到任何远程代理。或者在本地系统上安装kafkacat