生产者和消费者无法写入或读取主题

jk9hmnmh  于 2021-06-08  发布在  Kafka
关注(0)|答案(1)|浏览(382)

我无法在kafka 0.10.2中使用ssl配置授权。我正在为代理、生产者和消费者使用命令行客户机。在下列情况下,生产者和使用者不能对测试主题进行写入或读取: allow.everyone.if.no.acl.found=true 在kafka服务器配置文件中被注解掉(否则,它们可以读写)
我已经浏览了官方文档、这个symantec设置、合流文档和各种堆栈溢出帖子,但是我仍然无法获得授权(尽管我通过tls进行了身份验证)。
我的证书来自identrust/letsencrypt。如果我取消注解 allow.everyone.if.no.acl.found=true ,当生产者连接时,我在代理日志中看到:

DEBUG SslTransportLayer:358 - SSL handshake completed successfully with
 peerHost 'devel-2.sjml.com' peerPort 56099 peerPrincipal 'CN=testkafkaconsumer1.eigenroute.com' cipherSuite 
'TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384'

生产者能够写信给消费者,消费者能够阅读测试主题。但是,当前面提到的行被注解掉时,这个输出不会出现在日志中。在这种情况下,producer命令行客户端输出以下内容:

WARN Error while fetching metadata with correlation id 10588 :
 {test100=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)

下面是主题test100的zookeeper的acl,以及列出它的命令:

$ bin/kafka-acls.sh --list --authorizer-properties zookeeper.connect=localhost:2181 --topic test100
Current ACLs for resource `Topic:test100`:
    User:CN=testkafkaconsumer1.eigenroute.com has Allow permission for operations: Read from hosts: *
    User:CN=kafka.eigenroute.com has Allow permission for operations: All from hosts: *
    User:CN=testkafkaproducer1.eigenroute.com has Allow permission for operations: Write from hosts: *

以下是我用于将用户添加到acl的命令:

./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=kafka.eigenroute.com --operation All --topic test100
./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=testkafkaproducer1.eigenroute.com --operation Write --topic test100
./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=testkafkaconsumer1.eigenroute.com --operation Read --topic test100

操作系统:debian 8杰西
以下是代理、使用者和生产者的配置文件:
代理配置:


# secure-server-letsencrypt.properties

broker.id=0

delete.topic.enable=true

listeners=SSL://kafka.eigenroute.com:9093
port=9093
advertised.host.name=kafka.eigenroute.com
ssl.keystore.location=/home/kafka/keystore/kafka.keystore.jks
ssl.keystore.password=some-password
ssl.key.password=some-password
ssl.truststore.location=/usr/lib/jvm/java-8-oracle/jre/lib/security/cacerts
ssl.truststore.password=some-password
ssl.endpoint.identification.algorithm=HTTPS
ssl.client.auth=required
security.inter.broker.protocol=SSL

authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:CN=testkafkaproducer1.eigenroute.com

# allow.everyone.if.no.acl.found=true

advertised.listeners=SSL://kafka.eigenroute.com:9093

num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600

log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000

使用者配置:


# secure-consumer.properties

zookeeper.connect=127.0.0.1:2181

# timeout in ms for connecting to zookeeper

zookeeper.connection.timeout.ms=6000

# consumer group id

group.id=test-consumer-group

# consumer timeout

# consumer.timeout.ms=5000

security.protocol=SSL
ssl.truststore.location=/usr/lib/jvm/java-8-oracle/jre/lib/security/cacerts
ssl.truststore.password=some-password
ssl.keystore.location=/home/kafka/keystore/testkafkaconsumer1.keystore.jks
ssl.keystore.password=some-password
ssl.key.password=some-password

生产者配置:

bootstrap.servers=kafka.eigenroute.com:9093

security.protocol=SSL
ssl.truststore.location=/usr/lib/jvm/java-8-oracle/jre/lib/security/cacerts
ssl.truststore.password=some-password
ssl.keystore.location=/home/kafka/keystore/testkafkaproducer1.keystore.jks
ssl.keystore.password=some-password
ssl.key.password=some-password

compression.type=none

我认为将producer用户设置为超级用户,就像我在broker/server配置中所做的那样,应该允许producer写入主题;唉,事实并非如此。代理似乎无法从zookeeper中找到acl。有人能建议怎么解决这个问题吗?谢谢!

jq6vz3qz

jq6vz3qz1#

如的cli文档中所述 --consumer 以及 --producer 选项,您需要允许对消费者的主题进行描述和阅读,也可以对消费者组进行阅读
对于生产者,你需要描述并写下生产者的主题,同时在集群上创建。

相关问题