aws胶水流作业无法连接到Kafka

uajslkp6  于 2021-07-13  发布在  Spark
关注(0)|答案(0)|浏览(181)

我正在尝试用自我管理的kafka集群(在ec2示例上)实现glue etl作业。正如我在本地机器上用简单的poducer和consumer脚本检查的那样,集群正在工作。问题是,在glue job中,我在日志中收到以下警告:

Feb 11, 2021, 5:31:06 PM 21/02/11 11:31:06 WARN NetworkClient: [Consumer clientId=consumer-1, groupId=spark-kafka-source-3e37027f-1b50-4aac-bbbf-68c11f4db9bd-1062155183-driver-0] Connection to node -1 could not be established. Broker may not be available.
Feb 11, 2021, 5:33:17 PM 21/02/11 11:33:17 WARN NetworkClient: [Consumer clientId=consumer-1, groupId=spark-kafka-source-3e37027f-1b50-4aac-bbbf-68c11f4db9bd-1062155183-driver-0] Connection to node -3 could not be established. Broker may not be available.
Feb 11, 2021, 5:35:28 PM 21/02/11 11:35:28 WARN NetworkClient: [Consumer clientId=consumer-1, groupId=spark-kafka-source-3e37027f-1b50-4aac-bbbf-68c11f4db9bd-1062155183-driver-0] Connection to node -2 could not be established. Broker may not be available.
Feb 11, 2021, 5:37:39 PM 21/02/11 11:37:39 WARN NetworkClient: [Consumer clientId=consumer-1, groupId=spark-kafka-source-3e37027f-1b50-4aac-bbbf-68c11f4db9bd-1062155183-driver-0] Connection to node -1 could not be established. Broker may not be available.
Feb 11, 2021, 5:39:50 PM 21/02/11 11:39:50 WARN NetworkClient: [Consumer clientId=consumer-1, groupId=spark-kafka-source-3e37027f-1b50-4aac-bbbf-68c11f4db9bd-1062155183-driver-0] Connection to node -3 could not be established. Broker may not be available.

我检查了sgs的ec2示例,胶水连接的设置,但它仍然不工作

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题