当debezium connector从您的sql服务器获取数据时,有没有方法限制kafka connect堆空间

jm2pwxwz  于 2021-06-04  发布在  Kafka
关注(0)|答案(1)|浏览(468)

我正在尝试设置一个连接器,从sql服务器获取数据以与ApacheKafka一起使用。我已经用docker compose文件设置了所有kafka服务,但是sql服务器在另一台服务器上。这是我在ksqldb中的debezium连接器的配置:

create source connector sql_connector with 
('connector.class'='io.debezium.connector.sqlserver.SqlServerConnector',
'database.server.name'='sqlserver',
'database.hostname'= 'xxxxx',
'database.port'='1433',
'database.user'= 'xxxx',
'database.password'= 'xxxxxx',
'database.dbname'='xxxxxxxxx',
'database.history.kafka.bootstrap.servers'='broker:29092',
'database.history.kafka.topic'='dbz_dbhistory.sqlserver.asgard-01');

执行此操作时,我得到一个响应,即连接器已成功创建。但是,当我通过对连接器使用“show connectors”查询ksqldb时,我得到以下错误消息:

io.confluent.ksql.util.KsqlServerException: org.apache.http.conn.HttpHostConnectException: Connect to connect:8083 [connect/172.18.0.6] failed: Connection refused (Connection refused)
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to
        connect:8083 [connect/172.18.0.6] failed: Connection refused (Connection
        refused)
Caused by: Could not connect to the server.
Caused by: Could not connect to the server.

当我检查我的kafka connect日志时,我可以看到它正在向服务器发出select语句,但是过了一段时间,我发现以下错误,我的kafka connect关闭了:

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000b5400000, 118489088, 0) failed; error='Not enough space' (errno=12)

# 

# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (mmap) failed to map 118489088 bytes for committing reserved memory.

# An error report file with more information is saved as:

有没有办法解决这个问题?除了给我的服务器更多的内存。

pes8fvy9

pes8fvy91#

您的计算机的可用内存不足~118mb:

Native memory allocation (mmap) failed to map 118489088 bytes for committing reserved memory

您需要增加或释放机器上的内存,以启动jvm。如果正在运行,可以使用以下环境变量更改jvm的堆内存设置:

KAFKA_HEAP_OPTS="-Xms256M -Xmx2G"

相关问题