Mariadb+Canal+Kafka数据异常

q9yhzks0  于 7个月前  发布在  Kafka
关注(0)|答案(2)|浏览(217)

Question

Mariadb: 10.4.13
Canal: 1.1.5

用docker-compose搭建运行环境

Mariadb,配置如下:
MariaDB [(none)]> show variables like 'binlog%';
+-----------------------------------------+--------+
| Variable_name | Value |
+-----------------------------------------+--------+
| binlog_annotate_row_events | ON |
| binlog_cache_size | 32768 |
| binlog_checksum | CRC32 |
| binlog_commit_wait_count | 0 |
| binlog_commit_wait_usec | 100000 |
| binlog_direct_non_transactional_updates | OFF |
| binlog_file_cache_size | 16384 |
| binlog_format | ROW |
| binlog_optimize_thread_scheduling | ON |
| binlog_row_image | FULL |
| binlog_stmt_cache_size | 32768 |
+-----------------------------------------+--------+

canal-server设置环境变量如下:

  1. environment:
  2. - canal.auto.scan=false
  3. - canal.instance.master.address=db:3306
  4. - canal.instance.dbUsername=canal
  5. - canal.instance.dbPassword=QWqw12!@
  6. - canal.instance.filter.regex=.*\\..*
  7. - canal.instance.connectionCharset=UTF-8
  8. - canal.instance.tsdb.enable=true
  9. - canal.instance.filter.rows=false
  10. - canal.instance.filter.query.dml=true
  11. - canal.serverMode=kafka
  12. - canal.mq.servers=kafka:9092
  13. - canal.mq.topic=db-xxt
  14. - canal.mq.partition=0

这时kafka收不到任何数据。

如果将 canal.instance.filter.query.dml=true 注释掉。可以收到如下数据:

  1. {"data":null,"database":"","es":1641177155000,"id":3,"isDdl":false,"mysqlType":null,"old":null,"pkNames":null,"sql":"insert into ttt(id,name) values(6,'fff')","sqlType":null,"table":"","ts":1641177155476,"type":"QUERY"}
  1. {"data":null,"database":"","es":1641177220000,"id":4,"isDdl":false,"mysqlType":null,"old":null,"pkNames":null,"sql":"update ttt set name='ffa' where id = 6","sqlType":null,"table":"","ts":1641177220487,"type":"QUERY"}
  1. {"data":null,"database":"","es":1641177266000,"id":5,"isDdl":false,"mysqlType":null,"old":null,"pkNames":null,"sql":"delete from ttt where id = 6","sqlType":null,"table":"","ts":1641177266776,"type":"QUERY"}

但是,其中很多重要的字段都是空的,例如:data,database和table。

是哪些参数设置的不对?

t3irkdon

t3irkdon1#

用python的客户端可以正常访问。而且参数canal.instance.filter.query.dml似乎没有什么影响。

41ik7eoe

41ik7eoe2#

有同样的问题

相关问题