Mariadb+Canal+Kafka数据异常

q9yhzks0  于 4个月前  发布在  Kafka
关注(0)|答案(2)|浏览(116)

Question

Mariadb: 10.4.13
Canal: 1.1.5

用docker-compose搭建运行环境

Mariadb,配置如下:
MariaDB [(none)]> show variables like 'binlog%';
+-----------------------------------------+--------+
| Variable_name | Value |
+-----------------------------------------+--------+
| binlog_annotate_row_events | ON |
| binlog_cache_size | 32768 |
| binlog_checksum | CRC32 |
| binlog_commit_wait_count | 0 |
| binlog_commit_wait_usec | 100000 |
| binlog_direct_non_transactional_updates | OFF |
| binlog_file_cache_size | 16384 |
| binlog_format | ROW |
| binlog_optimize_thread_scheduling | ON |
| binlog_row_image | FULL |
| binlog_stmt_cache_size | 32768 |
+-----------------------------------------+--------+

canal-server设置环境变量如下:

environment:
      - canal.auto.scan=false
      - canal.instance.master.address=db:3306
      - canal.instance.dbUsername=canal
      - canal.instance.dbPassword=QWqw12!@
      - canal.instance.filter.regex=.*\\..*
      - canal.instance.connectionCharset=UTF-8
      - canal.instance.tsdb.enable=true
      - canal.instance.filter.rows=false
      - canal.instance.filter.query.dml=true
      - canal.serverMode=kafka
      - canal.mq.servers=kafka:9092
      - canal.mq.topic=db-xxt
      - canal.mq.partition=0

这时kafka收不到任何数据。

如果将 canal.instance.filter.query.dml=true 注释掉。可以收到如下数据:

{"data":null,"database":"","es":1641177155000,"id":3,"isDdl":false,"mysqlType":null,"old":null,"pkNames":null,"sql":"insert into ttt(id,name) values(6,'fff')","sqlType":null,"table":"","ts":1641177155476,"type":"QUERY"}
{"data":null,"database":"","es":1641177220000,"id":4,"isDdl":false,"mysqlType":null,"old":null,"pkNames":null,"sql":"update ttt set name='ffa' where id = 6","sqlType":null,"table":"","ts":1641177220487,"type":"QUERY"}
{"data":null,"database":"","es":1641177266000,"id":5,"isDdl":false,"mysqlType":null,"old":null,"pkNames":null,"sql":"delete from  ttt  where id = 6","sqlType":null,"table":"","ts":1641177266776,"type":"QUERY"}

但是,其中很多重要的字段都是空的,例如:data,database和table。

是哪些参数设置的不对?

t3irkdon

t3irkdon1#

用python的客户端可以正常访问。而且参数canal.instance.filter.query.dml似乎没有什么影响。

相关问题