canal 启动，instance日志中一直抛出 java.lang.NullPointerException: null

mftmpeh8 于 5个月前发布在 Java

关注(0)|答案(9)|浏览(162)

@agapple@rewerma

I have searched the issues of this repository and believe that this is not a duplicate.
I have checked the FAQ of this repository and believe that this is not a duplicate.

environment

Canal .1.6-hotfix-1 和 1.1.5
Mysql 8.0.20
服务器Linux version 3.10.0-957.el7.x86_64

Issue Description

启动canal的时候，example日志一直报错，使用 canal-adaptor或者SimpleCanalClientPermanceTest等测试文件无法获取任何数据变更信息

Steps to reproduce

tar -zxvf canal.deployer-1.1.6.tar.gz
修改conf/example/instance.properties 内容如下

#################################################
## mysql serverId , v1.0.26+ will autoGen
# canal.instance.mysql.slaveId=0

# enable gtid use true/false
canal.instance.gtidon=false

# position info
canal.instance.master.address=${address}
canal.instance.master.journal.name=
canal.instance.master.position=
canal.instance.master.timestamp=
canal.instance.master.gtid=

# rds oss binlog
canal.instance.rds.accesskey=
canal.instance.rds.secretkey=
canal.instance.rds.instanceId=

# table meta tsdb info
canal.instance.tsdb.enable=true
#canal.instance.tsdb.url=jdbc:mysql://127.0.0.1:3306/canal_tsdb
#canal.instance.tsdb.dbUsername=canal
#canal.instance.tsdb.dbPassword=canal

#canal.instance.standby.address =
#canal.instance.standby.journal.name =
#canal.instance.standby.position =
#canal.instance.standby.timestamp =
#canal.instance.standby.gtid=

# username/password
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
canal.instance.connectionCharset = UTF-8
# enable druid Decrypt database password
canal.instance.enableDruid=false
#canal.instance.pwdPublicKey=MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALK4BUxdDltRRE5/zXpVEVPUgunvscYFtEip3pmLlhrWpacX7y7GCMo2/JM6LeHmiiNdH1FWgGCpUfircSwlWKUCAwEAAQ==

# table regex
canal.instance.filter.regex=.*\\..*
# table black regex
canal.instance.filter.black.regex=mysql\\.slave_.*
# table field filter(format: schema1.tableName1:field1/field2,schema2.tableName2:field1/field2)
#canal.instance.filter.field=test1.t_product:id/subject/keywords,test2.t_company:id/name/contact/ch
# table field black filter(format: schema1.tableName1:field1/field2,schema2.tableName2:field1/field2)
#canal.instance.filter.black.field=test1.t_product:subject/product_image,test2.t_company:id/name/contact/ch

# mq config
canal.mq.topic=example
# dynamic topic route by schema or table regex
#canal.mq.dynamicTopic=mytest1.user,topic2:mytest2\\..*,.*\\..*
canal.mq.partition=0
# hash partition config
#canal.mq.enableDynamicQueuePartition=false
#canal.mq.partitionsNum=3
#canal.mq.dynamicTopicPartitionNum=test.*:4,mycanal:6
#canal.mq.partitionHash=test.table:id^name,.*\\..*
#################################################

sh startup.sh
logs/example/example.log 一直打印错误日志

Expected behaviour

如何修复这个错误

Actual behaviour

一直报错

If there is an exception, please attach the exception trace:

2022-12-28 17:20:48.844 [destination = example , address = /${address} , EventParser] WARN  c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position just show master status
2022-12-28 17:20:52.050 [destination = example , address = /${address} , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - dump address /${address} has an error, retrying. caused by 
java.lang.NullPointerException: null
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:5397) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.ast.statement.SQLCheck.accept0(SQLCheck.java:47) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:48) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.printTableElements(SQLASTOutputVisitor.java:3530) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:9568) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:133) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:93) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:48) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:89) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:74) ~[druid-1.2.11.jar:1.2.11]
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.snapshot(MemoryTableMeta.java:151) ~[canal.parse-1.1.6.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.applySnapshotToDB(DatabaseTableMeta.java:303) ~[canal.parse-1.1.6.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.rollback(DatabaseTableMeta.java:176) ~[canal.parse-1.1.6.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.processTableMeta(AbstractMysqlEventParser.java:142) ~[canal.parse-1.1.6.jar:na]
	at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$1.run(AbstractEventParser.java:197) ~[canal.parse-1.1.6.jar:na]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191]
2022-12-28 17:20:52.052 [destination = example , address = /${address} , EventParser] ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:example[java.lang.NullPointerException
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:5397)
	at com.alibaba.druid.sql.ast.statement.SQLCheck.accept0(SQLCheck.java:47)
	at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:48)
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.printTableElements(SQLASTOutputVisitor.java:3530)
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:9568)
	at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:133)
	at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:93)
	at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:48)
	at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:89)
	at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:74)
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.snapshot(MemoryTableMeta.java:151)
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.applySnapshotToDB(DatabaseTableMeta.java:303)
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.rollback(DatabaseTableMeta.java:176)
	at com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.processTableMeta(AbstractMysqlEventParser.java:142)
	at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$1.run(AbstractEventParser.java:197)
	at java.lang.Thread.run(Thread.java:748)

canal

来源：https://github.com/alibaba/canal/issues/4547

9条答案

按热度按时间

vybvopom1#

canal 账号权限足够的

错误代码定位

public Map<String, String> snapshot() {
        Map<String, String> schemaDdls = new HashMap<>();
        for (Schema schema : repository.getSchemas()) {
            StringBuffer data = new StringBuffer(4 * 1024);
            for (String table : schema.showTables()) {
                SchemaObject schemaObject = schema.findTable(table);
                schemaObject.getStatement().output(data);
                data.append("; \n");
            }
            schemaDdls.put(schema.getName(), data.toString());
        }

        return schemaDdls;
    }

schemaObject.getStatement().output(data); 报出null point , 调试的时候schemaObject.getStatement 非空，output(data)报错

赞(0）回复(0）举报 5个月前

3vpjnl9f2#

canal.instance.master.address=${address}
canal.instance.master.journal.name=
canal.instance.master.position=

关键参数没配置，name与position在数据库中执行show master status得到
@mjzfrank

赞(0）回复(0）举报 5个月前

sqyvllje3#

@zhng1456
一样的报错

2022-12-29 09:05:40.141 [destination = example , address = /${address} , EventParser] WARN  c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position mysql-bin.002908:230082980:1672275861000
2022-12-29 09:05:41.490 [destination = example , address = /${address} , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - dump address /${address} has an error, retrying. caused by 
java.lang.NullPointerException: null
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:5298) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.ast.statement.SQLCheck.accept0(SQLCheck.java:49) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:49) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.printTableElements(SQLASTOutputVisitor.java:3493) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:9390) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:125) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:85) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:49) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:90) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:75) ~[druid-1.2.6.jar:1.2.6]
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.snapshot(MemoryTableMeta.java:151) ~[canal.parse-1.1.5.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.applySnapshotToDB(DatabaseTableMeta.java:290) ~[canal.parse-1.1.5.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.rollback(DatabaseTableMeta.java:175) ~[canal.parse-1.1.5.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.processTableMeta(AbstractMysqlEventParser.java:142) ~[canal.parse-1.1.5.jar:na]
	at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$1.run(AbstractEventParser.java:197) ~[canal.parse-1.1.5.jar:na]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191]
2022-12-29 09:05:41.492 [destination = example , address = /${address} , EventParser] ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:example[java.lang.NullPointerException
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:5298)
	at com.alibaba.druid.sql.ast.statement.SQLCheck.accept0(SQLCheck.java:49)
	at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:49)
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.printTableElements(SQLASTOutputVisitor.java:3493)
	at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:9390)
	at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:125)
	at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:85)
	at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:49)
	at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:90)
	at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:75)
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.snapshot(MemoryTableMeta.java:151)

赞(0）回复(0）举报 5个月前

3bygqnnd4#

调试源码中

当表ddl 包含 CONSTRAINT ，MySqlCreateTableStatement.accept0 报错， bug存在 https://github.com/alibaba/druid 中

CREATE TABLE `test` (
  `id` bigint NOT NULL DEFAULT '0',
  `test_no` bigint NOT NULL DEFAULT '0',
  PRIMARY KEY (`id`),
  CONSTRAINT `test_chk_1` CHECK ((`test_no` is not null))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_as_cs

赞(0）回复(0）举报 5个月前

093gszye5#

参考 #4159
遇到类似异常后禁用tsdb正常
canal.instance.tsdb.enable=false

赞(0）回复(0）举报 5个月前

pqwbnv8z6#

解决了吗？我也遇到了这个问题

赞(0）回复(0）举报 5个月前

50few1ms7#

禁用tsdb后恢复正常配置文件canal.instance.tsdb.enable=false…

------------------ 原始邮件 ------------------ 发件人: ***@***.***>; 发送时间: 2023年4月18日(星期二) 中午11:24 收件人: ***@***.***>; 抄送: ***@***.***>; ***@***.***>; 主题: Re: [alibaba/canal] canal 启动，instance日志中一直抛出 java.lang.NullPointerException: null (Issue #4547) 解决了吗？我也遇到了这个问题 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

赞(0）回复(0）举报 5个月前

vxqlmq5t8#

这个一般是遇到druid的ddl解析异常

赞(0）回复(0）举报 5个月前

vof42yt19#

我禁用了canal.instance.tsdb.enable=false参数，仍然报错呢

赞(0）回复(0）举报 5个月前