如何配置flink SQL客户端以提交启用检查点的作业?

xytpbqjk  于 2023-08-01  发布在  Apache
关注(0)|答案(3)|浏览(229)

以下是启用检查点的flinksql-client配置文件的示例:
当我通过flink sql客户端提交流作业时,检查点没有启用。

a11xaf1n

a11xaf1n1#

检查点配置不能在flink sql客户端配置文件中设置,但可以在群集配置文件(flink-conf.yaml)中设置。
您至少应该配置execution.checkpointing.intervalstate.backendstate.checkpoints.dir
比如说:

execution.checkpointing.interval: 10000
state.backend: filesystem
state.checkpoints.dir: file:///tmp/flink-checkpoints-directory
state.savepoints.dir: file:///tmp/flink-savepoints-directory

字符串
有关详细信息,请参阅检查点、容错和状态后端的配置文档。

uurv41yg

uurv41yg2#

就我在文档中看到的,你应该能够从sql客户端通过设置这样的属性来做到这一点:

SET 'state.checkpoints.dir' = 'hdfs:///bar/foo/';
SET 'execution.checkpointing.mode' = 'EXACTLY_ONCE';
SET 'execution.checkpointing.interval' = '30min';
SET 'execution.checkpointing.min-pause' = '20min';
SET 'execution.checkpointing.max-concurrent-checkpoints' = '1';
SET 'execution.checkpointing.prefer-checkpoint-for-recovery' = 'true';

字符串

o75abkj4

o75abkj43#

在调用sql客户端时,您实际上可以传入一个init sql文件,而不必更新群集配置文件。
./sql-client

-i,--init <initialization file>            Script file that used to init
                                            the session context. If get
                                            error in execution, the sql
                                            client will exit. Notice it's
                                            not allowed to add query or
                                            insert into the init file.

字符串
然后,您可以在文件中指定

SET 'execution.checkpointing.interval' = '1min';


https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/

相关问题