Question
我的conf/es/test.yml内容如下:
dataSourceKey: defaultDS
destination: example1
groupId: g1
esMapping:
_index: test6-subaccount_history
_type: _doc
_id: _id
upsert: true
#pk: id
sql: "select f_id as _id, f_id, f_version, f_user_id, f_account_id, f_subaccount_id, f_account_type, f_subaccount_type, f_action_type, f_currency, f_amount, f_balance, f_suspense, f_clearing, f_total, f_action_id, f_source_id, f_source_type, f_source_type_code, f_source_ext, f_created_at, f_updated_at from t_subaccount_history"
#objFields:
#_labels: array:;
etlCondition: "where f_version>3"
commitBatch: 3000
不加etl条件可以正常的同步至es,但加上etl条件,依旧是把所有mysql更新或新增记录都同步到es,仿佛etl条件无效
9条答案
按热度按时间ebdffaop1#
我也遇到了同样的问题
gojuced72#
Question
我的conf/es/test.yml内容如下:
dataSourceKey: defaultDS
destination: example1
groupId: g1
esMapping:
_index: test6-subaccount_history
_type: _doc
_id: _id
upsert: true
#pk: id
sql: "select f_id as _id, f_id, f_version, f_user_id, f_account_id, f_subaccount_id, f_account_type, f_subaccount_type, f_action_type, f_currency, f_amount, f_balance, f_suspense, f_clearing, f_total, f_action_id, f_source_id, f_source_type, f_source_type_code, f_source_ext, f_created_at, f_updated_at from t_subaccount_history"
#objFields:
#_labels: array:;
etlCondition: "where f_version>3"
commitBatch: 3000
不加etl条件可以正常的同步至es,但加上etl条件,依旧是把所有mysql更新或新增记录都同步到es,仿佛etl条件无效
请问你这个问题解决了吗?
ny6fqffe3#
etlCondition:条件无效
mjqavswn4#
同问
q0qdq0h25#
我也遇到了,1.1.5版本
s3fp2yjn6#
可能是yaml格式的问题,etlCondition字段需要在 esMapping子级
ubof19bj7#
也是醉了,这文档写的跟啥一样
etlCondition不支持直接写死的方式,如下
etlCondition: "where f_version>3"
只能这样etlCondition: "where f_version>{}"
然后再调用etl全量同步接口的时候
加上参数 ?params=3
这样才会生效
t1qtbnec8#
也是醉了,这文档写的跟啥一样 etlCondition不支持直接写死的方式,如下 etlCondition: "where f_version>3"
只能这样etlCondition: "where f_version>{}" 然后再调用etl全量同步接口的时候 加上参数 ?params=3 这样才会生效
请问如果有多个条件呢?etlcondition 怎么写?还有etl指令怎么拼接参数?
b5lpy0ml9#
也是醉了,这文档写的跟啥一样 etlCondition不支持直接写死的方式,如下 etlCondition: "where f_version>3"
只能这样etlCondition: "where f_version>{}" 然后再调用etl全量同步接口的时候 加上参数 ?params=3 这样才会生效
请问如果有多个条件呢?etlcondition 怎么写?还有etl指令怎么拼接参数?
用分号分隔
etlCondition: "where id>={} and age = {}"
curl http://127.0.0.1:8081/etl/rdb/mysql1/mytest_user.yml?params=4;44 -X POST