DataX datax 有基于influx2.0的读写插件吗

nbysray5  于 2021-11-29  发布在  Java
关注(0)|答案(9)|浏览(613)

看了现在的influxdbreader和wirte 只提供了1.0api的配置,2.0的连接方式应该如何配置?

70gysomp

70gysomp1#

里面没有人回答啊!一点都不活跃啊!

qvsjd97n

qvsjd97n2#

都不维护了,pr都不合并了,100多个pr

kqqjbcuj

kqqjbcuj4#

我 fork 的版本 (https://github.com/wgzhao/Addax/tree/plugins/add_support_influxdb2) 正在创建支持 InfluxDB 2.0 版本的读写插件,预计本月可以完成,届时欢迎测试。

icnyk63a

icnyk63a5#

大佬辛苦了!期待ing

pu3pd22g

pu3pd22g6#

初步完成

bin/addax.sh  job/influx2stream.json

  ___      _     _
 / _ \    | |   | |
/ /_\ \ __| | __| | __ ___  __
|  _  |/ _` |/ _` |/ _` \ \/ /
| | | | (_| | (_| | (_| |>  <
\_| |_/\__,_|\__,_|\__,_/_/\_\

:: Addax version ::    (v4.0.3-SNAPSHOT)

2021-10-16 21:48:04.373 [        main] INFO  VMInfo               - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2021-10-16 21:48:04.395 [        main] INFO  Engine               -
{
"content":[
{
"reader":{
"parameter":{
"password":"*****",
"column":[
"location",
"height",
"wet"
],
"range":[
"-5h"
],
"connection":[
{
"bucket":"test",
"endpoint":"http://localhost:8086",
"org":"com.wgzhao",
"table":[
"temperature"
]
}
],
"token":"*",
"username":"influx"
},
"name":"influxdb2reader"
},
"writer":{
"parameter":{
"print":"true"
},
"name":"streamwriter"
}
}
],
"setting":{
"speed":{
"bytes":-1,
"channel":1
}
}
}

2021-10-16 21:48:04.412 [        main] INFO  PerfTrace            - PerfTrace traceId=job_-1, isEnable=false, priority=0
2021-10-16 21:48:04.412 [        main] INFO  JobContainer         - Addax jobContainer starts job.
2021-10-16 21:48:04.413 [        main] INFO  JobContainer         - Set jobId = 0
2021-10-16 21:48:04.422 [       job-0] INFO  InfluxDB2Reader$Job  - query sql:
from(bucket:"test")
|> range(start: -5h)
|> filter(fn: (r) =>  r._measurement =="temperature")
|> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value")

2021-10-16 21:48:04.922 [       job-0] INFO  JobContainer         - Addax Reader.Job [influxdb2reader] do prepare work .
2021-10-16 21:48:04.923 [       job-0] INFO  JobContainer         - Addax Writer.Job [streamwriter] do prepare work .
2021-10-16 21:48:04.923 [       job-0] INFO  JobContainer         - Job set Channel-Number to 1 channels.
2021-10-16 21:48:04.931 [       job-0] INFO  JobContainer         - Addax Reader.Job [influxdb2reader] splits to [1] tasks.
2021-10-16 21:48:04.931 [       job-0] INFO  JobContainer         - Addax Writer.Job [streamwriter] splits to [1] tasks.
2021-10-16 21:48:04.949 [       job-0] INFO  JobContainer         - Scheduler starts [1] taskGroups.
2021-10-16 21:48:04.955 [ taskGroup-0] INFO  TaskGroupContainer   - taskGroupId=[0] start [1] channels for [1] tasks.
2021-10-16 21:48:04.959 [ taskGroup-0] INFO  Channel              - Channel set byte_speed_limit to -1, No bps activated.
2021-10-16 21:48:04.959 [ taskGroup-0] INFO  Channel              - Channel set record_speed_limit to -1, No tps activated.
2021-10-16 21:48:04.971 [0-0-0-reader] INFO  InfluxDB2Reader$Task - query sql:
from(bucket:"test")
|> range(start: -5h)
|> filter(fn: (r) =>  r._measurement =="temperature")
|> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value")

south	0.0	98.79917211498994
south	11.213378550330416	128.2914766263023
south	18.227557041986085	143.31154450222954
south	20.197607365998625	73.42110223770723
south	34.24995322408072	137.60217891448792
south	12.097268438446765	99.65679452053762
south	6.5705939115079115	165.1786225169218

....

south	20.091793933660185	78.18571421443588
south	76.14518221584392	154.75183073851687
south	15.233804393893424	140.9468537079755
south	109.10254263467125	146.93918477063326
2021-10-16 21:48:07.966 [       job-0] INFO  AbstractScheduler    - Scheduler accomplished all tasks.
2021-10-16 21:48:07.967 [       job-0] INFO  JobContainer         - Addax Writer.Job [streamwriter] do post work.
2021-10-16 21:48:07.967 [       job-0] INFO  JobContainer         - Addax Reader.Job [influxdb2reader] do post work.
2021-10-16 21:48:07.969 [       job-0] INFO  JobContainer         - PerfTrace not enable!
2021-10-16 21:48:07.971 [       job-0] INFO  StandAloneJobContainerCommunicator - Total 120 records, 4768 bytes | Speed 1.55KB/s, 40 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.041s | Percentage 100.00%
2021-10-16 21:48:07.974 [       job-0] INFO  JobContainer         -
任务启动时刻                    : 2021-10-16 21:48:04
任务结束时刻                    : 2021-10-16 21:48:07
任务总计耗时                    :                  3s
任务平均流量                    :            1.55KB/s
记录写入速度                    :             40rec/s
读出记录总数                    :                 120
读写失败总数                    :                   0
5jvtdoz2

5jvtdoz27#

@edwar172038 已经合并到主分支了

n8ghc7c1

n8ghc7c18#

@wgzhao influxdb2 我已成功合并到自己项目 并验证读取成功了 感谢大佬支持!

f0brbegy

f0brbegy9#

实时数据同步,可以了解一下CloudCanal工具。目前社区版永久免费使用

相关问题