flume httpsource中的空事件

jdg4fx2g  于 2021-06-04  发布在  Flume
关注(0)|答案(1)|浏览(330)

我正在尝试在flume中设置一个非常简单的工作流,它可以执行以下操作:
json数据->httpsource->内存通道->记录器
我正在尝试发布此数据:
[{“string”:“这是一个测试桩”}]
但这里是我在记录器中得到的:
18/08/09 17:45:30 info sink.loggersink:事件:{标题:{}正文:}
我可以看出httpsource正在接收数据,因为如果我发送一个无效的json消息,我会得到以下错误:
原因:java.lang.illegalstateexception:应为begin\数组,但在第1行第3列为begin\对象
这是我的Flume配置:

  1. # Agent ============================================================================
  2. booking.sources = httpsource
  3. booking.channels = channel
  4. booking.sinks = hdfssink
  5. # Source ============================================================================
  6. booking.sources.httpsource.type = org.apache.flume.source.http.HTTPSource
  7. booking.sources.httpsource.port = 8090
  8. booking.sources.httpsource.bind = localhost
  9. # Channel ============================================================================
  10. booking.channels.channel.type = memory
  11. booking.channels.channel.capacity = 50000
  12. booking.channels.channel.transactionCapacity = 5000
  13. booking.sources.httpsource.channels = channel
  14. booking.sinks.hdfssink.channel = channel
  15. # Sink ============================================================================
  16. booking.sinks.hdfssink.type = logger

我用以下命令运行它:

  1. flume-ng agent --conf conf --conf-file conf/test.conf --name booking -Dflume.root.logger=DEBUG,console

下面是我用来发布json数据的命令:

  1. curl --verbose --header "Content-Type: application/json" -k -d "@test.txt" -X POST http://localhost:8091

有人知道数据可能会流向哪里吗?或者是一些额外的故障排除步骤?

fcg9iug3

fcg9iug31#

这只是我没有读足够的文件。jsonhandler需要的json格式是[{headers:{},body:“}],我的测试不符合这个格式。
以下是有帮助的文档:http://flume.apache.org/flumeuserguide.html#jsonhandler

相关问题