大家好,我正在尝试使用tshark捕获网络流量,并使用apacheflume将这些结果发送给spark。
问题是,当我在flume的配置中使用exec源时,flume不工作,它在启动后立即停止
我的配置:
agent.sources = tsharkSource
agent.channels = memoryChannel
agentasinks = avroSink
# Configuring the sources to pull the bashes from Tshark
agent.sources.tsharkSource.type = exec
agent.sources.tsharkSource.command = tshark -T json
agent.sources.tsharkSource.channels = memoryChannel
# Configuring the sink to push logs to spark change hostname to 116's ip adress
agent.sinks.avroSink.type = avro
agent.sinks.avroSink.channel = memoryChannel
agent.sinks.avroSink.hostname = 192.168.1.112
agent.sinks.avroSink.port= 6969
# Configuring the memory channel
agent.channels.memoryChannel.type = memory
agent.channels.memoryChannel.capacity = 1000
agent.channels.memoryChannel.transactionCapacity = 100
shell输出:
flume-ng agent -f conf/flume-conf.properties -n agent
Warning: No configuration directory set! Use --conf <dir> to override.
Info: Including Hive libraries found via () for Hive access
+ exec /usr/lib/jum/jaua-1.11.0-openjdk-amd64//bin/jaua -Xmx20m -cp
1/home/oshiflume/flume/apache-flume-1.8.0-bin/lib/*:/libfle -
Djava.library.path= org.apache.flume.node.Application -f conf/flume-
conf.properties -n agent
log4j:WARN No appenders could be found for logger (org.apache.flume.node.Application).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.htmlDnoconfig for more info.
1条答案
按热度按时间p4rjhz4m1#
你的执行命令是
我看到两个错误。首先,必须使用
-c conf
flume配置文件通常命名为some-config.conf
控制台上的警告是No configuration directory set! Use --conf
、-c和--conf是同一件事。您可能需要将配置文件从flume-conf.properties重命名为flume.conf
作为解决方案,您可以尝试以下命令:
如果要在执行后显示日志,请使用以下命令
要显示日志,log4j.properties必须在conf目录中作为conf/log4j.properties。
我的财产如下: