如何配置fluentd配置文件来向kafka发送日志?

1cosmwyk  于 2021-06-06  发布在  Kafka
关注(0)|答案(0)|浏览(613)

我一直在努力创建一个syslog应用程序,在这个应用程序中,我必须收集使用fluentd生成的不同类型的日志,并将它们发送给cassandra,以便稍后进行分析。我必须使用Kafka作为一种缓冲或流处理器。为了将kafka与fluentd结合使用,我必须配置fluentd。
我已经安装了fluentd,cassandra,kafka和zookeeper,它们都独立运行,没有任何错误。但是当我试图用 fluent.conf 文件,我遇到多个错误。
我的 fluent.conf 文件是:

<source>
  @type syslog
  bind 0.0.0.0
  port 5140
  tag default
  protocol_type udp
  source_hostname_key source_host
  <parse>
   message_format auto
  </parse>
</source>

<match**>
  @type stdout
</match>

<match app.**>
  @type kafka_buffered
  zookeeper             localhost:2181
  zookeeper_path        root/attackfence/kafka_2.11-2.2.0/zookeeeper
  partition_key         partition
  partition_key_key     partition_key
  message_key_key       message_key
  default_topic         nil
  default_partition_key nil
  default_message_key   nil
  output_data_type      json
  output_include_tag    false
  output_include_time   false
  exclude_topic_key     false
  exclude_partition_key false
  get_kafka_client_log  true
</match>

我得到的错误是:

2019-05-23 18:22:44 +0530 [error]: #0 unexpected error error_class=NoMethodError error="undefined method `each' for nil:NilClass"
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluent-plugin-kafka-0.9.4/lib/fluent/plugin/out_kafka_buffered.rb:117:in `refresh_client'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluent-plugin-kafka-0.9.4/lib/fluent/plugin/out_kafka_buffered.rb:208:in `start'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/compat/call_super_mixin.rb:42:in `start'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:165:in `block in start'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:154:in `block (2 levels) in lifecycle'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:153:in `each'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:153:in `block in lifecycle'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:140:in `each'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:140:in `lifecycle'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:164:in `start'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/engine.rb:274:in `start'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/engine.rb:219:in `run'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/supervisor.rb:799:in `run_engine'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/supervisor.rb:549:in `block in run_worker'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/supervisor.rb:724:in `main_process'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/supervisor.rb:544:in `run_worker'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/command/fluentd.rb:316:in `<top (required)>'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/bin/fluentd:8:in `<top (required)>'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/bin/fluentd:23:in `load'
  2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/bin/fluentd:23:in `<main>'

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题