使用logstash的多个kafka主题具有不同的hdfs输出不起作用

g52tjvyc  于 2021-06-08  发布在  Kafka
关注(0)|答案(0)|浏览(273)

我使用logstash来消耗kafka数据并将日志保存到hdfs,我想用不同的kafka主题分割hdfs输出,我的logstash配置如下:

input {
  kafka {
    bootstrap_servers => "192.168.1.14:9092"
    group_id => "logstash_act"
    topics => ["act", "log"]
    decorate_events => true
    consumer_threads => 5
    }
}
output {
    if [kafka][topic] == "act" {
        webhdfs {
            host => "192.168.1.8"
            port => 50070
            user => "root"
            path => "/ubuntu/input/logstash-%{+YYYY}-%{+MM}-%{+dd}.act"
            codec => plain { format => "%{message}" }
            idle_flush_time => 10
            retry_interval => 0.5 flush_size => 500
        }
    }

    else if [kafka][topic] == "log" {
        webhdfs {
            host => "192.168.1.8"
            port => 50070
            user => "root"
            path => "/ubuntu/input/logstash-%{+YYYY}-%{+MM}-%{+dd}.log"
            codec => plain { format => "%{message}" }
            idle_flush_time => 10
            retry_interval => 0.5 flush_size => 500
        }
    }
}

没有错误,只是根本没有输出。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题