使用kibana和logstash在网格中显示XML数据

carvr3hs  于 2022-12-09  发布在  Kibana
关注(0)|答案(1)|浏览(198)

我想使用logstash和Kibana以网格格式显示XML数据。使用下面的conf文件,我可以将数据显示到网格中,但不能拆分行数据。示例:

输出量

logstash.conf文件:

input {
 file {
path => "C:/ELK Stack/logstash-8.2.0-windows-x86_64/logstash-8.2.0/Test.xml"
start_position => "beginning"  
sincedb_path => "NUL"
codec => multiline {
pattern => "^<?stations.*>"
negate => "true"
what => "previous"
auto_flush_interval => 1
max_lines => 3000   
}}}

filter
{
 xml
   {
    source => "message"
    target => "parsed"
    store_xml => "false"
    xpath => [
        "/stations/station/id/text()", "station_id",
        "/stations/station/name/text()", "station_name"
    ]
}
mutate {
   remove_field => [ "message"]
}
}
output {  
elasticsearch {
    action => "index"
    hosts => "localhost:9200"
     index => "logstash_index123xml"
    workers => 1
}
stdout {
codec => rubydebug
}

}

zbq4xfa0

zbq4xfa01#

xpath总是返回数组,要关联两个数组的成员,你需要使用一个ruby过滤器。要获得多个事件,你可以使用一个split过滤器来拆分你在ruby过滤器中构建的数组。如果你以

<stations> 
<station> 
<id>1</id> 
<name>a</name> 
<id>2</id>
<name>b</name> 
</station> 
</stations>

那么如果您使用

xml {
        source => "message"
        store_xml => "false"
        xpath => {
            "/stations/station/id/text()" => "[@metadata][station_id]"
            "/stations/station/name/text()" => "[@metadata][station_name]"
        }
        remove_field => [ "message" ]
    }
    ruby {
        code => '
            ids = event.get("[@metadata][station_id]")
            names = event.get("[@metadata][station_name]")
            if ids.is_a? Array and names.is_a? Array y and ids.length == names.length
                a = []
                ids.each_index { |x|
                    a << { "station_name" => names[x], "station_id" => ids[x] }
                }
                event.set("[@metadata][theData]", a)
            end
        '
    }
    if [@metadata][theData] {
        split {
            field => "[@metadata][theData]"
            add_field => {
                "station_name" => "%{[@metadata][theData][station_name]}"
                "station_id" => "%{[@metadata][theData][station_id]}"
            }
        }
    }

您将获得两个事件

{
    "station_name" => "a",
      "station_id" => "1",
    ...
}
{
    "station_name" => "b",
      "station_id" => "2",
    ...
}

相关问题