logstash 替换日志存储中的@timestamp

wvyml7n5  于 2023-02-27  发布在  Logstash
关注(0)|答案(1)|浏览(266)

我对我的logstash配置越来越着迷了。
我找不到将@timestamp字段替换为另一个字段的方法:
下面是logstash收到的内容:

{
                          "offset" => 6718968,
                    "Varnish_txid" => "639657758",
                       "plateform" => "cdnfronts",
                         "Referer" => "-",
                      "input_type" => "log",
                        "respsize" => "281",
                          "source" => "/var/log/varnish/varnish4xx-5xx.log",
                              "UA" => "Microsoft-WebDAV-MiniRedir/5.1.2600",
                            "type" => "varnish-logs",
                            "tags" => [
        [0] "json",
        [1] "varnish",
        [2] "beats_input_codec_json_applied",
        [3] "_dateparsefailure"
    ],
            "st_snt2c_or_sntfromb" => "405",
                      "RemoteHost" => "32.26.21.21",
                      "@timestamp" => 2017-02-14T13:38:47.808Z,
                "Varnish.Handling" => "pass",
    "tot_bytes_rcvby_c_or_sntby_b" => "-",
         "time_req_rcv4c_or_snt4b" => "[14/Feb/2017:14:38:44 +0100]",
                        "@version" => "1",
                            "beat" => {
        "hostname" => "cdn1",
            "name" => "cdn1",
         "version" => "5.1.2"
    },
                            "host" => "cdn1",
                   "time_1st_byte" => "0.010954",
                    "Varnish_side" => "c",
                    "reqfirstline" => "OPTIONS http://a.toto.com/ HTTP/1.1"
}

下面是我的logstash配置文件:

input {
    beats {
        port => 5000
        codec => "json"
        ssl => true
        ssl_certificate => "/etc/logstash/ssl/logstash-forwarder.crt"
        ssl_key => "/etc/logstash/ssl/logstash-forwarder.key"

    }
}
filter {
  if "json" in [tags] {
    json {
      source => "message"
    }
    if "varnish" in [tags] {
     date {
       locale => "en"
       match => [ "[time_req_rcv4c_or_snt4b]","dd/MMM/yyyy:HH:mm:ss Z" ]
       remove_field => "[time_req_rcv4c_or_snt4b]"
      }
    }
  }
}
output {
  if "varnish" in [tags] {
      elasticsearch {
        hosts => ["elasticsearch:9200"]
        index => "logstash-varnish-%{+YYYY.MM.dd}"
    }
  } else {
      elasticsearch {
        hosts => ["elasticsearch:9200"]
      }
  }
  
    stdout {
        codec => rubydebug
    }
}

我试过:

match => [ "time_req_rcv4c_or_snt4b","dd/MMM/yyyy:HH:mm:ss Z" ]
   remove_field => "time_req_rcv4c_or_snt4b"

以及

match => [ "[time_req_rcv4c_or_snt4b]","dd/MMM/yyyy:HH:mm:ss Z" ]
   remove_field => "[time_req_rcv4c_or_snt4]

任何人都可以解释我错过了什么。我没有找到任何相关的谷歌时刻。

pkmbmrz7

pkmbmrz71#

根据您的输出:

"time_req_rcv4c_or_snt4b" => "[14/Feb/2017:14:38:44 +0100]",

您的日期字段两边有[],因此您需要在日期模式中匹配这些字段,或者在第一次匹配日期时去掉它们。

相关问题