我对我的logstash配置越来越着迷了。
我找不到将@timestamp字段替换为另一个字段的方法:
下面是logstash收到的内容:
{
"offset" => 6718968,
"Varnish_txid" => "639657758",
"plateform" => "cdnfronts",
"Referer" => "-",
"input_type" => "log",
"respsize" => "281",
"source" => "/var/log/varnish/varnish4xx-5xx.log",
"UA" => "Microsoft-WebDAV-MiniRedir/5.1.2600",
"type" => "varnish-logs",
"tags" => [
[0] "json",
[1] "varnish",
[2] "beats_input_codec_json_applied",
[3] "_dateparsefailure"
],
"st_snt2c_or_sntfromb" => "405",
"RemoteHost" => "32.26.21.21",
"@timestamp" => 2017-02-14T13:38:47.808Z,
"Varnish.Handling" => "pass",
"tot_bytes_rcvby_c_or_sntby_b" => "-",
"time_req_rcv4c_or_snt4b" => "[14/Feb/2017:14:38:44 +0100]",
"@version" => "1",
"beat" => {
"hostname" => "cdn1",
"name" => "cdn1",
"version" => "5.1.2"
},
"host" => "cdn1",
"time_1st_byte" => "0.010954",
"Varnish_side" => "c",
"reqfirstline" => "OPTIONS http://a.toto.com/ HTTP/1.1"
}
下面是我的logstash配置文件:
input {
beats {
port => 5000
codec => "json"
ssl => true
ssl_certificate => "/etc/logstash/ssl/logstash-forwarder.crt"
ssl_key => "/etc/logstash/ssl/logstash-forwarder.key"
}
}
filter {
if "json" in [tags] {
json {
source => "message"
}
if "varnish" in [tags] {
date {
locale => "en"
match => [ "[time_req_rcv4c_or_snt4b]","dd/MMM/yyyy:HH:mm:ss Z" ]
remove_field => "[time_req_rcv4c_or_snt4b]"
}
}
}
}
output {
if "varnish" in [tags] {
elasticsearch {
hosts => ["elasticsearch:9200"]
index => "logstash-varnish-%{+YYYY.MM.dd}"
}
} else {
elasticsearch {
hosts => ["elasticsearch:9200"]
}
}
stdout {
codec => rubydebug
}
}
我试过:
match => [ "time_req_rcv4c_or_snt4b","dd/MMM/yyyy:HH:mm:ss Z" ]
remove_field => "time_req_rcv4c_or_snt4b"
以及
match => [ "[time_req_rcv4c_or_snt4b]","dd/MMM/yyyy:HH:mm:ss Z" ]
remove_field => "[time_req_rcv4c_or_snt4]
任何人都可以解释我错过了什么。我没有找到任何相关的谷歌时刻。
1条答案
按热度按时间pkmbmrz71#
根据您的输出:
您的日期字段两边有[],因此您需要在日期模式中匹配这些字段,或者在第一次匹配日期时去掉它们。