我有一个奇怪的问题与logstash(v8.3.3)没有正确应用标签。
/conf.d/中当前只有两个配置文件,一个用于beats,一个用于syslog
01-节拍.conf
input {
beats {
port => 5044
ssl => false
tags => [ 'beat' ]
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGLINE}" }
}
date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
if 'beat' in [tags] {
elasticsearch {
hosts => localhost
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}
}
stdout {
codec => rubydebug
}
}
02-系统日志文件配置
input {
tcp {
port => 514
type => syslog
tags => [ 'syslog' ]
}
udp {
port => 514
type => syslog
tags => [ 'syslog' ]
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
if [message] =~ /(?i)crc/ {
mutate { add_tag => ["switch"] }
}
}
output {
if 'syslog' in [tags] {
elasticsearch {
hosts => ["localhost:9200"]
index => "syslog-%{+YYYY.MM}"
}
}
stdout {
codec => rubydebug
}
如果我通过以下方式运行logstash,则会在02-syslog.conf文件中出现问题:/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/02-syslog.conf
已成功添加switch标记。
范例:
root@elk-test ~ $ logger -n localhost -T -P 514 "crc error on port 48"
{
"type" => "syslog",
"tags" => [
[0] "syslog",
[1] "_grokparsefailure",
[2] "switch"
],
"@timestamp" => 2022-07-31T17:42:36.408104Z,
"@version" => "1",
"event" => {
"original" => "<13>1 2022-07-31T19:42:36.314411+02:00 srv-elk-hck.localdomain root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"88217\"] crc error on port 48"
},
"message" => "<13>1 2022-07-31T19:42:36.314411+02:00 srv-elk-hck.localdomain root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"88217\"] crc error on port 48"
}
而当我通过以下方式启动服务时:systemctl start logstash.service
不添加标记。
范例:
root@elk-test ~ $ logger -n localhost -T -P 514 "crc error on port 48"
{
"_index": "syslog-2022.07",
"_id": "mcJQVYIBNEboR1irs878",
"_version": 1,
"_score": 0,
"_source": {
"@timestamp": "2022-07-31T17:32:08.058612Z",
"message": [
"<13>1 2022-07-31T19:32:07.935622+02:00 srv-elk-hck.localdomain root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48",
"root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48"
],
"timestamp": "2022-07-31T19:32:07.935622+02:00",
"tags": [
"syslog",
"_dateparsefailure",
"_grokparsefailure"
],
"host": {
"hostname": "srv-elk-hck.localdomain"
},
"@version": "1",
"type": "syslog",
"event": {
"original": "<13>1 2022-07-31T19:32:07.935622+02:00 srv-elk-hck.localdomain root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48"
}
},
"fields": {
"event.original": [
"<13>1 2022-07-31T19:32:07.935622+02:00 srv-elk-hck.localdomain root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48"
],
"tags.keyword": [
"syslog",
"_dateparsefailure",
"_grokparsefailure"
],
"@version.keyword": [
"1"
],
"host.hostname": [
"srv-elk-hck.localdomain"
],
"type": [
"syslog"
],
"message": [
"<13>1 2022-07-31T19:32:07.935622+02:00 srv-elk-hck.localdomain root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48",
"root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48"
],
"tags": [
"syslog",
"_dateparsefailure",
"_grokparsefailure"
],
"@timestamp": [
"2022-07-31T17:32:08.058Z"
],
"type.keyword": [
"syslog"
],
"message.keyword": [
"<13>1 2022-07-31T19:32:07.935622+02:00 srv-elk-hck.localdomain root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48",
"root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48"
],
"event.original.keyword": [
"<13>1 2022-07-31T19:32:07.935622+02:00 srv-elk-hck.localdomain root - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"15780\"] crc error on port 48"
],
"host.hostname.keyword": [
"srv-elk-hck.localdomain"
],
"@version": [
"1"
],
"timestamp": [
"2022-07-31T17:32:07.935Z"
]
}
}
我错过了什么?
1条答案
按热度按时间eivnm1vs1#
服务必须使用不同的配置。在事件中,您显示的[message]是一个数组,这表明grok模式以
%{GREEDYDATA:message}
(message,而不是syslog_message)结尾。如果您修复了该问题,则会添加您的标记。否则,请尝试