使用grok过滤logstash中的apache错误日志

wpx232ag  于 2021-06-14  发布在  ElasticSearch
关注(0)|答案(1)|浏览(641)

我需要使用grok过滤apache错误日志。请帮助我的模式,我无法创建!。
我的示例日志:

2020-10-07T01:21:26.403-0400    ERROR   [reload]        cfgfile/list.go:96  Error creating runner from config: Error getting config for fileset system/auth: Error interpreting the template of the inp$
2020-10-07T01:21:36.404-0400    ERROR   [reload]        cfgfile/list.go:96  Error creating runner from config: Error getting config for fileset system/auth: Error interpreting the template of the inp$
2020-10-07T01:21:38.925-0400    ERROR   pipeline/output.go:100  Failed to connect to backoff(async(tcp://IP:5044)): dial tcp IP:5044: i/o timeout
2020-10-07T01:21:38.925-0400    INFO    pipeline/output.go:93   Attempting to reconnect to backoff(async(tcp://IP:5044)) with 26743 reconnect attempt(s)
2020-10-07T01:21:38.925-0400    INFO    [publish]   pipeline/retry.go:189   retryer: send unwait-signal to consumer
2020-10-07T01:21:38.925-0400    INFO    [publish]   pipeline/retry.go:191     done
2020-10-07T01:21:38.925-0400    INFO    [publish]   pipeline/retry.go:166   retryer: send wait signal to consumer
2020-10-07T01:21:38.925-0400    INFO    [publish]   pipeline/retry.go:168     done

我理解通过grok模式,我们可以使用以下这些,但我不理解如何在grok模式中使用这些:


# Error logs

HTTPD20_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{LOGLEVEL:loglevel}\] (?:\[client %{IPORHOST:clientip}\] ){0,1}%{GREEDYDATA:message}
HTTPD24_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{WORD:module}:%{LOGLEVEL:loglevel}\] \[pid %{POSINT:pid}(:tid %{NUMBER:tid})?\]( \(%{POSINT:proxy_errorcode}\)%{DATA:proxy_message}:)?( \[client %{IPORHOST:clientip}:%{POSINT:clientport}\])?( %{DATA:errorcode}:)? %{GREEDYDATA:message}
HTTPD_ERRORLOG %{HTTPD20_ERRORLOG}|%{HTTPD24_ERRORLOG}

谁能帮帮我!提前谢谢!

kcrjzv8t

kcrjzv8t1#

在处理示例数据之后,这种grok模式必须起作用:

filter {
    grok {
            match => { "message" => "%{TIMESTAMP_ISO8601}%{SPACE}%{LOGLEVEL}(%{SPACE}\[%{WORD:action}\])?%{SPACE}%{WORD:package}/%{WORD:class}.go:%{INT:line:number}%{SPACE}%{GREEDYDATA:message}$" }
    }
}

你的数据不完全是http,所以需要一个自定义模式,我想我的grok必须更容易阅读,没有空间,我建议你使用mutate gsub来统一空间(请记住我的解决方案中最后一个名为“message”的句子)。
这里有更多关于这个模式和其他模式的细节。

相关问题