我正在使用以下6.x堆栈:
filebeat->logstash->elastic->kibana公司
我正在尝试配置filebeat/logstash,以便只向elastic发送特定的日志文件。现在它似乎正在发送所有信息。
当我查看kibana时,我看到大多数来自/var/log/messages的消息。
filebeat配置:
filebeat.inputs:
- type: log
enabled: true
paths:
#- /var/log/*.log
- /var/log/nginx/app.access.log
#- c:\programdata\elasticsearch\logs\*
output.logstash:
hosts: ["localhost:5044"]
index: logstash
日志存储配置:
input {
beats {
port => "5044"
}
}
filter {
grok {
match => {"message" => '%{IP:client} - %{USERNAME:username} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{DATA: http_x_forwarded_for} %{DATA:gzip_ratio} rt=%{NUMBER:request_time} uct=%{NUMBER:upstream_connect_time} uht=%{NUMBER:upstream_header_time} urt=%{NUMBER:upstream_response_time} %{NUMBER:user_id}'}
}
grok {
match => {"request" => "\/dataset\/%{NUMBER:dataset_id}"}
}
kv {
source => "request"
field_split => "&?"
transform_key => "lowercase"
}
if "/search" in [request] and [q] {
mutate {add_field => {"search_action" => "search" }}
}
if [request] =~ "\/dataset\/%{NUMBER:dataset_id}" {
mutate {add_field => {"dataset_id" => dataset_id}}
}
mutate {
convert => {
"user_id" => "integer"
"dataset_id" => "integer"
}
# lowercase => [ "request" ]
}
}
output {
elasticsearch {
codec => "json"
hosts => ["127.0.0.1:9200"]
}
stdout { codec => rubydebug }
}
使用此配置,我应该只获取发送到elastic的/var/log/nginx/app.access.log数据?
在kibana中,我看到多个日志文件中的所有活动:
/var/log/messages/var/log/secure
你知道为什么我找不到解决办法吗?
谢谢
暂无答案!
目前还没有任何答案,快来回答吧!