下面是我正在处理的一个示例日志条目(JSON):
{
"Order": {
"content": {
"seqnum": "107",
"type": "DAIRY",
"section": "A1",
"time": "2022-03-02T14:21:45",
"version": "24",
"src": "EAST",
"status": "3"
},
"crc": {
"crcvalue": "45BD2E93"
}
}
},
下面是我的logstash过滤器。
filter {
json {
source => "message"
}
mutate { add_field => { "Order_Version" => "%{Order.content.version}" } }
mutate { add_field => { "Order_State" => "%{Order.content.status}" } }
mutate { convert => { "Order_Version" => "integer" } }
mutate { convert => { "Order_State" => "integer" } }
if [Order_State] == 3
{
mutate {
add_field => { "Order_State" => "Processing" }
}
}
if [Order_State] == 1
{
mutate {
add_field => { "Order_State" => "Shipped" }
}
}
if [Order_State] == 4
{
mutate {
add_field => { "Order_State" => "Pending" }
}
}
mutate { remove_field => ["message", "@timestamp", "path", "host", "@version", "Order.content.version", "Order.content.status", "Order.content.seqnum", "crcvalue"] }
}
当我应用上面的过滤器时,我能够正确地将所有数据解析到字段中,但是当我尝试改变数据时,我无法使用“if”条件添加新字段。我不确定,但是“version”和“status”字段中的数据没有添加到新字段中。虽然正在创建新字段,但是它们中只有“0”,这可能是因为我将它们转换为整数。“remove_field”只适用于由elastic生成的字段,而不适用于json消息字段。
#_Data in Elastic
{
"_index" : "order-sample-1",
"_type" : "_doc",
"_id" : "uXh-a38ByXB23tWqi2r3",
"_score" : 1.0,
"_source" : {
"Order" : {
"content" : {
"section" : "A1",
"src" : "EAST",
"status" : "3",
"seqnum" : "107",
"time" : "2022-03-07T14:21:45",
"version" : "24"
},
"crc" : {
"crcvalue" : "45BD2E93"
}
},
"Order_State" : 0,
"Order_Version" : 0
}
},
#_fields on the kibana
Order.content.seqnum: 107
Order.content.type: DAIRY
Order.content.section: A1
Order.content.time: 2022-03-02T14:21:45
Order.content.version: 24
Order.content.src: EAST
Order.content.status: 3
Order.crc.crcvalue: 45BD2E93
我如何复制/使用json字段的内容并将其放入一个新字段?或者我应该只使用gsub并替换字段的内容?
所有我试图实现的是像下面,并删除其余的领域.
section: A1
time: 2022-03-02T14:21:45
src: EAST
Order_Version: 24
Order_State: Processing
任何帮助都是感激的。谢谢。
1条答案
按热度按时间hi3rlvi21#
对于
{ "Order" : { "content" : { "section" : "A1" } } }
对象,elasticsearch和kibana将使用Order. content. section引用它。logstash使用语法[Order][content][section],这样做的好处是,如果您有另一个对象
{ "Order" : { "content.section" : "A1" } }
,那么logstash可以明确地区分[Order][content][section]和[Order][content.section]。因此,在变体中使用“%{[订单][内容][版本]}”等。