elasticsearch Logstash opensearch输出没有获取document_id

czfnxgou 于 2023-03-22 发布在 ElasticSearch

关注(0)|答案(2)|浏览(151)

我正在尝试使用以下命令在两个opensearch示例之间传输数据：

input {
    opensearch {
        hosts =>  ["http://192.168.1.245:8443"]
        index =>  "crawl"
        query =>  '{ "query": { "match_all": {}} }'
    }
}

output {
  opensearch {
    hosts => ["http://192.168.1.245:9200"]
    index => "crawl"
    document_id => "%{[id]}"
    doc_as_upsert => true 
  }
  stdout { codec => rubydebug { metadata => true } }
}

我遇到的问题是它没有将输入document_id插入到输出document_id中：
我试过：

document_id => "%{[id]}"
document_id => "%{id}"
document_id => "%{[_id]}"
document_id => "%{_id}"
document_id => "%{[document_id]}"
document_id => "%{document_id}"
document_id => "%{[metadata][id]}"

它将其保存到索引中作为查找：

"_index": "crawl",
"_id": "%{document_id}",
"_score": 1.0,

有人知道怎么让它工作吗？

elasticsearch

来源：https://stackoverflow.com/questions/75801157/logstash-opensearch-output-no-getting-document-id

2条答案

按热度按时间

jogvjijk1#

你需要这样做：

在输入中添加docinfo => true
在[@metadata][_id]字段中查找文档的_id

配置：

input {
    opensearch {
        hosts =>  ["http://192.168.1.245:8443"]
        index =>  "crawl"
        query =>  '{ "query": { "match_all": {}} }'
        docinfo => true                           <----------
    }
}

output {
  opensearch {
    hosts => ["http://192.168.1.245:9200"]
    index => "crawl"
    document_id => "%{[@metadata][input][opensearch][_id]}"          <----------
    doc_as_upsert => true 
  }
  stdout { codec => rubydebug { metadata => true } }
}

赞(0）回复(0）举报 2023-03-22

9lowa7mx2#

根据Val给出的建议，我设法让它与以下内容一起工作：

input {
    opensearch {
        hosts =>  ["http://192.168.1.245:8443"]
        index =>  "crawl"
        query =>  '{ "query": { "match_all": {}} }'
        docinfo => true <----
        docinfo_target => "[@metadata][doc]" <----
    }
}

output {
  opensearch {
    hosts => ["http://192.168.1.245:9200"]
    index => "crawl"
    document_id => "%{[@metadata][doc][_id]}" <----
    doc_as_upsert => true 
  }
  stdout { codec => rubydebug { metadata => true } }
}

赞(0）回复(0）举报 2023-03-22

我来回答

elasticsearch Logstash opensearch输出没有获取document_id

2条答案

相关问题

热门标签

最新问答