如何自动删除aws中超过1个月的ElasticSearch记录

kpbpu008  于 2021-06-14  发布在  ElasticSearch
关注(0)|答案(2)|浏览(452)

我有一个功能,我必须删除超过1个月的elasticsearch记录。我可以通过一个cron作业在ElasticSearch上运行delete查询来实现这一点,但我想自动实现这一点。
就像aws中的s3文件一样,我可以设置ttl。类似地,在elastic search>7.1中查找类似于\u ttl的内容

f1tvaqid

f1tvaqid1#

您可以使用专门为考虑这些用例而设计的索引生命周期管理选项。
它也是基本许可证的一部分,因此您不必购买它,请参阅弹性订阅了解更多详细信息。

pgccezyw

pgccezyw2#

Posting the complete answer:
1. Create the policy to rollover based on max_age and delete after 10 mins
policy = {
    "policy": {
        "phases": {
            "hot": {
                "actions": {
                    "rollover": {
                        "max_age": "11m"
                    }
                }
            },
            "delete": {
                "min_age": "10m",
                "actions": {
                    "delete": {}
                }
            }
        }
    }
}

2. Insert the policy:
IlmClient.put_lifecycle(es, "datastream_policy", policy)

3. Create index template to apply policy to all rollover index being created so that they will be deleted after 10 mins:

template = {
    "index_patterns": ["datastream-*"],
    "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 0,
        "index.lifecycle.name": "datastream_policy",
        "index.lifecycle.rollover_alias": "datastream"
    },
    "mappings": {
        "email": "keyword"
    }
}

4. Put the index template in elastic search
es.indices.put_template(name="datastream_template", body=json.dumps(template))

5. Create a starter index:
indexd = {
    "aliases": {
        "datastream": {
            "is_write_index": True
        }
    }
}

es.indices.create("datastream-000001", body=indexd)

相关问题