elasticsearch索引生命周期策略不滚动的问题

qvtsj1bj  于 2021-06-14  发布在  ElasticSearch
关注(0)|答案(1)|浏览(1435)

为了评估其对我们日常运营的帮助潜力，我部署了elastic search和kibana（7.7.1，具有基本许可证），并为ntopng（我们的监控平台）创建了一个索引模板。
由于索引不断增长，我想删除超过20天左右的ntopng索引，因此我创建了一个名为ntopng的生命周期策略，其中时间戳索引应在1天后滚动（出于测试目的），然后在滚动2天后删除：

接下来，我选择了当天创建的时间戳索引，并将生命周期策略应用于它：

在此之前，我必须为该索引创建一个别名，因此我手动执行了以下操作：

  1. POST /_aliases
  2. {
  3.   "actions" : [
  4.     { "add" : { "index" : "ntopng-2020.09.09", "alias" : "ntopng_Alias" } }
  5.   ]
  6. }

在那之后（我猜）一切看起来都很好，因为没有显示错误或警报：

  1. "indices" : {
  2.     "ntopng-2020.09.09" : {
  3.       "index" : "ntopng-2020.09.09",
  4.       "managed" : true,
  5.       "policy" : "ntopng",
  6.       "lifecycle_date_millis" : 1599609600433,
  7.       "age" : "20.14h",
  8.       "phase" : "hot",
  9.       "phase_time_millis" : 1599681721821,
  10.       "action" : "rollover",
  11.       "action_time_millis" : 1599680521920,
  12.       "step" : "check-rollover-ready",
  13.       "step_time_millis" : 1599681721821,
  14.       "is_auto_retryable_error" : true,
  15.       "failed_step_retry_count" : 1,
  16.       "phase_execution" : {
  17.         "policy" : "ntopng",
  18.         "phase_definition" : {
  19.           "min_age" : "0ms",
  20.           "actions" : {
  21.             "rollover" : {
  22.               "max_age" : "1d"
  23.             },
  24.             "set_priority" : {
  25.               "priority" : 100
  26.             }
  27.           }
  28.         },
  29.         "version" : 4,
  30.         "modified_date_in_millis" : 1599509572867
  31.       }
  32.     }

我的预期是，在第二天，该政策将自动滚动到下一个指数（ntopng-2020.10.10），这样最初的指数最终将在未来两天被删除。
相反，我得到了以下错误：

  1. GET ntopng-*/_ilm/explain
  2. {
  3.   "indices" : {
  4.     "ntopng-2020.09.09" : {
  5.       "index" : "ntopng-2020.09.09",
  6.       "managed" : true,
  7.       "policy" : "ntopng",
  8.       "lifecycle_date_millis" : 1599609600433,
  9.       "age" : "1.94d",
  10.       "phase" : "hot",
  11.       "phase_time_millis" : 1599776521822,
  12.       "action" : "rollover",
  13.       "action_time_millis" : 1599680521920,
  14.       "step" : "ERROR",
  15.       "step_time_millis" : 1599777121822,
  16.       "failed_step" : "check-rollover-ready",
  17.       "is_auto_retryable_error" : true,
  18.       "failed_step_retry_count" : 80,
  19.       "step_info" : {
  20.         "type" : "illegal_argument_exception",
  21.         "reason" : """index name [ntopng-2020.09.09] does not match pattern '^.*-\d+$'""",
  22.         "stack_trace" : """java.lang.IllegalArgumentException: index name [ntopng-2020.09.09] does not match pattern '^.*-\d+$'
  23.     at org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.generateRolloverIndexName(TransportRolloverAction.java:241)
  24.     at org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.masterOperation(TransportRolloverAction.java:133)
  25.     at org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.masterOperation(TransportRolloverAction.java:73)
  26.     at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.lambda$doStart$3(TransportMasterNodeAction.java:170)
  27.     at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73)
  28.     at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  29.     at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:225)
  30.     at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.doStart(TransportMasterNodeAction.java:170)
  31.     at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.start(TransportMasterNodeAction.java:133)
  32.     at org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:110)
  33.     at org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:59)
  34.     at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:153)
  35.     at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:123)
  36.     at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:151)
  37.     at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:129)
  38.     at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:64)
  39.     at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83)
  40.     at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72)
  41.     at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:399)
  42.     at org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:92)
  43.     at org.elasticsearch.xpack.core.ClientHelper.executeWithHeadersAsync(ClientHelper.java:155)
  44.     at org.elasticsearch.xpack.ilm.LifecyclePolicySecurityClient.doExecute(LifecyclePolicySecurityClient.java:51)
  45.     at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:399)
  46.     at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1234)
  47.     at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.rolloverIndex(AbstractClient.java:1736)
  48.     at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:127)
  49.     at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:173)
  50.     at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:329)
  51.     at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:267)
  52.     at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:183)
  53.     at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:211)
  54.     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
  55.     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  56.     at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
  57.     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  58.     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
  59.     at java.base/java.lang.Thread.run(Thread.java:832)
  60. """
  61.       },
  62.       "phase_execution" : {
  63.         "policy" : "ntopng",
  64.         "phase_definition" : {
  65.           "min_age" : "0ms",
  66.           "actions" : {
  67.             "rollover" : {
  68.               "max_age" : "1d"
  69.             },
  70.             "set_priority" : {
  71.               "priority" : 100
  72.             }
  73.           }
  74.         },
  75.         "version" : 4,
  76.         "modified_date_in_millis" : 1599509572867
  77.       }
  78.     }
  80.   "ntopng-2020.09.10" : {
  81.       "index" : "ntopng-2020.09.10",
  82.       "managed" : true,
  83.       "policy" : "ntopng",
  84.       "lifecycle_date_millis" : 1599696000991,
  85.       "age" : "22.57h",
  86.       "phase" : "hot",
  87.       "phase_time_millis" : 1599776521844,
  88.       "action" : "rollover",
  89.       "action_time_millis" : 1599696122033,
  90.       "step" : "ERROR",
  91.       "step_time_millis" : 1599777121839,
  92.       "failed_step" : "check-rollover-ready",
  93.       "is_auto_retryable_error" : true,
  94.       "failed_step_retry_count" : 67,
  95.       "step_info" : {
  96.         "type" : "illegal_argument_exception",
  97.         "reason" : "index.lifecycle.rollover_alias [ntopng_Alias] does not point to index [ntopng-2020.09.10]",
  98.         "stack_trace" : """java.lang.IllegalArgumentException: index.lifecycle.rollover_alias [ntopng_Alias] does not point to index [ntopng-2020.09.10]
  99.     at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:104)
  100.     at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:173)
  101.     at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:329)
  102.     at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:267)
  103.     at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:183)
  104.     at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:211)
  105.     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
  106.     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  107.     at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
  108.     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
  109.     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
  110.     at java.base/java.lang.Thread.run(Thread.java:832)
  111. """
  112.       },
  113.       "phase_execution" : {
  114.         "policy" : "ntopng",
  115.         "phase_definition" : {
  116.           "min_age" : "0ms",
  117.           "actions" : {
  118.             "rollover" : {
  119.               "max_age" : "1d"
  120.             },
  121.             "set_priority" : {
  122.               "priority" : 100
  123.             }
  124.           }
  125.         }

第一个索引错误为“索引名[ntopng-2020.09.09]与模式'^.*-\d+$不匹配”
第二个显示：“index.lifecycle.rollover\u alias[ntopng\u alias]未指向索引[ntopng-2020.09.10]”
请注意，我正在学习es索引管理的基础知识，因此我非常感谢您提供任何有关问题的线索。

elasticsearchkibanaelastic-stackelk

1条答案

按热度按时间
tf7tbtn2

tf7tbtn21#

好的，我刚刚发现索引名必须以0001这样的数字模式结束，而不是2020.09.09，所以我可能需要找到一种替代方法来实现它。

赞(0）分享  回复(0）举报  2021-06-14
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com