健康的Elasticsearch集群在打开关闭的索引后变为红色

jecbmhm3  于 2023-06-29  发布在  ElasticSearch
关注(0)|答案(1)|浏览(162)

我有一个托管集群,由elastio.co托管。下面是配置|Platform => Amazon Web Services||Memory => 4 GB||Storage => 96 GB||SSD => Yes||High availability => Yes 2 data centers|
此群集中的每个索引都包含正好一天的日志数据。平均索引大小为15 mb,平均文档数为15000。集群没有任何压力(JVM、索引和搜索时间、磁盘空间都在非常舒适的区域)
当我打开一个以前关闭的索引时,簇变成了红色。下面是我在查询elasticsearch时发现的一些矩阵。

GET /_cluster/allocation/explain
{
  "index": "some_index_name",    # 1 Primary shard , 1 replica shard 
  "shard": 0,
  "primary": true
}

回复:

"unassigned_info": {
"reason": "ALLOCATION_FAILED"
"failed_allocation_attempts": 3,
"details": "failed recovery, failure RecoveryFailedException[[some_index_name][0]: Recovery failed on {instance-*****}{Hash}{HASH}{IP}{IP}{logical_availability_zone=zone-1, availability_zone=***, region=***}]; nested: IndexShardRecoveryException[failed to fetch index version after copying it over]; nested: IndexShardRecoveryException[shard allocated for local recovery (post api), should exist, but doesn't, current files: []]; nested: IndexNotFoundException[no segments* file found in store(mmapfs(/app/data/nodes/0/indices/MFIFAQO2R_ywstzqrfbY4w/0/index)): files: []]; ",
"last_allocation_status": "no_valid_shard_copy"
}, 
"can_allocate": "no_valid_shard_copy",
"allocate_explanation": "cannot allocate because all found copies of the shard are either stale or corrupt",
"node_allocation_decisions": [
  {
    "node_name": "instance-***",
    "node_decision": "no",
    "store": {
      "in_sync": false,
      "allocation_id": "RANDOM_HASH",
      "store_exception": {
        "type": "index_not_found_exception",
        "reason": "no segments* file found in SimpleFSDirectory@/app/data/nodes/0/indices/RANDOM_HASH/0/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@346e1b99: files: []"
      }
  }
},
{
  "node_name": "instance-***",
  "node_attributes": {
    "logical_availability_zone": "zone-0",
  },
  "node_decision": "no",
  "store": {
    "found": false
  }
}

我试过把碎片重新路由到一个节点。即使设置数据丢失标志为真。

POST _cluster/reroute
{
  "commands" : [
  {"allocate_stale_primary" : {
  "index" : "some_index_name", "shard" : 0,
  "node" : "instance-***",
  "accept_data_loss" : true
    }
  }
  ]
}

回复:

"acknowledged": true,
"state": {
"version": 338190,
"state_uuid": "RANDOM_HASH",
"master_node": "RANDOM_HASH",
"blocks": {
  "indices": {
    "restored_**: {
      "4": {
        "description": "index closed",
        "retryable": false,
        "levels": [
          "read",
          "write"
        ]
      }
    },
    "restored_**": {
      "4": {
        "description": "index closed",
        "retryable": false,
        "levels": [
          "read",
          "write"
        ]
      }
    }
  }
},
"routing_table": {
  "indices": {
    "SOME_INDEX_NAME": {
      "shards": {
        "0": [
          {
            "state": "INITIALIZING",
            "primary": true,
            "relocating_node": null,
            "shard": 0,
            "index": "SOME_INDEX_NAME",
            "recovery_source": {
              "type": "EXISTING_STORE"
            },
            "allocation_id": {
              "id": "HASH"
            },
            "unassigned_info": {
              "reason": "ALLOCATION_FAILED",
              "failed_attempts": 4,
              "delayed": false,
              "details": "same as explanation above ^ ",
              "allocation_status": "no_valid_shard_copy"
            }
          },
          {
            "state": "UNASSIGNED",
            "primary": false,
            "node": null,
            "relocating_node": null,
            "shard": 0,
            "index": "some_index_name",
            "recovery_source": {
              "type": "PEER"
            },
            "unassigned_info": {
              "reason": "INDEX_REOPENED",
              "delayed": false,
              "allocation_status": "no_attempt"
            }
          }
        ]
      }
    },

欢迎任何建议。感谢和问候。

im9ewurl

im9ewurl1#

这在主节点突然关闭时发生。
以下是我为解决我遇到的相同问题而采取的步骤,

  • 步骤1:检查分配
  • curl -XGET http://localhost:9200/_cat/allocation?v
  • 步骤2:检查分片存储
  • curl -XGET http://localhost:9200/_shard_stores?pretty请注意“index”、“shard”和“node”是否出现了您显示的错误。错误应该是-->“no segments* file found in SimpleFSDirectory@/....”
  • 步骤3:现在重新路由索引,如下所示
  • curl -XPOST 'http://localhost:9200/_cluster/reroute?master_timeout=5m' -H'Content-Type:application/json' -d '{“commands”:[ {“allocate_empty_primary”:{“index”:“IndexFromStep2”,“shard”:ShardFromStep 2,“节点”:“NodeFromStep2”,“accept_data_loss”:联系我们
  • 步骤4:重复步骤2和步骤3,直到看到此输出。
  • curl -XGET 'http://localhost:9200/_shard_stores?漂亮的

{“indices”:关于我们
您的群集应该很快会变为绿色。

相关问题