elasticsearch批量插入导致writestateexception和未分配碎片

juud5qan  于 2021-06-14  发布在  ElasticSearch
关注(0)|答案(0)|浏览(245)

尝试在我的elasticsearch示例中创建约250万个文档失败,原因是: WriteStateException 我正在使用 elasticsearch-pyhelpers.bulk 函数对文档进行索引。代码如下所示:

actions = [
    {
        "_op_type": "create",
        "_index": index_name,
        "_id": _get_id(document),
        "doc": asdict(document),
        "doc_as_upsert": False,
    }
    for document in documents
]

helpers.bulk(
    client=es_session,
    actions=actions,
    refresh=_get_refresh_setting(),
    chunk_size=chunk_size,
    request_timeout=request_timeout,
)

我尝试了500到10.000之间的各种块大小,还将请求超时增加到3600(=1h)。
我使用的是es 7.8.1版本,docker容器中只有一个节点。在我开始操作之前,只有一个索引是完全空的。创建索引时使用的Map类似于:

{
    "settings": {
        "analysis": {
            "normalizer": {
                "lowercase_unaccent_normalizer": {
                    "type": "custom", "char_filter": [], "filter": ["lowercase", "asciifolding"],
                }
            }
        }
    },
    "mappings": {
        "properties": {
            "identifier": {"type": "keyword"},
            "name": {"type": "keyword"},
            "vector": {"type": "dense_vector", "dims": HIDDEN_CELLS},
        }
    }
}

最后,elasticsearch日志如下:

{
    "type": "server",
    "timestamp": "2020-10-28T06:39:38,320Z",
    "level": "WARN",
    "component": "o.e.i.c.IndicesClusterStateService",
    "cluster.name": "docker-cluster",
    "node.name": "elasticsearch-cb6858695-9qdh2",
    "message": "[.tasks][0] marking and sending shard failed due to [failed to create shard]",
    "cluster.uuid": "8-BFXGMcSSaBcUAvqyefbg",
    "node.id": "EplobYNeTK213O8soSVIbQ",
    "stacktrace": [
        "org.elasticsearch.gateway.WriteStateException: failed to rename tmp file to final name in the first state location /usr/share/elasticsearch/data/nodes/0/indices/8TyPjAZnRM2jInEXBmWghA/0/state-0.st.tmp",
        "at org.elasticsearch.gateway.MetadataStateFormat.performRenames(MetadataStateFormat.java:155) ~[elasticsearch-7.8.1.jar:7.8.1]",
        "at org.elasticsearch.gateway.MetadataStateFormat.write(MetadataStateFormat.java:245) ~[elasticsearch-7.8.1.jar:7.8.1]",
        "at org.elasticsearch.gateway.MetadataStateFormat.writeAndCleanup(MetadataStateFormat.java:185) ~[elasticsearch-7.8.1.jar:7.8.1]",
        "at org.elasticsearch.index.shard.IndexShard.persistMetadata(IndexShard.java:2730) ~[elasticsearch-7.8.1.jar:7.8.1]",
        "at org.elasticsearch.index.shard.IndexShard.<init>(IndexShard.java:372) ~[elasticsearch-7.8.1.jar:7.8.1]",
        ...

        "Caused by: java.nio.file.FileSystemException: /usr/share/elasticsearch/data/nodes/0/indices/8TyPjAZnRM2jInEXBmWghA/0/_state/state-0.st.tmp -> /usr/share/elasticsearch/data/nodes/0/indices/8TyPjAZnRM2jInEXBmWghA/0/_state/state-0.st: Interrupted system call",
        "at sun.nio.fs.UnixException.translateToIOException(UnixException.java:100) ~[?:?]",
        "at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]",
        "at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:415) ~[?:?]",
        "at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267) ~[?:?]",
...

现在,每当我尝试查询elasticsearch时,都会遇到以下异常:

> curl localhost:9200/my_index/_search
> {"error":{"root_cause":[],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[]},"status":503}

此外,检查群集运行状况会显示红色状态:

> {"cluster_name":"docker-cluster","status":"red","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":2,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":3,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":40.0}

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题