elastic search edge ngram未返回所有预期结果

uajslkp6  于 2021-06-10  发布在  ElasticSearch
关注(0)|答案(3)|浏览(425)

我很难找到ElasticSearch查询意外的结果。将下列文档编入ElasticSearch。

{
"group": "J00-I99", codes: [
   { "id": "J15", "description": "hello world" },
   { "id": "J15.0", "description": "test one world" },
   { "id": "J15.1", "description": "test two world J15.0" },
   { "id": "J15.2", "description": "test two three world J15" },
   { "id": "J15.3", "description": "hello world J18 " },
    ............................ // Similar records here
   { "id": "J15.9", "description": "hello world new" },
   { "id": "J16.0", "description": "new description" }
]
}

这里我的目标是实现自动完成功能,为此我使用了n-gram方法。我不想使用完全的暗示方法。
目前我有两个问题:
搜索查询(id和描述字段):j15
预期结果:包括j15在内的所有上述结果实际结果:得到的结果很少(j15.0、j15.1、j15.8)
搜索查询(id和description字段):测试2
预期结果:

{ "id": "J15.1", "description": "test two world J15.0" },
{ "id": "J15.2", "description": "test two three world J15" },

实际结果:

{ "id": "J15.0", "description": "test one world" },
   { "id": "J15.1", "description": "test two world J15.0" },
   { "id": "J15.2", "description": "test two three world J15" },

然后像这样Map。

{

                settings: {
                    number_of_shards: 1,
                    analysis: {
                        filter: {
                            ngram_filter: {
                                type: 'edge_ngram',
                                min_gram: 2,
                                max_gram: 20
                            }
                        },
                        analyzer: {
                            ngram_analyzer: {
                                type: 'custom',
                                tokenizer: 'standard',
                                filter: [
                                    'lowercase', 'ngram_filter'
                                ]
                            }
                        }
                    }
                },
                mappings: {
                    properties: {
                        group: {
                            type: 'text'
                        },
                        codes: {
                            type: 'nested',
                            properties: {
                                id: {
                                    type: 'text',
                                    analyzer: 'ngram_analyzer',
                                    search_analyzer: 'standard'
                                },
                                description: {
                                    type: 'text',
                                    analyzer: 'ngram_analyzer',
                                    search_analyzer: 'standard'
                                }
                            }
                        }
                    }
                }
            }

搜索查询:

GET myindex/_search
{
  "_source": {
    "excludes": [
      "codes"
    ]
  },
  "query": {
    "nested": {
      "path": "codes",
      "query": {
        "bool": {
          "should": [
            {
              "match": {
                "codes.description": "J15"
              }
            },
            {
              "match": {
                "codes.id": "J15"
              }
            }
          ]
        }
      },
      "inner_hits": {}
    }
  }
}

注意:文档索引的大小会很大。这里只提到样本数据。
对于第二个问题,我可以使用multi\u match with and操作符吗?

GET myindex/_search
{
  "_source": {
    "excludes": [
      "codes"
    ]
  },
  "query": {
    "nested": {
      "path": "codes",
      "query": {
        "bool": {
          "should": [
            {
              "multi_match": {
                    "query": "J15",
                    "fields": ["codes.id", "codes.description"],
                    "operator": and
                }
            }
          ]
        }
      },
      "inner_hits": {}
    }
  }
}

任何帮助将非常感谢,因为我有困难的时间来解决这个问题。

arknldoa

arknldoa1#

添加索引Map、搜索查询和搜索结果的工作示例
索引Map:

{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "tokenizer": "my_tokenizer"
        }
      },
      "tokenizer": {
        "my_tokenizer": {
          "type": "edge_ngram",
          "min_gram": 2,
          "max_gram": 20,
          "token_chars": [
            "letter",
            "digit"
          ]
        }
      }
    },
    "max_ngram_diff": 50
  },
  "mappings": {
    "properties": {
      "group": {
        "type": "text"
      },
      "codes": {
        "type": "nested",
        "properties": {
          "id": {
            "type": "text",
            "analyzer": "my_analyzer"
          }
        }
      }
    }
  }
}

索引数据:

{
    "group": "J00-I99", 
    "codes": [
        {
            "id": "J15",
            "description": "hello world"
        },
        {
            "id": "J15.0",
            "description": "test one world"
        },
        {
            "id": "J15.1",
            "description": "test two world J15.0"
        },
        {
            "id": "J15.2",
            "description": "test two three world J15"
        },
        {
            "id": "J15.3",
            "description": "hello world J18 "
        },
        {
            "id": "J15.9",
            "description": "hello world new"
        },
        {
            "id": "J16.0",
            "description": "new description"
        }
    ]
}

搜索查询:

{
    "_source": {
        "excludes": [
            "codes"
        ]
    },
    "query": {
        "nested": {
            "path": "codes",
            "query": {
                "bool": {
                    "should": [
                        {
                            "match": {
                                "codes.description": "J15"
                            }
                        },
                        {
                            "match": {
                                "codes.id": "J15"
                            }
                        }
                    ],
                    "must": {
                        "multi_match": {
                            "query": "test two",
                            "fields": [
                                "codes.id",
                                "codes.description"
                            ],
                            "type": "phrase"
                        }
                    }
                }
            },
            "inner_hits": {}
        }
    }
}

搜索结果:

"inner_hits": {
          "codes": {
            "hits": {
              "total": {
                "value": 2,
                "relation": "eq"
              },
              "max_score": 3.2227304,
              "hits": [
                {
                  "_index": "stof_64170045",
                  "_type": "_doc",
                  "_id": "1",
                  "_nested": {
                    "field": "codes",
                    "offset": 3
                  },
                  "_score": 3.2227304,
                  "_source": {
                    "id": "J15.2",
                    "description": "test two three world J15"
                  }
                },
                {
                  "_index": "stof_64170045",
                  "_type": "_doc",
                  "_id": "1",
                  "_nested": {
                    "field": "codes",
                    "offset": 2
                  },
                  "_score": 2.0622847,
                  "_source": {
                    "id": "J15.1",
                    "description": "test two world J15.0"
                  }
                }
              ]
            }
          }
        }
      }
xzlaal3s

xzlaal3s2#

添加另一个答案,因为这是一个不同的问题和第一个答案是集中在第一个问题。
问题是你的第二个问题 test two 退货 test one world 以及在索引时使用 ngram_analyzer 它使用的是标准的分析器,它将文本拆分为空格,而您的搜索分析器也是 standard 因此,如果在索引文档和搜索词上使用analyze api,您将看到它与标记匹配:

{
   "text" : "test one world",
   "analyzer" : "standard"
}

和生成的代币

{
    "tokens": [
        {
            "token": "test",
            "start_offset": 0,
            "end_offset": 4,
            "type": "<ALPHANUM>",
            "position": 0
        },
        {
            "token": "one",
            "start_offset": 5,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 1
        },
        {
            "token": "world",
            "start_offset": 9,
            "end_offset": 14,
            "type": "<ALPHANUM>",
            "position": 2
        }
    ]
}

你的搜索词呢 test two ```
{
"tokens": [
{
"token": "test",
"start_offset": 0,
"end_offset": 4,
"type": "",
"position": 0
},
{
"token": "two",
"start_offset": 5,
"end_offset": 8,
"type": "",
"position": 1
}
]
}

如你所见 `test` 令牌存在于您的文档中,因此您将获得该搜索结果。在查询中使用and运算符可以解决这个问题,如下所示
搜索查询

{
"_source": {
"excludes": [
"codes"
]
},
"query": {
"nested": {
"path": "codes",
"query": {
"bool": {
"must": {
"multi_match": {
"query": "test two",
"fields": [
"codes.id",
"codes.description"
],
"operator" :"AND"
}
}
}
},
"inner_hits": {}
}
}
}

和搜索结果

"hits": [
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 2
},
"_score": 2.6901608,
"_source": {
"id": "J15.1",
"description": "test two world J15.0"
}
},
{
"_index": "myindexedge64170045",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "codes",
"offset": 3
},
"_score": 2.561376,
"_source": {
"id": "J15.2",
"description": "test two three world J15"
}
}
]
}
}
}
}

falq053o

falq053o3#

问题是默认情况下 inner_hits 只返回3个匹配的文档,如官方文档中所述,
大小
每次内部点击返回的最大点击数。默认情况下,返回前三个匹配的命中。
只需添加 size 参数在你的内部点击得到所有的搜索结果。

"inner_hits": {
                "size": 10 // note this
            }

在您的示例数据上尝试了此操作,并查看了第一个查询的搜索结果,该查询仅返回3个搜索结果
第一个查询搜索结果

"hits": [
                                {
                                    "_index": "myindexedge64170045",
                                    "_type": "_doc",
                                    "_id": "1",
                                    "_nested": {
                                        "field": "codes",
                                        "offset": 2
                                    },
                                    "_score": 1.8687118,
                                    "_source": {
                                        "id": "J15.1",
                                        "description": "test two world J15.0"
                                    }
                                },
                                {
                                    "_index": "myindexedge64170045",
                                    "_type": "_doc",
                                    "_id": "1",
                                    "_nested": {
                                        "field": "codes",
                                        "offset": 3
                                    },
                                    "_score": 1.7934312,
                                    "_source": {
                                        "id": "J15.2",
                                        "description": "test two three world J15"
                                    }
                                },
                                {
                                    "_index": "myindexedge64170045",
                                    "_type": "_doc",
                                    "_id": "1",
                                    "_nested": {
                                        "field": "codes",
                                        "offset": 0
                                    },
                                    "_score": 0.29618382,
                                    "_source": {
                                        "id": "J15",
                                        "description": "hello world"
                                    }
                                },
                                {
                                    "_index": "myindexedge64170045",
                                    "_type": "_doc",
                                    "_id": "1",
                                    "_nested": {
                                        "field": "codes",
                                        "offset": 1
                                    },
                                    "_score": 0.29618382,
                                    "_source": {
                                        "id": "J15.0",
                                        "description": "test one world"
                                    }
                                },
                                {
                                    "_index": "myindexedge64170045",
                                    "_type": "_doc",
                                    "_id": "1",
                                    "_nested": {
                                        "field": "codes",
                                        "offset": 4
                                    },
                                    "_score": 0.29618382,
                                    "_source": {
                                        "id": "J15.3",
                                        "description": "hello world J18 "
                                    }
                                },
                                {
                                    "_index": "myindexedge64170045",
                                    "_type": "_doc",
                                    "_id": "1",
                                    "_nested": {
                                        "field": "codes",
                                        "offset": 5
                                    },
                                    "_score": 0.29618382,
                                    "_source": {
                                        "id": "J15.9",
                                        "description": "hello world new"
                                    }
                                }
                            ]
                        }
                    }
                }
            }

相关问题