lucene 根据结果返回的文档获取ElasticSearch插件中的字段值

9wbgstp7  于 2022-11-23  发布在  Lucene
关注(0)|答案(1)|浏览(217)

我的要求是从基于模糊匹配的ElasticSearch中搜索文档,然后通过比较文档的值和输入字符串对文档进行“rescore”,例如,如果查询返回3个文档(doc:1,2,3),那么为了比较常量值“星星Wars”,比较结果应该是:

doc:1, MovieName:"Star Wars" (compare ('Star Wars','Star Wars'))
doc:2, MovieName:"Starr Warz" (compare ('Star Wars','Starr Warz'))
doc:3, MovieName:"The Star Wars" (compare ('Star Wars','The Star Wars'))

我找到了下面的elasticsearch rescore插件示例,并实现了它来实现上面的功能。
我可以在插件中传递和访问输入“星星大战”,但是我在获取结果(topdocs)中返回的文档的MovieName字段的值时遇到了麻烦。
我的查询:

GET movie-idx/_search?
    {
      "query": {
        "bool": {
          "must": [
            {
              "query_string": {
                "fields": [
                  "MovieName"
                ],
                "query": "Star Wars",
                "minimum_should_match": "61%",
                "fuzziness": 1,
                "_name": "fuzzy"
              }
            }
          ]
        }
      },
      "rescore": {
        "calculateMovieScore": {
          "MovieName": "Star Wars"
        }
      }
    }

我的rescorer类看起来像:

private static class DocsRescorer implements Rescorer {
        private static final DocsRescorer INSTANCE = new DocsRescorer();

        @Override
        public TopDocs rescore(TopDocs topDocs, IndexSearcher searcher, RescoreContext rescoreContext) throws IOException {
            DocRescoreContext context = (DocRescoreContext) rescoreContext;
            int end = Math.min(topDocs.scoreDocs.length, rescoreContext.getWindowSize());

            MovieScorer MovieScorer = new MovieScorerBuilder()
                    .withInputName(context.MovieName)
                    .build();

            for (int i = 0; i < end; i++) {
                String name = <get MovieName values from actual document returned by topdocs>
                float score = MovieScorer.calculateScore(name);
                topDocs.scoreDocs[i].score = score;
            }

            List<ScoreDoc> scoreDocList =  Stream.of(topDocs.scoreDocs).filter((a) -> a.score >= context.threshold).sorted(
                    (a, b) -> {
                        if (a.score > b.score) {
                            return -1;
                        }
                        if (a.score < b.score) {
                            return 1;
                        }
                        // Safe because doc ids >= 0
                        return a.doc - b.doc;
                    }
            ).collect(Collectors.toList());
            ScoreDoc[] scoreDocs = scoreDocList.toArray(new ScoreDoc[scoreDocList.size()]);
            topDocs.scoreDocs = scoreDocs;
            return topDocs;
        }

        @Override
        public Explanation explain(int topLevelDocId, IndexSearcher searcher, RescoreContext rescoreContext,
                                   Explanation sourceExplanation) throws IOException {
            DocRescoreContext context = (DocRescoreContext) rescoreContext;
            // Note that this is inaccurate because it ignores factor field
            return Explanation.match(context.factor, "test", singletonList(sourceExplanation));
        }

        @Override
        public void extractTerms(IndexSearcher searcher, RescoreContext rescoreContext, Set<Term> termsSet) {
            // Since we don't use queries there are no terms to extract.
        }
    }

我的理解是,插件代码将执行一次,它将从初始查询(本例中的模糊搜索)和for(int i = 0; i〈end; i++)将遍历结果中返回的每个文档。我需要帮助的地方是:

String name = <get MovieName value from actual document returned by topdocs>
uqcuzwp8

uqcuzwp81#

我知道这已经超过2年了,但我遇到了同样的问题,并找到了一个解决方案,所以我把它张贴在这里。这是为ES 7.8.0中的Rescorer插件完成的。我使用的基本示例是分组插件Link
这是一堆我不完全理解的代码,但主要的原理是您需要一个您想要获取的字段的IFD(IndexFieldData〈?〉)示例。在我的示例中,我只需要hits的_id。它看起来像这样:
1.提前准备IFD并将其传递给RescoreContext:将一个成员添加到扩展RescoreContext的类中,以将此IFD保留在上下文中,我们将其称为"idField"(稍后在第3节中使用)。

@Override
public RescoreContext innerBuildContext(int windowSize, QueryShardContext queryShardContext) throws IOException {
    return new MyRescoreContext(windowSize, queryShardContext.getForField(queryShardContext.fieldMapper("_id")));
}

1.接下来,在Rescorer本身中:(方法rescore(...))
2.1)首先按scoreDoc.doc排序

ScoreDoc[] hits = topDocs.scoreDocs; 
 Arrays.sort(hits, Comparator.comparingInt((d) -> d.doc));

2.2)执行黑色魔术(代码我不明白)

List<LeafReaderContext> readerContexts = searcher.getIndexReader().leaves();
int currentReaderIx = -1;
int currentReaderEndDoc = 0;
LeafReaderContext currentReaderContext = null;
    
for (int i = 0; i < end; i++) {
ScoreDoc hit = hits[i];
    
    // find segment that contains current document 
   while (hit.doc >= currentReaderEndDoc) {  
      currentReaderIx++;
      currentReaderContext = readerContexts.get(currentReaderIx);
      currentReaderEndDoc = currentReaderContext.docBase + currentReaderContext.reader().maxDoc();
   }

   int docId = hit.doc - currentReaderContext.docBase;

   // code from section 3 goes here //
}

1.现在,有了这个神奇的"docId",您可以从For循环内的IFD中获取:

SortedBinaryDocValues values = rescoreContext.idField.load(currentReaderContext).getBytesValues();
 values.advanceExact(docId);
 String id = values.nextValue().utf8ToString();

在您的示例中,获取所需字段的IFD(而不是_id字段),并在For循环中从docId-〉string值创建一个Hashmap。然后在应用分数的同一个For循环中使用此Map。
希望这对每个人都有帮助!这种技术根本没有文档记录,任何地方都没有解释!

相关问题