org.apache.lucene.util.fst.Util.toUTF16()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(6.4k)|赞(0)|评价(0)|浏览(168)

本文整理了Java中org.apache.lucene.util.fst.Util.toUTF16()方法的一些代码示例，展示了Util.toUTF16()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Util.toUTF16()方法的具体详情如下：
包路径：org.apache.lucene.util.fst.Util
类名称：Util
方法名：toUTF16

Util.toUTF16介绍

[英]Just maps each UTF16 unit (char) to the ints in an IntsRef.
[中]只需将每个UTF16单元（字符）映射到IntsRef中的INT。

代码示例

代码示例来源：origin: org.apache.lucene/lucene-analyzers-common

  1. private FST<CharsRef> parseConversions(LineNumberReader reader, int num) throws IOException, ParseException {
  2.  Map<String,String> mappings = new TreeMap<>();
  3.  
  4.  for (int i = 0; i < num; i++) {
  5.   String line = reader.readLine();
  6.   String parts[] = line.split("\\s+");
  7.   if (parts.length != 3) {
  8.    throw new ParseException("invalid syntax: " + line, reader.getLineNumber());
  9.   }
  10.   if (mappings.put(parts[1], parts[2]) != null) {
  11.    throw new IllegalStateException("duplicate mapping specified for: " + parts[1]);
  12.   }
  13.  }
  14.  
  15.  Outputs<CharsRef> outputs = CharSequenceOutputs.getSingleton();
  16.  Builder<CharsRef> builder = new Builder<>(FST.INPUT_TYPE.BYTE2, outputs);
  17.  IntsRefBuilder scratchInts = new IntsRefBuilder();
  18.  for (Map.Entry<String,String> entry : mappings.entrySet()) {
  19.   Util.toUTF16(entry.getKey(), scratchInts);
  20.   builder.add(scratchInts.get(), new CharsRef(entry.getValue()));
  21.  }
  22.  
  23.  return builder.finish();
  24. }

代码示例来源：origin: org.apache.lucene/lucene-analyzers-common

  1. /** Builds the NormalizeCharMap; call this once you
  2.   *  are done calling {@link #add}. */
  3.  public NormalizeCharMap build() {
  4.   final FST<CharsRef> map;
  5.   try {
  6.    final Outputs<CharsRef> outputs = CharSequenceOutputs.getSingleton();
  7.    final org.apache.lucene.util.fst.Builder<CharsRef> builder = new org.apache.lucene.util.fst.Builder<>(FST.INPUT_TYPE.BYTE2, outputs);
  8.    final IntsRefBuilder scratch = new IntsRefBuilder();
  9.    for(Map.Entry<String,String> ent : pendingPairs.entrySet()) {
  10.     builder.add(Util.toUTF16(ent.getKey(), scratch),
  11.           new CharsRef(ent.getValue()));
  12.    }
  13.    map = builder.finish();
  14.    pendingPairs.clear();
  15.   } catch (IOException ioe) {
  16.    // Bogus FST IOExceptions!!  (will never happen)
  17.    throw new RuntimeException(ioe);
  18.   }
  19.   return new NormalizeCharMap(map);
  20.  }
  21. }

代码示例来源：origin: harbby/presto-connectors

  1. private FST<CharsRef> parseConversions(LineNumberReader reader, int num) throws IOException, ParseException {
  2.  Map<String,String> mappings = new TreeMap<>();
  3.  
  4.  for (int i = 0; i < num; i++) {
  5.   String line = reader.readLine();
  6.   String parts[] = line.split("\\s+");
  7.   if (parts.length != 3) {
  8.    throw new ParseException("invalid syntax: " + line, reader.getLineNumber());
  9.   }
  10.   if (mappings.put(parts[1], parts[2]) != null) {
  11.    throw new IllegalStateException("duplicate mapping specified for: " + parts[1]);
  12.   }
  13.  }
  14.  
  15.  Outputs<CharsRef> outputs = CharSequenceOutputs.getSingleton();
  16.  Builder<CharsRef> builder = new Builder<>(FST.INPUT_TYPE.BYTE2, outputs);
  17.  IntsRefBuilder scratchInts = new IntsRefBuilder();
  18.  for (Map.Entry<String,String> entry : mappings.entrySet()) {
  19.   Util.toUTF16(entry.getKey(), scratchInts);
  20.   builder.add(scratchInts.get(), new CharsRef(entry.getValue()));
  21.  }
  22.  
  23.  return builder.finish();
  24. }

代码示例来源：origin: org.infinispan/infinispan-embedded-query

  1. private FST<CharsRef> parseConversions(LineNumberReader reader, int num) throws IOException, ParseException {
  2.  Map<String,String> mappings = new TreeMap<>();
  3.  
  4.  for (int i = 0; i < num; i++) {
  5.   String line = reader.readLine();
  6.   String parts[] = line.split("\\s+");
  7.   if (parts.length != 3) {
  8.    throw new ParseException("invalid syntax: " + line, reader.getLineNumber());
  9.   }
  10.   if (mappings.put(parts[1], parts[2]) != null) {
  11.    throw new IllegalStateException("duplicate mapping specified for: " + parts[1]);
  12.   }
  13.  }
  14.  
  15.  Outputs<CharsRef> outputs = CharSequenceOutputs.getSingleton();
  16.  Builder<CharsRef> builder = new Builder<>(FST.INPUT_TYPE.BYTE2, outputs);
  17.  IntsRefBuilder scratchInts = new IntsRefBuilder();
  18.  for (Map.Entry<String,String> entry : mappings.entrySet()) {
  19.   Util.toUTF16(entry.getKey(), scratchInts);
  20.   builder.add(scratchInts.get(), new CharsRef(entry.getValue()));
  21.  }
  22.  
  23.  return builder.finish();
  24. }

代码示例来源：origin: NationalSecurityAgency/datawave

  1. Util.toUTF16(value, irBuilder);
  2. final IntsRef ints = irBuilder.get();
  3. synchronized (this.fst) {

代码示例来源：origin: org.infinispan/infinispan-embedded-query

  1. /** Builds the NormalizeCharMap; call this once you
  2.   *  are done calling {@link #add}. */
  3.  public NormalizeCharMap build() {
  4.   final FST<CharsRef> map;
  5.   try {
  6.    final Outputs<CharsRef> outputs = CharSequenceOutputs.getSingleton();
  7.    final org.apache.lucene.util.fst.Builder<CharsRef> builder = new org.apache.lucene.util.fst.Builder<>(FST.INPUT_TYPE.BYTE2, outputs);
  8.    final IntsRefBuilder scratch = new IntsRefBuilder();
  9.    for(Map.Entry<String,String> ent : pendingPairs.entrySet()) {
  10.     builder.add(Util.toUTF16(ent.getKey(), scratch),
  11.           new CharsRef(ent.getValue()));
  12.    }
  13.    map = builder.finish();
  14.    pendingPairs.clear();
  15.   } catch (IOException ioe) {
  16.    // Bogus FST IOExceptions!!  (will never happen)
  17.    throw new RuntimeException(ioe);
  18.   }
  19.   return new NormalizeCharMap(map);
  20.  }
  21. }

代码示例来源：origin: harbby/presto-connectors

  1. /** Builds the NormalizeCharMap; call this once you
  2.   *  are done calling {@link #add}. */
  3.  public NormalizeCharMap build() {
  4.   final FST<CharsRef> map;
  5.   try {
  6.    final Outputs<CharsRef> outputs = CharSequenceOutputs.getSingleton();
  7.    final org.apache.lucene.util.fst.Builder<CharsRef> builder = new org.apache.lucene.util.fst.Builder<>(FST.INPUT_TYPE.BYTE2, outputs);
  8.    final IntsRefBuilder scratch = new IntsRefBuilder();
  9.    for(Map.Entry<String,String> ent : pendingPairs.entrySet()) {
  10.     builder.add(Util.toUTF16(ent.getKey(), scratch),
  11.           new CharsRef(ent.getValue()));
  12.    }
  13.    map = builder.finish();
  14.    pendingPairs.clear();
  15.   } catch (IOException ioe) {
  16.    // Bogus FST IOExceptions!!  (will never happen)
  17.    throw new RuntimeException(ioe);
  18.   }
  19.   return new NormalizeCharMap(map);
  20.  }
  21. }

代码示例来源：origin: NationalSecurityAgency/datawave

  1. public static FST<?> getFST(SortedSet<String> values) throws IOException {
  2.   final IntsRefBuilder irBuilder = new IntsRefBuilder();
  3.   // The builder options with defaults
  4.   FST.INPUT_TYPE inputType = FST.INPUT_TYPE.BYTE1;
  5.   int minSuffixCount1 = 0;
  6.   int minSuffixCount2 = 0;
  7.   boolean doShareSuffix = true;
  8.   boolean doShareNonSingletonNodes = true;
  9.   int shareMaxTailLength = Integer.MAX_VALUE;
  10.   
  11.   boolean allowArrayArcs = true;
  12.   int bytesPageBits = 15;
  13.   final Outputs<Object> outputs = NoOutputs.getSingleton();
  14.   
  15.   // create the FST from the values
  16.   org.apache.lucene.util.fst.Builder<Object> fstBuilder = new org.apache.lucene.util.fst.Builder<>(inputType, minSuffixCount1, minSuffixCount2,
  17.           doShareSuffix, doShareNonSingletonNodes, shareMaxTailLength, outputs, allowArrayArcs, bytesPageBits);
  18.   
  19.   for (String value : values) {
  20.     Util.toUTF16(value, irBuilder);
  21.     final IntsRef scratchInt = irBuilder.get();
  22.     fstBuilder.add(scratchInt, outputs.getNoOutput());
  23.   }
  24.   return fstBuilder.finish();
  25. }

内容来源于网络，如有侵权，请联系作者删除！

Util

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com