org.apache.lucene.analysis.Token.setPayload()方法的使用及代码示例

x33g5p2x  于2022-01-30 转载在 其他  
字(8.8k)|赞(0)|评价(0)|浏览(149)

本文整理了Java中org.apache.lucene.analysis.Token.setPayload()方法的一些代码示例，展示了Token.setPayload()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Token.setPayload()方法的具体详情如下：
包路径：org.apache.lucene.analysis.Token
类名称：Token
方法名：setPayload

Token.setPayload介绍

[英]Sets this Token's payload.
[中]设置此令牌的有效负载。

代码示例

代码示例来源：origin: org.compass-project/compass

public Token next() throws IOException {
  // we put the payload on the last token. It has already been indexed
  // and it will be used on the all property later on
  if (lastToken != null && payload != null) {
    lastToken.setPayload(payload);
  }
  lastToken = tokenStream.next();
  if (lastToken != null) {
    tokens.add(lastToken);
  }
  return lastToken;
}

代码示例来源：origin: org.compass-project/compass

public void close() throws IOException {
    if (lastToken != null && payload != null) {
      lastToken.setPayload(payload);
    }
    tokenStream.close();
  }
}

代码示例来源：origin: org.apache.lucene/lucene-analyzers

/**
 * Stores a 32 bit float in the payload, or set it to null if 1f;
 * @param token
 * @param weight
 */
@Override
public void setWeight(Token token, float weight) {
 if (weight == 1f) {
  token.setPayload(null);
 } else {
  token.setPayload(new Payload(PayloadHelper.encodeFloat(weight)));
 }
}

代码示例来源：origin: org.apache.lucene/com.springsource.org.apache.lucene

public Object clone() {
 try {
  Token t = (Token)super.clone();
  // Do a deep clone
  if (termBuffer != null) {
   t.termBuffer = (char[]) termBuffer.clone();
  }
  if (payload != null) {
   t.setPayload((Payload) payload.clone());
  }
  return t;
 } catch (CloneNotSupportedException e) {
  throw new RuntimeException(e);  // shouldn't happen
 }
}

代码示例来源：origin: org.apache.lucene/lucene-core-jfrog

public Object clone() {
 try {
  Token t = (Token)super.clone();
  // Do a deep clone
  if (termBuffer != null) {
   t.termBuffer = (char[]) termBuffer.clone();
  }
  if (payload != null) {
   t.setPayload((Payload) payload.clone());
  }
  return t;
 } catch (CloneNotSupportedException e) {
  throw new RuntimeException(e);  // shouldn't happen
 }
}

代码示例来源：origin: org.apache.lucene/lucene-core-jfrog

/** Returns the next token in the stream, or null at EOS.
 *  @deprecated The returned Token is a "full private copy" (not
 *  re-used across calls to next()) but will be slower
 *  than calling {@link #next(Token)} instead.. */
public Token next() throws IOException {
 final Token reusableToken = new Token();
 Token nextToken = next(reusableToken);
 if (nextToken != null) {
  Payload p = nextToken.getPayload();
  if (p != null) {
   nextToken.setPayload((Payload) p.clone());
  }
 }
 return nextToken;
}

代码示例来源：origin: org.apache.lucene/com.springsource.org.apache.lucene

/** Returns the next token in the stream, or null at EOS.
 *  @deprecated The returned Token is a "full private copy" (not
 *  re-used across calls to next()) but will be slower
 *  than calling {@link #next(Token)} instead.. */
public Token next() throws IOException {
 final Token reusableToken = new Token();
 Token nextToken = next(reusableToken);
 if (nextToken != null) {
  Payload p = nextToken.getPayload();
  if (p != null) {
   nextToken.setPayload((Payload) p.clone());
  }
 }
 return nextToken;
}

代码示例来源：origin: org.compass-project/compass

public void updateMapping(InternalResource resource, ResourcePropertyMapping resourcePropertyMapping) {
  if (lastToken != null && payload != null) {
    lastToken.setPayload(payload);
    lastToken = null;
  }
  if (boostSupport) {
    if (resourcePropertyMapping != null && resourcePropertyMapping.getBoost() != 1.0f) {
      payload = AllBoostUtils.writeFloat(resourcePropertyMapping.getBoost());
    } else if (resource.getBoost() != 1.0f) {
      // we get the boost from the resource thus taking into account any resource property mapping
      // and/or resource mapping boost level
      payload = AllBoostUtils.writeFloat(resource.getBoost());
    } else {
      payload = null;
    }
  }
}

代码示例来源：origin: org.apache.lucene/lucene-analyzers

private Token getNextSuffixInputToken(Token token) throws IOException {
 if (!suffix.incrementToken()) return null;
 token.copyBuffer(termAtt.buffer(), 0, termAtt.length());
 token.setPositionIncrement(posIncrAtt.getPositionIncrement());
 token.setFlags(flagsAtt.getFlags());
 token.setOffset(offsetAtt.startOffset(), offsetAtt.endOffset());
 token.setType(typeAtt.type());
 token.setPayload(payloadAtt.getPayload());
 return token;
}

代码示例来源：origin: org.apache.lucene/lucene-analyzers

private Token getNextInputToken(Token token) throws IOException {
 if (!input.incrementToken()) return null;
 token.copyBuffer(in_termAtt.buffer(), 0, in_termAtt.length());
 token.setPositionIncrement(in_posIncrAtt.getPositionIncrement());
 token.setFlags(in_flagsAtt.getFlags());
 token.setOffset(in_offsetAtt.startOffset(), in_offsetAtt.endOffset());
 token.setType(in_typeAtt.type());
 token.setPayload(in_payloadAtt.getPayload());
 return token;
}

代码示例来源：origin: org.infinispan/infinispan-embedded-query

private Token getNextSuffixInputToken(Token token) throws IOException {
 if (!suffix.incrementToken()) return null;
 token.copyBuffer(termAtt.buffer(), 0, termAtt.length());
 token.setPositionIncrement(posIncrAtt.getPositionIncrement());
 token.setFlags(flagsAtt.getFlags());
 token.setOffset(offsetAtt.startOffset(), offsetAtt.endOffset());
 token.setType(typeAtt.type());
 token.setPayload(payloadAtt.getPayload());
 return token;
}

代码示例来源：origin: org.apache.lucene/lucene-analyzers

private Token getNextPrefixInputToken(Token token) throws IOException {
 if (!prefix.incrementToken()) return null;
 token.copyBuffer(p_termAtt.buffer(), 0, p_termAtt.length());
 token.setPositionIncrement(p_posIncrAtt.getPositionIncrement());
 token.setFlags(p_flagsAtt.getFlags());
 token.setOffset(p_offsetAtt.startOffset(), p_offsetAtt.endOffset());
 token.setType(p_typeAtt.type());
 token.setPayload(p_payloadAtt.getPayload());
 return token;
}

代码示例来源：origin: org.infinispan/infinispan-embedded-query

private Token getNextPrefixInputToken(Token token) throws IOException {
 if (!prefix.incrementToken()) return null;
 token.copyBuffer(p_termAtt.buffer(), 0, p_termAtt.length());
 token.setPositionIncrement(p_posIncrAtt.getPositionIncrement());
 token.setFlags(p_flagsAtt.getFlags());
 token.setOffset(p_offsetAtt.startOffset(), p_offsetAtt.endOffset());
 token.setType(p_typeAtt.type());
 token.setPayload(p_payloadAtt.getPayload());
 return token;
}

代码示例来源：origin: org.apache.lucene/lucene-analyzers

@Override
public final boolean incrementToken() throws IOException {
 if (!prefixExhausted) {
  Token nextToken = getNextPrefixInputToken(reusableToken);
  if (nextToken == null) {
   prefixExhausted = true;
  } else {
   previousPrefixToken.reinit(nextToken);
   // Make it a deep copy
   Payload p = previousPrefixToken.getPayload();
   if (p != null) {
    previousPrefixToken.setPayload((Payload) p.clone());
   }
   setCurrentToken(nextToken);
   return true;
  }
 }
 Token nextToken = getNextSuffixInputToken(reusableToken);
 if (nextToken == null) {
  return false;
 }
 nextToken = updateSuffixToken(nextToken, previousPrefixToken);
 setCurrentToken(nextToken);
 return true;
}

代码示例来源：origin: org.infinispan/infinispan-embedded-query

@Override
public final boolean incrementToken() throws IOException {
 if (!prefixExhausted) {
  Token nextToken = getNextPrefixInputToken(reusableToken);
  if (nextToken == null) {
   prefixExhausted = true;
  } else {
   previousPrefixToken.reinit(nextToken);
   // Make it a deep copy
   BytesRef p = previousPrefixToken.getPayload();
   if (p != null) {
    previousPrefixToken.setPayload(p.clone());
   }
   setCurrentToken(nextToken);
   return true;
  }
 }
 Token nextToken = getNextSuffixInputToken(reusableToken);
 if (nextToken == null) {
  return false;
 }
 nextToken = updateSuffixToken(nextToken, previousPrefixToken);
 setCurrentToken(nextToken);
 return true;
}

代码示例来源：origin: org.apache.lucene/lucene-analyzers

private Token getNextToken(Token token) throws IOException {
 if (!this.incrementToken()) return null;
 token.copyBuffer(termAtt.buffer(), 0, termAtt.length());
 token.setPositionIncrement(posIncrAtt.getPositionIncrement());
 token.setFlags(flagsAtt.getFlags());
 token.setOffset(offsetAtt.startOffset(), offsetAtt.endOffset());
 token.setType(typeAtt.type());
 token.setPayload(payloadAtt.getPayload());
 return token;
}

代码示例来源：origin: DiceTechJobs/SolrPlugins

private Collection<Token> getTokens(String q, Analyzer analyzer) throws IOException {
  Collection<Token> result = new ArrayList<Token>();
  assert analyzer != null;
  TokenStream ts = analyzer.tokenStream("", q);
  try {
    ts.reset();
    // TODO: support custom attributes
    CharTermAttribute termAtt = ts.addAttribute(CharTermAttribute.class);
    OffsetAttribute offsetAtt = ts.addAttribute(OffsetAttribute.class);
    TypeAttribute typeAtt = ts.addAttribute(TypeAttribute.class);
    FlagsAttribute flagsAtt = ts.addAttribute(FlagsAttribute.class);
    PayloadAttribute payloadAtt = ts.addAttribute(PayloadAttribute.class);
    PositionIncrementAttribute posIncAtt = ts.addAttribute(PositionIncrementAttribute.class);
    while (ts.incrementToken()){
      Token token = new Token();
      token.copyBuffer(termAtt.buffer(), 0, termAtt.length());
      token.setOffset(offsetAtt.startOffset(), offsetAtt.endOffset());
      token.setType(typeAtt.type());
      token.setFlags(flagsAtt.getFlags());
      token.setPayload(payloadAtt.getPayload());
      token.setPositionIncrement(posIncAtt.getPositionIncrement());
      result.add(token);
    }
    ts.end();
    return result;
  } finally {
    IOUtils.closeWhileHandlingException(ts);
  }
}

代码示例来源：origin: org.compass-project/compass

if (value != null) {
    Token t = new Token(value, 0, value.length());
    t.setPayload(payload);
    tokens.add(t);
  t.setPayload(payload);
  tokens.add(t);
} else if (resourcePropertyMapping.getExcludeFromAll() == ExcludeFromAll.NO_ANALYZED) {
    Token token = ts.next();
    while (token != null) {
      token.setPayload(payload);
      tokens.add(token);
      token = ts.next();

内容来源于网络，如有侵权，请联系作者删除！

token

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com