org.antlr.v4.runtime.Parser.consume()方法的使用及代码示例

x33g5p2x  于2022-01-26 转载在 其他  
字(9.6k)|赞(0)|评价(0)|浏览(237)

本文整理了Java中org.antlr.v4.runtime.Parser.consume()方法的一些代码示例，展示了Parser.consume()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Parser.consume()方法的具体详情如下：
包路径：org.antlr.v4.runtime.Parser
类名称：Parser
方法名：consume

Parser.consume介绍

[英]Consume and return the #getCurrentToken.

E.g., given the following input with A being the current lookahead symbol, this function moves the cursor to B and returns A.

  1. A B 
  2. ^

If the parser is not in error recovery mode, the consumed symbol is added to the parse tree using ParserRuleContext#addChild(Token), and ParseTreeListener#visitTerminal is called on any parse listeners. If the parser is in error recovery mode, the consumed symbol is added to the parse tree using ParserRuleContext#addErrorNode(Token), and ParseTreeListener#visitErrorNode is called on any parse listeners.
[中]使用并返回#getCurrentToken。
例如，给定以下输入，A是当前的先行符号，此函数将光标移动到B并返回A。

  1. A B 
  2. ^

如果解析器未处于错误恢复模式，则使用ParserRunContext#addChild（令牌）将使用的符号添加到解析树中，并在任何解析侦听器上调用ParserTreeListener#VisiterMinal。如果解析器*处于错误恢复模式，则使用ParserRuleContext#addErrorNode（令牌）将消耗的符号添加到解析树中，并在任何解析侦听器上调用ParserTreeListener#visitErrorNode。

代码示例

代码示例来源：origin: apache/incubator-shardingsphere

  1. /**
  2.    * Get matched token by token type.
  3.    *
  4.    * @param tokenType token type
  5.    * @return matched token
  6.    * @throws RecognitionException mismatch throw exception
  7.    */
  8.   public Token getMatchedToken(final int tokenType) throws RecognitionException {
  9.     Token result = parser.getCurrentToken();
  10.     boolean isIdentifierCompatible = false;
  11.     if (identifierTokenIndex == tokenType && identifierTokenIndex > result.getType()) {
  12.       isIdentifierCompatible = true;
  13.     }
  14.     if (result.getType() == tokenType || isIdentifierCompatible) {
  15.       if (Token.EOF != tokenType && isIdentifierCompatible && result instanceof CommonToken) {
  16.         ((CommonToken) result).setType(identifierTokenIndex);
  17.       }
  18.       parser.getErrorHandler().reportMatch(parser);
  19.       parser.consume();
  20.     } else {
  21.       result = parser.getErrorHandler().recoverInline(parser);
  22.       if (parser.getBuildParseTree() && -1 == result.getTokenIndex()) {
  23.         parser.getContext().addErrorNode(parser.createErrorNode(parser.getContext(), result));
  24.       }
  25.     }
  26.     return result;
  27.   }
  28. }

代码示例来源：origin: org.antlr/antlr4-runtime

  1. /** Consume tokens until one matches the given token set. */
  2.   protected void consumeUntil(Parser recognizer, IntervalSet set) {
  3. //        System.err.println("consumeUntil("+set.toString(recognizer.getTokenNames())+")");
  4.     int ttype = recognizer.getInputStream().LA(1);
  5.     while (ttype != Token.EOF && !set.contains(ttype) ) {
  6.       //System.out.println("consume during recover LA(1)="+getTokenNames()[input.LA(1)]);
  7. //            recognizer.getInputStream().consume();
  8.       recognizer.consume();
  9.       ttype = recognizer.getInputStream().LA(1);
  10.     }
  11.   }
  12. }

代码示例来源：origin: org.antlr/antlr4-runtime

  1. if (t.getType() > 0) {
  2.   _errHandler.reportMatch(this);
  3.   consume();

代码示例来源：origin: org.antlr/antlr4-runtime

  1. recognizer.consume(); // simply delete extra token

代码示例来源：origin: org.antlr/antlr4-runtime

  1. consume();

代码示例来源：origin: org.antlr/antlr4-runtime

  1. recognizer.consume();
  2. return matchedSymbol;

代码示例来源：origin: org.antlr/antlr4-runtime

  1. recognizer.consume();

代码示例来源：origin: net.sf.cssbox/jstyleparser

  1. /**
  2.  * Consumes token until lexer state is balanced and
  3.  * token from follow is matched. Matched token is also consumed
  4.  */
  5. protected void consumeUntilGreedy(Parser recognizer, IntervalSet follow) {
  6.   logger.trace("CONSUME UNTIL GREEDY {}", follow.toString());
  7.   for (int ttype = recognizer.getInputStream().LA(1); ttype != -1 && !follow.contains(ttype); ttype = recognizer.getInputStream().LA(1)) {
  8.     Token t = recognizer.consume();
  9.     logger.trace("Skipped greedy: {}", t.getText());
  10.   }
  11.   Token t = recognizer.consume();
  12.   logger.trace("Skipped greedy: {} follow: {}", t.getText(), follow);
  13. }

代码示例来源：origin: antlr/intellij-plugin-v4

  1. @Override
  2.   protected void consumeUntil(Parser recognizer, IntervalSet set) {
  3. //        System.err.println("consumeUntil("+set.toString(recognizer.getTokenNames())+")");
  4.     int ttype = recognizer.getInputStream().LA(1);
  5.     while (ttype != Token.EOF ) {
  6.       //System.out.println("consume during recover LA(1)="+getTokenNames()[input.LA(1)]);
  7. //            recognizer.getInputStream().consume();
  8.       recognizer.consume();
  9.       ttype = recognizer.getInputStream().LA(1);
  10.     }
  11.   }
  12. }

代码示例来源：origin: radkovo/jStyleParser

  1. /**
  2.  * Consumes token until lexer state is balanced and
  3.  * token from follow is matched. Matched token is also consumed
  4.  */
  5. protected void consumeUntilGreedy(Parser recognizer, IntervalSet follow) {
  6.   logger.trace("CONSUME UNTIL GREEDY {}", follow.toString());
  7.   for (int ttype = recognizer.getInputStream().LA(1); ttype != -1 && !follow.contains(ttype); ttype = recognizer.getInputStream().LA(1)) {
  8.     Token t = recognizer.consume();
  9.     logger.trace("Skipped greedy: {}", t.getText());
  10.   }
  11.   Token t = recognizer.consume();
  12.   logger.trace("Skipped greedy: {} follow: {}", t.getText(), follow);
  13. }

代码示例来源：origin: uk.co.nichesolutions/antlr4-runtime

  1. /** Consume tokens until one matches the given token set. */
  2.   protected void consumeUntil(Parser recognizer, IntervalSet set) {
  3. //        System.err.println("consumeUntil("+set.toString(recognizer.getTokenNames())+")");
  4.     int ttype = recognizer.getInputStream().LA(1);
  5.     while (ttype != Token.EOF && !set.contains(ttype) ) {
  6.       //System.out.println("consume during recover LA(1)="+getTokenNames()[input.LA(1)]);
  7. //            recognizer.getInputStream().consume();
  8.       recognizer.consume();
  9.       ttype = recognizer.getInputStream().LA(1);
  10.     }
  11.   }
  12. }

代码示例来源：origin: com.impetus.fabric/fabric-jdbc-driver-shaded

  1. /** Consume tokens until one matches the given token set. */
  2.   protected void consumeUntil(Parser recognizer, IntervalSet set) {
  3. //        System.err.println("consumeUntil("+set.toString(recognizer.getTokenNames())+")");
  4.     int ttype = recognizer.getInputStream().LA(1);
  5.     while (ttype != Token.EOF && !set.contains(ttype) ) {
  6.       //System.out.println("consume during recover LA(1)="+getTokenNames()[input.LA(1)]);
  7. //            recognizer.getInputStream().consume();
  8.       recognizer.consume();
  9.       ttype = recognizer.getInputStream().LA(1);
  10.     }
  11.   }
  12. }

代码示例来源：origin: io.virtdata/virtdata-lib-realer

  1. /** Consume tokens until one matches the given token set. */
  2.   protected void consumeUntil(Parser recognizer, IntervalSet set) {
  3. //        System.err.println("consumeUntil("+set.toString(recognizer.getTokenNames())+")");
  4.     int ttype = recognizer.getInputStream().LA(1);
  5.     while (ttype != Token.EOF && !set.contains(ttype) ) {
  6.       //System.out.println("consume during recover LA(1)="+getTokenNames()[input.LA(1)]);
  7. //            recognizer.getInputStream().consume();
  8.       recognizer.consume();
  9.       ttype = recognizer.getInputStream().LA(1);
  10.     }
  11.   }
  12. }

代码示例来源：origin: com.tunnelvisionlabs/antlr4-runtime

  1. /** Consume tokens until one matches the given token set. */
  2.   protected void consumeUntil(@NotNull Parser recognizer, @NotNull IntervalSet set) {
  3. //        System.err.println("consumeUntil("+set.toString(recognizer.getTokenNames())+")");
  4.     int ttype = recognizer.getInputStream().LA(1);
  5.     while (ttype != Token.EOF && !set.contains(ttype) ) {
  6.       //System.out.println("consume during recover LA(1)="+getTokenNames()[input.LA(1)]);
  7. //            recognizer.getInputStream().consume();
  8.       recognizer.consume();
  9.       ttype = recognizer.getInputStream().LA(1);
  10.     }
  11.   }
  12. }

代码示例来源：origin: org.antlr/antlr4

  1. @Override
  2.     public void recover(Parser recognizer, RecognitionException e) {
  3.       int errIndex = recognizer.getInputStream().index();
  4.       if ( firstErrorTokenIndex == -1 ) {
  5.         firstErrorTokenIndex = errIndex; // latch
  6.       }
  7. //            System.err.println("recover: error at " + errIndex);
  8.       TokenStream input = recognizer.getInputStream();
  9.       if ( input.index()<input.size()-1 ) { // don't consume() eof
  10.         recognizer.consume(); // just kill this bad token and let it continue.
  11.       }
  12.     }

代码示例来源：origin: com.tunnelvisionlabs/antlr4

  1. @Override
  2.     public void recover(Parser recognizer, RecognitionException e) {
  3.       int errIndex = recognizer.getInputStream().index();
  4.       if ( firstErrorTokenIndex == -1 ) {
  5.         firstErrorTokenIndex = errIndex; // latch
  6.       }
  7. //            System.err.println("recover: error at " + errIndex);
  8.       TokenStream input = recognizer.getInputStream();
  9.       if ( input.index()<input.size()-1 ) { // don't consume() eof
  10.         recognizer.consume(); // just kill this bad token and let it continue.
  11.       }
  12.     }

代码示例来源：origin: uk.co.nichesolutions/antlr4

  1. @Override
  2.     public void recover(Parser recognizer, RecognitionException e) {
  3.       int errIndex = recognizer.getInputStream().index();
  4.       if ( firstErrorTokenIndex == -1 ) {
  5.         firstErrorTokenIndex = errIndex; // latch
  6.       }
  7. //            System.err.println("recover: error at " + errIndex);
  8.       TokenStream input = recognizer.getInputStream();
  9.       if ( input.index()<input.size()-1 ) { // don't consume() eof
  10.         recognizer.consume(); // just kill this bad token and let it continue.
  11.       }
  12.     }

代码示例来源：origin: antlr/intellij-plugin-v4

  1. @Override
  2.     public void recover(Parser recognizer, RecognitionException e) {
  3.       int errIndex = recognizer.getInputStream().index();
  4.       if ( firstErrorTokenIndex == -1 ) {
  5.         firstErrorTokenIndex = errIndex; // latch
  6.       }
  7. //            System.err.println("recover: error at " + errIndex);
  8.       TokenStream input = recognizer.getInputStream();
  9.       if ( input.index()<input.size()-1 ) { // don't consume() eof
  10.         recognizer.consume(); // just kill this bad token and let it continue.
  11.       }
  12.     }

代码示例来源：origin: io.virtdata/virtdata-lib-realer

  1. @Override
  2.     public void recover(Parser recognizer, RecognitionException e) {
  3.       int errIndex = recognizer.getInputStream().index();
  4.       if ( firstErrorTokenIndex == -1 ) {
  5.         firstErrorTokenIndex = errIndex; // latch
  6.       }
  7. //            System.err.println("recover: error at " + errIndex);
  8.       TokenStream input = recognizer.getInputStream();
  9.       if ( input.index()<input.size()-1 ) { // don't consume() eof
  10.         recognizer.consume(); // just kill this bad token and let it continue.
  11.       }
  12.     }

代码示例来源：origin: org.ballerinalang/language-server-core

  1. private void removePendingTokensAfterThisToken(Parser recognizer, Token token,
  2.                         TokenRemovalStrategy currentStrategy) {
  3.   int currentTokenIndex = recognizer.getCurrentToken().getTokenIndex();
  4.   if (token != null && (isInLastTermination(recognizer) ||
  5.       TokenRemovalStrategy.SYNC.equals(currentStrategy) && currentTokenIndex < token.getTokenIndex()
  6.       || TokenRemovalStrategy.MATCH.equals(currentStrategy) && currentTokenIndex <= token.getTokenIndex())) {
  7.     return;
  8.   }
  9.   while (removeTokenCount > 0) {
  10.     forceConsumedTokens.push(recognizer.consume());
  11.     removeTokenCount--;
  12.   }
  13.   this.context.put(CompletionKeys.FORCE_CONSUMED_TOKENS_KEY, forceConsumedTokens);
  14. }

内容来源于网络，如有侵权，请联系作者删除！

Parser

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多

    Parser类方法

    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com