org.apache.flink.streaming.api.operators.Output.emitWatermark()方法的使用及代码示例

x33g5p2x  于2022-01-26 转载在 其他  
字(5.9k)|赞(0)|评价(0)|浏览(82)

本文整理了Java中org.apache.flink.streaming.api.operators.Output.emitWatermark()方法的一些代码示例,展示了Output.emitWatermark()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Output.emitWatermark()方法的具体详情如下:
包路径:org.apache.flink.streaming.api.operators.Output
类名称:Output
方法名:emitWatermark

Output.emitWatermark介绍

[英]Emits a Watermark from an operator. This watermark is broadcast to all downstream operators.

A watermark specifies that no element with a timestamp lower or equal to the watermark timestamp will be emitted in the future.
[中]从操作员处发出水印。该水印将广播给所有下游运营商。
水印指定将来不会发射时间戳小于或等于水印时间戳的元素。

代码示例

代码示例来源:origin: apache/flink

@Override
public void emitWatermark(Watermark mark) {
  output.emitWatermark(mark);
}

代码示例来源:origin: apache/flink

/** This will only be called if allowWatermark returned {@code true}. */
@Override
protected void processAndEmitWatermark(Watermark mark) {
  nextWatermarkTime = Long.MAX_VALUE;
  output.emitWatermark(mark);
  // we can shutdown the watermark timer now, no watermarks will be needed any more.
  // Note that this procedure actually doesn't need to be synchronized with the lock,
  // but since it's only a one-time thing, doesn't hurt either
  final ScheduledFuture<?> nextWatermarkTimer = this.nextWatermarkTimer;
  if (nextWatermarkTimer != null) {
    nextWatermarkTimer.cancel(true);
  }
}

代码示例来源:origin: apache/flink

public void processWatermark(Watermark mark) throws Exception {
  if (timeServiceManager != null) {
    timeServiceManager.advanceWatermark(mark);
  }
  output.emitWatermark(mark);
}

代码示例来源:origin: apache/flink

@Override
  public void processWatermark(Watermark mark) throws Exception {
    output.emitWatermark(mark);
  }
}

代码示例来源:origin: apache/flink

@Override
  public void processWatermark(Watermark mark) {
    output.emitWatermark(mark);
  }
}

代码示例来源:origin: apache/flink

@Override
  public void processWatermark(Watermark mark) throws Exception {
    output.emitWatermark(mark);
  }
}

代码示例来源:origin: apache/flink

@Override
  public void processWatermark(Watermark mark) throws Exception {
    output.emitWatermark(mark);
  }
}

代码示例来源:origin: apache/flink

protected void handleWatermark(Watermark mark) {
    output.emitWatermark(mark);
  }
}

代码示例来源:origin: apache/flink

@Override
public void emitWatermark(Watermark mark) {
  watermarkGauge.setCurrentWatermark(mark.getTimestamp());
  for (Output<StreamRecord<OUT>> out : allOutputs) {
    out.emitWatermark(mark);
  }
}

代码示例来源:origin: apache/flink

@Override
  public void processWatermark(Watermark mark) throws Exception {
    // if we receive a Long.MAX_VALUE watermark we forward it since it is used
    // to signal the end of input and to not block watermark progress downstream
    if (mark.getTimestamp() == Long.MAX_VALUE && mark.getTimestamp() > currentWatermark) {
      currentWatermark = Long.MAX_VALUE;
      output.emitWatermark(mark);
    }
  }
}

代码示例来源:origin: apache/flink

@Override
  protected void handleWatermark(Watermark mark) {
    if (mark.equals(Watermark.MAX_WATERMARK)) {
      output.emitWatermark(mark);
      lastWatermark = Long.MAX_VALUE;
    }
  }
}

代码示例来源:origin: apache/flink

@Override
  public void close() throws Exception {
    super.close();

    // emit a final watermark
    Watermark newWatermark = userFunction.getCurrentWatermark();
    if (newWatermark != null && newWatermark.getTimestamp() > currentWatermark) {
      currentWatermark = newWatermark.getTimestamp();
      // emit watermark
      output.emitWatermark(newWatermark);
    }
  }
}

代码示例来源:origin: apache/flink

@Override
public void emitWatermark(Watermark mark) {
  watermarkGauge.setCurrentWatermark(mark.getTimestamp());
  if (streamStatusProvider.getStreamStatus().isActive()) {
    for (Output<StreamRecord<T>> output : outputs) {
      output.emitWatermark(mark);
    }
  }
}

代码示例来源:origin: apache/flink

@Override
protected void handleElement(StreamRecord<String> element) {
  long timestamp = Long.valueOf(element.getValue());
  if (timestamp > lastWatermark) {
    output.emitWatermark(new Watermark(timestamp));
    lastWatermark = timestamp;
  }
}

代码示例来源:origin: apache/flink

@Override
  public void processWatermark(Watermark mark) throws Exception {
    output.emitWatermark(new Watermark(mark.getTimestamp() * 2));
  }
}

代码示例来源:origin: apache/flink

@Override
public void onProcessingTime(long timestamp) throws Exception {
  // register next timer
  long newWatermark = userFunction.getCurrentWatermark();
  if (newWatermark > currentWatermark) {
    currentWatermark = newWatermark;
    // emit watermark
    output.emitWatermark(new Watermark(currentWatermark));
  }
  long now = getProcessingTimeService().getCurrentProcessingTime();
  getProcessingTimeService().registerTimer(now + watermarkInterval, this);
}

代码示例来源:origin: apache/flink

@Override
public void onProcessingTime(long timestamp) throws Exception {
  // register next timer
  Watermark newWatermark = userFunction.getCurrentWatermark();
  if (newWatermark != null && newWatermark.getTimestamp() > currentWatermark) {
    currentWatermark = newWatermark.getTimestamp();
    // emit watermark
    output.emitWatermark(newWatermark);
  }
  long now = getProcessingTimeService().getCurrentProcessingTime();
  getProcessingTimeService().registerTimer(now + watermarkInterval, this);
}

代码示例来源:origin: apache/flink

@Override
public void processWatermark(Watermark mark) throws Exception {
  super.processWatermark(mark);
  for (Watermark previousMark: watermarks) {
    assertTrue(previousMark.getTimestamp() < mark.getTimestamp());
  }
  watermarks.add(mark);
  latch.trigger();
  output.emitWatermark(mark);
}

代码示例来源:origin: apache/flink

@Override
public void processElement(StreamRecord<T> element) throws Exception {
  long newTimestamp = userFunction.extractTimestamp(element.getValue(), element.getTimestamp());
  output.collect(element.replace(element.getValue(), newTimestamp));
  long watermark = userFunction.extractWatermark(element.getValue(), newTimestamp);
  if (watermark > currentWatermark) {
    currentWatermark = watermark;
    output.emitWatermark(new Watermark(currentWatermark));
  }
}

代码示例来源:origin: apache/flink

@Override
public void processElement(StreamRecord<T> element) throws Exception {
  final T value = element.getValue();
  final long newTimestamp = userFunction.extractTimestamp(value,
      element.hasTimestamp() ? element.getTimestamp() : Long.MIN_VALUE);
  output.collect(element.replace(element.getValue(), newTimestamp));
  final Watermark nextWatermark = userFunction.checkAndGetNextWatermark(value, newTimestamp);
  if (nextWatermark != null && nextWatermark.getTimestamp() > currentWatermark) {
    currentWatermark = nextWatermark.getTimestamp();
    output.emitWatermark(nextWatermark);
  }
}

相关文章