org.apache.flink.runtime.execution.Environment.getIOManager()方法的使用及代码示例

x33g5p2x  于2022-01-19 转载在 其他  
字(10.0k)|赞(0)|评价(0)|浏览(182)

本文整理了Java中org.apache.flink.runtime.execution.Environment.getIOManager()方法的一些代码示例，展示了Environment.getIOManager()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Environment.getIOManager()方法的具体详情如下：
包路径：org.apache.flink.runtime.execution.Environment
类名称：Environment
方法名：getIOManager

Environment.getIOManager介绍

[英]Returns the current IOManager.
[中]返回当前IOManager。

代码示例

代码示例来源：origin: apache/flink

  1. initializedDbBasePaths = env.getIOManager().getSpillingDirectories();

代码示例来源：origin: apache/flink

  1. configuration.getCheckpointMode(),
  2. getCheckpointLock(),
  3. getEnvironment().getIOManager(),
  4. getEnvironment().getTaskManagerInfo().getConfiguration(),
  5. getStreamStatusMaintainer(),

代码示例来源：origin: apache/flink

  1. @Override
  2. public void init() throws Exception {
  3.   StreamConfig configuration = getConfiguration();
  4.   TypeSerializer<IN> inSerializer = configuration.getTypeSerializerIn1(getUserCodeClassLoader());
  5.   int numberOfInputs = configuration.getNumberOfInputs();
  6.   if (numberOfInputs > 0) {
  7.     InputGate[] inputGates = getEnvironment().getAllInputGates();
  8.     inputProcessor = new StreamInputProcessor<>(
  9.         inputGates,
  10.         inSerializer,
  11.         this,
  12.         configuration.getCheckpointMode(),
  13.         getCheckpointLock(),
  14.         getEnvironment().getIOManager(),
  15.         getEnvironment().getTaskManagerInfo().getConfiguration(),
  16.         getStreamStatusMaintainer(),
  17.         this.headOperator,
  18.         getEnvironment().getMetricGroup().getIOMetricGroup(),
  19.         inputWatermarkGauge);
  20.   }
  21.   headOperator.getMetricGroup().gauge(MetricNames.IO_CURRENT_INPUT_WATERMARK, this.inputWatermarkGauge);
  22.   // wrap watermark gauge since registered metrics must be unique
  23.   getEnvironment().getMetricGroup().gauge(MetricNames.IO_CURRENT_INPUT_WATERMARK, this.inputWatermarkGauge::getValue);
  24. }

代码示例来源：origin: org.apache.flink/flink-runtime

  1. @Override
  2. public IOManager getIOManager() {
  3.   return getEnvironment().getIOManager();
  4. }

代码示例来源：origin: com.alibaba.blink/flink-runtime

  1. @Override
  2. public IOManager getIOManager() {
  3.   return getEnvironment().getIOManager();
  4. }

代码示例来源：origin: org.apache.flink/flink-runtime_2.10

  1. @Override
  2. public IOManager getIOManager() {
  3.   return getEnvironment().getIOManager();
  4. }

代码示例来源：origin: org.apache.flink/flink-runtime_2.11

  1. @Override
  2. public IOManager getIOManager() {
  3.   return getEnvironment().getIOManager();
  4. }

代码示例来源：origin: org.apache.flink/flink-statebackend-rocksdb_2.11

  1. initializedDbBasePaths = env.getIOManager().getSpillingDirectories();

代码示例来源：origin: org.apache.flink/flink-statebackend-rocksdb

  1. initializedDbBasePaths = env.getIOManager().getSpillingDirectories();

代码示例来源：origin: org.apache.flink/flink-statebackend-rocksdb_2.10

  1. initializedDbBasePaths = env.getIOManager().getSpillingDirectories();

代码示例来源：origin: org.apache.flink/flink-streaming-java_2.10

  1. configuration.getCheckpointMode(),
  2. getCheckpointLock(),
  3. getEnvironment().getIOManager(),
  4. getEnvironment().getTaskManagerInfo().getConfiguration(),
  5. getStreamStatusMaintainer(),

代码示例来源：origin: org.apache.flink/flink-streaming-java_2.11

  1. configuration.getCheckpointMode(),
  2. getCheckpointLock(),
  3. getEnvironment().getIOManager(),
  4. getEnvironment().getTaskManagerInfo().getConfiguration(),
  5. getStreamStatusMaintainer(),

代码示例来源：origin: com.alibaba.blink/flink-table

  1. @Override
  2. public void open() throws Exception {
  3.   super.open();
  4.   isFinished1 = false;
  5.   isFinished2 = false;
  6.   collector = new StreamRecordCollector<>(output);
  7.   serializer1 = (AbstractRowSerializer) getOperatorConfig().getTypeSerializerIn1(getUserCodeClassloader());
  8.   serializer2 = (AbstractRowSerializer) getOperatorConfig().getTypeSerializerIn2(getUserCodeClassloader());
  9.   leftNullRow = new GenericRow(serializer1.getNumFields());
  10.   rightNullRow = new GenericRow(serializer2.getNumFields());
  11.   joinedRow = new JoinedRow();
  12.   CookedClasses classes = cookGeneratedClasses(getContainingTask().getUserCodeClassLoader());
  13.   condFunc = classes.condFuncClass.newInstance();
  14.   keyComparator = classes.keyComparatorClass.newInstance();
  15.   keyComparator.init(keyGSorter.serializers(), keyGSorter.comparators());
  16.   memManager = getContainingTask().getEnvironment().getMemoryManager();
  17.   ioManager = getContainingTask().getEnvironment().getIOManager();
  18.   Projection<BaseRow, BinaryRow> projection1 = classes.projectionClass1.newInstance();
  19.   Projection<BaseRow, BinaryRow> projection2 = classes.projectionClass2.newInstance();
  20.   int pageNum1 = (int) (leftBufferMemory / memManager.getPageSize());
  21.   List<MemorySegment> mem1 = memManager.allocatePages(getContainingTask(), pageNum1);
  22.   buffer1 = new WrappedBuffer(mem1, serializer1, projection1);
  23.   int pageNum2 = (int) (rightBufferMemory / memManager.getPageSize());
  24.   List<MemorySegment> mem2 = memManager.allocatePages(getContainingTask(), pageNum2);
  25.   buffer2 = new WrappedBuffer(mem2, serializer2, projection2);
  26.   initGauge();
  27.   initJoin();
  28. }

代码示例来源：origin: com.alibaba.blink/flink-table

  1. @Override
  2. public void open() throws Exception {
  3.   super.open();
  4.   LOG.info("Opening SortOperator");
  5.   cookGeneratedClasses(getContainingTask().getUserCodeClassLoader());
  6.   TypeSerializer<BaseRow> inputSerializer = getOperatorConfig().getTypeSerializerIn1(getUserCodeClassloader());
  7.   this.binarySerializer =
  8.       new BinaryRowSerializer(((AbstractRowSerializer) inputSerializer).getTypes());
  9.   MemoryManager memManager = this.getContainingTask().getEnvironment().getMemoryManager();
  10.   IOManager ioManager = this.getContainingTask().getEnvironment().getIOManager();
  11.   NormalizedKeyComputer computer = computerClass.newInstance();
  12.   RecordComparator comparator = comparatorClass.newInstance();
  13.   computer.init(gSorter.serializers(), gSorter.comparators());
  14.   comparator.init(gSorter.serializers(), gSorter.comparators());
  15.   this.sorter = new BinaryExternalSorter(this.getContainingTask(), memManager, reservedMemorySize,
  16.       maxMemorySize, perRequestMemorySize, ioManager, inputSerializer, binarySerializer,
  17.       computer, comparator, getSqlConf());
  18.   this.sorter.startThreads();
  19.   gSorter = null;
  20.   collector = new StreamRecordCollector<>(output);
  21.   //register the the metrics.
  22.   getMetricGroup().gauge("memoryUsedSizeInBytes", (Gauge<Long>) sorter::getUsedMemoryInBytes);
  23.   getMetricGroup().gauge("numSpillFiles", (Gauge<Long>) sorter::getNumSpillFiles);
  24.   getMetricGroup().gauge("spillInBytes", (Gauge<Long>) sorter::getSpillInBytes);
  25. }

代码示例来源：origin: org.apache.flink/flink-streaming-java_2.10

  1. @Override
  2. public void init() throws Exception {
  3.   StreamConfig configuration = getConfiguration();
  4.   TypeSerializer<IN> inSerializer = configuration.getTypeSerializerIn1(getUserCodeClassLoader());
  5.   int numberOfInputs = configuration.getNumberOfInputs();
  6.   if (numberOfInputs > 0) {
  7.     InputGate[] inputGates = getEnvironment().getAllInputGates();
  8.     inputProcessor = new StreamInputProcessor<>(
  9.         inputGates,
  10.         inSerializer,
  11.         this,
  12.         configuration.getCheckpointMode(),
  13.         getCheckpointLock(),
  14.         getEnvironment().getIOManager(),
  15.         getEnvironment().getTaskManagerInfo().getConfiguration(),
  16.         getStreamStatusMaintainer(),
  17.         this.headOperator);
  18.     // make sure that stream tasks report their I/O statistics
  19.     inputProcessor.setMetricGroup(getEnvironment().getMetricGroup().getIOMetricGroup());
  20.   }
  21. }

代码示例来源：origin: com.alibaba.blink/flink-table

  1. IOManager ioManager = getContainingTask().getEnvironment().getIOManager();

代码示例来源：origin: com.alibaba.blink/flink-table

  1. @Override
  2. public void open() throws Exception {
  3.   super.open();
  4.   isFinished1 = false;
  5.   isFinished2 = false;
  6.   classes = cookGeneratedClasses(getContainingTask().getUserCodeClassLoader());
  7.   Collector<BaseRow> collector = new StreamRecordCollector<>(output);
  8.   inputSerializer1 = (AbstractRowSerializer) getOperatorConfig()
  9.     .getTypeSerializerIn1(getUserCodeClassloader());
  10.   serializer1 = new BinaryRowSerializer(inputSerializer1.getTypes());
  11.   inputSerializer2 = (AbstractRowSerializer) getOperatorConfig()
  12.     .getTypeSerializerIn2(getUserCodeClassloader());
  13.   serializer2 = new BinaryRowSerializer(inputSerializer2.getTypes());
  14.   memManager = this.getContainingTask().getEnvironment().getMemoryManager();
  15.   ioManager = this.getContainingTask().getEnvironment().getIOManager();
  16.   JoinConditionFunction condFunc = classes.condFuncClass.newInstance();
  17.   leftNullRow = new GenericRow(serializer1.getNumFields());
  18.   rightNullRow = new GenericRow(serializer2.getNumFields());
  19.   JoinedRow joinedRow = new JoinedRow();
  20.   helper =
  21.     new SortMergeJoinHelper(collector, condFunc, leftNullRow, rightNullRow, joinedRow);
  22.   initSorter();
  23.   probeBuffer =
  24.     newBuffer(probeBufferMemory, leftNeedsSort ? inputSerializer2 : inputSerializer1);
  25.   initGauge();
  26. }

代码示例来源：origin: org.apache.flink/flink-streaming-java_2.11

  1. @Override
  2. public void init() throws Exception {
  3.   StreamConfig configuration = getConfiguration();
  4.   TypeSerializer<IN> inSerializer = configuration.getTypeSerializerIn1(getUserCodeClassLoader());
  5.   int numberOfInputs = configuration.getNumberOfInputs();
  6.   if (numberOfInputs > 0) {
  7.     InputGate[] inputGates = getEnvironment().getAllInputGates();
  8.     inputProcessor = new StreamInputProcessor<>(
  9.         inputGates,
  10.         inSerializer,
  11.         this,
  12.         configuration.getCheckpointMode(),
  13.         getCheckpointLock(),
  14.         getEnvironment().getIOManager(),
  15.         getEnvironment().getTaskManagerInfo().getConfiguration(),
  16.         getStreamStatusMaintainer(),
  17.         this.headOperator,
  18.         getEnvironment().getMetricGroup().getIOMetricGroup(),
  19.         inputWatermarkGauge);
  20.   }
  21.   headOperator.getMetricGroup().gauge(MetricNames.IO_CURRENT_INPUT_WATERMARK, this.inputWatermarkGauge);
  22.   // wrap watermark gauge since registered metrics must be unique
  23.   getEnvironment().getMetricGroup().gauge(MetricNames.IO_CURRENT_INPUT_WATERMARK, this.inputWatermarkGauge::getValue);
  24. }

代码示例来源：origin: org.apache.flink/flink-streaming-java

  1. @Override
  2. public void init() throws Exception {
  3.   StreamConfig configuration = getConfiguration();
  4.   TypeSerializer<IN> inSerializer = configuration.getTypeSerializerIn1(getUserCodeClassLoader());
  5.   int numberOfInputs = configuration.getNumberOfInputs();
  6.   if (numberOfInputs > 0) {
  7.     InputGate[] inputGates = getEnvironment().getAllInputGates();
  8.     inputProcessor = new StreamInputProcessor<>(
  9.         inputGates,
  10.         inSerializer,
  11.         this,
  12.         configuration.getCheckpointMode(),
  13.         getCheckpointLock(),
  14.         getEnvironment().getIOManager(),
  15.         getEnvironment().getTaskManagerInfo().getConfiguration(),
  16.         getStreamStatusMaintainer(),
  17.         this.headOperator,
  18.         getEnvironment().getMetricGroup().getIOMetricGroup(),
  19.         inputWatermarkGauge);
  20.   }
  21.   headOperator.getMetricGroup().gauge(MetricNames.IO_CURRENT_INPUT_WATERMARK, this.inputWatermarkGauge);
  22.   // wrap watermark gauge since registered metrics must be unique
  23.   getEnvironment().getMetricGroup().gauge(MetricNames.IO_CURRENT_INPUT_WATERMARK, this.inputWatermarkGauge::getValue);
  24. }

代码示例来源：origin: com.alibaba.blink/flink-table

  1. this.ioManager = this.getContainingTask().getEnvironment().getIOManager();

内容来源于网络，如有侵权，请联系作者删除！

Environment

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com