cascading.tap.Tap.openForWrite()方法的使用及代码示例

x33g5p2x  于2022-01-30 转载在 其他  
字(6.1k)|赞(0)|评价(0)|浏览(160)

本文整理了Java中cascading.tap.Tap.openForWrite()方法的一些代码示例,展示了Tap.openForWrite()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Tap.openForWrite()方法的具体详情如下:
包路径:cascading.tap.Tap
类名称:Tap
方法名:openForWrite

Tap.openForWrite介绍

[英]Method openForWrite opens the resource represented by this Tap instance for writing.

This method is for user application use and does honor the SinkMode#REPLACE settings. That is, if SinkMode is set to SinkMode#REPLACE the underlying resource will be deleted.

Note if SinkMode#UPDATE is set, the resource will not be deleted.
[中]方法openForWrite打开此Tap实例表示的资源进行写入。
此方法供用户应用程序使用,并遵循SinkMode#REPLACE设置。也就是说,如果SinkMode被设置为SinkMode#REPLACE,那么底层资源将被删除。
注意:如果设置了SinkMode#UPDATE,资源将不会被删除。

代码示例

代码示例来源:origin: elastic/elasticsearch-hadoop

@Override
public TupleEntryCollector openForWrite(FlowProcess<Object> flowProcess, Object output) throws IOException {
  initInnerTapIfNotSetFromFlowProcess(flowProcess);
  return actualTap.openForWrite(flowProcess, output);
}

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
 {
 return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
 }

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openTrapForWrite( Tap trap ) throws IOException
 {
 return trap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
 }

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openForWrite( FlowProcess<? extends Config> flowProcess, Output output ) throws IOException
 {
 return original.openForWrite( flowProcess, output );
 }

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
 {
 return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
 }

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openForWrite( FlowProcess<? extends Config> flowProcess ) throws IOException
 {
 return original.openForWrite( flowProcess );
 }

代码示例来源:origin: cascading/cascading-hadoop2-mr1

@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
 {
 return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
 }

代码示例来源:origin: cascading/cascading-hadoop2-tez

@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
 {
 return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
 }

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
 {
 return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
 }

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openTrapForWrite( Tap trap ) throws IOException
 {
 return trap.openForWrite( this );
 }

代码示例来源:origin: dataArtisans/cascading-flink

@Override
public TupleEntryCollector openTapForWrite(Tap tap) throws IOException {
  return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
}

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openForWrite( FlowProcess<? extends TConfig> flowProcess ) throws IOException
 {
 return original.openForWrite( processProvider.apply( flowProcess ) );
 }

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openForWrite( FlowProcess<? extends TConfig> flowProcess, TOutput output ) throws IOException
 {
 return original.openForWrite( processProvider.apply( flowProcess ), null );
 }

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
 {
 return tap.openForWrite( getFlowProcess() );
 }

代码示例来源:origin: org.elasticsearch/elasticsearch-hadoop

@Override
public TupleEntryCollector openForWrite(FlowProcess<Object> flowProcess, Object output) throws IOException {
  initInnerTapIfNotSetFromFlowProcess(flowProcess);
  return actualTap.openForWrite(flowProcess, output);
}

代码示例来源:origin: com.hotels/plunger

private void writeToHadoopTap(Tap<?, ?, ?> tap) throws IOException {
 @SuppressWarnings("unchecked")
 Tap<JobConf, ?, ?> hadoopTap = (Tap<JobConf, ?, ?>) tap;
 JobConf conf = new JobConf();
 HadoopFlowProcess flowProcess = new HadoopFlowProcess(conf);
 hadoopTap.sinkConfInit(flowProcess, conf);
 TupleEntryCollector collector = hadoopTap.openForWrite(flowProcess);
 for (TupleEntry tuple : data.asTupleEntryList()) {
  collector.add(tuple);
 }
 collector.close();
}

代码示例来源:origin: LiveRamp/cascading_ext

public static void writeToStore(CascadingUtil util, TupleDataStore store, Tuple ... tuples) throws IOException{
 TupleEntryCollector output = store.getTap().openForWrite(util.getFlowProcess());
 for(Tuple t: tuples){
  output.add(t);
 }
 output.close();
}

代码示例来源:origin: cwensel/cascading

@Override
public TupleEntryCollector openTrapForWrite( Tap trap ) throws IOException
 {
 TezConfiguration jobConf = new TezConfiguration( getConfiguration() );
 int stepNum = jobConf.getInt( "cascading.flow.step.num", 0 );
 int nodeNum = jobConf.getInt( "cascading.flow.node.num", 0 );
 String partname = String.format( "-%05d-%05d-", stepNum, nodeNum );
 jobConf.set( "cascading.tapcollector.partname", "%s%spart" + partname + "%05d" );
 return trap.openForWrite( new Hadoop2TezFlowProcess( this, jobConf ), null ); // do not honor sinkmode as this may be opened across tasks
 }

代码示例来源:origin: cascading/cascading-hadoop2-tez

@Override
public TupleEntryCollector openTrapForWrite( Tap trap ) throws IOException
 {
 TezConfiguration jobConf = new TezConfiguration( getConfiguration() );
 int stepNum = jobConf.getInt( "cascading.flow.step.num", 0 );
 int nodeNum = jobConf.getInt( "cascading.flow.node.num", 0 );
 String partname = String.format( "-%05d-%05d-", stepNum, nodeNum );
 jobConf.set( "cascading.tapcollector.partname", "%s%spart" + partname + "%05d" );
 return trap.openForWrite( new Hadoop2TezFlowProcess( this, jobConf ), null ); // do not honor sinkmode as this may be opened across tasks
 }

代码示例来源:origin: cascading/lingual-core

private void writeValuesTuple( PlatformBroker platformBroker, Ref head ) throws IOException
 {
 SchemaCatalogManager catalog = platformBroker.getCatalogManager();
 String identifier = getIdentifierFor( platformBroker, head );
 TableDef tableDef = createTableFor( catalog, head, identifier );
 TupleEntryCollector collector = catalog.createTapFor( tableDef, SinkMode.KEEP ).openForWrite( platformBroker.getFlowProcess() );
 for( List<RexLiteral> values : head.tuples )
  collector.add( EnumerableUtil.createTupleFrom( values ) );
 collector.close();
 }

相关文章