本文整理了Java中cascading.tap.Tap.openForWrite()
方法的一些代码示例,展示了Tap.openForWrite()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Tap.openForWrite()
方法的具体详情如下:
包路径:cascading.tap.Tap
类名称:Tap
方法名:openForWrite
[英]Method openForWrite opens the resource represented by this Tap instance for writing.
This method is for user application use and does honor the SinkMode#REPLACE settings. That is, if SinkMode is set to SinkMode#REPLACE the underlying resource will be deleted.
Note if SinkMode#UPDATE is set, the resource will not be deleted.
[中]方法openForWrite打开此Tap实例表示的资源进行写入。
此方法供用户应用程序使用,并遵循SinkMode#REPLACE设置。也就是说,如果SinkMode被设置为SinkMode#REPLACE,那么底层资源将被删除。
注意:如果设置了SinkMode#UPDATE,资源将不会被删除。
代码示例来源:origin: elastic/elasticsearch-hadoop
@Override
public TupleEntryCollector openForWrite(FlowProcess<Object> flowProcess, Object output) throws IOException {
initInnerTapIfNotSetFromFlowProcess(flowProcess);
return actualTap.openForWrite(flowProcess, output);
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
{
return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openTrapForWrite( Tap trap ) throws IOException
{
return trap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openForWrite( FlowProcess<? extends Config> flowProcess, Output output ) throws IOException
{
return original.openForWrite( flowProcess, output );
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
{
return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openForWrite( FlowProcess<? extends Config> flowProcess ) throws IOException
{
return original.openForWrite( flowProcess );
}
代码示例来源:origin: cascading/cascading-hadoop2-mr1
@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
{
return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cascading/cascading-hadoop2-tez
@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
{
return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
{
return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openTrapForWrite( Tap trap ) throws IOException
{
return trap.openForWrite( this );
}
代码示例来源:origin: dataArtisans/cascading-flink
@Override
public TupleEntryCollector openTapForWrite(Tap tap) throws IOException {
return tap.openForWrite( this, null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openForWrite( FlowProcess<? extends TConfig> flowProcess ) throws IOException
{
return original.openForWrite( processProvider.apply( flowProcess ) );
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openForWrite( FlowProcess<? extends TConfig> flowProcess, TOutput output ) throws IOException
{
return original.openForWrite( processProvider.apply( flowProcess ), null );
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openTapForWrite( Tap tap ) throws IOException
{
return tap.openForWrite( getFlowProcess() );
}
代码示例来源:origin: org.elasticsearch/elasticsearch-hadoop
@Override
public TupleEntryCollector openForWrite(FlowProcess<Object> flowProcess, Object output) throws IOException {
initInnerTapIfNotSetFromFlowProcess(flowProcess);
return actualTap.openForWrite(flowProcess, output);
}
代码示例来源:origin: com.hotels/plunger
private void writeToHadoopTap(Tap<?, ?, ?> tap) throws IOException {
@SuppressWarnings("unchecked")
Tap<JobConf, ?, ?> hadoopTap = (Tap<JobConf, ?, ?>) tap;
JobConf conf = new JobConf();
HadoopFlowProcess flowProcess = new HadoopFlowProcess(conf);
hadoopTap.sinkConfInit(flowProcess, conf);
TupleEntryCollector collector = hadoopTap.openForWrite(flowProcess);
for (TupleEntry tuple : data.asTupleEntryList()) {
collector.add(tuple);
}
collector.close();
}
代码示例来源:origin: LiveRamp/cascading_ext
public static void writeToStore(CascadingUtil util, TupleDataStore store, Tuple ... tuples) throws IOException{
TupleEntryCollector output = store.getTap().openForWrite(util.getFlowProcess());
for(Tuple t: tuples){
output.add(t);
}
output.close();
}
代码示例来源:origin: cwensel/cascading
@Override
public TupleEntryCollector openTrapForWrite( Tap trap ) throws IOException
{
TezConfiguration jobConf = new TezConfiguration( getConfiguration() );
int stepNum = jobConf.getInt( "cascading.flow.step.num", 0 );
int nodeNum = jobConf.getInt( "cascading.flow.node.num", 0 );
String partname = String.format( "-%05d-%05d-", stepNum, nodeNum );
jobConf.set( "cascading.tapcollector.partname", "%s%spart" + partname + "%05d" );
return trap.openForWrite( new Hadoop2TezFlowProcess( this, jobConf ), null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cascading/cascading-hadoop2-tez
@Override
public TupleEntryCollector openTrapForWrite( Tap trap ) throws IOException
{
TezConfiguration jobConf = new TezConfiguration( getConfiguration() );
int stepNum = jobConf.getInt( "cascading.flow.step.num", 0 );
int nodeNum = jobConf.getInt( "cascading.flow.node.num", 0 );
String partname = String.format( "-%05d-%05d-", stepNum, nodeNum );
jobConf.set( "cascading.tapcollector.partname", "%s%spart" + partname + "%05d" );
return trap.openForWrite( new Hadoop2TezFlowProcess( this, jobConf ), null ); // do not honor sinkmode as this may be opened across tasks
}
代码示例来源:origin: cascading/lingual-core
private void writeValuesTuple( PlatformBroker platformBroker, Ref head ) throws IOException
{
SchemaCatalogManager catalog = platformBroker.getCatalogManager();
String identifier = getIdentifierFor( platformBroker, head );
TableDef tableDef = createTableFor( catalog, head, identifier );
TupleEntryCollector collector = catalog.createTapFor( tableDef, SinkMode.KEEP ).openForWrite( platformBroker.getFlowProcess() );
for( List<RexLiteral> values : head.tuples )
collector.add( EnumerableUtil.createTupleFrom( values ) );
collector.close();
}
内容来源于网络,如有侵权,请联系作者删除!