cascading.util.Util.returnInstanceFieldIfExistsSafe()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(7.4k)|赞(0)|评价(0)|浏览(168)

本文整理了Java中cascading.util.Util.returnInstanceFieldIfExistsSafe()方法的一些代码示例，展示了Util.returnInstanceFieldIfExistsSafe()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Util.returnInstanceFieldIfExistsSafe()方法的具体详情如下：
包路径：cascading.util.Util
类名称：Util
方法名：returnInstanceFieldIfExistsSafe

Util.returnInstanceFieldIfExistsSafe介绍

暂无

代码示例

代码示例来源：origin: cwensel/cascading

  1. protected static Job getJob( RunningJob runningJob )
  2.  {
  3.  if( runningJob == null ) // if null, job hasn't been submitted
  4.   return null;
  5.  Job job = Util.returnInstanceFieldIfExistsSafe( runningJob, "job" );
  6.  if( job == null )
  7.   {
  8.   LOG.warn( "unable to get underlying org.apache.hadoop.mapreduce.Job from org.apache.hadoop.mapred.RunningJob, task level task counters will be unavailable" );
  9.   return null;
  10.   }
  11.  return job;
  12.  }

代码示例来源：origin: cascading/cascading-hadoop2-mr1

  1. protected static Job getJob( RunningJob runningJob )
  2.  {
  3.  if( runningJob == null ) // if null, job hasn't been submitted
  4.   return null;
  5.  Job job = Util.returnInstanceFieldIfExistsSafe( runningJob, "job" );
  6.  if( job == null )
  7.   {
  8.   LOG.warn( "unable to get underlying org.apache.hadoop.mapreduce.Job from org.apache.hadoop.mapred.RunningJob, task level task counters will be unavailable" );
  9.   return null;
  10.   }
  11.  return job;
  12.  }

代码示例来源：origin: cwensel/cascading

  1. private static FrameworkClient getFrameworkClient( DAGClient dagClient )
  2.  {
  3.  if( dagClient instanceof TezTimelineClient )
  4.   return ( (TezTimelineClient) dagClient ).getFrameworkClient();
  5.  return Util.returnInstanceFieldIfExistsSafe( dagClient, "frameworkClient" );
  6.  }
  7. }

代码示例来源：origin: cascading/cascading-hadoop2-tez-stats

  1. private static FrameworkClient getFrameworkClient( DAGClient dagClient )
  2.  {
  3.  if( dagClient instanceof TezTimelineClient )
  4.   return ( (TezTimelineClient) dagClient ).getFrameworkClient();
  5.  return Util.returnInstanceFieldIfExistsSafe( dagClient, "frameworkClient" );
  6.  }
  7. }

代码示例来源：origin: cwensel/cascading

  1. static void injectIdentityMap( AbstractGraph graph )
  2.  {
  3.  // this overcomes jgrapht 0.9.0 using a LinkedHashMap vs an IdentityHashMap
  4.  // vertex not found errors will be thrown if this fails
  5.  Object specifics = Util.returnInstanceFieldIfExistsSafe( graph, "specifics" );
  6.  if( specifics == null )
  7.   {
  8.   LOG.warn( "unable to get jgrapht Specifics for identity map injection, may be using an incompatible jgrapht version" );
  9.   return;
  10.   }
  11.  boolean success = Util.setInstanceFieldIfExistsSafe( specifics, "vertexMapDirected", new IdentityHashMap<>() );
  12.  if( !success )
  13.   LOG.warn( "unable to set IdentityHashMap on jgrapht Specifics, may be using an incompatible jgrapht version" );
  14.  }
  15. }

代码示例来源：origin: cwensel/cascading

  1. public static String id( FlowElement flowElement )
  2.  {
  3.  if( flowElement instanceof Pipe )
  4.   return Pipe.id( (Pipe) flowElement );
  5.  if( flowElement instanceof Tap )
  6.   return Tap.id( (Tap) flowElement );
  7.  String id = Util.returnInstanceFieldIfExistsSafe( flowElement, "id" );
  8.  if( id != null )
  9.   return id;
  10.  throw new IllegalArgumentException( "id not supported for: " + flowElement.getClass().getCanonicalName() );
  11.  }

代码示例来源：origin: cwensel/cascading

  1. };
  2. ApplicationId appId = Util.returnInstanceFieldIfExistsSafe( dagClient, "appId" );
  3. String dagId = Util.returnInstanceFieldIfExistsSafe( dagClient, "dagId" );
  4. TezConfiguration conf = Util.returnInstanceFieldIfExistsSafe( dagClient, "conf" );
  5. FrameworkClient frameworkClient = Util.returnInstanceFieldIfExistsSafe( dagClient, "frameworkClient" );

代码示例来源：origin: cascading/cascading-hadoop2-tez-stats

  1. };
  2. ApplicationId appId = Util.returnInstanceFieldIfExistsSafe( dagClient, "appId" );
  3. String dagId = Util.returnInstanceFieldIfExistsSafe( dagClient, "dagId" );
  4. TezConfiguration conf = Util.returnInstanceFieldIfExistsSafe( dagClient, "conf" );
  5. FrameworkClient frameworkClient = Util.returnInstanceFieldIfExistsSafe( dagClient, "frameworkClient" );

代码示例来源：origin: cwensel/cascading

  1. public static void setSourcePathForSplit( MRInput input, MRReader reader, Configuration configuration )
  2.  {
  3.  Path path = null;
  4.  if( Util.returnInstanceFieldIfExistsSafe( input, "useNewApi" ) )
  5.   {
  6.   org.apache.hadoop.mapreduce.InputSplit newInputSplit = (org.apache.hadoop.mapreduce.InputSplit) reader.getSplit();
  7.   if( newInputSplit instanceof org.apache.hadoop.mapreduce.lib.input.FileSplit )
  8.    path = ( (org.apache.hadoop.mapreduce.lib.input.FileSplit) newInputSplit ).getPath();
  9.   }
  10.  else
  11.   {
  12.   org.apache.hadoop.mapred.InputSplit oldInputSplit = (org.apache.hadoop.mapred.InputSplit) reader.getSplit();
  13.   if( oldInputSplit instanceof org.apache.hadoop.mapred.FileSplit )
  14.    path = ( (org.apache.hadoop.mapred.FileSplit) oldInputSplit ).getPath();
  15.   }
  16.  if( path != null )
  17.   configuration.set( FileType.CASCADING_SOURCE_PATH, path.toString() );
  18.  }

代码示例来源：origin: cascading/cascading-hadoop2-tez

  1. public static void setSourcePathForSplit( MRInput input, MRReader reader, Configuration configuration )
  2.  {
  3.  Path path = null;
  4.  if( Util.returnInstanceFieldIfExistsSafe( input, "useNewApi" ) )
  5.   {
  6.   org.apache.hadoop.mapreduce.InputSplit newInputSplit = (org.apache.hadoop.mapreduce.InputSplit) reader.getSplit();
  7.   if( newInputSplit instanceof org.apache.hadoop.mapreduce.lib.input.FileSplit )
  8.    path = ( (org.apache.hadoop.mapreduce.lib.input.FileSplit) newInputSplit ).getPath();
  9.   }
  10.  else
  11.   {
  12.   org.apache.hadoop.mapred.InputSplit oldInputSplit = (org.apache.hadoop.mapred.InputSplit) reader.getSplit();
  13.   if( oldInputSplit instanceof org.apache.hadoop.mapred.FileSplit )
  14.    path = ( (org.apache.hadoop.mapred.FileSplit) oldInputSplit ).getPath();
  15.   }
  16.  if( path != null )
  17.   configuration.set( FileType.CASCADING_SOURCE_PATH, path.toString() );
  18.  }

代码示例来源：origin: cascading/cascading-hadoop2-tez

  1. protected void internalNonBlockingStart() throws IOException
  2.  {
  3.  try
  4.   {
  5.   if( !isTimelineServiceEnabled( jobConfiguration ) )
  6.    flowStep.logWarn( "'" + YarnConfiguration.TIMELINE_SERVICE_ENABLED + "' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon" );
  7.   TezConfiguration workingConf = new TezConfiguration( jobConfiguration );
  8.   // this could be problematic
  9.   flowStep.logInfo( "tez session mode enabled: " + workingConf.getBoolean( TezConfiguration.TEZ_AM_SESSION_MODE, TezConfiguration.TEZ_AM_SESSION_MODE_DEFAULT ) );
  10.   prepareEnsureStagingDir( workingConf );
  11.   tezClient = TezClient.create( flowStep.getName(), workingConf, ( (Hadoop2TezFlowStep) flowStep ).getAllLocalResources(), null );
  12.   tezClient.start();
  13.   dagClient = tezClient.submitDAG( dag );
  14.   dagId = Util.returnInstanceFieldIfExistsSafe( dagClient, "dagId" );
  15.   flowStep.logInfo( "submitted tez dag to app master: {}, with dag id: {}", tezClient.getAppMasterApplicationId(), dagId );
  16.   }
  17.  catch( TezException exception )
  18.   {
  19.   this.throwable = exception;
  20.   throw new CascadingException( exception );
  21.   }
  22.  }

代码示例来源：origin: cwensel/cascading

  1. protected void internalNonBlockingStart() throws IOException
  2.  {
  3.  try
  4.   {
  5.   if( !isTimelineServiceEnabled( jobConfiguration ) )
  6.    flowStep.logWarn( "'" + YarnConfiguration.TIMELINE_SERVICE_ENABLED + "' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon" );
  7.   TezConfiguration workingConf = new TezConfiguration( jobConfiguration );
  8.   // this could be problematic
  9.   flowStep.logInfo( "tez session mode enabled: " + workingConf.getBoolean( TezConfiguration.TEZ_AM_SESSION_MODE, TezConfiguration.TEZ_AM_SESSION_MODE_DEFAULT ) );
  10.   prepareEnsureStagingDir( workingConf );
  11.   tezClient = TezClient.create( flowStep.getName(), workingConf, ( (Hadoop2TezFlowStep) flowStep ).getAllLocalResources(), null );
  12.   tezClient.start();
  13.   dagClient = tezClient.submitDAG( dag );
  14.   dagId = Util.returnInstanceFieldIfExistsSafe( dagClient, "dagId" );
  15.   flowStep.logInfo( "submitted tez dag to app master: {}, with dag id: {}", tezClient.getAppMasterApplicationId(), dagId );
  16.   }
  17.  catch( TezException exception )
  18.   {
  19.   this.throwable = exception;
  20.   throw new CascadingException( exception );
  21.   }
  22.  }

内容来源于网络，如有侵权，请联系作者删除！

Util

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com