org.apache.gobblin.source.workunit.Extract.getExtractId()方法的使用及代码示例

x33g5p2x  于2022-01-19 转载在 其他  
字(7.6k)|赞(0)|评价(0)|浏览(149)

本文整理了Java中org.apache.gobblin.source.workunit.Extract.getExtractId()方法的一些代码示例，展示了Extract.getExtractId()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Extract.getExtractId()方法的具体详情如下：
包路径：org.apache.gobblin.source.workunit.Extract
类名称：Extract
方法名：getExtractId

Extract.getExtractId介绍

[英]Get a (non-globally) unique ID for this Extract.
[中]获取此提取的（非全局）唯一ID。

代码示例

代码示例来源：origin: apache/incubator-gobblin

  1. private Extract getExtractForFile(PartitionAwareFileRetriever.FileInfo file,
  2.   String topicName,
  3.   String namespace,
  4.   Map<Long, Extract> extractMap) {
  5.  Extract extract = extractMap.get(file.getWatermarkMsSinceEpoch());
  6.  if (extract == null) {
  7.   // Create an extract object for the dayPath
  8.   extract = new Extract(this.tableType, namespace, topicName);
  9.   LOG.info("Created extract: " + extract.getExtractId() + " for path " + topicName);
  10.   extractMap.put(file.getWatermarkMsSinceEpoch(), extract);
  11.  }
  12.  return extract;
  13. }

代码示例来源：origin: apache/incubator-gobblin

  1. @Override
  2. public int hashCode() {
  3.  return (this.getNamespace() + this.getTable() + this.getExtractId()).hashCode();
  4. }

代码示例来源：origin: apache/incubator-gobblin

  1. @Override
  2. public boolean equals(Object object) {
  3.  if (!(object instanceof Extract)) {
  4.   return false;
  5.  }
  6.  Extract other = (Extract) object;
  7.  return super.equals(other) && this.getNamespace().equals(other.getNamespace())
  8.    && this.getTable().equals(other.getTable()) && this.getExtractId().equals(other.getExtractId());
  9. }

代码示例来源：origin: apache/incubator-gobblin

  1. /**
  2.  * Get the writer output file path corresponding to this {@link Extract}.
  3.  *
  4.  * @return writer output file path corresponding to this {@link Extract}
  5.  * @deprecated As {@code this.getIsFull} is deprecated.
  6.  */
  7. @Deprecated
  8. public String getOutputFilePath() {
  9.  return this.getNamespace().replaceAll("\\.", "/") + "/" + this.getTable() + "/" + this.getExtractId() + "_"
  10.    + (this.getIsFull() ? "full" : "append");
  11. }

代码示例来源：origin: apache/incubator-gobblin

  1. /**
  2.   * Returns a unique {@link Extract} instance.
  3.   * Any two calls of this method from the same {@link ExtractFactory} instance guarantees to
  4.   * return {@link Extract}s with different IDs.
  5.   *
  6.   * @param type {@link TableType}
  7.   * @param namespace dot separated namespace path
  8.   * @param table table name
  9.   * @return a unique {@link Extract} instance
  10.   */
  11.  public synchronized Extract getUniqueExtract(TableType type, String namespace, String table) {
  12.   Extract newExtract = new Extract(type, namespace, table);
  13.   while (this.createdInstances.contains(newExtract)) {
  14.    if (Strings.isNullOrEmpty(newExtract.getExtractId())) {
  15.     newExtract.setExtractId(this.dtf.print(new DateTime()));
  16.    } else {
  17.     DateTime extractDateTime = this.dtf.parseDateTime(newExtract.getExtractId());
  18.     newExtract.setExtractId(this.dtf.print(extractDateTime.plusSeconds(1)));
  19.    }
  20.   }
  21.   this.createdInstances.add(newExtract);
  22.   return newExtract;
  23.  }
  24. }

代码示例来源：origin: apache/incubator-gobblin

  1. /**
  2.  * Create a new properly populated {@link Extract} instance.
  3.  *
  4.  * <p>
  5.  *   This method should always return a new unique {@link Extract} instance.
  6.  * </p>
  7.  *
  8.  * @param type {@link org.apache.gobblin.source.workunit.Extract.TableType}
  9.  * @param namespace namespace of the table this extract belongs to
  10.  * @param table name of the table this extract belongs to
  11.  * @return a new unique {@link Extract} instance
  12.  *
  13.  * @Deprecated Use {@link org.apache.gobblin.source.extractor.extract.AbstractSource#createExtract(
  14.  *org.apache.gobblin.source.workunit.Extract.TableType, String, String)}
  15.  */
  16. @Deprecated
  17. public synchronized Extract createExtract(Extract.TableType type, String namespace, String table) {
  18.  Extract extract = new Extract(this, type, namespace, table);
  19.  while (EXTRACT_SET.contains(extract)) {
  20.   if (Strings.isNullOrEmpty(extract.getExtractId())) {
  21.    extract.setExtractId(DTF.print(new DateTime()));
  22.   } else {
  23.    DateTime extractDateTime = DTF.parseDateTime(extract.getExtractId());
  24.    extract.setExtractId(DTF.print(extractDateTime.plusSeconds(1)));
  25.   }
  26.  }
  27.  EXTRACT_SET.add(extract);
  28.  return extract;
  29. }

代码示例来源：origin: apache/incubator-gobblin

  1. /**
  2.   * Verify that each {@link Extract} created by an {@ExtractFactory} has a unique ID.
  3.   */
  4.  @Test
  5.  public void testGetUniqueExtract() {
  6.   ExtractFactory extractFactory = new ExtractFactory("yyyyMMddHHmmss");
  7.   Set<String> extractIDs = Sets.newHashSet();
  8.   int numOfExtracts = 100;
  9.   for (int i = 0; i < numOfExtracts; i++) {
  10.    extractIDs
  11.      .add(extractFactory.getUniqueExtract(Extract.TableType.APPEND_ONLY, "namespace", "table").getExtractId());
  12.   }
  13.   Assert.assertEquals(extractIDs.size(), numOfExtracts);
  14.  }
  15. }

代码示例来源：origin: org.apache.gobblin/gobblin-api

  1. @Override
  2. public boolean equals(Object object) {
  3.  if (!(object instanceof Extract)) {
  4.   return false;
  5.  }
  6.  Extract other = (Extract) object;
  7.  return super.equals(other) && this.getNamespace().equals(other.getNamespace())
  8.    && this.getTable().equals(other.getTable()) && this.getExtractId().equals(other.getExtractId());
  9. }

代码示例来源：origin: org.apache.gobblin/gobblin-api

  1. @Override
  2. public int hashCode() {
  3.  return (this.getNamespace() + this.getTable() + this.getExtractId()).hashCode();
  4. }

代码示例来源：origin: org.apache.gobblin/gobblin-core

  1. private Extract getExtractForFile(PartitionAwareFileRetriever.FileInfo file,
  2.   String topicName,
  3.   String namespace,
  4.   Map<Long, Extract> extractMap) {
  5.  Extract extract = extractMap.get(file.getWatermarkMsSinceEpoch());
  6.  if (extract == null) {
  7.   // Create an extract object for the dayPath
  8.   extract = new Extract(this.tableType, namespace, topicName);
  9.   LOG.info("Created extract: " + extract.getExtractId() + " for path " + topicName);
  10.   extractMap.put(file.getWatermarkMsSinceEpoch(), extract);
  11.  }
  12.  return extract;
  13. }

代码示例来源：origin: org.apache.gobblin/gobblin-api

  1. /**
  2.  * Get the writer output file path corresponding to this {@link Extract}.
  3.  *
  4.  * @return writer output file path corresponding to this {@link Extract}
  5.  * @deprecated As {@code this.getIsFull} is deprecated.
  6.  */
  7. @Deprecated
  8. public String getOutputFilePath() {
  9.  return this.getNamespace().replaceAll("\\.", "/") + "/" + this.getTable() + "/" + this.getExtractId() + "_"
  10.    + (this.getIsFull() ? "full" : "append");
  11. }

代码示例来源：origin: org.apache.gobblin/gobblin-api

  1. /**
  2.   * Returns a unique {@link Extract} instance.
  3.   * Any two calls of this method from the same {@link ExtractFactory} instance guarantees to
  4.   * return {@link Extract}s with different IDs.
  5.   *
  6.   * @param type {@link TableType}
  7.   * @param namespace dot separated namespace path
  8.   * @param table table name
  9.   * @return a unique {@link Extract} instance
  10.   */
  11.  public synchronized Extract getUniqueExtract(TableType type, String namespace, String table) {
  12.   Extract newExtract = new Extract(type, namespace, table);
  13.   while (this.createdInstances.contains(newExtract)) {
  14.    if (Strings.isNullOrEmpty(newExtract.getExtractId())) {
  15.     newExtract.setExtractId(this.dtf.print(new DateTime()));
  16.    } else {
  17.     DateTime extractDateTime = this.dtf.parseDateTime(newExtract.getExtractId());
  18.     newExtract.setExtractId(this.dtf.print(extractDateTime.plusSeconds(1)));
  19.    }
  20.   }
  21.   this.createdInstances.add(newExtract);
  22.   return newExtract;
  23.  }
  24. }

代码示例来源：origin: org.apache.gobblin/gobblin-api

  1. /**
  2.  * Create a new properly populated {@link Extract} instance.
  3.  *
  4.  * <p>
  5.  *   This method should always return a new unique {@link Extract} instance.
  6.  * </p>
  7.  *
  8.  * @param type {@link org.apache.gobblin.source.workunit.Extract.TableType}
  9.  * @param namespace namespace of the table this extract belongs to
  10.  * @param table name of the table this extract belongs to
  11.  * @return a new unique {@link Extract} instance
  12.  *
  13.  * @Deprecated Use {@link org.apache.gobblin.source.extractor.extract.AbstractSource#createExtract(
  14.  *org.apache.gobblin.source.workunit.Extract.TableType, String, String)}
  15.  */
  16. @Deprecated
  17. public synchronized Extract createExtract(Extract.TableType type, String namespace, String table) {
  18.  Extract extract = new Extract(this, type, namespace, table);
  19.  while (EXTRACT_SET.contains(extract)) {
  20.   if (Strings.isNullOrEmpty(extract.getExtractId())) {
  21.    extract.setExtractId(DTF.print(new DateTime()));
  22.   } else {
  23.    DateTime extractDateTime = DTF.parseDateTime(extract.getExtractId());
  24.    extract.setExtractId(DTF.print(extractDateTime.plusSeconds(1)));
  25.   }
  26.  }
  27.  EXTRACT_SET.add(extract);
  28.  return extract;
  29. }

内容来源于网络，如有侵权，请联系作者删除！

Extract

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com