org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader()方法的使用及代码示例

x33g5p2x  于2022-01-20 转载在 其他  
字(5.9k)|赞(0)|评价(0)|浏览(167)

本文整理了Java中org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader()方法的一些代码示例,展示了HStore.createStoreFileAndReader()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。HStore.createStoreFileAndReader()方法的具体详情如下:
包路径:org.apache.hadoop.hbase.regionserver.HStore
类名称:HStore
方法名:createStoreFileAndReader

HStore.createStoreFileAndReader介绍

暂无

代码示例

代码示例来源:origin: apache/hbase

  1. public void bulkLoadHFile(StoreFileInfo fileInfo) throws IOException {
  2. HStoreFile sf = createStoreFileAndReader(fileInfo);
  3. bulkLoadHFile(sf);
  4. }

代码示例来源:origin: apache/hbase

  1. /**
  2. * Validates a store file by opening and closing it. In HFileV2 this should not be an expensive
  3. * operation.
  4. * @param path the path to the store file
  5. */
  6. private void validateStoreFile(Path path) throws IOException {
  7. HStoreFile storeFile = null;
  8. try {
  9. storeFile = createStoreFileAndReader(path);
  10. } catch (IOException e) {
  11. LOG.error("Failed to open store file : {}, keeping it in tmp location", path, e);
  12. throw e;
  13. } finally {
  14. if (storeFile != null) {
  15. storeFile.closeStoreFile(false);
  16. }
  17. }
  18. }

代码示例来源:origin: apache/hbase

  1. @VisibleForTesting
  2. protected HStoreFile createStoreFileAndReader(final Path p) throws IOException {
  3. StoreFileInfo info = new StoreFileInfo(conf, this.getFileSystem(), p);
  4. return createStoreFileAndReader(info);
  5. }

代码示例来源:origin: apache/hbase

  1. HStoreFile moveFileIntoPlace(Path newFile) throws IOException {
  2. validateStoreFile(newFile);
  3. // Move the file into the right spot
  4. Path destPath = fs.commitStoreFile(getColumnFamilyName(), newFile);
  5. return createStoreFileAndReader(destPath);
  6. }

代码示例来源:origin: apache/hbase

  1. public Path bulkLoadHFile(byte[] family, String srcPathStr, Path dstPath) throws IOException {
  2. Path srcPath = new Path(srcPathStr);
  3. try {
  4. fs.commitStoreFile(srcPath, dstPath);
  5. } finally {
  6. if (this.getCoprocessorHost() != null) {
  7. this.getCoprocessorHost().postCommitStoreFile(family, srcPath, dstPath);
  8. }
  9. }
  10. LOG.info("Loaded HFile " + srcPath + " into store '" + getColumnFamilyName() + "' as "
  11. + dstPath + " - updating store file list.");
  12. HStoreFile sf = createStoreFileAndReader(dstPath);
  13. bulkLoadHFile(sf);
  14. LOG.info("Successfully loaded store file {} into store {} (new location: {})",
  15. srcPath, this, dstPath);
  16. return dstPath;
  17. }

代码示例来源:origin: apache/hbase

  1. for (StoreFileInfo storeFileInfo : files) {
  2. completionService.submit(() -> this.createStoreFileAndReader(storeFileInfo));
  3. totalValidStoreFile++;

代码示例来源:origin: apache/hbase

  1. HStoreFile storeFile = createStoreFileAndReader(storeFileInfo);
  2. outputStoreFiles.add(storeFile);

代码示例来源:origin: apache/hbase

  1. /**
  2. * @param path The pathname of the tmp file into which the store was flushed
  3. * @param logCacheFlushId
  4. * @param status
  5. * @return store file created.
  6. * @throws IOException
  7. */
  8. private HStoreFile commitFile(Path path, long logCacheFlushId, MonitoredTask status)
  9. throws IOException {
  10. // Write-out finished successfully, move into the right spot
  11. Path dstPath = fs.commitStoreFile(getColumnFamilyName(), path);
  12. status.setStatus("Flushing " + this + ": reopening flushed file");
  13. HStoreFile sf = createStoreFileAndReader(dstPath);
  14. StoreFileReader r = sf.getReader();
  15. this.storeSize.addAndGet(r.length());
  16. this.totalUncompressedBytes.addAndGet(r.getTotalUncompressedBytes());
  17. if (LOG.isInfoEnabled()) {
  18. LOG.info("Added " + sf + ", entries=" + r.getEntries() +
  19. ", sequenceid=" + logCacheFlushId +
  20. ", filesize=" + TraditionalBinaryPrefix.long2String(r.length(), "", 1));
  21. }
  22. return sf;
  23. }

代码示例来源:origin: harbby/presto-connectors

  1. @Override
  2. public void bulkLoadHFile(StoreFileInfo fileInfo) throws IOException {
  3. StoreFile sf = createStoreFileAndReader(fileInfo);
  4. bulkLoadHFile(sf);
  5. }

代码示例来源:origin: harbby/presto-connectors

  1. private StoreFile createStoreFileAndReader(final Path p) throws IOException {
  2. StoreFileInfo info = new StoreFileInfo(conf, this.getFileSystem(), p);
  3. return createStoreFileAndReader(info);
  4. }

代码示例来源:origin: harbby/presto-connectors

  1. /**
  2. * Validates a store file by opening and closing it. In HFileV2 this should
  3. * not be an expensive operation.
  4. *
  5. * @param path the path to the store file
  6. */
  7. private void validateStoreFile(Path path)
  8. throws IOException {
  9. StoreFile storeFile = null;
  10. try {
  11. storeFile = createStoreFileAndReader(path);
  12. } catch (IOException e) {
  13. LOG.error("Failed to open store file : " + path
  14. + ", keeping it in tmp location", e);
  15. throw e;
  16. } finally {
  17. if (storeFile != null) {
  18. storeFile.closeReader(false);
  19. }
  20. }
  21. }

代码示例来源:origin: harbby/presto-connectors

  1. StoreFile moveFileIntoPlace(final Path newFile) throws IOException {
  2. validateStoreFile(newFile);
  3. // Move the file into the right spot
  4. Path destPath = fs.commitStoreFile(getColumnFamilyName(), newFile);
  5. return createStoreFileAndReader(destPath);
  6. }

代码示例来源:origin: harbby/presto-connectors

  1. @Override
  2. public Path bulkLoadHFile(String srcPathStr, long seqNum) throws IOException {
  3. Path srcPath = new Path(srcPathStr);
  4. Path dstPath = fs.bulkLoadStoreFile(getColumnFamilyName(), srcPath, seqNum);
  5. LOG.info("Loaded HFile " + srcPath + " into store '" + getColumnFamilyName() + "' as "
  6. + dstPath + " - updating store file list.");
  7. StoreFile sf = createStoreFileAndReader(dstPath);
  8. bulkLoadHFile(sf);
  9. LOG.info("Successfully loaded store file " + srcPath + " into store " + this
  10. + " (new location: " + dstPath + ")");
  11. return dstPath;
  12. }

代码示例来源:origin: harbby/presto-connectors

  1. StoreFile storeFile = createStoreFileAndReader(storeFileInfo);
  2. outputStoreFiles.add(storeFile);

代码示例来源:origin: harbby/presto-connectors

  1. for (Path newFile : newFiles) {
  2. StoreFile sf = createStoreFileAndReader(newFile);
  3. sf.closeReader(evictOnClose);
  4. sfs.add(sf);

代码示例来源:origin: harbby/presto-connectors

  1. private StoreFile commitFile(final Path path, final long logCacheFlushId, MonitoredTask status)
  2. throws IOException {
  3. // Write-out finished successfully, move into the right spot
  4. Path dstPath = fs.commitStoreFile(getColumnFamilyName(), path);
  5. status.setStatus("Flushing " + this + ": reopening flushed file");
  6. StoreFile sf = createStoreFileAndReader(dstPath);
  7. StoreFile.Reader r = sf.getReader();
  8. this.storeSize += r.length();
  9. this.totalUncompressedBytes += r.getTotalUncompressedBytes();
  10. if (LOG.isInfoEnabled()) {
  11. LOG.info("Added " + sf + ", entries=" + r.getEntries() +
  12. ", sequenceid=" + logCacheFlushId +
  13. ", filesize=" + TraditionalBinaryPrefix.long2String(r.length(), "", 1));
  14. }
  15. return sf;
  16. }

相关文章

HStore类方法