org.apache.hadoop.hbase.regionserver.HStore.getColumnFamilyName()方法的使用及代码示例

x33g5p2x  于2022-01-20 转载在 其他  
字(13.2k)|赞(0)|评价(0)|浏览(196)

本文整理了Java中org.apache.hadoop.hbase.regionserver.HStore.getColumnFamilyName()方法的一些代码示例，展示了HStore.getColumnFamilyName()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。HStore.getColumnFamilyName()方法的具体详情如下：
包路径：org.apache.hadoop.hbase.regionserver.HStore
类名称：HStore
方法名：getColumnFamilyName

HStore.getColumnFamilyName介绍

暂无

代码示例

代码示例来源：origin: apache/hbase

  1. @Override
  2. public String toString() {
  3.  return this.getColumnFamilyName();
  4. }

代码示例来源：origin: apache/hbase

  1. /**
  2.  * This method should only be called from Region. It is assumed that the ranges of values in the
  3.  * HFile fit within the stores assigned region. (assertBulkLoadHFileOk checks this)
  4.  *
  5.  * @param srcPathStr
  6.  * @param seqNum sequence Id associated with the HFile
  7.  */
  8. public Pair<Path, Path> preBulkLoadHFile(String srcPathStr, long seqNum) throws IOException {
  9.  Path srcPath = new Path(srcPathStr);
  10.  return fs.bulkLoadStoreFile(getColumnFamilyName(), srcPath, seqNum);
  11. }

代码示例来源：origin: apache/hbase

  1. /**
  2.  * Replaces the store files that the store has with the given files. Mainly used by secondary
  3.  * region replicas to keep up to date with the primary region files.
  4.  * @throws IOException
  5.  */
  6. public void refreshStoreFiles(Collection<String> newFiles) throws IOException {
  7.  List<StoreFileInfo> storeFiles = new ArrayList<>(newFiles.size());
  8.  for (String file : newFiles) {
  9.   storeFiles.add(fs.getStoreFileInfo(getColumnFamilyName(), file));
  10.  }
  11.  refreshStoreFilesInternal(storeFiles);
  12. }

代码示例来源：origin: apache/hbase

  1. /**
  2.  * Creates an unsorted list of StoreFile loaded in parallel
  3.  * from the given directory.
  4.  * @throws IOException
  5.  */
  6. private List<HStoreFile> loadStoreFiles() throws IOException {
  7.  Collection<StoreFileInfo> files = fs.getStoreFiles(getColumnFamilyName());
  8.  return openStoreFiles(files);
  9. }

代码示例来源：origin: apache/hbase

  1. @Override
  2. public void refreshStoreFiles() throws IOException {
  3.  Collection<StoreFileInfo> newFiles = fs.getStoreFiles(getColumnFamilyName());
  4.  refreshStoreFilesInternal(newFiles);
  5. }

代码示例来源：origin: apache/hbase

  1. /**
  2.  * Adds a value to the memstore
  3.  */
  4. public void add(final Cell cell, MemStoreSizing memstoreSizing) {
  5.  lock.readLock().lock();
  6.  try {
  7.   if (this.currentParallelPutCount.getAndIncrement() > this.parallelPutCountPrintThreshold) {
  8.    LOG.trace(this.getTableName() + "tableName={}, encodedName={}, columnFamilyName={} is " +
  9.     "too busy!", this.getRegionInfo().getEncodedName(), this .getColumnFamilyName());
  10.   }
  11.   this.memstore.add(cell, memstoreSizing);
  12.  } finally {
  13.   lock.readLock().unlock();
  14.   currentParallelPutCount.decrementAndGet();
  15.  }
  16. }

代码示例来源：origin: apache/hbase

  1. /**
  2.  * Adds the specified value to the memstore
  3.  */
  4. public void add(final Iterable<Cell> cells, MemStoreSizing memstoreSizing) {
  5.  lock.readLock().lock();
  6.  try {
  7.   if (this.currentParallelPutCount.getAndIncrement() > this.parallelPutCountPrintThreshold) {
  8.    LOG.trace(this.getTableName() + "tableName={}, encodedName={}, columnFamilyName={} is " +
  9.      "too busy!", this.getRegionInfo().getEncodedName(), this .getColumnFamilyName());
  10.   }
  11.   memstore.add(cells, memstoreSizing);
  12.  } finally {
  13.   lock.readLock().unlock();
  14.   currentParallelPutCount.decrementAndGet();
  15.  }
  16. }

代码示例来源：origin: apache/hbase

  1. HStoreFile moveFileIntoPlace(Path newFile) throws IOException {
  2.  validateStoreFile(newFile);
  3.  // Move the file into the right spot
  4.  Path destPath = fs.commitStoreFile(getColumnFamilyName(), newFile);
  5.  return createStoreFileAndReader(destPath);
  6. }

代码示例来源：origin: apache/hbase

  1. public DefaultMobStoreFlusher(Configuration conf, HStore store) throws IOException {
  2.  super(conf, store);
  3.  if (!(store instanceof HMobStore)) {
  4.   throw new IllegalArgumentException("The store " + store + " is not a HMobStore");
  5.  }
  6.  mobCellValueSizeThreshold = store.getColumnFamilyDescriptor().getMobThreshold();
  7.  this.targetPath = MobUtils.getMobFamilyPath(conf, store.getTableName(),
  8.    store.getColumnFamilyName());
  9.  if (!this.store.getFileSystem().exists(targetPath)) {
  10.   this.store.getFileSystem().mkdirs(targetPath);
  11.  }
  12.  this.mobStore = (HMobStore) store;
  13. }

代码示例来源：origin: apache/hbase

  1. /**
  2.  * Be careful, this method will drop all data in the memstore of this region.
  3.  * Currently, this method is used to drop memstore to prevent memory leak
  4.  * when replaying recovered.edits while opening region.
  5.  */
  6. public MemStoreSize dropMemStoreContents() throws IOException {
  7.  MemStoreSizing totalFreedSize = new NonThreadSafeMemStoreSizing();
  8.  this.updatesLock.writeLock().lock();
  9.  try {
  10.   for (HStore s : stores.values()) {
  11.    MemStoreSize memStoreSize = doDropStoreMemStoreContentsForSeqId(s, HConstants.NO_SEQNUM);
  12.    LOG.info("Drop memstore for Store " + s.getColumnFamilyName() + " in region "
  13.        + this.getRegionInfo().getRegionNameAsString()
  14.        + " , dropped memstoresize: [" + memStoreSize + " }");
  15.    totalFreedSize.incMemStoreSize(memStoreSize);
  16.   }
  17.   return totalFreedSize.getMemStoreSize();
  18.  } finally {
  19.   this.updatesLock.writeLock().unlock();
  20.  }
  21. }

代码示例来源：origin: apache/hbase

  1. public Path bulkLoadHFile(byte[] family, String srcPathStr, Path dstPath) throws IOException {
  2.  Path srcPath = new Path(srcPathStr);
  3.  try {
  4.   fs.commitStoreFile(srcPath, dstPath);
  5.  } finally {
  6.   if (this.getCoprocessorHost() != null) {
  7.    this.getCoprocessorHost().postCommitStoreFile(family, srcPath, dstPath);
  8.   }
  9.  }
  10.  LOG.info("Loaded HFile " + srcPath + " into store '" + getColumnFamilyName() + "' as "
  11.    + dstPath + " - updating store file list.");
  12.  HStoreFile sf = createStoreFileAndReader(dstPath);
  13.  bulkLoadHFile(sf);
  14.  LOG.info("Successfully loaded store file {} into store {} (new location: {})",
  15.    srcPath, this, dstPath);
  16.  return dstPath;
  17. }

代码示例来源：origin: apache/hbase

  1. /**
  2.  * Utility method broken out of internalPrepareFlushCache so that method is smaller.
  3.  */
  4. private void logFatLineOnFlush(Collection<HStore> storesToFlush, long sequenceId) {
  5.  if (!LOG.isInfoEnabled()) {
  6.   return;
  7.  }
  8.  // Log a fat line detailing what is being flushed.
  9.  StringBuilder perCfExtras = null;
  10.  if (!isAllFamilies(storesToFlush)) {
  11.   perCfExtras = new StringBuilder();
  12.   for (HStore store: storesToFlush) {
  13.    MemStoreSize mss = store.getFlushableSize();
  14.    perCfExtras.append("; ").append(store.getColumnFamilyName());
  15.    perCfExtras.append("={dataSize=")
  16.      .append(StringUtils.byteDesc(mss.getDataSize()));
  17.    perCfExtras.append(", heapSize=")
  18.      .append(StringUtils.byteDesc(mss.getHeapSize()));
  19.    perCfExtras.append(", offHeapSize=")
  20.      .append(StringUtils.byteDesc(mss.getOffHeapSize()));
  21.    perCfExtras.append("}");
  22.   }
  23.  }
  24.  MemStoreSize mss = this.memStoreSizing.getMemStoreSize();
  25.  LOG.info("Flushing " + + storesToFlush.size() + "/" + stores.size() + " column families," +
  26.    " dataSize=" + StringUtils.byteDesc(mss.getDataSize()) +
  27.    " heapSize=" + StringUtils.byteDesc(mss.getHeapSize()) +
  28.    ((perCfExtras != null && perCfExtras.length() > 0)? perCfExtras.toString(): "") +
  29.    ((wal != null) ? "" : "; WAL is null, using passed sequenceid=" + sequenceId));
  30. }

代码示例来源：origin: apache/hbase

  1. private void bulkLoadHFile(HStoreFile sf) throws IOException {
  2.  StoreFileReader r = sf.getReader();
  3.  this.storeSize.addAndGet(r.length());
  4.  this.totalUncompressedBytes.addAndGet(r.getTotalUncompressedBytes());
  5.  // Append the new storefile into the list
  6.  this.lock.writeLock().lock();
  7.  try {
  8.   this.storeEngine.getStoreFileManager().insertNewFiles(Lists.newArrayList(sf));
  9.  } finally {
  10.   // We need the lock, as long as we are updating the storeFiles
  11.   // or changing the memstore. Let us release it before calling
  12.   // notifyChangeReadersObservers. See HBASE-4485 for a possible
  13.   // deadlock scenario that could have happened if continue to hold
  14.   // the lock.
  15.   this.lock.writeLock().unlock();
  16.  }
  17.  LOG.info("Loaded HFile " + sf.getFileInfo() + " into store '" + getColumnFamilyName());
  18.  if (LOG.isTraceEnabled()) {
  19.   String traceMessage = "BULK LOAD time,size,store size,store files ["
  20.     + EnvironmentEdgeManager.currentTime() + "," + r.length() + "," + storeSize
  21.     + "," + storeEngine.getStoreFileManager().getStorefileCount() + "]";
  22.   LOG.trace(traceMessage);
  23.  }
  24. }

代码示例来源：origin: apache/hbase

  1. public CompactingMemStore(Configuration conf, CellComparator c,
  2.   HStore store, RegionServicesForStores regionServices,
  3.   MemoryCompactionPolicy compactionPolicy) throws IOException {
  4.  super(conf, c, regionServices);
  5.  this.store = store;
  6.  this.regionServices = regionServices;
  7.  this.pipeline = new CompactionPipeline(getRegionServices());
  8.  this.compactor = createMemStoreCompactor(compactionPolicy);
  9.  if (conf.getBoolean(MemStoreLAB.USEMSLAB_KEY, MemStoreLAB.USEMSLAB_DEFAULT)) {
  10.   // if user requested to work with MSLABs (whether on- or off-heap), then the
  11.   // immutable segments are going to use CellChunkMap as their index
  12.   indexType = IndexType.CHUNK_MAP;
  13.  } else {
  14.   indexType = IndexType.ARRAY_MAP;
  15.  }
  16.  // initialization of the flush size should happen after initialization of the index type
  17.  // so do not transfer the following method
  18.  initInmemoryFlushSize(conf);
  19.  LOG.info("Store={}, in-memory flush size threshold={}, immutable segments index type={}, " +
  20.      "compactor={}", this.store.getColumnFamilyName(),
  21.    StringUtils.byteDesc(this.inmemoryFlushSize), this.indexType,
  22.    (this.compactor == null? "NULL": this.compactor.toString()));
  23. }

代码示例来源：origin: apache/hbase

  1. /**
  2.  * @param path The pathname of the tmp file into which the store was flushed
  3.  * @param logCacheFlushId
  4.  * @param status
  5.  * @return store file created.
  6.  * @throws IOException
  7.  */
  8. private HStoreFile commitFile(Path path, long logCacheFlushId, MonitoredTask status)
  9.   throws IOException {
  10.  // Write-out finished successfully, move into the right spot
  11.  Path dstPath = fs.commitStoreFile(getColumnFamilyName(), path);
  12.  status.setStatus("Flushing " + this + ": reopening flushed file");
  13.  HStoreFile sf = createStoreFileAndReader(dstPath);
  14.  StoreFileReader r = sf.getReader();
  15.  this.storeSize.addAndGet(r.length());
  16.  this.totalUncompressedBytes.addAndGet(r.getTotalUncompressedBytes());
  17.  if (LOG.isInfoEnabled()) {
  18.   LOG.info("Added " + sf + ", entries=" + r.getEntries() +
  19.    ", sequenceid=" + logCacheFlushId +
  20.    ", filesize=" + TraditionalBinaryPrefix.long2String(r.length(), "", 1));
  21.  }
  22.  return sf;
  23. }

代码示例来源：origin: apache/hbase

  1. private void archiveStoreFile(int index) throws IOException {
  2.  Collection<HStoreFile> files = this.store.getStorefiles();
  3.  HStoreFile sf = null;
  4.  Iterator<HStoreFile> it = files.iterator();
  5.  for (int i = 0; i <= index; i++) {
  6.   sf = it.next();
  7.  }
  8.  store.getRegionFileSystem().removeStoreFiles(store.getColumnFamilyName(), Lists.newArrayList(sf));
  9. }

代码示例来源：origin: apache/hbase

  1. @Override
  2. protected boolean shouldSplit() {
  3.  boolean force = region.shouldForceSplit();
  4.  boolean foundABigStore = false;
  5.  // Get count of regions that have the same common table as this.region
  6.  int tableRegionsCount = getCountOfCommonTableRegions();
  7.  // Get size to check
  8.  long sizeToCheck = getSizeToCheck(tableRegionsCount);
  9.  for (HStore store : region.getStores()) {
  10.   // If any of the stores is unable to split (eg they contain reference files)
  11.   // then don't split
  12.   if (!store.canSplit()) {
  13.    return false;
  14.   }
  15.   // Mark if any store is big enough
  16.   long size = store.getSize();
  17.   if (size > sizeToCheck) {
  18.    LOG.debug("ShouldSplit because " + store.getColumnFamilyName() +
  19.     " size=" + StringUtils.humanSize(size) +
  20.     ", sizeToCheck=" + StringUtils.humanSize(sizeToCheck) +
  21.     ", regionsWithCommonTable=" + tableRegionsCount);
  22.    foundABigStore = true;
  23.   }
  24.  }
  25.  return foundABigStore || force;
  26. }

代码示例来源：origin: apache/hbase

  1. @Test
  2. public void testSkipRecoveredEditsReplayAllIgnored() throws Exception {
  3.  byte[] family = Bytes.toBytes("family");
  4.  this.region = initHRegion(tableName, method, CONF, family);
  5.  Path regiondir = region.getRegionFileSystem().getRegionDir();
  6.  FileSystem fs = region.getRegionFileSystem().getFileSystem();
  7.  Path recoveredEditsDir = WALSplitter.getRegionDirRecoveredEditsDir(regiondir);
  8.  for (int i = 1000; i < 1050; i += 10) {
  9.   Path recoveredEdits = new Path(recoveredEditsDir, String.format("%019d", i));
  10.   FSDataOutputStream dos = fs.create(recoveredEdits);
  11.   dos.writeInt(i);
  12.   dos.close();
  13.  }
  14.  long minSeqId = 2000;
  15.  Path recoveredEdits = new Path(recoveredEditsDir, String.format("%019d", minSeqId - 1));
  16.  FSDataOutputStream dos = fs.create(recoveredEdits);
  17.  dos.close();
  18.  Map<byte[], Long> maxSeqIdInStores = new TreeMap<>(Bytes.BYTES_COMPARATOR);
  19.  for (HStore store : region.getStores()) {
  20.   maxSeqIdInStores.put(Bytes.toBytes(store.getColumnFamilyName()), minSeqId);
  21.  }
  22.  long seqId = region.replayRecoveredEditsIfAny(maxSeqIdInStores, null, null);
  23.  assertEquals(minSeqId, seqId);
  24. }

代码示例来源：origin: apache/hbase

  1. protected boolean shouldFlush(HStore store) {
  2.   if (store.getMemStoreSize().getHeapSize()
  3.     + store.getMemStoreSize().getOffHeapSize() > this.flushSizeLowerBound) {
  4.    LOG.debug("Flush {} of {}; "
  5.        + "heap memstoreSize={} +"
  6.        + "off heap memstoreSize={} > memstore lowerBound={}"
  7.      , store.getColumnFamilyName()
  8.      , region.getRegionInfo().getEncodedName()
  9.      , store.getMemStoreSize().getHeapSize()
  10.      , store.getMemStoreSize().getOffHeapSize()
  11.      , this.flushSizeLowerBound
  12.    );
  13.    return true;
  14.   }
  15.   return false;
  16.  }
  17. }

代码示例来源：origin: apache/hbase

  1. @Override
  2.  public Object run() throws Exception {
  3.   // Make sure it worked (above is sensitive to caching details in hadoop core)
  4.   FileSystem fs = FileSystem.get(conf);
  5.   assertEquals(FaultyFileSystem.class, fs.getClass());
  6.   // Initialize region
  7.   init(name.getMethodName(), conf);
  8.   LOG.info("Adding some data");
  9.   store.add(new KeyValue(row, family, qf1, 1, (byte[])null), null);
  10.   store.add(new KeyValue(row, family, qf2, 1, (byte[])null), null);
  11.   store.add(new KeyValue(row, family, qf3, 1, (byte[])null), null);
  12.   LOG.info("Before flush, we should have no files");
  13.   Collection<StoreFileInfo> files =
  14.    store.getRegionFileSystem().getStoreFiles(store.getColumnFamilyName());
  15.   assertEquals(0, files != null ? files.size() : 0);
  16.   //flush
  17.   try {
  18.    LOG.info("Flushing");
  19.    flush(1);
  20.    fail("Didn't bubble up IOE!");
  21.   } catch (IOException ioe) {
  22.    assertTrue(ioe.getMessage().contains("Fault injected"));
  23.   }
  24.   LOG.info("After failed flush, we should still have no files!");
  25.   files = store.getRegionFileSystem().getStoreFiles(store.getColumnFamilyName());
  26.   assertEquals(0, files != null ? files.size() : 0);
  27.   store.getHRegion().getWAL().close();
  28.   return null;
  29.  }
  30. });

内容来源于网络，如有侵权，请联系作者删除！

HStore

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多

    HStore类方法

    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com