org.rocksdb.Options.setWriteBufferSize()方法的使用及代码示例

x33g5p2x  于2022-01-25 转载在 其他  
字(8.7k)|赞(0)|评价(0)|浏览(213)

本文整理了Java中org.rocksdb.Options.setWriteBufferSize()方法的一些代码示例，展示了Options.setWriteBufferSize()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Options.setWriteBufferSize()方法的具体详情如下：
包路径：org.rocksdb.Options
类名称：Options
方法名：setWriteBufferSize

Options.setWriteBufferSize介绍

暂无

代码示例

代码示例来源：origin: alibaba/jstorm

public RocksDbCacheOperator(TopologyContext context, String cacheDir) {
  this.stormConf = context.getStormConf();
  this.maxFlushSize = ConfigExtension.getTransactionCacheBatchFlushSize(stormConf);
  Options rocksDbOpt = new Options();
  rocksDbOpt.setCreateMissingColumnFamilies(true).setCreateIfMissing(true);
  long bufferSize =
      ConfigExtension.getTransactionCacheBlockSize(stormConf) != null ? ConfigExtension.getTransactionCacheBlockSize(stormConf) : (1 * SizeUnit.GB);
  rocksDbOpt.setWriteBufferSize(bufferSize);
  int maxBufferNum = ConfigExtension.getTransactionMaxCacheBlockNum(stormConf) != null ? ConfigExtension.getTransactionMaxCacheBlockNum(stormConf) : 3;
  rocksDbOpt.setMaxWriteBufferNumber(maxBufferNum);
  // Config for log of RocksDb
  rocksDbOpt.setMaxLogFileSize(1073741824); // 1G
  rocksDbOpt.setKeepLogFileNum(1);
  rocksDbOpt.setInfoLogLevel(InfoLogLevel.WARN_LEVEL);
  
  try {
    Map<Object, Object> conf = new HashMap<Object, Object>();
    conf.put(ROCKSDB_ROOT_DIR, cacheDir);
    conf.put(ROCKSDB_RESET, true);
    initDir(conf);
    initDb(null, rocksDbOpt);
  } catch (Exception e) {
    throw new RuntimeException(e);
  }
  kryo = new Kryo();
  output = new Output(200, 2000000000);
  input = new Input(1);
  LOG.info("Finished rocksDb cache init: maxFlushSize={}, bufferSize={}, maxBufferNum={}", maxFlushSize, bufferSize, maxBufferNum);
}

代码示例来源：origin: apache/kylin

public RocksDBLookupBuilder(TableDesc tableDesc, String[] keyColumns, String dbPath) {
  this.tableDesc = tableDesc;
  this.encoder = new RocksDBLookupRowEncoder(tableDesc, keyColumns);
  this.dbPath = dbPath;
  this.writeBatchSize = 500;
  this.options = new Options();
  options.setCreateIfMissing(true).setWriteBufferSize(8 * SizeUnit.KB).setMaxWriteBufferNumber(3)
      .setMaxBackgroundCompactions(5).setCompressionType(CompressionType.SNAPPY_COMPRESSION)
      .setCompactionStyle(CompactionStyle.UNIVERSAL);
}

代码示例来源：origin: ltsopensource/light-task-scheduler

@Override
protected void init() throws FailStoreException {
  try {
    options = new Options();
    options.setCreateIfMissing(true)
        .setWriteBufferSize(8 * SizeUnit.KB)
        .setMaxWriteBufferNumber(3)
        .setMaxBackgroundCompactions(10)
        .setCompressionType(CompressionType.SNAPPY_COMPRESSION)
        .setCompactionStyle(CompactionStyle.UNIVERSAL);
    Filter bloomFilter = new BloomFilter(10);
    BlockBasedTableConfig tableConfig = new BlockBasedTableConfig();
    tableConfig.setBlockCacheSize(64 * SizeUnit.KB)
        .setFilter(bloomFilter)
        .setCacheNumShardBits(6)
        .setBlockSizeDeviation(5)
        .setBlockRestartInterval(10)
        .setCacheIndexAndFilterBlocks(true)
        .setHashIndexAllowCollision(false)
        .setBlockCacheCompressedSize(64 * SizeUnit.KB)
        .setBlockCacheCompressedNumShardBits(10);
    options.setTableFormatConfig(tableConfig);
  } catch (Exception e) {
    throw new FailStoreException(e);
  }
}

代码示例来源：origin: ltsopensource/light-task-scheduler

@Override
protected void init() throws FailStoreException {
  try {
    options = new Options();
    options.setCreateIfMissing(true)
        .setWriteBufferSize(8 * SizeUnit.KB)
        .setMaxWriteBufferNumber(3)
        .setMaxBackgroundCompactions(10)
        .setCompressionType(CompressionType.SNAPPY_COMPRESSION)
        .setCompactionStyle(CompactionStyle.UNIVERSAL);
    Filter bloomFilter = new BloomFilter(10);
    BlockBasedTableConfig tableConfig = new BlockBasedTableConfig();
    tableConfig.setBlockCacheSize(64 * SizeUnit.KB)
        .setFilter(bloomFilter)
        .setCacheNumShardBits(6)
        .setBlockSizeDeviation(5)
        .setBlockRestartInterval(10)
        .setCacheIndexAndFilterBlocks(true)
        .setHashIndexAllowCollision(false)
        .setBlockCacheCompressedSize(64 * SizeUnit.KB)
        .setBlockCacheCompressedNumShardBits(10);
    options.setTableFormatConfig(tableConfig);
  } catch (Exception e) {
    throw new FailStoreException(e);
  }
}

代码示例来源：origin: alibaba/jstorm

currentOptions.setWriteBufferSize(64 * SizeUnit.MB);

代码示例来源：origin: org.rocksdb/rocksdbjni

@Override
public Options setWriteBufferSize(final long writeBufferSize) {
 assert(isOwningHandle());
 setWriteBufferSize(nativeHandle_, writeBufferSize);
 return this;
}

代码示例来源：origin: org.locationtech.geogig/geogig-core

.setWriteBufferSize(16 * 1024 * 1024)//
.setMaxWriteBufferNumber(8)//
.setMaxBackgroundCompactions(2)//

代码示例来源：origin: com.alibaba.jstorm/jstorm-core

long bufferSize = ConfigExtension.getTransactionCacheBlockSize(stormConf) != null ? 
    ConfigExtension.getTransactionCacheBlockSize(stormConf) : (1 * SizeUnit.GB);
rocksDbOpt.setWriteBufferSize(bufferSize);
int maxBufferNum = ConfigExtension.getTransactionMaxCacheBlockNum(stormConf) != null ?
    ConfigExtension.getTransactionMaxCacheBlockNum(stormConf) : 3;

代码示例来源：origin: pravega/pravega

private Options createDatabaseOptions() {
  BlockBasedTableConfig tableFormatConfig = new BlockBasedTableConfig()
      .setBlockSize(cacheBlockSizeKB * 1024L)
      .setBlockCacheSize(readCacheSizeMB * 1024L * 1024L)
      .setCacheIndexAndFilterBlocks(true);
  return new Options()
      .setCreateIfMissing(true)
      .setDbLogDir(Paths.get(this.dbDir, DB_LOG_DIR).toString())
      .setWalDir(Paths.get(this.dbDir, DB_WRITE_AHEAD_LOG_DIR).toString())
      .setWalTtlSeconds(0)
      .setWalSizeLimitMB(MAX_WRITE_AHEAD_LOG_SIZE_MB)
      .setWriteBufferSize(writeBufferSizeMB * 1024L * 1024L)
      .setMaxWriteBufferNumber(MAX_WRITE_BUFFER_NUMBER)
      .setMinWriteBufferNumberToMerge(MIN_WRITE_BUFFER_NUMBER_TO_MERGE)
      .setTableFormatConfig(tableFormatConfig)
      .setOptimizeFiltersForHits(true)
      .setUseDirectReads(true);
}

代码示例来源：origin: org.apache.kylin/kylin-core-dictionary

public RocksDBLookupBuilder(TableDesc tableDesc, String[] keyColumns, String dbPath) {
  this.tableDesc = tableDesc;
  this.encoder = new RocksDBLookupRowEncoder(tableDesc, keyColumns);
  this.dbPath = dbPath;
  this.writeBatchSize = 500;
  this.options = new Options();
  options.setCreateIfMissing(true).setWriteBufferSize(8 * SizeUnit.KB).setMaxWriteBufferNumber(3)
      .setMaxBackgroundCompactions(5).setCompressionType(CompressionType.SNAPPY_COMPRESSION)
      .setCompactionStyle(CompactionStyle.UNIVERSAL);
}

代码示例来源：origin: org.apache.kafka/kafka-streams

options.setWriteBufferSize(WRITE_BUFFER_SIZE);
options.setCompressionType(COMPRESSION_TYPE);
options.setCompactionStyle(COMPACTION_STYLE);

代码示例来源：origin: com.github.ddth/ddth-commons-core

/**
 * Builds RocksDb {@link Options}.
 * 
 * @param maxBackgroundThreads
 * @param levelZeloFileNumCompactionTrigger
 * @param writeBufferSize
 * @param targetFileSizeBase
 * @return
 */
public static Options buildOptions(int maxBackgroundThreads,
    int levelZeloFileNumCompactionTrigger, long writeBufferSize, long targetFileSizeBase) {
  Options rocksOptions = new Options();
  rocksOptions.setCreateIfMissing(true).getEnv().setBackgroundThreads(1, Env.FLUSH_POOL)
      .setBackgroundThreads(maxBackgroundThreads, Env.COMPACTION_POOL);
  rocksOptions.setMaxBackgroundFlushes(1).setMaxBackgroundCompactions(maxBackgroundThreads);
  rocksOptions.setWriteBufferSize(writeBufferSize).setMinWriteBufferNumberToMerge(2)
      .setLevelZeroFileNumCompactionTrigger(levelZeloFileNumCompactionTrigger)
      .setTargetFileSizeBase(targetFileSizeBase);
  rocksOptions.setMemTableConfig(new SkipListMemTableConfig());
  // rocksOptions.setMemTableConfig(new HashSkipListMemTableConfig());
  // rocksOptions.setMemTableConfig(new HashLinkedListMemTableConfig());
  return rocksOptions;
}

代码示例来源：origin: org.apache.samza/samza-kv-rocksdb

Long writeBufSize = storeConfig.getLong("container.write.buffer.size.bytes", 32 * 1024 * 1024);
options.setWriteBufferSize((int) (writeBufSize / numTasksForContainer));

代码示例来源：origin: apache/samza

Long writeBufSize = storeConfig.getLong("container.write.buffer.size.bytes", 32 * 1024 * 1024);
options.setWriteBufferSize((int) (writeBufSize / numTasksForContainer));

代码示例来源：origin: opendedup/sdfs

options.setWriteBufferSize(bufferSize);

代码示例来源：origin: org.apache.samza/samza-kv-rocksdb_2.11

Long writeBufSize = storeConfig.getLong("container.write.buffer.size.bytes", 32 * 1024 * 1024);
options.setWriteBufferSize((int) (writeBufSize / numTasksForContainer));

代码示例来源：origin: com.github.ltsopensource/lts-core

@Override
protected void init() throws FailStoreException {
  try {
    options = new Options();
    options.setCreateIfMissing(true)
        .setWriteBufferSize(8 * SizeUnit.KB)
        .setMaxWriteBufferNumber(3)
        .setMaxBackgroundCompactions(10)
        .setCompressionType(CompressionType.SNAPPY_COMPRESSION)
        .setCompactionStyle(CompactionStyle.UNIVERSAL);
    Filter bloomFilter = new BloomFilter(10);
    BlockBasedTableConfig tableConfig = new BlockBasedTableConfig();
    tableConfig.setBlockCacheSize(64 * SizeUnit.KB)
        .setFilter(bloomFilter)
        .setCacheNumShardBits(6)
        .setBlockSizeDeviation(5)
        .setBlockRestartInterval(10)
        .setCacheIndexAndFilterBlocks(true)
        .setHashIndexAllowCollision(false)
        .setBlockCacheCompressedSize(64 * SizeUnit.KB)
        .setBlockCacheCompressedNumShardBits(10);
    options.setTableFormatConfig(tableConfig);
  } catch (Exception e) {
    throw new FailStoreException(e);
  }
}

代码示例来源：origin: org.apache.bookkeeper/bookkeeper-server

options.setCompressionType(CompressionType.LZ4_COMPRESSION);
options.setWriteBufferSize(writeBufferSizeMB * 1024 * 1024);
options.setMaxWriteBufferNumber(4);
if (numLevels > 0) {

内容来源于网络，如有侵权，请联系作者删除！

Options

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多

    Options类方法

    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com