com.yahoo.memory.WritableMemory.writableRegion()方法的使用及代码示例

x33g5p2x  于2022-02-03 转载在 其他  
字(12.9k)|赞(0)|评价(0)|浏览(103)

本文整理了Java中com.yahoo.memory.WritableMemory.writableRegion()方法的一些代码示例,展示了WritableMemory.writableRegion()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。WritableMemory.writableRegion()方法的具体详情如下:
包路径:com.yahoo.memory.WritableMemory
类名称:WritableMemory
方法名:writableRegion

WritableMemory.writableRegion介绍

[英]A writable region is a writable view of this object. This returns a new WritableMemory representing the defined writable region with the given offsetBytes and capacityBytes.

  • Returned object's origin = this objects' origin + offsetBytes
  • Returned object's capacity = capacityBytes
    If the given capacityBytes is zero, the returned object is effectively immutable and the backing storage and byte order are unspecified.
    [中]可写区域是此对象的可写视图。这将返回一个新的可写内存,该内存使用给定的offsetBytes和capacityBytes表示定义的可写区域。
    *返回对象的原点=此对象的原点+偏移字节
    *返回对象的容量=容量字节
    如果给定的capacityBytes为零,则返回的对象实际上是不可变的,并且未指定备份存储和字节顺序。

代码示例

代码示例来源:origin: apache/incubator-druid

@SuppressWarnings("ResultOfObjectAllocationIgnored")
@Override
public void init(final ByteBuffer buf, final int position)
{
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN).writableRegion(position, size);
 // Not necessary to keep the constructed object since it is cheap to reconstruct by wrapping the memory.
 // The objects are not cached as in BuildBufferAggregator since they never exceed the max size and never move.
 // So it is easier to reconstruct them by wrapping memory then to keep position-to-object mappings. 
 new Union(lgK, mem);
}

代码示例来源:origin: apache/incubator-druid

@Override
public void init(final ByteBuffer buf, final int position)
{
 final WritableMemory mem = getMemory(buf).writableRegion(position, size);
 putSketchIntoCache(buf, position, new HllSketch(lgK, tgtHllType, mem));
}

代码示例来源:origin: apache/incubator-druid

@Override
public Object get(final ByteBuffer buf, final int position)
{
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN).writableRegion(position, size);
 final Lock lock = stripedLock.getAt(HllSketchBuildBufferAggregator.lockIndex(position)).readLock();
 lock.lock();
 try {
  final Union union = Union.writableWrap(mem);
  return union.getResult(tgtHllType);
 }
 finally {
  lock.unlock();
 }
}

代码示例来源:origin: apache/incubator-druid

/**
 * In very rare cases sketches can exceed given memory, request on-heap memory and move there.
 * We need to identify such sketches and reuse the same objects as opposed to wrapping new memory regions.
 */
@Override
public void relocate(final int oldPosition, final int newPosition, final ByteBuffer oldBuf, final ByteBuffer newBuf)
{
 HllSketch sketch = sketchCache.get(oldBuf).get(oldPosition);
 final WritableMemory oldMem = getMemory(oldBuf).writableRegion(oldPosition, size);
 if (sketch.isSameResource(oldMem)) { // sketch has not moved
  final WritableMemory newMem = getMemory(newBuf).writableRegion(newPosition, size);
  sketch = HllSketch.writableWrap(newMem);
 }
 putSketchIntoCache(newBuf, newPosition, sketch);
}

代码示例来源:origin: apache/incubator-druid

@Override
public synchronized void relocate(int oldPosition, int newPosition, ByteBuffer oldBuffer, ByteBuffer newBuffer)
{
 UpdateDoublesSketch sketch = sketches.get(oldBuffer).get(oldPosition);
 final WritableMemory oldRegion = getMemory(oldBuffer).writableRegion(oldPosition, maxIntermediateSize);
 if (sketch.isSameResource(oldRegion)) { // sketch was not relocated on heap
  final WritableMemory newRegion = getMemory(newBuffer).writableRegion(newPosition, maxIntermediateSize);
  sketch = UpdateDoublesSketch.wrap(newRegion);
 }
 putSketch(newBuffer, newPosition, sketch);
 final Int2ObjectMap<UpdateDoublesSketch> map = sketches.get(oldBuffer);
 map.remove(oldPosition);
 if (map.isEmpty()) {
  sketches.remove(oldBuffer);
  memCache.remove(oldBuffer);
 }
}

代码示例来源:origin: apache/incubator-druid

@Override
public synchronized void relocate(int oldPosition, int newPosition, ByteBuffer oldBuffer, ByteBuffer newBuffer)
{
 DoublesUnion union = unions.get(oldBuffer).get(oldPosition);
 final WritableMemory oldMem = getMemory(oldBuffer).writableRegion(oldPosition, maxIntermediateSize);
 if (union.isSameResource(oldMem)) { // union was not relocated on heap
  final WritableMemory newMem = getMemory(newBuffer).writableRegion(newPosition, maxIntermediateSize);
  union = DoublesUnion.wrap(newMem);
 }
 putUnion(newBuffer, newPosition, union);
 Int2ObjectMap<DoublesUnion> map = unions.get(oldBuffer);
 map.remove(oldPosition);
 if (map.isEmpty()) {
  unions.remove(oldBuffer);
  memCache.remove(oldBuffer);
 }
}

代码示例来源:origin: apache/incubator-druid

/**
 * This method uses locks because it can be used during indexing,
 * and Druid can call aggregate() and get() concurrently
 * https://github.com/apache/incubator-druid/pull/3956
 * The returned sketch is a separate instance of ArrayOfDoublesCompactSketch
 * representing the current state of the aggregation, and is not affected by consequent
 * aggregate() calls
 */
@Override
public Object get(final ByteBuffer buf, final int position)
{
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN);
 final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
 final Lock lock = stripedLock.getAt(lockIndex(position)).readLock();
 lock.lock();
 try {
  final ArrayOfDoublesUpdatableSketch sketch = (ArrayOfDoublesUpdatableSketch) ArrayOfDoublesSketches
    .wrapSketch(region);
  return sketch.compact();
 }
 finally {
  lock.unlock();
 }
}

代码示例来源:origin: apache/incubator-druid

@Override
public void aggregate(final ByteBuffer buf, final int position)
{
 final HllSketch sketch = selector.getObject();
 if (sketch == null) {
  return;
 }
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN).writableRegion(position, size);
 final Lock lock = stripedLock.getAt(HllSketchBuildBufferAggregator.lockIndex(position)).writeLock();
 lock.lock();
 try {
  final Union union = Union.writableWrap(mem);
  union.update(sketch);
 }
 finally {
  lock.unlock();
 }
}

代码示例来源:origin: apache/incubator-druid

/**
 * This method uses locks because it can be used during indexing,
 * and Druid can call aggregate() and get() concurrently
 * https://github.com/apache/incubator-druid/pull/3956
 * The returned sketch is a separate instance of ArrayOfDoublesCompactSketch
 * representing the current state of the aggregation, and is not affected by consequent
 * aggregate() calls
 */
@Override
public Object get(final ByteBuffer buf, final int position)
{
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN);
 final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
 final Lock lock = stripedLock.getAt(ArrayOfDoublesSketchBuildBufferAggregator.lockIndex(position)).readLock();
 lock.lock();
 try {
  final ArrayOfDoublesUnion union = ArrayOfDoublesSketches.wrapUnion(region);
  return union.getResult();
 }
 finally {
  lock.unlock();
 }
}

代码示例来源:origin: apache/incubator-druid

private Union createNewUnion(ByteBuffer buf, int position, boolean isWrapped)
{
 WritableMemory mem = getMemory(buf).writableRegion(position, maxIntermediateSize);
 Union union = isWrapped
        ? (Union) SetOperation.wrap(mem)
        : (Union) SetOperation.builder().setNominalEntries(size).build(Family.UNION, mem);
 Int2ObjectMap<Union> unionMap = unions.get(buf);
 if (unionMap == null) {
  unionMap = new Int2ObjectOpenHashMap<>();
  unions.put(buf, unionMap);
 }
 unionMap.put(position, union);
 return union;
}

代码示例来源:origin: apache/incubator-druid

/**
 * This method uses locks because it can be used during indexing,
 * and Druid can call aggregate() and get() concurrently
 * https://github.com/apache/incubator-druid/pull/3956
 */
@Override
public void aggregate(final ByteBuffer buf, final int position)
{
 final ArrayOfDoublesSketch update = selector.getObject();
 if (update == null) {
  return;
 }
 // Wrapping memory and ArrayOfDoublesUnion is inexpensive compared to union operations.
 // Maintaining a cache of wrapped objects per buffer position like in Theta sketch aggregator
 // might might be considered, but it would increase complexity including relocate() support.
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN);
 final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
 final Lock lock = stripedLock.getAt(ArrayOfDoublesSketchBuildBufferAggregator.lockIndex(position)).writeLock();
 lock.lock();
 try {
  final ArrayOfDoublesUnion union = ArrayOfDoublesSketches.wrapUnion(region);
  union.update(update);
 }
 finally {
  lock.unlock();
 }
}

代码示例来源:origin: apache/incubator-druid

@Override
public void init(final ByteBuffer buf, final int position)
{
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN);
 final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
 new ArrayOfDoublesSetOperationBuilder().setNominalEntries(nominalEntries)
   .setNumberOfValues(numberOfValues).buildUnion(region);
}

代码示例来源:origin: apache/incubator-druid

@Override
public synchronized void init(final ByteBuffer buffer, final int position)
{
 final WritableMemory mem = getMemory(buffer);
 final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
 final UpdateDoublesSketch sketch = DoublesSketch.builder().setK(size).build(region);
 putSketch(buffer, position, sketch);
}

代码示例来源:origin: apache/incubator-druid

@Override
public synchronized void init(final ByteBuffer buffer, final int position)
{
 final WritableMemory mem = getMemory(buffer);
 final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
 final DoublesUnion union = DoublesUnion.builder().setMaxK(k).build(region);
 putUnion(buffer, position, union);
}

代码示例来源:origin: apache/incubator-druid

@Override
public void init(final ByteBuffer buf, final int position)
{
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN);
 final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
 new ArrayOfDoublesUpdatableSketchBuilder().setNominalEntries(nominalEntries)
   .setNumberOfValues(valueSelectors.length)
   .setNumberOfValues(valueSelectors.length).build(region);
}

代码示例来源:origin: apache/incubator-druid

/**
 * This method uses locks because it can be used during indexing,
 * and Druid can call aggregate() and get() concurrently
 * https://github.com/apache/incubator-druid/pull/3956
 */
@Override
public void aggregate(final ByteBuffer buf, final int position)
{
 for (int i = 0; i < valueSelectors.length; i++) {
  values[i] = valueSelectors[i].getDouble();
 }
 final IndexedInts keys = keySelector.getRow();
 // Wrapping memory and ArrayOfDoublesSketch is inexpensive compared to sketch operations.
 // Maintaining a cache of wrapped objects per buffer position like in Theta sketch aggregator
 // might might be considered, but it would increase complexity including relocate() support.
 final WritableMemory mem = WritableMemory.wrap(buf, ByteOrder.LITTLE_ENDIAN);
 final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
 final Lock lock = stripedLock.getAt(lockIndex(position)).writeLock();
 lock.lock();
 try {
  final ArrayOfDoublesUpdatableSketch sketch = ArrayOfDoublesSketches.wrapUpdatableSketch(region);
  for (int i = 0, keysSize = keys.size(); i < keysSize; i++) {
   final String key = keySelector.lookupName(keys.get(i));
   sketch.update(key, values);
  }
 }
 finally {
  lock.unlock();
 }
}

代码示例来源:origin: DataSketches/sketches-core

private static double directUnionTrial2(
  WritableMemory heapMem, int[] heapLayout, int sketchNomEntries, int unionNomEntries) {
 WritableMemory unionMem = heapMem.writableRegion(heapLayout[0], heapLayout[1]-heapLayout[0]);
 WritableMemory sketch1mem = heapMem.writableRegion(heapLayout[1], heapLayout[2]-heapLayout[1]);
 WritableMemory sketch2mem = heapMem.writableRegion(heapLayout[2], heapLayout[3]-heapLayout[2]);
 WritableMemory sketch3mem = heapMem.writableRegion(heapLayout[3], heapLayout[4]-heapLayout[3]);
 WritableMemory resultMem = heapMem.writableRegion(heapLayout[4], heapLayout[5]-heapLayout[4]);
 //Recover the 3 sketches
 UpdateSketch sk1 = (UpdateSketch) Sketch.wrap(sketch1mem);
 UpdateSketch sk2 = (UpdateSketch) Sketch.wrap(sketch2mem);
 UpdateSketch sk3 = (UpdateSketch) Sketch.wrap(sketch3mem);
 //confirm that each of these 3 sketches is exact.
 assertEquals(sk1.getEstimate(), sketchNomEntries, 0.0);
 assertEquals(sk2.getEstimate(), sketchNomEntries, 0.0);
 assertEquals(sk3.getEstimate(), sketchNomEntries, 0.0);
 //Create a new union in the same space with a smaller size.
 unionMem.clear();
 Union union = SetOperation.builder().setNominalEntries(unionNomEntries).buildUnion(unionMem);
 union.update(sk1);
 union.update(sk2);
 union.update(sk3);
 Sketch resSk = union.getResult(true, resultMem);
 double est = resSk.getEstimate();
 return est;
}

代码示例来源:origin: DataSketches/sketches-core

/**
 * @return a byte array representation of this object
 */
public byte[] toByteArray() {
 final int sizeBytes = PREAMBLE_SIZE_BYTES + sketch_.getSerializedSizeBytes();
 final byte[] byteArray = new byte[sizeBytes];
 final WritableMemory mem = WritableMemory.wrap(byteArray);
 mem.putByte(PREAMBLE_LONGS_BYTE, (byte) 1); // unused, always 1
 mem.putByte(SERIAL_VERSION_BYTE, serialVersionUID);
 mem.putByte(FAMILY_ID_BYTE, (byte) Family.TUPLE.getID());
 mem.putByte(SKETCH_TYPE_BYTE, (byte) SerializerDeserializer.SketchType.ArrayOfDoublesUnion.ordinal());
 mem.putLong(THETA_LONG, theta_);
 sketch_.serializeInto(mem.writableRegion(PREAMBLE_SIZE_BYTES, mem.getCapacity() - PREAMBLE_SIZE_BYTES));
 return byteArray;
}

代码示例来源:origin: DataSketches/sketches-core

WritableMemory unionMem = heapMem.writableRegion(offset, bytes);
WritableMemory sketch1mem = heapMem.writableRegion(heapLayout[1], heapLayout[2]-heapLayout[1]);
WritableMemory sketch2mem = heapMem.writableRegion(heapLayout[2], heapLayout[3]-heapLayout[2]);
WritableMemory sketch3mem = heapMem.writableRegion(heapLayout[3], heapLayout[4]-heapLayout[3]);
WritableMemory resultMem = heapMem.writableRegion(heapLayout[4], heapLayout[5]-heapLayout[4]);

代码示例来源:origin: DataSketches/sketches-core

final WritableMemory sketchMem = mem.writableRegion(ArrayOfDoublesUnion.PREAMBLE_SIZE_BYTES,
  mem.getCapacity() - ArrayOfDoublesUnion.PREAMBLE_SIZE_BYTES);
sketch = new DirectArrayOfDoublesQuickSelectSketch(sketchMem, seed);

相关文章