com.yahoo.memory.WritableMemory.allocateDirect()方法的使用及代码示例

x33g5p2x  于2022-02-03 转载在 其他  
字(11.5k)|赞(0)|评价(0)|浏览(123)

本文整理了Java中com.yahoo.memory.WritableMemory.allocateDirect()方法的一些代码示例，展示了WritableMemory.allocateDirect()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。WritableMemory.allocateDirect()方法的具体详情如下：
包路径：com.yahoo.memory.WritableMemory
类名称：WritableMemory
方法名：allocateDirect

WritableMemory.allocateDirect介绍

[英]Allocates and provides access to capacityBytes directly in native (off-heap) memory leveraging the WritableMemory API. Native byte order is assumed. The allocated memory will be 8-byte aligned, but may not be page aligned. If capacityBytes is zero, byte order, backing storage and read-only status of the WritableMemory object, returned from WritableHandle#get() are unspecified.

The default MemoryRequestServer, which allocates any request for memory onto the heap, will be used.

NOTE: Native/Direct memory acquired using Unsafe may have garbage in it. It is the responsibility of the using class to clear this memory, if required, and to call close() when done.
[中]利用WritableMemory API直接在本机（堆外）内存中分配和提供对容量字节的访问。假定为本机字节顺序。分配的内存将按8字节对齐，但可能不按页面对齐。如果capacityBytes为零，则未指定从WritableHandle#get（）返回的WritableMemory对象的字节顺序、后备存储和只读状态。
将使用默认的MemoryRequestServer，它将任何内存请求分配到堆中。
注意：使用“不安全”获取的本机/直接内存中可能有垃圾。如果需要，使用类负责清除该内存，并在完成时调用close（）。

代码示例

代码示例来源：origin: DataSketches/sketches-core

private static WritableDirectHandle makeNativeMemory(int k) {
 return WritableMemory.allocateDirect(getMaxBytes(k));
}

代码示例来源：origin: DataSketches/sketches-core

private static WritableDirectHandle makeNativeMemory(int k) {
 return WritableMemory.allocateDirect(getMaxBytes(k));
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkEmptyDirect() {
 try (WritableDirectHandle wdh = WritableMemory.allocateDirect(1000)) {
  WritableMemory mem = wdh.get();
  UpdateDoublesSketch sketch = DoublesSketch.builder().build(mem);
  sketch.toByteArray(); //exercises a specific path
 }
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void directSketchShouldMoveOntoHeapEventually() {
 try (WritableDirectHandle wdh = WritableMemory.allocateDirect(1000)) {
  WritableMemory mem = wdh.get();
  UpdateDoublesSketch sketch = DoublesSketch.builder().build(mem);
  Assert.assertTrue(sketch.isSameResource(mem));
  for (int i = 0; i < 1000; i++) {
   sketch.update(i);
  }
  Assert.assertFalse(sketch.isSameResource(mem));
 }
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkMoveAndResize() {
 int k = 1 << 12;
 int u = 2 * k;
 int bytes = Sketches.getMaxUpdateSketchBytes(k);
  try (WritableDirectHandle wdh = WritableMemory.allocateDirect(bytes/2)) { //will request
  WritableMemory wmem = wdh.get();
  UpdateSketch sketch = Sketches.updateSketchBuilder().setNominalEntries(k).build(wmem);
  assertTrue(sketch.isSameResource(wmem));
  for (int i = 0; i < u; i++) { sketch.update(i); }
  assertFalse(sketch.isSameResource(wmem));
 }
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkMoveAndResize() {
 int k = 1 << 12;
 int u = 2 * k;
 int bytes = Sketches.getMaxUpdateSketchBytes(k);
 try (WritableDirectHandle wdh = WritableMemory.allocateDirect(bytes/2);
    WritableDirectHandle wdh2 = WritableMemory.allocateDirect(bytes/2) ) {
  WritableMemory wmem = wdh.get();
  UpdateSketch sketch = Sketches.updateSketchBuilder().setNominalEntries(k).build(wmem);
  assertTrue(sketch.isSameResource(wmem));
  WritableMemory wmem2 = wdh2.get();
  Union union = SetOperation.builder().buildUnion(wmem2);
  assertTrue(union.isSameResource(wmem2));
  for (int i = 0; i < u; i++) { union.update(i); }
  assertFalse(union.isSameResource(wmem));
  Union union2 = SetOperation.builder().buildUnion(); //on-heap union
  assertFalse(union2.isSameResource(wmem2));  //obviously not
 }
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkEstModeNativeMemory() {
 int k = 4096;
 int u = 2*k;
 int memCapacity = (k << 4) + (Family.QUICKSELECT.getMinPreLongs() << 3);
 try(WritableDirectHandle memHandler = WritableMemory.allocateDirect(memCapacity)) {
  UpdateSketch usk = UpdateSketch.builder().setNominalEntries(k).build(memHandler.get());
  DirectQuickSelectSketch sk1 = (DirectQuickSelectSketch)usk; //for internal checks
  assertTrue(usk.isEmpty());
  for (int i = 0; i< u; i++) { usk.update(i); }
  double est = usk.getEstimate();
  println(""+est);
  assertEquals(usk.getEstimate(), u, u*.05);
  assertTrue(sk1.getRetainedEntries(false) > k);
 }
}

代码示例来源：origin: DataSketches/memory

/**
 * Allocates and provides access to capacityBytes directly in native (off-heap) memory
 * leveraging the WritableMemory API. Native byte order is assumed.
 * The allocated memory will be 8-byte aligned, but may not be page aligned.
 * If capacityBytes is zero, byte order, backing storage and read-only status
 * of the WritableMemory object, returned from {@link WritableHandle#get()} are unspecified.
 *
 * <p>The default MemoryRequestServer, which allocates any request for memory onto the heap,
 * will be used.</p>
 *
 * <p><b>NOTE:</b> Native/Direct memory acquired using Unsafe may have garbage in it.
 * It is the responsibility of the using class to clear this memory, if required,
 * and to call <i>close()</i> when done.</p>
 *
 * @param capacityBytes the size of the desired memory in bytes.
 * @return WritableDirectHandle for this off-heap resource.
 * Please read Javadocs for {@link Handle}.
 */
public static WritableDirectHandle allocateDirect(final long capacityBytes) {
 return allocateDirect(capacityBytes, null);
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkEstModeNativeMemory() {
 lgK = 12;
 int k = 1 << lgK;
 int u = 2*k;
 int memCapacity = (k << 4) + (Family.QUICKSELECT.getMinPreLongs() << 3);
 try(WritableDirectHandle memHandler = WritableMemory.allocateDirect(memCapacity)) {
  final ConcurrentThetaBuilder bldr = configureBuilder();
  //must build shared first
  shared = bldr.build(memHandler.get());
  UpdateSketch usk = bldr.build();
  assertTrue(usk.isEmpty());
  for (int i = 0; i< u; i++) { usk.update(i); }
  waitForPropagationToComplete();
  double est = usk.getEstimate();
  println(""+est);
  assertEquals(usk.getEstimate(), u, u*.05);
  assertTrue(shared.getSharedRetainedEntries(false) > k);
 }
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkGrowBaseBuf() {
 final int k = 128;
 final int u = 32; // don't need the BB to fill here
 final int initBytes = (4 + (u / 2)) << 3; // not enough to hold everything
 try (WritableDirectHandle memHandler = WritableMemory.allocateDirect(initBytes)) {
  //final MemoryManager memMgr = new MemoryManager();
  //final WritableMemory mem1 = memMgr.request(initBytes);
  final WritableMemory mem1 = memHandler.get();
  println("Initial mem size: " + mem1.getCapacity());
  final UpdateDoublesSketch usk1 = DoublesSketch.builder().setK(k).build(mem1);
  for (int i = 1; i <= u; i++) {
   usk1.update(i);
  }
  final int currentSpace = usk1.getCombinedBufferItemCapacity();
  println("curCombBufItemCap: " + currentSpace);
  assertEquals(currentSpace, 2 * k);
 }
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkGrowFromWrappedEmptySketch() {
 final int k = 16;
 final int n = 0;
 final int initBytes = DoublesSketch.getUpdatableStorageBytes(k, n); //8 bytes
 final UpdateDoublesSketch usk1 = DoublesSketch.builder().setK(k).build();
 final Memory origSketchMem = Memory.wrap(usk1.toByteArray());
 try (WritableDirectHandle memHandle = WritableMemory.allocateDirect(initBytes)) {
  WritableMemory mem = memHandle.get();
  origSketchMem.copyTo(0, mem, 0, initBytes);
  UpdateDoublesSketch usk2 = DirectUpdateDoublesSketch.wrapInstance(mem);
  assertTrue(mem.isSameResource(usk2.getMemory()));
  assertEquals(mem.getCapacity(), initBytes);
  assertTrue(mem.isDirect());
  assertTrue(usk2.isEmpty());
  //update the sketch forcing it to grow on-heap
  for (int i = 1; i <= 5; i++) { usk2.update(i); }
  assertEquals(usk2.getN(), 5);
  WritableMemory mem2 = usk2.getMemory();
  assertFalse(mem.isSameResource(mem2));
  assertFalse(mem2.isDirect()); //should now be on-heap
  final int expectedSize = COMBINED_BUFFER + ((2 * k) << 3);
  assertEquals(mem2.getCapacity(), expectedSize);
 }
}

代码示例来源：origin: DataSketches/sketches-core

try (WritableDirectHandle hand = WritableMemory.allocateDirect(bytes)) {
 wmem = hand.get();

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkReadOnlyRebuildResize() {
 int k = 1 << 12;
 int u = 2 * k;
 int bytes = Sketches.getMaxUpdateSketchBytes(k);
 try (WritableDirectHandle wdh = WritableMemory.allocateDirect(bytes/2)) { //will request
  WritableMemory wmem = wdh.get();
  UpdateSketch sketch = Sketches.updateSketchBuilder().setNominalEntries(k).build(wmem);
  for (int i = 0; i < u; i++) { sketch.update(i); }
  double est1 = sketch.getEstimate();
  byte[] ser = sketch.toByteArray();
  Memory mem = Memory.wrap(ser);
  UpdateSketch roSketch = (UpdateSketch) Sketches.wrapSketch(mem);
  double est2 = roSketch.getEstimate();
  assertEquals(est2, est1);
  try {
   roSketch.rebuild();
   fail();
  } catch (SketchesReadOnlyException e) {
   //expected
  }
  try {
   roSketch.reset();
   fail();
  } catch (SketchesReadOnlyException e) {
   //expected
  }
 }
}

代码示例来源：origin: DataSketches/sketches-core

Memory heapROMem;
try (WritableDirectHandle wdh = WritableMemory.allocateDirect(bytes)) {
 directMem = wdh.get();

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkUnionCompactOrderedSource() {
 int k = 1 << 12;
 UpdateSketch sk = Sketches.updateSketchBuilder().build();
 for (int i = 0; i < k; i++) { sk.update(i); }
 double est1 = sk.getEstimate();
 int bytes = Sketches.getMaxCompactSketchBytes(sk.getRetainedEntries());
 try (WritableDirectHandle h = WritableMemory.allocateDirect(bytes)) {
  WritableMemory wmem = h.get();
  CompactSketch csk = sk.compact(true, wmem); //ordered, direct
  Union union = Sketches.setOperationBuilder().buildUnion();
  union.update(csk);
  double est2 = union.getResult().getEstimate();
  assertEquals(est2, est1);
 }
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkLimitedMemoryScenarios() { //Requesting application
 final int k = 128;
 final int u = 40 * k;
 final int initBytes = ((2 * k) + 4) << 3; //just the BB
 //########## Owning Implementation
 // This part would actually be part of the Memory owning implemention so it is faked here
 try (WritableDirectHandle wdh = WritableMemory.allocateDirect(initBytes)) {
  final WritableMemory wmem = wdh.get();
  println("Initial mem size: " + wmem.getCapacity());
  //########## Receiving Application
  // The receiving application has been given wmem to use for a sketch,
  // but alas, it is not ultimately large enough.
  final UpdateDoublesSketch usk1 = DoublesSketch.builder().setK(k).build(wmem);
  assertTrue(usk1.isEmpty());
  //Load the sketch
  for (int i = 0; i < u; i++) {
   // The sketch uses The MemoryRequest, acquired from wmem, to acquire more memory as
   // needed, and requests via the MemoryRequest to free the old allocations.
   usk1.update(i);
  }
  final double result = usk1.getQuantile(0.5);
  println("Result: " + result);
  assertEquals(result, u / 2.0, 0.05 * u); //Success
  //########## Owning Implementation
  //The actual Memory has been re-allocated several times,
  // so the above wmem reference is invalid.
  println("\nFinal mem size: " + wmem.getCapacity());
 }
}

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkGrowCombBuf() {
 final int k = 128;
 final int u = (2 * k) - 1; //just to fill the BB
 final int initBytes = ((2 * k) + 4) << 3; //just room for BB
 try (WritableDirectHandle memHandler = WritableMemory.allocateDirect(initBytes)) {
  //final MemoryManager memMgr = new MemoryManager();
  //final WritableMemory mem1 = memMgr.request(initBytes);
  final WritableMemory mem1 = memHandler.get();
  println("Initial mem size: " + mem1.getCapacity());
  final UpdateDoublesSketch usk1 = DoublesSketch.builder().setK(k).build(mem1);
  for (int i = 1; i <= u; i++) {
   usk1.update(i);
  }
  final int currentSpace = usk1.getCombinedBufferItemCapacity();
  println("curCombBufItemCap: " + currentSpace);
  final double[] newCB = usk1.growCombinedBuffer(currentSpace, 3 * k);
  final int newSpace = usk1.getCombinedBufferItemCapacity();
  println("newCombBurItemCap: " + newSpace);
  assertEquals(newCB.length, 3 * k);
  //memMgr.free(mem1);
 }
}

代码示例来源：origin: DataSketches/sketches-core

int bytes = HllSketch.getMaxUpdatableSerializationBytes(lgConfigK, tgtHllType);
HllSketch hllSketch;
try (WritableDirectHandle handle = WritableMemory.allocateDirect(bytes)) {
 WritableMemory wmem = handle.get();
 hllSketch = new HllSketch(lgConfigK, tgtHllType, wmem);

代码示例来源：origin: DataSketches/sketches-core

@Test
public void checkInsertsAndExtracts() {
 final int bytes = 32;
 try (WritableDirectHandle offHeapMemHandler = WritableMemory.allocateDirect(bytes)) {
  final WritableMemory offHeapMem = offHeapMemHandler.get();
  final WritableMemory onHeapMem = WritableMemory.wrap(new byte[bytes]);

内容来源于网络，如有侵权，请联系作者删除！

WritableMemory

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com