本文整理了Java中com.clearspring.analytics.stream.cardinality.HyperLogLog.merge()
方法的一些代码示例,展示了HyperLogLog.merge()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。HyperLogLog.merge()
方法的具体详情如下:
包路径:com.clearspring.analytics.stream.cardinality.HyperLogLog
类名称:HyperLogLog
方法名:merge
暂无
代码示例来源:origin: addthis/stream-lib
@Test
public void testMerge() throws CardinalityMergeException {
int numToMerge = 5;
int bits = 16;
int cardinality = 1000000;
HyperLogLog[] hyperLogLogs = new HyperLogLog[numToMerge];
HyperLogLog baseline = new HyperLogLog(bits);
for (int i = 0; i < numToMerge; i++) {
hyperLogLogs[i] = new HyperLogLog(bits);
for (int j = 0; j < cardinality; j++) {
double val = Math.random();
hyperLogLogs[i].offer(val);
baseline.offer(val);
}
}
long expectedCardinality = numToMerge * cardinality;
HyperLogLog hll = hyperLogLogs[0];
hyperLogLogs = Arrays.asList(hyperLogLogs).subList(1, hyperLogLogs.length).toArray(new HyperLogLog[0]);
long mergedEstimate = hll.merge(hyperLogLogs).cardinality();
long baselineEstimate = baseline.cardinality();
double se = expectedCardinality * (1.04 / Math.sqrt(Math.pow(2, bits)));
System.out.println("Baseline estimate: " + baselineEstimate);
System.out.println("Expect estimate: " + mergedEstimate + " is between " + (expectedCardinality - (3 * se)) + " and " + (expectedCardinality + (3 * se)));
assertTrue(mergedEstimate >= expectedCardinality - (3 * se));
assertTrue(mergedEstimate <= expectedCardinality + (3 * se));
assertEquals(mergedEstimate, baselineEstimate);
}
代码示例来源:origin: addthis/stream-lib
/**
* should not fail with HyperLogLogMergeException: "Cannot merge estimators of different sizes"
*/
@Test
public void testMergeWithRegisterSet() throws CardinalityMergeException {
HyperLogLog first = new HyperLogLog(16, new RegisterSet(1 << 20));
HyperLogLog second = new HyperLogLog(16, new RegisterSet(1 << 20));
first.offer(0);
second.offer(1);
first.merge(second);
}
代码示例来源:origin: Big-Data-Manning/big-data-code
public void operate(FlowProcess process, BufferCall call) {
Iterator<TupleEntry> it = call.getArgumentsIterator();
HyperLogLog curr = null;
try {
while(it.hasNext()) {
TupleEntry tuple = it.next();
byte[] serialized = (byte[]) tuple.getObject(0);
HyperLogLog hll = HyperLogLog.Builder.build(
serialized);
if(curr==null) {
curr = hll;
} else {
curr = (HyperLogLog) curr.merge(hll);
}
}
call.getOutputCollector().add(
new Tuple(curr.getBytes()));
} catch (IOException e) {
throw new RuntimeException(e);
} catch(CardinalityMergeException e) {
throw new RuntimeException(e);
}
}
}
代码示例来源:origin: LiveRamp/cascading_ext
HyperLogLog merged = (HyperLogLog)new HyperLogLog(hllError).merge(countParts.toArray(new ICardinality[countParts.size()]));
long cardinality = merged.cardinality();
内容来源于网络,如有侵权,请联系作者删除!