标准差

8ljdwjyq  于 2021-07-13  发布在  Java
关注(0)|答案(2)|浏览(362)

我希望澄清我正在寻找一种方法来计算标准偏差使用流(我有一个工作方法,目前计算和返回sd,但不使用流)。
我正在使用的数据集与link中看到的非常匹配。如此链接所示,我可以对我的数据进行分组&获得平均值,但无法找出如何获得sd。
代码

outPut.stream()
            .collect(Collectors.groupingBy(e -> e.getCar(),
                    Collectors.averagingDouble(e -> (e.getHigh() - e.getLow()))))
            .forEach((car,avgHLDifference) -> System.out.println(car+ "\t" + avgHLDifference));

我还检查了doublesummarystatistics上的链接,但它似乎对sd没有帮助。

e4eetjau

e4eetjau1#

您可以为此任务使用自定义收集器来计算平方和。内置的 DoubleSummaryStatistics 收集器不跟踪它。Maven组对此进行了讨论,但最终没有实施。计算平方和时的困难在于对中间结果求平方时的潜在溢出。

static class DoubleStatistics extends DoubleSummaryStatistics {

    private double sumOfSquare = 0.0d;
    private double sumOfSquareCompensation; // Low order bits of sum
    private double simpleSumOfSquare; // Used to compute right sum for non-finite inputs

    @Override
    public void accept(double value) {
        super.accept(value);
        double squareValue = value * value;
        simpleSumOfSquare += squareValue;
        sumOfSquareWithCompensation(squareValue);
    }

    public DoubleStatistics combine(DoubleStatistics other) {
        super.combine(other);
        simpleSumOfSquare += other.simpleSumOfSquare;
        sumOfSquareWithCompensation(other.sumOfSquare);
        sumOfSquareWithCompensation(other.sumOfSquareCompensation);
        return this;
    }

    private void sumOfSquareWithCompensation(double value) {
        double tmp = value - sumOfSquareCompensation;
        double velvel = sumOfSquare + tmp; // Little wolf of rounding error
        sumOfSquareCompensation = (velvel - sumOfSquare) - tmp;
        sumOfSquare = velvel;
    }

    public double getSumOfSquare() {
        double tmp =  sumOfSquare + sumOfSquareCompensation;
        if (Double.isNaN(tmp) && Double.isInfinite(simpleSumOfSquare)) {
            return simpleSumOfSquare;
        }
        return tmp;
    }

    public final double getStandardDeviation() {
        return getCount() > 0 ? Math.sqrt((getSumOfSquare() / getCount()) - Math.pow(getAverage(), 2)) : 0.0d;
    }

}

然后,您可以将这个类与

Map<String, Double> standardDeviationMap =
    list.stream()
        .collect(Collectors.groupingBy(
            e -> e.getCar(),
            Collectors.mapping(
                e -> e.getHigh() - e.getLow(),
                Collector.of(
                    DoubleStatistics::new,
                    DoubleStatistics::accept,
                    DoubleStatistics::combine,
                    d -> d.getStandardDeviation()
                )
            )
        ));

这将把输入列表收集到一个Map中,其中的值对应于 high - low 同一把钥匙。

y53ybaqx

y53ybaqx2#

您可以使用此自定义收集器:

private static final Collector<Double, double[], Double> VARIANCE_COLLECTOR = Collector.of( // See https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
        () -> new double[3], // {count, mean, M2}
        (acu, d) -> { // See chapter about Welford's online algorithm and https://math.stackexchange.com/questions/198336/how-to-calculate-standard-deviation-with-streaming-inputs
            acu[0]++; // Count
            double delta = d - acu[1];
            acu[1] += delta / acu[0]; // Mean
            acu[2] += delta * (d - acu[1]); // M2
        },
        (acuA, acuB) -> { // See chapter about "Parallel algorithm" : only called if stream is parallel ...
            double delta = acuB[1] - acuA[1];
            double count = acuA[0] + acuB[0];
            acuA[2] = acuA[2] + acuB[2] + delta * delta * acuA[0] * acuB[0] / count; // M2
            acuA[1] += delta * acuB[0] / count;  // Mean
            acuA[0] = count; // Count
            return acuA;
        },
        acu -> acu[2] / (acu[0] - 1.0), // Var = M2 / (count - 1)
        UNORDERED);

然后只需在流中调用此收集器:

double stdDev = Math.sqrt(outPut.stream().boxed().collect(VARIANCE_COLLECTOR));

相关问题