pandas 嫩皮阵法重塑需要3个小时,有没有办法加快速度?

6rqinv9w  于 2022-11-05  发布在  其他
关注(0)|答案(1)|浏览(163)

我写了一个函数,它可以根据给定的窗口长度对numpy数组进行整形。

我的功能为:

def w_s(df, l):
    """
    Convert numpy array into desired shape with lag 1.
    Args:
        df (numpy.ndarray): Numpy array.
        l (integer): Length of the sample window.

    Returns:
        Returns numpy array in a desired shape to be used in decision trees.
    """
    data = np.zeros((l, 1))
    data = np.append(data, df)
    data = data[l:]
    for i in range(1, l):
        s1 = np.roll(df,0-i)
        data = np.append(data,s1)
    data = data.reshape(l, len(df)).T

    return data[:-(l-1)]

我有两个长度为1780000的数组,这个函数大约需要3个小时。
CPU times: user 5min 20s, sys: 43min 45s, total: 49min 6s Wall time: 3h 5min 46s
我的机器是Mac M1。我在Jupyter cell上运行这个,而服务器在Firefox上运行。我如何才能更快地完成这个任务?

ergxz8rk

ergxz8rk1#

这就是sliding_window_view

>>> import numpy as np

>>> arr = np.arange(10)
>>> np.lib.stride_tricks.sliding_window_view(arr, 5)
array([[0, 1, 2, 3, 4],
       [1, 2, 3, 4, 5],
       [2, 3, 4, 5, 6],
       [3, 4, 5, 6, 7],
       [4, 5, 6, 7, 8],
       [5, 6, 7, 8, 9]])

相关问题