scipy Python中多元正态分布的极大似然估计

ie3xauqp  于 2024-01-09  发布在  Python
关注(0)|答案(1)|浏览(272)

我试图用极大似然估计拟合多元正态分布的参数。

  1. import numpy as np
  2. from scipy.stats import norm
  3. from scipy.optimize import minimize
  4. from scipy.stats import multivariate_normal as mnorm
  5. def estimation(obs,fun,init,method='Nelder-Mead'):
  6. mle = lambda param: -np.sum(fun(*[obs,param])) ## negate since we will minimize
  7. result = minimize(mle,init,method=method)
  8. return result.x

字符串
拟合一元正态分布很好:

  1. obs = np.random.normal(1,4,50000)
  2. ini = [0,1]
  3. print(estimation(obs,lambda ob,p:norm.logpdf(ob,p[0],p[1]),ini))


但是遇到了一些多变量的问题(将数组分配给变量时出错):

  1. obs_m = np.random.multivariate_normal([0,0],[[1,0],[0,100]],50000)
  2. ini_m = [[0,0],[[1,0],[0,100]]]
  3. print(estimation(obs_m,lambda ob,p:mnorm.logpdf(ob,p[0],p[1],ini_m))


看来优化算法对任意数组/矩阵都不起作用。我必须将均值数组和协方差矩阵解压缩成一个平面数组来最小化。

  1. ini_m = [0,0,1,0,0,100]
  2. print(estimation(obs_m,lambda ob,p:mnorm.logpdf(ob,[p[0],p[1]],[[p[2],p[3]],[p[4],p[5]]]),ini_m))


显然,当维数增加时,或者对于一些没有封闭形式解的更复杂的分布,这会很快失控,在这里最好怎么做呢?谢谢。

dtcbnfnu

dtcbnfnu1#

假设optimize.minimize * 确实 * 允许你不加修改地传入meancov的猜测值--以你想要的任何形状--它会传入相同形状的meancov数组到你的负对数似然函数。cov的一些元素会是冗余的,因为它必须是对称的,你需要增加一些约束来保持cov的正定性。
下面的解决方案需要编写特定于分布的函数,用于在决策变量的一维数组和特定于分布的参数之间进行转换,但它可以扩展到任意维数,并且对于multivariate_normal,它解决了冗余决策变量的问题并确保了正定性。

  1. import numpy as np
  2. from scipy import stats, optimize
  3. rng = np.random.default_rng(747458538356)
  4. def mle(obs, dist, params0, method='slsqp'):
  5. # Distribution-independent MLE function
  6. dim = obs.shape[-1]
  7. def nllf(x):
  8. params = x2params(x, dim)
  9. logpdf = dist.logpdf(obs, *params)
  10. return -np.sum(logpdf)
  11. x0 = params2x(params0)
  12. res = optimize.minimize(nllf, x0, method=method)
  13. return x2params(res.x, dim)
  14. dist = stats.multivariate_normal
  15. def x2params(x, dim):
  16. # multivariate_normal-specific function to convert decision variable
  17. # array to distribution parameters
  18. mean = x[:dim]
  19. # Decision variables are elements of Cholesky decomposition
  20. # to ensure positive definite covariance matrix
  21. L = np.zeros((dim, dim))
  22. indices = np.tril_indices(dim)
  23. L[indices] = x[dim:]
  24. cov = L @ L.T
  25. return mean, cov
  26. def params2x(params):
  27. # multivariate_normal-specific function to convert distribution parameters
  28. # to decision variable array
  29. mean, cov = params
  30. dim = len(mean)
  31. indices = np.tril_indices(dim)
  32. L = np.linalg.cholesky(cov)
  33. return np.concatenate((np.ravel(mean), L[indices]))
  34. # generate observations
  35. size = 500 # number of observations
  36. mean = [0, 0]
  37. cov = np.array([[5, 2], [2, 10]])
  38. params0 = (mean, cov)
  39. obs = dist.rvs(mean, cov, size=size, random_state=rng)
  40. # compare our function against reference
  41. res = mle(obs, dist, params0)
  42. ref = stats.multivariate_normal.fit(obs)
  43. np.testing.assert_allclose(res[0], ref[0], rtol=1e-3)
  44. np.testing.assert_allclose(res[1], ref[1], rtol=1e-3)
  45. mean, cov = res
  46. print(mean)
  47. # [0.09533408 0.04011049]
  48. print(cov)
  49. # [[ 4.91358864 2.25704525]
  50. # [ 2.25704525 10.59167997]]

字符串

展开查看全部

相关问题