python-3.x VowpalWabbit上下文强盗模型未按预期收敛

我在模拟一个场景，其中有两个选项（体育/政治），两个转换率（c_0，c_1）。为了决定向客户显示哪个选项，我使用了一个上下文强盗模型。
我已经生成了100个具有固定上下文（user=Tom）的数据点，其格式如下

“共享”|上下文用户=Tom\n0：{cost}：0.5|动作选择=运动\n|行动选择=政治'*

需要注意的一点是，成本是随机生成的价值。在这种情况下：

P（成本=-1|选择=体育）= 0.6
P（成本=-1|选择=政治）= 0.7
P（成本=0.2|选择=体育）= 0.4
P（成本=0.2|选择=政治）= 0.3

在这个训练数据集中，选项 “体育” 的平均成本是-0.52，而 * 政治 * 的平均成本是-0.65。因此，我希望模型更喜欢选项B（即，在这100个样本上进行训练后，选择A。然而，在训练后，在'shared |Context user=Tom \n|Action choice=sports \n|Action choice=politics '上运行预测，我得到PMF [0.9，0.1]。
这一点令人担忧，原因有很多：
1.我期待相反的输出，其中P（B）> P（A）。
1.模型不仅认为选项A更好，而且非常有信心。我希望概率在40- 60%左右，但它收敛到90%（！！）.
我试过调整模型更改参数可以使模型更好地用于特定数据集，但是重新生成数据集很容易产生模型行为如上所述的状态。
该模型正在运行，

import vowpalwabbit as vw

model = vw.Workspace(
    "--cb_explore_adf   --passes 1000 -l 0.2 --cb_type ips --holdout_off --epsilon 0.2 --cache -k"
)

for sample in total_training:
        x = model.parse(
            sample,
            vw.LabelType.CONTEXTUAL_BANDIT
        )
        model.learn(x)
model.predict('shared |Context user=Tom \n|Action choice=sports \n|Action choice=politics ')

完整的训练集是：

'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:-1.0:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n|Action choice=sports \n0:0.2:0.5 |Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:0.2:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics ',
  'shared |Context user=Tom\n0:-1.0:0.5 |Action choice=sports \n|Action choice=politics '

python-3.x VowpalWabbit上下文强盗模型未按预期收敛

1条答案

相关问题

热门标签

最新问答