Paddle AttributeError: 'Variable' object has no attribute 'regularizer'

4dc9hkyq  于 2021-11-30  发布在  Java
关注(0)|答案(2)|浏览(517)

下面是我做gradients and clip_by_norm的代码,前面运行都正常,在apply_gradients时报错:
grad_clip = 5.0 # keep the same with the previous setting

generator pre-training

pretrain_opt = fluid.optimizer.AdamOptimizer(gpre_lr, beta1=0.9, beta2=0.999)
g_out = fluid.gradients(g_pretrain_loss, g_vars)
print(g_out)
#g_out = fluid.layers.stack([g_out], axis=0)
pretrain_grad_list = []
pretrain_g_var = []
for i, [g_grad, g_var] in enumerate(list(zip(g_out,g_vars))):
if g_grad is not None:
pretrain_grad = fluid.layers.clip_by_norm(g_grad, grad_clip) # gradient clipping
pretrain_grad_list.append(pretrain_grad)
pretrain_g_var.append(g_var)
print('i={}'.format(i))
print('the length of pretrain_grad_list and pretrain_g_var={} and {}'.format(len(pretrain_grad_list), len(pretrain_g_var)))
g_pretrain_op = pretrain_opt.apply_gradients(zip(pretrain_grad_list, pretrain_g_var))

错误信息如下:
the length of pretrain_grad_list and pretrain_g_var=18 and 18
Traceback (most recent call last):
File "run_dict.py", line 151, in
main()
File "run_dict.py", line 144, in main
real_train_dict(main_program, generator, generator_meta, mediator, discriminator, oracle_loader, config)
File "/media/data2/dingcheng/workspace/baidu/ccl/MetaCotRelGAN/real/real_gan/real_train_dict.py", line 295, in real_train_dict
md_pretrain_loss_fake)
File "/media/data2/dingcheng/workspace/baidu/ccl/MetaCotRelGAN/real/real_gan/real_train_dict.py", line 632, in get_train_ops
g_pretrain_op = pretrain_opt.apply_gradients(zip(pretrain_grad_list, pretrain_g_var))
File "/usr/local/lib/python3.6/dist-packages/paddle/fluid/optimizer.py", line 531, in apply_gradients
self.regularization)
File "/usr/local/lib/python3.6/dist-packages/paddle/fluid/regularizer.py", line 53, in append_regularization_ops
if param.regularizer is not None:
AttributeError: 'Variable' object has no attribute 'regularizer'
同样的代码,在tensorflow这边是没有问题的,请帮忙分析一下?

gpfsuwkq

gpfsuwkq1#

https://github.com/PaddlePaddle/models/blob/develop/PaddleNLP/language_model/train.py#L134
梯度裁剪的方法请参考这种

yuvru6vn

yuvru6vn2#

好的,我改成你说的方法了,还是有问题,不过现在能更加明确的告诉我问题出在哪里了,我继续debug吧。多谢,后面有问题再向你们请教哈。

相关问题