bug描述 Describe the Bug
问题:我利用rocketqa做self_training,因此在迭代过程中,需要多次load同一模型。先去做inference获得伪标签,再去利用伪标签做finetuning,根据我写的封装逻辑,这个过程要对同一模型加载两次,但paddle框架应该不支持这样操作,因此会报错
代码如下:
import rocketqa
ce_model = “zh_dureader_ce_v2”
ce_conf = {
"model": ce_model,
"use_cuda": True,
"device_id": 0,
"batch_size": 32
}
cross_encoder = rocketqa.load_model(**ce_conf)
cross_encoder = rocketqa.load_model(**ce_conf)
报错如下
Traceback (most recent call last):
File "/u01/bankQA/self_training/test_rkqa.py", line 297, in <module>
cross_encoder = rocketqa.load_model(**ce_conf)
File "/u01/miniconda3/envs/bankqa/lib/python3.8/site-packages/rocketqa/rocketqa.py", line 122, in load_model
encoder = CrossEncoder(**encoder_conf)
File "/u01/miniconda3/envs/bankqa/lib/python3.8/site-packages/rocketqa/encoder/cross_encoder.py", line 90, in __init__
self.test_pyreader, self.graph_vars = create_predict_model(
File "/u01/miniconda3/envs/bankqa/lib/python3.8/site-packages/rocketqa/model/cross_encoder_predict.py", line 39, in create_predict_model
pyreader = fluid.layers.py_reader(
File "/u01/miniconda3/envs/bankqa/lib/python3.8/site-packages/paddle/fluid/layers/io.py", line 723, in py_reader
return _py_reader(
File "/u01/miniconda3/envs/bankqa/lib/python3.8/site-packages/paddle/fluid/layers/io.py", line 440, in _py_reader
feed_queue = core.init_lod_tensor_blocking_queue(var, capacity, False)
RuntimeError: (AlreadyExists) LoDTensorBlockingQueueHolder::InitOnce() can only be called once
[Hint: Expected queue_ == nullptr, but received queue_ != nullptr.] (at /paddle/paddle/fluid/operators/reader/lod_tensor_blocking_queue.h:207)
其他补充信息 Additional Supplementary Information
No response
2条答案
按热度按时间fzwojiic1#
不需要同时new两个model,可以new一个model,然后再加载两次不同的参数。
5cg8jx4n2#
不需要同时new两个model,可以new一个model,然后再加载两次不同的参数。
ok,以上只是我举了一个简单例子,我实际遇到的问题是:在循环中,迭代训练模型时就会报错!以下是一个伪代码,以说明模型运行情况
在循环外(迭代外)加载teacher模型,然后进迭代(获取伪标签->训练新模型->加载新模型->评估-->获取伪标签),在第一次循环时(进行第一次训练),不报错。但是到第二次循环,就会报错:
下面是我的真实代码,,思路跟上面伪代码是一样的: