我试着在CNN上实现一个pytorch框架。
我确信代码是正确的,因为它来自一个教程,当我在GoogleDrive上的Jupyter Notebook上运行它时,它工作正常。
但是当我尝试将其本地化为.py
文件时,它提示了一个错误:AttributeError: Can't pickle local object 'pre_datasets.<locals>.<lambda>'
我知道这是关于在函数外部推理对象的,但是这个错误的确切原因是什么呢?
我该怎么补救呢?
下面是代码的主要部分。
def pre_datasets():
TRAIN_TFM = transforms.Compose(
[
transforms.Resize(size=(128, 128)),
# TODO
transforms.ToTensor(),
]
)
train_set = DatasetFolder(
root=CONFIG["train_set_path"],
loader=lambda x: Image.open(x),
extensions="jpg",
transform=TRAIN_TFM,
)
train_loader = DataLoader(
dataset=train_set,
batch_size=CONFIG["batch_size"],
shuffle=True,
num_workers=CONFIG["num_workers"],
pin_memory=True,
)
return train_loader
def train(train_loader):
...
for epoch in range(CONFIG["num_epochs"]):
...
for batch in train_loader: # error happened here
...
if __name__ == "__main__":
train_loader = pre_datasets()
train(train_loader)
下面是错误消息:
Traceback (most recent call last):
File "HW03_byCRZ.py", line 197, in <module>
train(train_loader, valid_loader)
File "HW03_byCRZ.py", line 157, in train
for batch in train_loader:
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 355, in __iter__
return self._get_iterator()
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 301, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 914, in __init__
w.start()
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Users/ceezous/opt/anaconda3/envs/pytorch_env/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'pre_datasets.<locals>.<lambda>'
2条答案
按热度按时间kb5ga3dv1#
我也遇到过类似的问题,我使用dill时是这样的:
而且开箱即用!
z31licg02#
Pickle'ing一个lambda函数需要额外的修改(序列化函数)更多关于如何做的信息请参见Can Python pickle lambda functions?。