pytorch 异常:无法添加DataPipe函数名sharding_filter,因为它已被使用

nbysray5  于 2023-04-06  发布在  其他
关注(0)|答案(1)|浏览(419)

torchdata.datapipes在Google Colab中不起作用。
即使在安装了torchdata库之后,当导入datapipes函数时,它也会引发异常。
我安装了依赖项
!pip install torchdata!pip install --pre torchdata -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html但是当我写
import torchdata.datapipes as dp

from torchdata.datapipes.iter import StreamReader,FileOpener,FileLister
 
from torchdata.dataloader2 import DataLoader2, MultiProcessingReadingService

它抛出一个错误:

Exception                                 Traceback (most recent call last)
<ipython-input-15-01a976fbf894> in <cell line: 1>()
----> 1 from torchdata.datapipes.iter import StreamReader,FileOpener,FileLister
      2 
      3 from torchdata.dataloader2 import DataLoader2, MultiProcessingReadingService

6 frames
/usr/local/lib/python3.9/dist-packages/torch/utils/data/datapipes/datapipe.py in register_datapipe_as_function(cls, function_name, cls_to_register, enable_df_api_tracing)
    134     def register_datapipe_as_function(cls, function_name, cls_to_register, enable_df_api_tracing=False):
    135         if function_name in cls.functions:
--> 136             raise Exception("Unable to add DataPipe function name {} as it is already taken".format(function_name))
    137 
    138         def class_function(cls, enable_df_api_tracing, source_dp, *args, **kwargs):

Exception: Unable to add DataPipe function name sharding_filter as it is already taken

可能的原因和解决方案是什么?

o2g1uqev

o2g1uqev1#

这个问题现在已经自动解决了。库工作正常。由于数据管道处于测试开发阶段,我猜他们可能已经更新了一些代码。

相关问题