PaddleNLP [Bug]: undefined symbol: _ZN6paddle17OpMetaInfoBuilderC1EOSsm

eufgjt7s  于 7个月前  发布在  其他
关注(0)|答案(1)|浏览(116)

软件环境

  1. paddle-bfloat              0.1.7
  2. paddle2onnx                1.2.1
  3. paddlefsl                  1.1.0
  4. paddlehub                  2.4.0
  5. paddlemix                  0.1.0               
  6. paddlenlp                  3.0.0b0.post20240728
  7. paddlepaddle-gpu           2.6.1.post112
  8. paddleslim                 2.6.0

重复问题

  • 我已搜索现有的问题

错误描述

  1. 在编译csrc 算子时报错未定义的符号：_ZN6paddle17OpMetaInfoBuilderC1EOSsm
  2. 在相关问题中并未得到解决：
  3. [按paddle教程自定义算子时链接库找不到](https://aistudio.baidu.com/paddle/forum/topic/show/1729464)
  4. [#8222](https://github.com/PaddlePaddle/PaddleDetection/issues/8222)
  5. [#9000](https://github.com/PaddlePaddle/PaddleDetection/issues/9000)
  6. [#4045](https://github.com/PaddlePaddle/PaddleNLP/issues/4045)
  7. [#43041](https://github.com/PaddlePaddle/Paddle/issues/43041)
  8. 其中[#8222]提到更换gcc版本，尝试过切换gcc 11.2 和gcc 8.5 无效
  10. ### 报错log输出
  12. [2024-07-31 13:40:37,079] [    INFO] file_util.py:130 - copying build/paddlenlp_ops/lib.linux-x86_64-cpython-39/write_cache_kv.cu.o -> build/paddlenlp_ops/bdist.linux-x86_64/egg        
  13. [2024-07-31 13:40:37,082] [    INFO] file_util.py:130 - copying build/paddlenlp_ops/lib.linux-x86_64-cpython-39/paddlenlp_ops.so -> build/paddlenlp_ops/bdist.linux-x86_64/egg           
  14. [2024-07-31 13:40:37,099] [    INFO] file_util.py:130 - copying build/paddlenlp_ops/lib.linux-x86_64-cpython-39/dequant_int8.cu.o -> build/paddlenlp_ops/bdist.linux-x86_64/egg          
  15. [2024-07-31 13:40:37,103] [    INFO] file_util.py:130 - copying build/paddlenlp_ops/lib.linux-x86_64-cpython-39/transpose_removing_padding.cu.o -> build/paddlenlp_ops/bdist.linux-x86_64
  16. /egg                                                                                                                                                                                     
  17. [2024-07-31 13:40:37,108] [    INFO] file_util.py:130 - copying build/paddlenlp_ops/lib.linux-x86_64-cpython-39/save_with_output_msg.o -> build/paddlenlp_ops/bdist.linux-x86_64/egg     
  18. [2024-07-31 13:40:37,183] [    INFO] bdist_egg.py:177 - creating stub loader for paddlenlp_ops.so                                                                                        
  19. Traceback (most recent call last):                                                                                                                                                       
  20.   File "/mnt/petrelfs/taitan/projects/PaddleMIX/PaddleNLP/csrc/setup_cuda.py", line 53, in <module>                                                                                     
  21.     setup(                                                                                                                                                                               
  22.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/paddle/utils/cpp_extension/cpp_extension.py", line 236, in setup                                        
  23.     setuptools.setup(**attr)                                                                                                                                                             
  24.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/__init__.py", line 104, in setup                                                             
  25.     return distutils.core.setup(**attrs)                                                                                                                                                 
  26.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
  27.     return run_commands(dist)
  28.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
  29.     dist.run_commands()
  30.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
  31.     self.run_command(cmd)
  32.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/dist.py", line 967, in run_command    super().run_command(command)  File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
  33.     cmd_obj.run()
  34.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/command/install.py", line 87, in run
  35.     self.do_egg_install()
  36.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/command/install.py", line 139, in do_egg_install
  37.     self.run_command('bdist_egg')
  38.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
  39.     self.distribution.run_command(command)
  40.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/dist.py", line 967, in run_command
  41.     super().run_command(command)
  42.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
  43.     cmd_obj.run()
  44.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 179, in run
  45.     write_stub(os.path.basename(ext_name), pyfile)
  46.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/paddle/utils/cpp_extension/extension_utils.py", line 222, in custom_write_stub
  47.     new_custom_ops = load_op_meta_info_and_register_op(so_path)
  48.   File "/mnt/petrelfs/taitan/miniconda3/envs/paddle/lib/python3.9/site-packages/paddle/utils/cpp_extension/extension_utils.py", line 160, in load_op_meta_info_and_register_op
  49.     core.load_op_meta_info_and_register_op(lib_filename)
  50. RuntimeError: (PreconditionNotMet) The third-party dynamic library (/mnt/petrelfs/taitan/projects/PaddleMIX/PaddleNLP/csrc/build/paddlenlp_ops/lib.linux-x86_64-cpython-39/paddlenlp_ops
  51. .so) that Paddle depends on is not configured correctly. (error code is /mnt/petrelfs/taitan/projects/PaddleMIX/PaddleNLP/csrc/build/paddlenlp_ops/lib.linux-x86_64-cpython-39/paddlenlp
  52. _ops.so: undefined symbol: _ZN6paddle17OpMetaInfoBuilderC1EOSsm)
  53.   Suggestions:
  54.   1. Check if the third-party dynamic library (e.g. CUDA, CUDNN) is installed correctly and its version is matched with paddlepaddle you installed.
  55.   2. Configure third-party dynamic library environment variables as follows:
  56.   - Linux: set LD_LIBRARY_PATH by `export LD_LIBRARY_PATH=...`
  57.   - Windows: set PATH by `set PATH=XXX; (at ../paddle/phi/backends/dynload/dynamic_loader.cc:312)
  1. ### 稳定复现步骤 & 代码
  3. Ubuntu 20.04.6 LTS
  4. A100 80G
  5. cuda 11.2
  6. gcc 8.5.0
  7. python=3.9.19
  9. cudnn
  10. #define CUDNN_MAJOR 8
  11. #define CUDNN_MINOR 1
  12. #define CUDNN_PATCHLEVEL 1
PaddleNLP

1条答案

按热度按时间
hivapdat

hivapdat1#

这个问题是由系统环境引起的，可能在不同的环境中有不同的原因。下面是一个检查方案：

  • 检查cmake版本，并与官方的docker环境进行对照。
  • 检查系统中是否存在python库，因为在编译过程中会用到python.h文件，这个文件在conda中可能不存在。因此，需要链接到系统的python库。可以通过检查编译过程的输出来查看。
  • 检查gcc版本，v8以上的版本应该都可以正常工作。
赞(0）分享  回复(0）举报  7个月前
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com