PaddleNLP [Bug]: 在设置多卡推理时，Chatglm3相关代码报错,

lvmkulzt 于 2个月前发布在其他

关注(0)|答案(1)|浏览(44)

软件环境

- paddlepaddle:最新
- paddlepaddle-gpu: 
- paddlenlp: 最新

重复问题

我已搜索现有的问题

错误描述

LAUNCH INFO 2024-06-06 10:08:32,637 ------------------------- ERROR LOG DETAIL -------------------------
06 10:08:31,074] [    INFO] - We are using <class 'paddlenlp.transformers.chatglm_v2.configuration.ChatGLMv2Config'> to load 'THUDM/chatglm3-6b'.
[2024-06-06 10:08:31,074] [    INFO] - Loading configuration file /home/xy/.paddlenlp/models/THUDM/chatglm3-6b/config.json
[2024-06-06 10:08:31,074] [ WARNING] - You are using a model of type chatglm2 to instantiate a model of type chatglm_v2. This is not supported for all configurations of models and can yield errors.
[2024-06-06 10:08:31,075] [    INFO] - We are using <class 'paddlenlp.transformers.chatglm_v2.modeling.ChatGLMv2ForCausalLM'> to load 'THUDM/chatglm3-6b'.
[2024-06-06 10:08:31,075] [    INFO] - Loading configuration file /home/xy/.paddlenlp/models/THUDM/chatglm3-6b/config.json
[2024-06-06 10:08:31,075] [ WARNING] - You are using a model of type chatglm2 to instantiate a model of type chatglm_v2. This is not supported for all configurations of models and can yield errors.
[2024-06-06 10:08:31,076] [    INFO] - Loading weights file from cache at /home/xy/.paddlenlp/models/THUDM/chatglm3-6b/model_state.pdparams
Traceback (most recent call last):
  File "/ssd2/xieyu/tasks/llm/predictor.py", line 1623, in <module>
    predict()
  File "/ssd2/xieyu/tasks/llm/predictor.py", line 1536, in predict
    predictor = create_predictor(predictor_args, model_args)
  File "/ssd2/xieyu/tasks/llm/predictor.py", line 1267, in create_predictor
    model = AutoModelForCausalLM.from_pretrained(
  File "/home/xy/env_new/lib/python3.9/site-packages/paddlenlp/transformers/auto/modeling.py", line 778, in from_pretrained
    return cls._from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
  File "/home/xy/env_new/lib/python3.9/site-packages/paddlenlp/transformers/auto/modeling.py", line 326, in _from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
  File "/home/xy/env_new/lib/python3.9/site-packages/paddlenlp/transformers/model_utils.py", line 2265, in from_pretrained
    state_dict = cls.convert_tensor_parallel(resolved_archive_file, config)
  File "/home/xy/env_new/lib/python3.9/site-packages/paddlenlp/transformers/conversion_utils.py", line 1226, in convert_tensor_parallel
    name_action_mappings = cls._get_tensor_parallel_mappings(config)
  File "/home/xy/env_new/lib/python3.9/site-packages/paddlenlp/transformers/conversion_utils.py", line 1304, in _get_tensor_parallel_mappings
    raise NotImplementedError
NotImplementedError