$Hi, how to set trust_remote_code=True in mii pipeline?
Code
from mii import pipeline
pipe = pipeline("/root/Qwen-7B-Chat")
pipe.tokenizer.tokenizer.eos_token_id = 151643
output = pipe(["DeepSpeed is"], max_new_tokens=128, do_sample=False)
print(output)
Output
The repository for /root/Qwen-7B-Chat contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co//root/Qwen-7B-Chat.
You can avoid this prompt in future by passing the argument `trust_remote_code=True`.
Do you wish to run the custom code? [y/N]
2条答案
按热度按时间vcirk6k61#
我认为目前没有暴露的接口来设置'trust_remote_code'参数。您可以通过直接修改deepspeed源代码中的load方法来设置它。
https://github.com/microsoft/DeepSpeed/blob/2a6c58df34ea04e0aa451a4623b7c7d49b087a50/deepspeed/inference/v2/checkpoint/huggingface_engine.py#L24
1u4esq0p2#
Hi @gujingit,不幸的是@ZonePG是正确的。我认为这个添加已经迟到了。让我整理一个PR,将
hf_kwargs
选项添加到我们的API中。这应该允许传递任意参数来从HuggingFace加载检查点权重。