Android Studio 这段代码中出现HFValidationError的原因是什么?如何解决这个错误?

ny6fqffe  于 2023-06-24  发布在  Android
关注(0)|答案(1)|浏览(709)

我在Chaquopy android studio项目中的python代码:

import torch as tc
from transformers import GPT2Tokenizer, GPT2Model


def generate_text(txt):
    """
    Generate chat
    https://huggingface.co/gpt2
    """

    #Load Model files
    tokenizer = GPT2Tokenizer.from_pretrained('assets/') #This line causing error
    model = GPT2Model.from_pretrained('assets/')
    #Move moel to GPU if avilable
    device = tc.device("cuda" if tc.cuda.is_available() else "cpu")
    model.to(device)

    encoded_input = tokenizer(txt, return_tensors='pt')
    output = model(**encoded_input)

    return str(output)

现在它显示以下错误:

E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.example.chaquopy_130application, PID: 4867
    com.chaquo.python.PyException: HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'assets/'.
        at <python>.huggingface_hub.utils._validators.validate_repo_id(_validators.py:164)
        at <python>.huggingface_hub.utils._validators._inner_fn(_validators.py:110)
        at <python>.huggingface_hub.utils._deprecation.inner_f(_deprecation.py:103)
        at <python>.transformers.file_utils.get_list_of_files(file_utils.py:2103)
        at <python>.transformers.tokenization_utils_base.get_fast_tokenizer_file(tokenization_utils_base.py:3486)
        at <python>.transformers.tokenization_utils_base.from_pretrained(tokenization_utils_base.py:1654)
        at <python>.pythonScript.generate_text(pythonScript.py:30)

我已经把124 M GPT-2模型的所有文件 checkpointencoder.jsonhparams.jsonmodel.ckpt.data-00000-of-00001model.ckpt.index,* model. ckpt. meta *,vocab.bpe 文件放在'assets'文件夹中。

qoefvg9y

qoefvg9y1#

from_pretrained文档并不完全清楚它如何区分huggingface存储库名称和本地路径,尽管所有本地路径示例都以斜杠结尾。在任何情况下,当使用Chaquopy加载数据文件时,必须始终使用绝对路径,正如FAQ中所说的那样。
假设你的“assets”目录和Python代码在同一层,你可以这样做:

from os.path import dirname
tokenizer = GPT2Tokenizer.from_pretrained(f'{dirname(__file__)}/assets/')

相关问题