系统信息 / 系統信息
cuda 12.0 xprobe/xinference:v0.13.1 docker image from dockerhub
是否使用 Docker 运行 Xinference?
- docker / 是
- pip install / 通过 pip install 安装
- installation from source / 从源码安装
版本信息 / 版本信息
v0.13.1
用以启动 xinference 的命令
xinference-local --host 0.0.0.0 --port 9997
复现过程
start container show the error msg:
2024-07-16 Traceback (most recent call last):
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/llama_cpp/llama_cpp.py", line 75, in _load_shared_library
2024-07-16 return ctypes.CDLL(str(_lib_path), **cdll_args) # type: ignore
2024-07-16 File "/usr/lib/python3.10/ctypes/init.py", line 374, in init
2024-07-16 self._handle = _dlopen(self._name, mode)
2024-07-16 OSError: libcublas.so.12: cannot open shared object file: No such file or directory
2024-07-16
2024-07-16 During handling of the above exception, another exception occurred:
2024-07-16
2024-07-16 Traceback (most recent call last):
2024-07-16 File "/usr/local/bin/xinference-local", line 5, in
2024-07-16 from xinference.deploy.cmdline import local
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/xinference/init.py", line 38, in
2024-07-16 _install()
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/xinference/init.py", line 35, in _install
2024-07-16 install_model()
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/xinference/model/init.py", line 17, in _install
2024-07-16 from .llm import _install as llm_install
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/xinference/model/llm/init.py", line 20, in
2024-07-16 from .core import (
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/xinference/model/llm/core.py", line 26, in
2024-07-16 from ...types import PeftModelConfig
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/xinference/types.py", line 399, in
2024-07-16 from llama_cpp import Llama
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/llama_cpp/init.py", line 1, in
2024-07-16 from .llama_cpp import *
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/llama_cpp/llama_cpp.py", line 88, in
2024-07-16 _lib = _load_shared_library(_lib_base_name)
2024-07-16 File "/usr/local/lib/python3.10/dist-packages/llama_cpp
4条答案
按热度按时间8oomwypt1#
你需要将你的CUDA版本升级到至少12.4。
yqyhoc1h2#
感谢您的回复。升级CUDA并非易事。是否可以保留两个版本的Docker镜像？
ckx4rj1h3#
如果你想坚持使用旧版本的镜像，你可以拉取旧版本的镜像，然后在docker内部使用
pip install xinference==0.13.1进行升级。3mpgtkmj4#
如果你想坚持使用旧版本的镜像，你可以拉取旧版本的镜像，然后在docker内部使用
pip install xinference==0.13.1进行升级。可以启动服务器，但是无法运行gemma-2,错误信息为"需要升级TensorFlow"。