模型描述
我不确定OpenGVLab的这些新模型在这里是否受支持。
OpenGVLab/InternVL2-8B
在执行docker run后，我得到了以下错误，
2024-08-22 05:51:08.649 | INFO | text_generation_server.utils.import_utils:<module>:75 - Detected system cuda
Traceback (most recent call last):
File "/opt/conda/bin/text-generation-server", line 8, in <module>
sys.exit(app())
File "/opt/conda/lib/python3.10/site-packages/text_generation_server/cli.py", line 118, in serve
server.serve(
File "/opt/conda/lib/python3.10/site-packages/text_generation_server/server.py", line 297, in serve
asyncio.run(
File "/opt/conda/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/opt/conda/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/opt/conda/lib/python3.10/site-packages/text_generation_server/server.py", line 231, in serve_inner
model = get_model(
File "/opt/conda/lib/python3.10/site-packages/text_generation_server/models/__init__.py", line 1117, in get_model
raise ValueError(f"Unsupported model type {model_type}")
ValueError: Unsupported model type internvl_chat
rank=0
2024-08-22T05:51:12.673589Z ERROR text_generation_launcher: Shard 0 failed to start
2024-08-22T05:51:12.673618Z INFO text_generation_launcher: Shutting down shards
Error: ShardCannotStart请添加支持，这些是表现最好的开源模型。
开源状态
- 模型实现可用
- 模型权重可用
为实现提供有用的链接
- 无响应*
1条答案
按热度按时间bqucvtff1#
你好，@dhandhalyabhavik 👋
感谢你的请求。目前我们的带宽受到限制，所以很遗憾无法满足所有人的需求。但是如果有很多其他人也需要这个模型在TGI(例如通过在问题中添加👍表情符号)我们会更好地了解如何进行优先级排序。
此外，如果你有时间和热情，请随时发起一个PR。我会尽力为你提供关于如何实施该模型的指导。