Ollama 悬挂在重采样上,因为标记17158:'< token>'不符合语法规则,

p8h8hvxi  于 1个月前  发布在  其他
关注(0)|答案(5)|浏览(30)

在Ubuntu 22.04上,当某个请求导致ollama陷入无限循环时,可以通过以下方法来确认调试标志是否设置正确:

  1. 首先,确保在环境变量中设置了正确的值。在~/.bashrc文件中添加以下内容:
export OLLAMA_DEBUG=

export OLLAMA_DEBUG=1
set | grep OLLAMA
OLLAMA_DEBUG=1

然后运行source ~/.bashrc使更改生效。

  1. ollama.service文件的[Service]部分添加以下内容:
Environment="OLLAMA_DEBUG=

[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3

Environment="PATH=/home/……:/snap/bin OLLAMA_DEBUG=1"

[Install]
WantedBy=default.target

"
  1. 重新启动服务器:
sudo systemctl restart ollama.service
  1. 检查日志。在终端中运行以下命令:
journalctl -u ollama.service --since "$(date -d '1 hour ago' +'%Y-%m-%dT%H:%M:%S')" --until "$(date +'%Y-%m-%dT%H:%M:%S')" | grep -i debug

这将显示过去一小时内所有包含“debug”关键字的日志条目。如果调试标志设置正确,你应该能看到更多的日志输出。

u5i3ibmn

u5i3ibmn1#

循环是否尊重 systemd 的 RestartSec=3 设置?
您可以通过更改 ollama.service 文件并将 ExecStart=ollama serve 设置为运行 Package 脚本来诊断,例如保持进程运行或转储其环境变量。
要查看正在运行的进程的环境并检查调试标志,只需从 procfs 读取:

cat /proc/$PID/environ | tr '\0' '\n' | less

编辑:而不是花时间在 systemd 的不便和开销上,您可以杀死服务并直接运行 sudo -u ollama /usr/local/bin/ollama serve,然后在单独的终端窗口中运行模型时监控日志输出。

g0czyy6m

g0czyy6m2#

啊,这太棒了,直接运行它。a) 正确设置环境变量;b) 现在我可以看到 level=DEBUG 在日志中。我想我还不清楚如何修改 ollama.service 以正确设置环境变量。

f4t66c6m

f4t66c6m3#

这非常有趣,似乎instructor正在生成提示,甚至mistral:7b也无法应对,但更有趣的是,以一种导致ollama呕吐的方式。我让ollama在这里卡住,完全不返回。

time=2024-02-23T15:58:59.224Z level=DEBUG source=routes.go:1225 msg="chat handler" prompt="[INST] \n                As a genius expert, your task is to understand the content and provide\n                the parsed objects in json that match the following json_schema:\n\n                {'messages': {'items': {'$ref': '#/$defs/MessagePair'}, 'title': 'Messages', 'type': 'array'}}\n                \nHere are some more definitions to adhere too:\n{'MessagePair': {'properties': {'respectful': {'title': 'Respectful', 'type': 'string'}, 'nondisrespectful': {'title': 'Nondisrespectful', 'type': 'string'}}, 'required': ['respectful', 'nondisrespectful'], 'title': 'MessagePair', 'type': 'object'}}\n\n\n                As a genius expert, your task is to understand the content and provide\n                the parsed objects in json that match the following json_schema:\n\n                {'messages': {'items': {'$ref': '#/$defs/MessagePair'}, 'title': 'Messages', 'type': 'array'}}\n                \nHere are some more definitions to adhere too:\n{'MessagePair': {'properties': {'respectful': {'title': 'Respectful', 'type': 'string'}, 'nondisrespectful': {'title': 'Nondisrespectful', 'type': 'string'}}, 'required': ['respectful', 'nondisrespectful'], 'title': 'MessagePair', 'type': 'object'}} Generate 5 pairs of short instant messages, where each pair contains a non-disrespectful (respectful or neutral) message and a corresponding disrespectful message exemplifying 'Dishonesty'. [/INST]" images=0
[1708703939] slot 0 is processing [task id: 0]
[1708703939] slot 0 : in cache: 0 tokens | to process: 370 tokens
[1708703939] slot 0 : kv cache rm - [0, end)
[1708703939] Resampling because token 17158: ' Based' does not meet grammar rules
[1708703941] Resampling because token 12069: 'Please' does not meet grammar rules
[1708703941] Resampling because token 12069: 'Please' does not meet grammar rules
[1708703941] Resampling because token 12069: 'Please' does not meet grammar rules
[1708703941] Resampling because token 12069: 'Please' does not meet grammar rules
[1708703941] Resampling because token 12069: 'Please' does not meet grammar rules
[1708703951] slot 0: context shift - n_keep = 0, n_left = 2046, n_discard = 1023
[1708703959] slot 0: context shift - n_keep = 0, n_left = 2046, n_discard = 1023
[1708703967] slot 0: context shift - n_keep = 0, n_left = 2046, n_discard = 1023
[1708703974] slot 0: context shift - n_keep = 0, n_left = 2046, n_discard = 1023
[1708703982] slot 0: context shift - n_keep = 0, n_left = 2046, n_discard = 1023

它一直这样做,直到我杀死它,阻塞线程和套接字。

iqjalb3h

iqjalb3h4#

你好,@boxabirds,你是否偶然使用了JSON模式?抱歉你遇到了这个问题。

zaqlnxep

zaqlnxep5#

Environment="PATH=/home/……:/snap/bin OLLAMA_DEBUG=1"
OLLAMA_DEBUG needs to be on its own Environment line

Environment="PATH=/home/…<various paths>…:/snap/bin"
Environment="OLLAMA_DEBUG=1"

相关问题