问题是什么？

在向/api/embeddings发送一些请求后，服务似乎挂起，需要重新启动才能恢复。

以下是一些日志：

[GIN] 2024/07/18 - 00:52:55 | 200 |  2.824880868s |   10.255.56.113 | POST     "/api/embeddings"
time=2024-07-18T00:52:55.388Z level=INFO source=routes.go:298 msg="embedding generation failed: do embedding request: Post \"http://127.0.0.1:35303/embedding\": context canceled"
[GIN] 2024/07/18 - 00:52:55 | 500 |   257.27018ms |   10.255.56.113 | POST     "/api/embeddings"
cuda driver library failed to get device context 800time=2024-07-18T00:57:55.395Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:55.649Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:55.898Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:56.148Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:56.399Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:56.649Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:56.899Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:57.148Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:57.399Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:57.649Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:57.898Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:58.148Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:58.398Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:58.648Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:58.899Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:59.148Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:59.398Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:59.649Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:57:59.898Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
cuda driver library failed to get device context 800time=2024-07-18T00:58:00.149Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
time=2024-07-18T00:58:00.396Z level=WARN source=sched.go:634 msg="gpu VRAM usage didn't recover within timeout" seconds=5.006997071 model=/root/.ollama/models/blobs/sha256-970aa74c0a90ef7482477cf803618e776e173c007bf957f635f1015bfcfef0e6
cuda driver library failed to get device context 800time=2024-07-18T00:58:00.398Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
time=2024-07-18T00:58:00.646Z level=WARN source=sched.go:634 msg="gpu VRAM usage didn't recover within timeout" seconds=5.257512846 model=/root/.ollama/models/blobs/sha256-970aa74c0a90ef7482477cf803618e776e173c007bf957f635f1015bfcfef0e6
cuda driver library failed to get device context 800time=2024-07-18T00:58:00.648Z level=WARN source=gpu.go:399 msg="error looking up nvidia GPU memory"
time=2024-07-18T00:58:00.895Z level=WARN source=sched.go:634 msg="gpu VRAM usage didn't recover within timeout" seconds=5.506938263 model=/root/.ollama/models/blobs/sha256-970aa74c0a90ef7482477cf803618e776e173c007bf957f635f1015bfcfef0e6

操作系统

Linux

GPU

Nvidia

CPU

Intel

Ollama版本

0.2.5