问题
- 在我们能解决403访问403 Forbidden #676的问题之前,无法从Ollama服务器拉取模型
- 在我写这篇的时候,我不认为Ollama注册表(具有不同类型的Docker注册表)可用
- 我设法绕过了403问题,在Food for thought用例:Github Actions
#1145中演示了,并将相同的digests
作为docker镜像层推送到Docker注册表:) - 因此,推送我私有控制的Docker注册表,拉取它,并在已经管理/扩展以运行模型的隔离Kubernetes环境中使模型可用。
由于我们已经建立了一个具有围绕Docker镜像、Docker注册表的自动化的DevSecOps平台,我想利用这一点并重用已经在现有基础设施中设置的Ollama模型。Ollama对于CLI和服务器功能也已经很好了!
设计
- 鉴于blobs的内部只是docker镜像,为什么不为Docker镜像提供导出/导入工具?
- 我可以想象模型存储得非常类似于Docker镜像
mediaType
与docker的vnd.docker.container.image.v1+json
相同
ollama export llama2 marcellodesales/genai/models/llama2
- 将模型导出为带有给定名称的docker镜像
ollama import llama2 marcellodesales/genai/models/llama2
- 将模型作为Docker镜像导入到ollama的模型中
实现
- 如果有人想要实现这个功能,可以参考以下一些命令序列
- 如果这个功能被接受,我可以在本周末发送PR...
- 像往常一样列出模型
$ docker run --network host -ti -v $(pwd):$(pwd) -w $(pwd) -v $HOME/.ollama:/root/.ollama ollama/ollama list
NAME ID SIZE MODIFIED
codellama:code fc84f39375bc 3.8 GB 16 hours ago
- 检查模型
$ cat models/manifests/registry.ollama.ai/library/codellama/code
{"schemaVersion":2,"mediaType":"application/vnd.docker.distribution.manifest.v2+json","config":{"mediaType":"application/vnd.docker.container.image.v1+json","digest":"sha256:23fbdb4ea003a1e1c38187539cc4cc8e85c6fb80160a659e25894ca60e781a33","size":455},"layers":[{"mediaType":"application/vnd.ollama.image.model","digest":"sha256:8b2eceb7b7a11c307bc9deed38b263e05015945dc0fa2f50c0744c5d49dd293e","size":3825898144},{"mediaType":"application/vnd.ollama.image.license","digest":"sha256:8c17c2ebb0ea011be9981cc3922db8ca8fa61e828c5d3f44cb6ae342bf80460b","size":7020},{"mediaType":"application/vnd.ollama.image.license","digest":"sha256:590d74a5569b8a20eb2a8b0aa869d1d1d3faf6a7fdda1955ae827073c7f502fc","size":4790},{"mediaType":"application/vnd.ollama.image.params","digest":"sha256:d2b44be9e12117ee2652e9a6c51df28ef408bf487e770b11ee0f7bce8790f3ca","size":31}]}%
ollama models export llama2 marcellodesales/genai/models/llama2
实现
- 为所选模型选择清单并复制到临时目录
mkdir selected-model/manifests/registry.ollama.ai/library/llama2/latest
cp models/manifests/registry.ollama.ai/library/llama2/latest selected-model/manifests/registry.ollama.ai/library/llama2/latest
cp -R listed layers blobs to selected-model/blobs
- 为数据生成docker镜像
FROM busybox AS data
WORKDIR /marcellodesales/platforms/vionix/genai/model
COPY selected-model .
- 然后运行命令
docker images | grep marcellodesales
marcellodesales/genai/models/llama2 latest da34440295ec 26 hours ago 3.83GB
ollama models import marcellodesales/genai/models/llama2
实现
注意:导入命令在manifests下有标签信息,因此无需参数
- 只需生成docker镜像并推送
- 解析给定的清单,选择文件系统中的blobs = 选定的项目
- 对于选定的项目,创建一个用作数据容器的Docker镜像并推送到您的注册表
- 在运行Ollama服务器的目标处,拉取docker镜像并将数据移回.ollama/models
- 通过绕过描述的限制验证新模型是否已安装
$ docker run -ti -v /Users/mdesales/.ollama/models:/data marcellodesales/genai/models/llama2 cp -Rv /marcellodesales/platforms/vionix/genai/model/blobs /data
'/marcellodesales/platforms/vionix/genai/model/blobs/sha256:2759286baa875dc22de5394b4a925701b1896a7e3f8e53275c36f75a877a82c9' -> '/data/blobs/sha256:2759286baa875dc22de5394b4a925701b1896a7e3f8e53275c36f75a877a82c9'
'/marcellodesales/platforms/vionix/genai/model/blobs/sha256:5407e3188df9a34504e2071e0743682d859b68b6128f5c90994d0eafae29f722' -> '/data/blobs/sha256:5407e3188df9a34504e2071e0743682d859b68b6128f5c90994d0eafae29f722'
'/marcellodesales/platforms/vionix/genai/model/blobs/sha256:7c23fb36d80141c4ab8cdbb61ee4790102ebd2bf7aeff414453177d4f2110e5d' -> '/data/blobs/sha256:7c23fb36d80141c4ab8cdbb61ee4790102ebd2bf7aeff414453177d4f2110e5d'
'/marcellodesales/platforms/vionix/genai/model/blobs/sha256:22f7f8ef5f4c791c1b03d7eb414399294764d7cc82c7e94aa81a1feb80a983a2' -> '/data/blobs/sha256:22f7f8ef5f4c791c1b03d7eb414399294764d7cc82c7e94aa81a1feb80a983a2'
'/marcellodesales/platforms/vionix/genai/model/blobs/sha256:2e0493f67d0c8c9c68a8aeacdf6a38a2151cb3c4c1d42accf296e19810527988' -> '/data/blobs/sha256:2e0493f67d0c8c9c68a8aeacdf6a38a2151cb3c4c1d42accf296e19810527988'
'/marcellodesales/platforms/vionix/genai/model/blobs/sha256:8c17c2ebb0ea011be9981cc3922db8ca8fa61e828c5d3f44cb6ae342bf80460b' -> '/data/blobs/sha256:8c17c2ebb0ea011be9981cc3922db8ca8fa61e828c5d3f44cb6ae342bf80460b'
'/marcellodesales/platforms/vionix/genai/model/blobs' -> '/data/blobs'
$ docker run -ti -v /Users/mdesales/.ollama/models:/data marcellodesales/genai/models/llama2 cp -Rv /marcellodesales/platforms/vionix/genai/model/manifests /data
'/marcellodesales/platforms/vionix/genai/model/manifests/registry.ollama.ai/library/llama2/latest' -> '/data/manifests/registry.ollama.ai/library/llama2/latest'
'/marcellodesales/platforms/vionix/genai/model/manifests/registry.ollama.ai/library/llama2' -> '/data/manifests/registry.ollama.ai/library/llama2'
'/marcellodesales/platforms/vionix/genai/model/manifests/registry.ollama.ai/library' -> '/data/manifests/registry.ollama.ai/library'
'/marcellodesales/platforms/vionix/genai/model/manifests/registry.ollama.ai' -> '/data/manifests/registry.ollama.ai'
'/marcellodesales/platforms/vionix/genai/model/manifests' -> '/data/manifests'
然后,只需验证最终结果:
$ docker run --network host -ti -v $(pwd):$(pwd) -w $(pwd) -v $HOME/.ollama:/root/.ollama ollama/ollama list
NAME ID SIZE MODIFIED
codellama:code fc84f39375bc 3.8 GB 16 hours ago
llama2:latest fe938a131f40 3.8 GB 6 seconds ago
问题
- 对这个功能有兴趣吗?有人在做类似的事情吗?
- 有计划开源注册表服务器吗?有可能部署我们自己的吗?
- 有计划修复在403 Forbidden #676中讨论的403问题吗?
1条答案
按热度按时间5ssjco0h1#
一个选项是使用 OCI Registry as a Service