ollama 最新版本(0.3.4)无法检测到AMD GPU(Instinct MI210),

dzjeubhm  于 2个月前  发布在  其他
关注(0)|答案(1)|浏览(21)

最新版本的Ollama(0.3.4)在Docker容器中运行时无法检测或利用AMD GPU(Instinct MI210)。
我尝试使用Docker运行它,因为我不希望更新Ollama的工作版本为0.2.8。
我按照这里的说明进行了操作:https://hub.docker.com/r/ollama/ollama
命令:docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm
请参阅下面的错误信息(我也尝试过在我服务器上更新Ollama到最新版本,但遇到了同样的问题)
任何见解或建议将不胜感激!

[genai1~]$ docker exec 92ebd80dcef1 /opt/rocm/bin/rocm-smi
========================================== ROCm System Management Interface ==========================================
==================================================== Concise Info ====================================================
Device  Node  IDs              Temp    Power   Partitions          SCLK     MCLK     Fan  Perf  PwrCap  VRAM%  GPU%  
              (DID,     GUID)  (Edge)  (Avg)   (Mem, Compute, ID)                                                    
======================================================================================================================
Traceback (most recent call last):
  File "/opt/rocm/bin/rocm-smi", line 4074, in <module>
    showAllConcise(deviceList)
  File "/opt/rocm/bin/rocm-smi", line 2033, in showAllConcise
    zip(range(len(max_widths)), values['card%s' % (str(device))])), None)
  File "/opt/rocm/bin/rocm-smi", line 872, in printLog
    print(logstr + '\n', end='')
UnicodeEncodeError: 'ascii' codec can't encode character '\xb0' in position 35: ordinal not in range(128)
[genai1~]$ docker exec 92ebd80dcef1 ls -la /root/.ollama
total 20
drwxr-xr-x 3 root root 4096 Aug 10 19:17 .
dr-xr-x--- 1 root root 4096 Aug 10 19:46 ..
-rw------- 1 root root  387 Aug 10 19:17 id_ed25519
-rw-r--r-- 1 root root   81 Aug 10 19:17 id_ed25519.pub
drwxr-xr-x 4 root root 4096 Aug 10 19:55 models
[genai1~]$ docker logs 92ebd80dcef1
2docker logs 92ebd80dcef1
2024/08/10 19:46:51 routes.go:1108: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-08-10T19:46:51.654Z level=INFO source=images.go:781 msg="total blobs: 0"
time=2024-08-10T19:46:51.654Z level=INFO source=images.go:788 msg="total unused blobs removed: 0"
time=2024-08-10T19:46:51.654Z level=INFO source=routes.go:1155 msg="Listening on [::]:11434 (version 0.3.4)"
time=2024-08-10T19:46:51.655Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama2612956243/runners
time=2024-08-10T19:46:54.488Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cuda_v11 rocm_v60102 cpu cpu_avx cpu_avx2]"
time=2024-08-10T19:46:54.488Z level=INFO source=gpu.go:204 msg="looking for compatible GPUs"
time=2024-08-10T19:46:54.491Z level=WARN source=amd_linux.go:59 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
time=2024-08-10T19:46:54.491Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=0 total="0 B"
time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=1 total="0 B"
time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=2 total="0 B"
time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=3 total="0 B"
time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:360 msg="no compatible amdgpu devices detected"
time=2024-08-10T19:46:54.492Z level=INFO source=gpu.go:347 msg="no compatible GPUs were discovered"
time=2024-08-10T19:46:54.492Z level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="250.9 GiB" available="230.3 GiB"

Error: something went wrong, please see the ollama server logs for details
[genai1~]$ docker exec 92ebd80dcef1 env | grep OLLAMA
OLLAMA_HOST=0.0.0.0
[genai1~]$ docker exec 92ebd80dcef1 ls -l /opt/rocm/bin
total 78928
-rwxr-xr-x 1 root root  3993744 May 28 21:10 MIOpenDriver
lrwxrwxrwx 1 root root       35 Jun 10 14:56 amd-smi -> ../libexec/amdsmi_cli/amdsmi_cli.py
lrwxrwxrwx 1 root root       24 Jun 10 14:51 amdclang -> ../lib/llvm/bin/amdclang
lrwxrwxrwx 1 root root       26 Jun 10 14:51 amdclang++ -> ../lib/llvm/bin/amdclang++
lrwxrwxrwx 1 root root       27 Jun 10 14:51 amdclang-cl -> ../lib/llvm/bin/amdclang-cl
lrwxrwxrwx 1 root root       28 Jun 10 14:51 amdclang-cpp -> ../lib/llvm/bin/amdclang-cpp
lrwxrwxrwx 1 root root       24 Jun 10 14:51 amdflang -> ../lib/llvm/bin/amdflang
lrwxrwxrwx 1 root root       22 Jun 10 14:51 amdlld -> ../lib/llvm/bin/amdlld
lrwxrwxrwx 1 root root       22 Jun 10 14:53 aompcc -> ../lib/llvm/bin/aompcc
-rwxr-xr-x 1 root root   107592 May 28 19:14 clinfo
-rwxrwxr-x 1 root root     2551 May 28 18:08 hipcc
-rwxr-xr-x 1 root root   575472 May 28 18:11 hipcc.bin
-rwxrwxr-x 1 root root    21889 May 28 18:08 hipcc.pl
-rwxrwxr-x 1 root root     1508 May 28 18:08 hipcc_cmake_linker_helper
-rwxrwxr-x 1 root root     2453 May 28 18:08 hipconfig
-rwxr-xr-x 1 root root   575480 May 28 18:11 hipconfig.bin
-rwxrwxr-x 1 root root     7539 May 28 18:08 hipconfig.pl
-rwxr-xr-x 1 root root      784 May 28 18:08 hipconvertinplace-perl.sh
-rwxr-xr-x 1 root root      674 May 28 18:08 hipconvertinplace.sh
-rwxrwxr-x 1 root root     1857 May 28 18:08 hipdemangleatp
-rwxr-xr-x 1 root root      407 May 28 18:08 hipexamine-perl.sh
-rwxr-xr-x 1 root root      557 May 28 18:08 hipexamine.sh
-rwxr-xr-x 1 root root 62142024 May 28 19:14 hipify-clang
-rwxr-xr-x 1 root root   659191 May 28 18:08 hipify-perl
-rw-rw-r-- 1 root root     6486 May 28 18:08 hipvars.pm
-rwxr-xr-x 1 root root     1332 May 28 19:30 install_precompiled_kernels.sh
lrwxrwxrwx 1 root root       21 Jun 10 14:53 mygpu -> ../lib/llvm/bin/mygpu
lrwxrwxrwx 1 root root       22 Jun 10 14:53 mymcpu -> ../lib/llvm/bin/mymcpu
lrwxrwxrwx 1 root root       28 Jun 10 14:51 offload-arch -> ../lib/llvm/bin/offload-arch
-rwxrwxr-x 1 root root    10047 May 28 18:08 roc-obj
-rwxrwxr-x 1 root root     8457 May 28 18:08 roc-obj-extract
-rwxrwxr-x 1 root root       71 May 28 18:08 roc-obj-extract.bat
-rwxrwxr-x 1 root root     6973 May 28 18:08 roc-obj-ls
-rwxrwxr-x 1 root root       56 May 28 18:08 roc-obj-ls.bat
-rwxr-xr-x 1 root root 12314744 May 28 18:57 rocgdb
lrwxrwxrwx 1 root root       31 Jun 10 14:52 rocm-smi -> ../libexec/rocm_smi/rocm_smi.py
-rwxr-xr-x 1 root root    10057 May 28 18:50 rocm_agent_enumerator
-rwxr-xr-x 1 root root   159560 May 28 18:50 rocminfo
-r-xr-xr-x 1 root root    22019 May 28 18:08 rocprof
-r-xr-xr-x 1 root root    11523 May 28 18:08 rocprofv2
-rwxr-xr-x 1 root root    96000 May 28 19:17 rocsys
-rwxr-xr-x 1 root root     9224 May 28 19:30 test_perf.py
[genai1@~]$ docker exec 92ebd80dcef1 ls -l /opt/rocm/lib
total 10486408
drwxr-xr-x 32 root root       4096 Jun 10 14:56 cmake
drwxr-xr-x  3 root root       4096 Jun 10 14:55 hipblaslt
drwxr-xr-x  3 root root       4096 Jun 10 14:54 hipsparselt
drwxr-xr-x  2 root root       4096 Jun 10 14:52 hsa-amd-aqlprofile
lrwxrwxrwx  1 root root         14 Jun 10 14:55 libMIOpen.so -> libMIOpen.so.1
lrwxrwxrwx  1 root root         22 Jun 10 14:54 libMIOpen.so.1 -> libMIOpen.so.1.0.60102
-rwxr-xr-x  1 root root 1120623856 May 28 21:10 libMIOpen.so.1.0.60102
lrwxrwxrwx  1 root root         14 Jun 10 14:52 libOpenCL.so -> libOpenCL.so.1
lrwxrwxrwx  1 root root         16 Jun 10 14:52 libOpenCL.so.1 -> libOpenCL.so.1.2
-rwxr-xr-x  1 root root      36968 May 28 19:13 libOpenCL.so.1.2
lrwxrwxrwx  1 root root         17 Jun 10 14:51 libamd_comgr.so -> libamd_comgr.so.2
lrwxrwxrwx  1 root root         25 Jun 10 14:51 libamd_comgr.so.2 -> libamd_comgr.so.2.7.60102
-rwxr-xr-x  1 root root  149865832 May 28 18:50 libamd_comgr.so.2.7.60102
lrwxrwxrwx  1 root root         16 Jun 10 14:56 libamd_smi.so -> libamd_smi.so.24
lrwxrwxrwx  1 root root         24 Jun 10 14:56 libamd_smi.so.24 -> libamd_smi.so.24.5.60102
-rwxr-xr-x  1 root root    1664088 May 28 18:11 libamd_smi.so.24.5.60102
lrwxrwxrwx  1 root root         16 Jun 10 14:53 libamdhip64.so -> libamdhip64.so.6
-rwxr-xr-x  1 root root   23232120 Oct  6  2023 libamdhip64.so.5
lrwxrwxrwx  1 root root         24 Jun 10 14:53 libamdhip64.so.6 -> libamdhip64.so.6.1.60102
-rwxr-xr-x  1 root root   27904952 May 28 19:13 libamdhip64.so.6.1.60102
-rwxr-xr-x  1 root root    1531472 May 28 19:14 libamdocl64.so
-rw-r--r--  1 root root  295230498 May 28 19:46 libdevice_contraction_operations.a
-rw-r--r--  1 root root 1367985582 May 28 19:46 libdevice_conv_operations.a
-rw-r--r--  1 root root 1908049166 May 28 19:46 libdevice_gemm_operations.a
-rw-r--r--  1 root root  376043220 May 28 19:46 libdevice_other_operations.a
-rw-r--r--  1 root root 1074021278 May 28 19:47 libdevice_reduction_operations.a
lrwxrwxrwx  1 root root         15 Jun 10 14:56 libhipblas.so -> libhipblas.so.2
lrwxrwxrwx  1 root root         23 Jun 10 14:54 libhipblas.so.2 -> libhipblas.so.2.1.60102
-rwxr-xr-x  1 root root    1073688 May 28 21:09 libhipblas.so.2.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 libhipblaslt.so -> libhipblaslt.so.0
lrwxrwxrwx  1 root root         25 Jun 10 14:54 libhipblaslt.so.0 -> libhipblaslt.so.0.7.60102
-rwxr-xr-x  1 root root    9946264 May 28 22:46 libhipblaslt.so.0.7.60102
lrwxrwxrwx  1 root root         14 Jun 10 14:52 libhipfft.so -> libhipfft.so.0
lrwxrwxrwx  1 root root         22 Jun 10 14:52 libhipfft.so.0 -> libhipfft.so.0.1.60102
-rwxr-xr-x  1 root root      61832 May 28 21:51 libhipfft.so.0.1.60102
lrwxrwxrwx  1 root root         15 Jun 10 14:52 libhiprand.so -> libhiprand.so.1
lrwxrwxrwx  1 root root         23 Jun 10 14:52 libhiprand.so.1 -> libhiprand.so.1.1.60102
-rwxr-xr-x  1 root root      18440 May 28 19:51 libhiprand.so.1.1.60102
lrwxrwxrwx  1 root root         23 Jun 10 14:53 libhiprtc-builtins.so -> libhiprtc-builtins.so.6
lrwxrwxrwx  1 root root         31 Jun 10 14:53 libhiprtc-builtins.so.6 -> libhiprtc-builtins.so.6.1.60102
-rwxr-xr-x  1 root root     559736 May 28 19:13 libhiprtc-builtins.so.6.1.60102
lrwxrwxrwx  1 root root         14 Jun 10 14:53 libhiprtc.so -> libhiprtc.so.6
lrwxrwxrwx  1 root root         22 Jun 10 14:53 libhiprtc.so.6 -> libhiprtc.so.6.1.60102
-rwxr-xr-x  1 root root     918128 May 28 19:13 libhiprtc.so.6.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 libhipsolver.so -> libhipsolver.so.0
lrwxrwxrwx  1 root root         25 Jun 10 14:56 libhipsolver.so.0 -> libhipsolver.so.0.1.60102
-rwxr-xr-x  1 root root     307056 May 28 21:09 libhipsolver.so.0.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 libhipsparse.so -> libhipsparse.so.1
lrwxrwxrwx  1 root root         27 Jun 10 14:54 libhipsparse.so.1 -> libhipsparse.so.1.0.0.60102
-rwxr-xr-x  1 root root     312176 May 28 20:01 libhipsparse.so.1.0.0.60102
lrwxrwxrwx  1 root root         19 Jun 10 14:56 libhipsparselt.so -> libhipsparselt.so.0
lrwxrwxrwx  1 root root         27 Jun 10 14:54 libhipsparselt.so.0 -> libhipsparselt.so.0.2.60102
-rwxr-xr-x  1 root root    8293856 May 28 20:08 libhipsparselt.so.0.2.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:52 libhiptensor.so -> libhiptensor.so.0
lrwxrwxrwx  1 root root         25 Jun 10 14:52 libhiptensor.so.0 -> libhiptensor.so.0.1.60102
-rwxr-xr-x  1 root root  334544520 May 28 20:09 libhiptensor.so.0.1.60102
lrwxrwxrwx  1 root root         28 Jun 10 14:52 libhsa-amd-aqlprofile64.so -> libhsa-amd-aqlprofile64.so.1
lrwxrwxrwx  1 root root         36 Jun 10 14:52 libhsa-amd-aqlprofile64.so.1 -> libhsa-amd-aqlprofile64.so.1.0.60102
-rwxr-xr-x  1 root root     583088 May 28 18:50 libhsa-amd-aqlprofile64.so.1.0.60102
lrwxrwxrwx  1 root root         21 Jun 10 14:52 libhsa-runtime64.so -> libhsa-runtime64.so.1
lrwxrwxrwx  1 root root         30 Jun 10 14:52 libhsa-runtime64.so.1 -> libhsa-runtime64.so.1.13.60102
-rwxr-xr-x  1 root root    3089448 May 28 18:50 libhsa-runtime64.so.1.13.60102
-rwxr-xr-x  1 root root     275208 May 28 18:10 libhsakmt.a
lrwxrwxrwx  1 root root         11 Jun 10 14:52 liboam.so -> liboam.so.1
lrwxrwxrwx  1 root root         19 Jun 10 14:52 liboam.so.1 -> liboam.so.1.0.60102
-rwxr-xr-x  1 root root    1383936 May 28 18:11 liboam.so.1.0.60102
lrwxrwxrwx  1 root root         12 Jun 10 14:55 librccl.so -> librccl.so.1
lrwxrwxrwx  1 root root         20 Jun 10 14:55 librccl.so.1 -> librccl.so.1.0.60102
-rwxr-xr-x  1 root root  231504192 May 28 19:52 librccl.so.1.0.60102
lrwxrwxrwx  1 root root         18 Jun 10 14:55 librocalution.so -> librocalution.so.1
lrwxrwxrwx  1 root root         26 Jun 10 14:54 librocalution.so.1 -> librocalution.so.1.0.60102
-rwxr-xr-x  1 root root   11570680 May 28 21:07 librocalution.so.1.0.60102
lrwxrwxrwx  1 root root         22 Jun 10 14:55 librocalution_hip.so -> librocalution_hip.so.1
lrwxrwxrwx  1 root root         32 Jun 10 14:54 librocalution_hip.so.1 -> librocalution_hip.so.1.0.0.60102
-rwxr-xr-x  1 root root   75440704 May 28 21:07 librocalution_hip.so.1.0.0.60102
lrwxrwxrwx  1 root root         15 Jun 10 14:56 librocblas.so -> librocblas.so.4
lrwxrwxrwx  1 root root         23 Jun 10 14:53 librocblas.so.4 -> librocblas.so.4.1.60102
-rwxr-xr-x  1 root root  881735968 May 28 21:00 librocblas.so.4.1.60102
lrwxrwxrwx  1 root root         14 Jun 10 14:52 librocfft.so -> librocfft.so.0
lrwxrwxrwx  1 root root         22 Jun 10 14:51 librocfft.so.0 -> librocfft.so.0.1.60102
-rwxr-xr-x  1 root root   10736160 May 28 19:55 librocfft.so.0.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:51 librocm-core.so -> librocm-core.so.1
lrwxrwxrwx  1 root root         25 Jun 10 14:51 librocm-core.so.1 -> librocm-core.so.1.0.60102
-rwxr-xr-x  1 root root      17000 May 28 19:21 librocm-core.so.1.0.60102
lrwxrwxrwx  1 root root         19 Jun 10 14:51 librocm-dbgapi.so -> librocm-dbgapi.so.0
lrwxrwxrwx  1 root root         24 Jun 10 14:51 librocm-dbgapi.so.0 -> librocm-dbgapi.so.0.71.0
-rwxr-xr-x  1 root root    1185632 May 28 18:51 librocm-dbgapi.so.0.71.0
lrwxrwxrwx  1 root root         28 Jun 10 14:52 librocm-debug-agent.so.2 -> librocm-debug-agent.so.2.0.3
-rwxr-xr-x  1 root root     151368 May 28 19:13 librocm-debug-agent.so.2.0.3
lrwxrwxrwx  1 root root         18 Jun 10 14:52 librocm_smi64.so -> librocm_smi64.so.7
lrwxrwxrwx  1 root root         26 Jun 10 14:52 librocm_smi64.so.7 -> librocm_smi64.so.7.2.60102
-rwxr-xr-x  1 root root    1363256 May 28 18:11 librocm_smi64.so.7.2.60102
lrwxrwxrwx  1 root root         28 Jun 10 14:52 librocprofiler-register.so -> librocprofiler-register.so.0
lrwxrwxrwx  1 root root         32 Jun 10 14:52 librocprofiler-register.so.0 -> librocprofiler-register.so.0.3.0
-rwxr-xr-x  1 root root    1049792 May 28 18:11 librocprofiler-register.so.0.3.0
lrwxrwxrwx  1 root root         21 Jun 10 14:56 librocprofiler64.so -> librocprofiler64.so.1
lrwxrwxrwx  1 root root         29 Jun 10 14:56 librocprofiler64.so.1 -> librocprofiler64.so.1.0.60102
-rwxr-xr-x  1 root root     334944 May 28 19:17 librocprofiler64.so.1.0.60102
lrwxrwxrwx  1 root root         29 Jun 10 14:56 librocprofiler64.so.2 -> librocprofiler64.so.2.0.60102
-rwxr-xr-x  1 root root    1129944 May 28 19:17 librocprofiler64.so.2.0.60102
lrwxrwxrwx  1 root root         21 Jun 10 14:56 librocprofiler64v2.so -> librocprofiler64.so.2
lrwxrwxrwx  1 root root         15 Jun 10 14:52 librocrand.so -> librocrand.so.1
lrwxrwxrwx  1 root root         23 Jun 10 14:52 librocrand.so.1 -> librocrand.so.1.1.60102
-rwxr-xr-x  1 root root   62393304 May 28 19:45 librocrand.so.1.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 librocsolver.so -> librocsolver.so.0
lrwxrwxrwx  1 root root         25 Jun 10 14:54 librocsolver.so.0 -> librocsolver.so.0.1.60102
-rwxr-xr-x  1 root root 1384333400 May 28 21:05 librocsolver.so.0.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 librocsparse.so -> librocsparse.so.1
lrwxrwxrwx  1 root root         25 Jun 10 14:54 librocsparse.so.1 -> librocsparse.so.1.0.60102
-rwxr-xr-x  1 root root 1367043336 May 28 19:52 librocsparse.so.1.0.60102
lrwxrwxrwx  1 root root         19 Jun 10 14:52 libroctracer64.so -> libroctracer64.so.4
lrwxrwxrwx  1 root root         27 Jun 10 14:52 libroctracer64.so.4 -> libroctracer64.so.4.1.60102
-rwxr-xr-x  1 root root     321512 May 28 19:14 libroctracer64.so.4.1.60102
lrwxrwxrwx  1 root root         15 Jun 10 14:52 libroctx64.so -> libroctx64.so.4
lrwxrwxrwx  1 root root         23 Jun 10 14:52 libroctx64.so.4 -> libroctx64.so.4.1.60102
-rwxr-xr-x  1 root root      15360 May 28 19:14 libroctx64.so.4.1.60102
-rw-r--r--  1 root root      38898 May 28 19:46 libutility.a
drwxr-xr-x  7 root root       4096 Jun 10 14:53 llvm
drwxr-xr-x  2 root root       4096 Jun 10 14:53 pkgconfig
drwxr-xr-x  3 root root       4096 Jun 10 14:53 rocblas
drwxr-xr-x  3 root root       4096 Jun 10 14:52 rocfft
-rw-r--r--  1 root root        512 May 28 19:21 rocmmod
drwxr-xr-x  2 root root       4096 Jun 10 14:56 rocprofiler
drwxr-xr-x  2 root root       4096 Jun 10 14:52 roctracer
[genai1~]$ docker exec 92ebd80dcef1 /opt/rocm/bin/rocminfo
ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.13
Runtime Ext Version:     1.4
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
DMAbuf Support:          NO

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    Intel(R) Xeon(R) Gold 5416S        
  Uuid:                    CPU-XX                             
  Marketing Name:          Intel(R) Xeon(R) Gold 5416S        
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    0                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      49152(0xc000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   4000                               
  BDFID:                   0                                  
  Internal Node ID:        0                                  
  Compute Unit:            32                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    130981244(0x7ce9d7c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    130981244(0x7ce9d7c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    130981244(0x7ce9d7c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 2                  
*******                  
  Name:                    Intel(R) Xeon(R) Gold 5416S        
  Uuid:                    CPU-XX                             
  Marketing Name:          Intel(R) Xeon(R) Gold 5416S        
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    1                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      49152(0xc000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   4000                               
  BDFID:                   0                                  
  Internal Node ID:        1                                  
  Compute Unit:            32                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    132109384(0x7dfd448) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    132109384(0x7dfd448) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    132109384(0x7dfd448) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 3                  
*******                  
  Name:                    gfx90a                             
  Uuid:                    GPU-12f7490bc7b8aaf0               
  Marketing Name:          AMD Instinct MI210                 
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    2                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      8192(0x2000) KB                    
  Chip ID:                 29711(0x740f)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1700                               
  BDFID:                   19456                              
  Internal Node ID:        2                                  
  Compute Unit:            104                                
  SIMDs per CU:            4                                  
  Shader Engines:          8                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 82                                 
  SDMA engine uCode::      8                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*******                  
Agent 4                  
*******                  
  Name:                    gfx90a                             
  Uuid:                    GPU-6c2be151a7afe1e5               
  Marketing Name:          AMD Instinct MI210                 
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    3                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      8192(0x2000) KB                    
  Chip ID:                 29711(0x740f)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1700                               
  BDFID:                   25344                              
  Internal Node ID:        3                                  
  Compute Unit:            104                                
  SIMDs per CU:            4                                  
  Shader Engines:          8                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 82                                 
  SDMA engine uCode::      8                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*******                  
Agent 5                  
*******                  
  Name:                    gfx90a                             
  Uuid:                    GPU-692c10cf88a75cd5               
  Marketing Name:          AMD Instinct MI210                 
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    4                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      8192(0x2000) KB                    
  Chip ID:                 29711(0x740f)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1700                               
  BDFID:                   52224                              
  Internal Node ID:        4                                  
  Compute Unit:            104                                
  SIMDs per CU:            4                                  
  Shader Engines:          8                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 82                                 
  SDMA engine uCode::      8                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*******                  
Agent 6                  
*******                  
  Name:                    gfx90a                             
  Uuid:                    GPU-9d1a65bc110f6055               
  Marketing Name:          AMD Instinct MI210                 
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    5                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      8192(0x2000) KB                    
  Chip ID:                 29711(0x740f)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1700                               
  BDFID:                   58112                              
  Internal Node ID:        5                                  
  Compute Unit:            104                                
  SIMDs per CU:            4                                  
  Shader Engines:          8                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 82                                 
  SDMA engine uCode::      8                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*** Done ***

OS

Linux

GPU

AMD

CPU

  • 无响应*

Ollama版本

0.3.4

db2dz4w8

db2dz4w81#

我有一个类似的情况,Mi300As不能使用。Ollama版本是0.3.5。
运行./ollama-linux-amd64 serve后得到:

2024/08/12 14:55:33 routes.go:1123: INFO server config env="map[CUDA_VISIBLE_DEVICES:0,1 GPU_DEVICE_ORDINAL:0,1 HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/hk-project-test-p0020932/sb7059/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:0,1]"
time=2024-08-12T14:55:34.541+02:00 level=INFO source=images.go:782 msg="total blobs: 19"
time=2024-08-12T14:55:34.545+02:00 level=INFO source=images.go:790 msg="total unused blobs removed: 0"
time=2024-08-12T14:55:34.545+02:00 level=INFO source=routes.go:1170 msg="Listening on [::]:11434 (version 0.3.5)"
time=2024-08-12T14:55:34.546+02:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/scratch/slurm_tmpdir/job_1139/ollama1144170004/runners
time=2024-08-12T14:55:41.044+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]"
time=2024-08-12T14:55:41.044+02:00 level=INFO source=gpu.go:204 msg="looking for compatible GPUs"
time=2024-08-12T14:55:41.053+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=0 total="0 B"
time=2024-08-12T14:55:41.053+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=1 total="0 B"
time=2024-08-12T14:55:41.054+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=2 total="0 B"
time=2024-08-12T14:55:41.055+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=3 total="0 B"
time=2024-08-12T14:55:41.055+02:00 level=INFO source=amd_linux.go:360 msg="no compatible amdgpu devices detected"
time=2024-08-12T14:55:41.055+02:00 level=INFO source=gpu.go:350 msg="no compatible GPUs were discovered"
time=2024-08-12T14:55:41.055+02:00 level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="502.2 GiB" available="469.6 GiB"

相关问题