Just a quickie: this is the command I’m using on my Acer Nitro latop to run Ollama in Docker with GPU acceleration:
group_id_video=$(getent group video | cut -d: -f3)group_id_render=$(getent group render | cut -d: -f3)docker run -d \--privileged \--device /dev/kfd \--device /dev/dri \--volume ollama:/root/.ollama \--volume "/some/path/ollama:/images" \--group-add $group_id_video --group-add $group_id_render \-p 11434:11434 \-e OLLAMA_VULKAN=1 \-e OLLAMA_KV_CACHE_TYPE=q4_0 \-e OLLAMA_FLASH_ATTENTION=1 \-e "OLLAMA_ORIGINS=*" \-e HSA_OVERRIDE_GFX_VERSION=10.3.0 \--name ollama ollama/ollama:rocmprivileged: easier access to host graphics devices. Might not be needed in your setup
device: rendering and video device access
groups: might not be needed, but I had permission issues
ports: useful when connecting to the model from outside
OLLAMA_VULKAN: use the vulkan driver
OLLAMA_KV_CACHE_TYPE: might skip that if you can afford the RAM
OLLAMA_FLASH_ATTENTION: same
OLLAMA_ORIGINS: network security stuff
HSA_OVERRIDE_GFX_VERSION= pick the GFX version closest to your GPU
How to find out GFX info:
sudo rocminfo
find your agent. I had two, one Ryzen (CPU) and one gfx1035 (GPU). The number translates to version 10.3.5, the closest supported version is 10.3.0. I couldn’t find a good list of supported versions, you’ll have to search & guess. Eg here: https://github.com/likelovewant/ollama-for-amd/wiki
George's Techblog 