platform-engineer-agent/k3s-cluster
1
0
Fork 0
forked from roger/k3s-cluster
Code Pull Requests Activity

fix llamacpp image

Browse Source
This commit is contained in:
Roger Oriol
2026-07-23 00:12:08 +02:00
parent 0143ebefd9
commit 70e4baf5e3
2 changed files with 8 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
llamacpp/README.md
View File

@@ -23,7 +23,8 @@ without re-downloading.
## GPU / Vulkan ## GPU / Vulkan
The `server-vulkan` image bundles the Mesa/RADV Vulkan driver, which supports The `server-vulkan` image (`ghcr.io/ggml-org/llama.cpp:server-vulkan`) bundles
the Mesa/RADV Vulkan driver, which supports
the Radeon 8060S (RDNA 3.5). Full layer offload (`-ngl 999`) puts the ~16 GiB the Radeon 8060S (RDNA 3.5). Full layer offload (`-ngl 999`) puts the ~16 GiB
Q4 model entirely in the 96 GiB VRAM pool. Q4 model entirely in the 96 GiB VRAM pool.

10
llamacpp/deployment-qwen36.yaml
View File

@@ -5,9 +5,11 @@
# (~16 GiB) is fully offloaded to the GPU via the Vulkan backend, leaving plenty # (~16 GiB) is fully offloaded to the GPU via the Vulkan backend, leaving plenty
# of VRAM for the KV cache. # of VRAM for the KV cache.
# #
# Image: ghcr.io/ggerganov/llama.cpp:server-vulkan bundles the Mesa/RADV Vulkan # Image: ghcr.io/ggml-org/llama.cpp:server-vulkan bundles the Mesa/RADV Vulkan
# driver, which supports the Radeon 8060S (RDNA 3.5). Pin to a digest for # driver, which supports the Radeon 8060S (RDNA 3.5). The project moved from the
# production once the rolling tag is confirmed working. # legacy `ggerganov/llama.cpp` namespace (which only has light/full tags) to
# `ggml-org/llama.cpp` (server-vulkan + pinned build tags like server-vulkan-bXXXX).
# Pin to a build tag (e.g. server-vulkan-b4738) for production reproducibility.
# #
# GPU access: the container mounts /dev/dri (the DRM render nodes) and runs # GPU access: the container mounts /dev/dri (the DRM render nodes) and runs
# privileged. This is the simplest reliable option on k3s without a Vulkan # privileged. This is the simplest reliable option on k3s without a Vulkan
@@ -66,7 +68,7 @@ spec:
mountPath: /models mountPath: /models
containers: containers:
- name: llama-server - name: llama-server
image: ghcr.io/ggerganov/llama.cpp:server-vulkan image: ghcr.io/ggml-org/llama.cpp:server-vulkan
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
args: args:
- -m=/models/Qwen3.6-27B-UD-Q4_K_XL.gguf - -m=/models/Qwen3.6-27B-UD-Q4_K_XL.gguf