roger/k3s-cluster
1
0
Fork 1
Code Issues Pull Requests 6 Actions Packages Projects Releases Wiki Activity

fix qwen3.8 deployment

Browse Source
This commit is contained in:
Roger Oriol
2026-08-18 19:35:21 +02:00
parent 8ce5df0018
commit 0527bbf149
3 changed files with 39 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
llamacpp/README.md
View File

@@ -13,13 +13,13 @@ Ollama endpoint.
|---|---|---|---|
| `qwen3.8-27b` | Qwen3.8-27B with MTP | `Q4_K_M` primary, `Q4_0` draft, 196k context, q8_0 K/V cache | `llamacpp-qwen38-27b.llamacpp:80` |
The active Deployment uses llama.cpp's Hugging Face downloader for both model
repositories:
An initContainer downloads both model files atomically before llama-server
starts:
- Primary: `ggml-org/Qwen3.8-27B-GGUF:Q4_K_M`
- Draft: `ggml-org/Qwen3.8-27B-GGUF:Q4_0`
The model cache is stored on the shared hostPath PVC at
The model files are stored on the shared hostPath PVC at
`/data/llamacpp/models` on the NUCBox. The server is configured with
`--spec-default --spec-type draft-mtp`, `--reasoning-preserve`, `--fit off`,
and `--agent`.