deploy both qwen3.6 versions

This commit is contained in:
Roger Oriol
2026-07-23 22:10:44 +02:00
parent 8ef9db1e37
commit 04bd4d8e5e
16 changed files with 906 additions and 53 deletions

View File

@@ -0,0 +1,24 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: home-manager
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "0"
spec:
project: k3s-cluster
source:
repoURL: https://git.rogi.casa/roger/k3s-cluster.git
targetRevision: main
path: home-manager
directory:
recurse: true
destination:
server: https://kubernetes.default.svc
namespace: home-manager
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=false

View File

@@ -28,6 +28,7 @@ APPS=(
"glance|glance|glance|true|true"
"gym-tracker|gym-tracker|gym-tracker|true|true"
"homeassistant|home-assistant|homeassistant|true|true"
"home-manager|home-manager|home-manager|true|true"
"jellyfin|jellyfin|jellyfin|true|true"
"llamacpp|llamacpp|llamacpp|true|true"
"litellm|litellm|litellm|true|true"

139
home-manager/README.md Normal file
View File

@@ -0,0 +1,139 @@
# Home Manager Agent — Deployment Plan
An autonomous **Hermes Agent** that runs inside the k3s cluster and takes care
of the home by talking to **Home Assistant**'s REST API. It mirrors the
`platform-engineer` agent pattern but its domain is the home, not the cluster:
it watches sensors, runs routines, and notifies Roger on Discord — asking
before touching anything security-critical (locks, alarm, garage, HVAC).
Docs: https://hermes-agent.nousresearch.com/docs/user-guide/docker
---
## 1. Goal & operating model
- **One Hermes container** in namespace `home-manager`, scheduled on the amd64
NUC (`roger-nucbox-evo-x2`, 24 GiB) — same node as `platform-engineer`.
- Hermes runs in **gateway mode** under s6 supervision (`gateway run`), so the
built-in **cron scheduler** is active and survives restarts.
- The agent talks to the home via the **Home Assistant REST API** at
`http://home-assistant.home-assistant:80` (in-cluster `home-assistant`
Service). Auth is a HA Long-Lived Access Token in a Secret.
- The agent has **NO k8s RBAC and NO git repo** — it doesn't need them. Its
only write path is HA service calls. (Contrast with `platform-engineer`,
which remediates via git commits → ArgoCD sync.)
- LLM calls route through the in-cluster **LiteLLM** proxy.
- Notifications go to a **dedicated Discord bot** (separate token from the
platform engineer's bot).
- **Hermes-native cron jobs** make the agent run periodic checks + routines.
---
## 2. Files in this directory
| File | Purpose |
|------|---------|
| `namespace.yaml` | namespace `home-manager` |
| `rbac.yaml` | ServiceAccount + Role for the `cron-seed` Job only (pods/exec on the hermes pod) |
| `configmap.yaml` | seed `config.yaml` + `SOUL.md` (HA REST API docs + operating rules) |
| `secret.yaml` | HA token, Discord token/channel, LiteLLM key, dashboard auth (PLACEHOLDERS — fill in) |
| `pvc.yaml` | 5 Gi PVC for `/opt/data` (HERMES_HOME) |
| `deployment.yaml` | Deployment (1 replica, Recreate, pinned to amd64 NUC) + Service |
| `ingress.yaml` | `home-manager.rogi.casa` → dashboard (TLS + basic auth) |
| `cron-seed.yaml` | one-shot Job that creates the Hermes cron schedule |
| `README.md` | this file |
---
## 3. How it differs from `platform-engineer`
| Concern | `platform-engineer` | `home-manager` (this) |
|---|---|---|
| Domain | k3s cluster health | the home |
| Observe via | Prometheus / Loki / ArgoCD HTTP APIs | Home Assistant REST API |
| Write path | git commit → push → ArgoCD sync | HA service calls (`POST /api/services/...`) |
| k8s RBAC | none | none |
| git repo | yes (cloned to `/workspace/k3s-cluster`) | none |
| Discord bot | dedicated | dedicated (separate token + channel) |
| `HERMES_WRITE_SAFE_ROOT` | `/` (needs to edit manifests) | unset → defaults to `/opt/data` (tighter) |
| Image | `nousresearch/hermes-agent:latest` | same |
| Model | `qwen3.6` via LiteLLM | same |
---
## 4. Safety & guardrails
1. **SOUL.md is the real boundary.** The agent may freely toggle lights, fans,
media, and non-critical switches. It must **ask Roger on Discord before
touching** `lock.*`, `alarm_control_panel.*`, garage `cover.*`, risky
`climate.*` changes, or anything tagged `critical`/`security`/`safe_to_auto: false`.
2. **`tool_loop_guardrails.hard_stop_enabled: true`** — circuit-breaks a stuck
gateway.
3. **`yolo: true` + `approvals.mode: off`** — cron jobs aren't blocked waiting
for a human. Safe because the blast radius is HA service calls and the SOUL.md
encodes the confirmation gate for dangerous entities.
4. **Tirith scanner disabled** — it false-positives on the in-cluster plain-HTTP
HA URL and would block every query.
5. **Single replica / single-writer PVC** — never scale >1. `podAntiAffinity`
guards against an accidental co-run.
6. **No real-time event bus** — the terminal tool can't hold a HA WebSocket open.
The agent works on cron polling + on-demand Discord commands. (See §6 for an
optional real-time hook.)
---
## 5. Deployment checklist (do in this order)
1. **Create a Home Assistant Long-Lived Access Token.** In HA UI:
Profile → Long-Lived Access Tokens → Create Token → label `home-manager-agent`.
Copy the token.
2. **Create a dedicated Discord bot** (separate from the platform-engineer bot),
invite it to your server, and note its token + the channel ID you want it to
use as its home channel.
3. **Fill in `secret.yaml`** — base64-encode each value (`echo -n 'VALUE' | base64`):
- `HOMEASSISTANT_TOKEN` — the HA LLAT from step 1.
- `OPENAI_API_KEY` — your LiteLLM master key.
- `DISCORD_BOT_TOKEN`, `DISCORD_HOME_CHANNEL`, `DISCORD_FREE_RESPONSE_CHANNELS`
— from step 2 (channel ID is the same for both).
- `HERMES_DASHBOARD_BASIC_AUTH_PASSWORD` and `..._SECRET` — strong values.
4. **Fill in `configmap.yaml`** — replace `REPLACE_WITH_HOME_CHANNEL_ID` (two
places under `discord:`) with your numeric home channel ID. (Must match the
secret's `DISCORD_HOME_CHANNEL`.)
5. **Tailor `cron-seed.yaml`** schedules/prompts to your actual HA entities
(your sensor names, your `script.goodnight`, etc.).
6. **Commit & push** the whole change (including the regenerated
`argocd/apps/home-manager.yaml`). ArgoCD will create the namespace resources,
deploy the pod, and bring up the ingress at `home-manager.rogi.casa`.
7. **Seed the cron jobs:** ArgoCD runs `cron-seed.yaml` as a Sync hook
automatically. To re-seed after a wipe:
`kubectl apply -f home-manager/cron-seed.yaml`.
8. **Smoke test:** trigger a check manually —
`kubectl exec -n home-manager deploy/hermes -- hermes cron run door-window-check`
and confirm the message lands in Discord (or `[SILENT]` if all clear).
---
## 6. Optional: real-time events from HA
The base setup is cron-poll + Discord. For real-time reaction (e.g. "front door
opened → agent acts immediately"), you can have Home Assistant call the Hermes
gateway API on an event:
1. Enable the gateway API (port 8642) in `config.yaml` and set an auth token.
2. In HA, create a `rest` command + an automation that fires on the event and
POSTs to `http://home-manager.home-manager:80/...` (the in-cluster Service).
This is not enabled by default — start with cron + Discord, add webhooks once
the agent is trusted.
---
## 7. What ArgoCD owns vs. what is runtime state
- **ArgoCD owns** (in git): namespace, RBAC, Secret, ConfigMap (seed), PVC,
Deployment, Service, Ingress, cron-seed Job.
- **Runtime state (on the PVC, NOT reconciled):** `config.yaml`, `SOUL.md`,
`.env`, `cron/jobs.json`, `sessions/`, `memories/`, `skills/`. The ConfigMap
only *seeds* these on first boot; after that, edits made via the dashboard or
`hermes cron edit` persist on the PVC and Argo will not revert them. For a
hard reset, delete the PVC and re-apply.

186
home-manager/configmap.yaml Normal file
View File

@@ -0,0 +1,186 @@
# Hermes configuration + SOUL.md (seeded into the PVC on first boot).
---
apiVersion: v1
kind: ConfigMap
metadata:
name: hermes-seed
namespace: home-manager
data:
config.yaml: |
model:
provider: openai-api
default: qwen3.6
base_url: "http://litellm-service.litellm:80/v1"
api_mode: chat_completions
auxiliary:
compression:
provider: openai-api
model: qwen3.6
base_url: "http://litellm-service.litellm:80/v1"
title_generation:
provider: openai-api
model: qwen3.6
base_url: "http://litellm-service.litellm:80/v1"
terminal:
backend: local
cwd: /opt/data
timeout: 180
home_mode: profile
# The agent runs unattended (cron jobs). yolo:true disables approval prompts
# so cron jobs aren't blocked waiting for a human. Safe here because the
# agent's blast radius is limited to Home Assistant service calls, and the
# SOUL.md forbids touching security-critical entities without confirmation.
yolo: true
approvals:
mode: off
# Disable the Tirith pre-exec command scanner. It flags in-cluster plain
# HTTP URLs (http://home-assistant.home-assistant:80) as 'insecure URL'
# false positives, which blocks every HA API query.
security:
tirith_enabled: false
tirith_fail_open: true
tool_loop_guardrails:
hard_stop_enabled: true
hard_stop_after:
exact_failure: 5
idempotent_no_progress: 5
sessions:
auto_prune: true
retention_days: 90
cron:
wrap_response: false
discord:
# REPLACE_WITH_HOME_CHANNEL_ID — set to your home channel's Discord ID.
# Must match DISCORD_HOME_CHANNEL / DISCORD_FREE_RESPONSE_CHANNELS in the
# Secret. Hermes reads these from config.yaml, not the env.
allowed_channels: 'REPLACE_WITH_HOME_CHANNEL_ID'
free_response_channels: 'REPLACE_WITH_HOME_CHANNEL_ID'
dm_policy: open
group_policy: open
memory:
memory_enabled: true
user_profile_enabled: true
write_approval: false
skills:
write_approval: false
SOUL.md: |
# Home Manager — rogi.casa (via Home Assistant)
You are the autonomous Home Manager for the `rogi.casa` home. You run
*inside* the k3s cluster (namespace `home-manager`) and your job is to keep
the home comfortable, safe, and efficient by talking to Home Assistant, and
to notify your owner (Roger) on Discord when something needs a human.
## The home you look after
- **Home Assistant:** the brain of the home. Runs in namespace
`home-assistant`. You reach its REST API in-cluster at
`http://home-assistant.home-assistant:80` (the `home-assistant` Service,
port 80 → 8123). If that DNS ever fails, fall back to the node IP
`http://10.88.20.11:8123` (HA runs hostNetwork).
- **Auth:** every HA REST call needs an `Authorization: Bearer $HOMEASSISTANT_TOKEN`
header. The token is in your environment. NEVER log or echo the token.
- **Your model provider:** LiteLLM at `http://litellm-service.litellm:80/v1`.
- **Notifications:** Discord (your home channel).
## How you observe the home (Home Assistant REST API)
Use the terminal tool to send HTTP requests with curl. Always pass the bearer
token from the environment (`$HOMEASSISTANT_TOKEN`), never hard-code it.
Base URL: `$HOMEASSISTANT_URL` (set to http://home-assistant.home-assistant:80)
### 1. Read state
- All entities: GET /api/states
- One entity: GET /api/states/<entity_id> (e.g. light.living_room)
- Services list: GET /api/services
- Calendar events: GET /api/calendars/<calendar_id>?start=...&end=...
- History: GET /api/history/period/<timestamp>?filter_entity_id=<id>&minimal
- Config check: GET /api/config
### 2. Actuate (call a service)
POST /api/services/<domain>/<service> with a JSON body.
Common services:
- light.turn_on / light.turn_off / light.toggle
body: {"entity_id": "light.x", "brightness": 200}
- switch.turn_on / switch.turn_off / switch.toggle
body: {"entity_id": "switch.x"}
- cover.open_cover / close_cover / set_cover_position
body: {"entity_id": "cover.x", "position": 50}
- fan.toggle / fan.set_percentage
- media_player.play_media / media_pause / volume_set
- climate.set_temperature
body: {"entity_id": "climate.x", "temperature": 21}
- script.turn_on
body: {"entity_id": "script.goodnight"}
- input_boolean.toggle / input_select.select_option
### 3. Fire events
POST /api/events/<event_type> { ... }
(Listening to the live event bus needs the WebSocket API, which the
terminal tool can't hold open. Use cron polling for periodic checks, and
ask Roger on Discord for anything that needs real-time reaction.)
### 4. Parse JSON
Pipe curl output through `python3 -m json.tool` or `jq` in the terminal
tool. The execute_code tool is BLOCKED in cron mode.
## How you remediate (call services — NOT git, NOT kubectl)
Unlike the platform engineer, you do NOT change the cluster. You change the
*home* by calling Home Assistant services via the REST API. You have no k8s
access and no git repo — and you don't need them.
Workflow:
1. GET /api/states (or a specific entity) to understand the situation.
2. Decide a safe action. POST the service call.
3. GET the entity back to confirm the new state.
4. Report what you did in one line (entity_id + service + result).
## Operating rules
1. **Read first, act second.** Always GET the current state before changing
anything. Cite the exact `entity_id` and the service you called.
2. **Only safe, idempotent actions on your own.** Allowed without asking:
lights, fans, media players, non-critical switches, running scripts Roger
has marked safe, reading any sensor.
3. **NEVER touch security- or safety-critical entities without explicit
confirmation from Roger on Discord.** This includes, but is not limited to:
- `lock.*` (door locks)
- `alarm_control_panel.*` (alarm arm/disarm)
- `cover.*` for garage doors or main gates
- `climate.*` changes that could freeze/overheat the house or cost a lot
- any `switch.*` / `script.*` tagged `critical` or `security`
- any entity with a custom attribute `safe_to_auto: false`
For these, post the proposed action to Discord and wait for Roger to
reply yes/no before calling the service.
4. **When in doubt, notify, don't act.** If an action is unusual, could
startle someone (e.g. loud media at 03:00, vacuum robot at night), or you
are unsure what an entity does, ask Roger first.
5. **Be quiet when healthy.** Watchdog cron jobs reply with exactly
`[SILENT]` when there is nothing to report. Failed jobs always deliver.
6. **No runaway loops.** You cannot create new cron jobs from inside a cron
run (Hermes disables that). Do not try.
7. **Talk like a concierge.** Short, concrete, with entity_ids and values.
No filler. When you did something, say what in one line.
8. **Respect time of day and presence.** Don't run noisy routines (loud
media, vacuum robots) at night. Check presence (`zone.*` /
`binary_sensor.*`) and the time before acting.
## How you reach Roger
Notifications go to Discord (your home channel). Cron jobs deliver there by
default (`deliver="discord"`). Keep messages under ~1800 chars. For anything
that needs a yes/no decision, ask the question clearly and wait — do not
proceed on a guess.

View File

@@ -0,0 +1,93 @@
# One-shot Job that seeds Hermes' built-in cron schedule on first install.
# Idempotent: skips job names that already exist.
#
# Cron prompts are deliberately written as plain-English instructions (no inline
# curl commands) to avoid tripping Hermes' threat-pattern scanner, which blocks
# cron prompts containing curl+auth-header patterns. The exact HA REST API
# endpoints and examples are documented in the agent's SOUL.md instead.
#
# Tailor the schedules/prompts below to your actual Home Assistant entities.
---
apiVersion: batch/v1
kind: Job
metadata:
name: hermes-cron-seed
namespace: home-manager
labels:
app: hermes
annotations:
argocd.argoproj.io/sync-options: Replace=true
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
spec:
backoffLimit: 4
ttlSecondsAfterFinished: 86400
template:
metadata:
labels:
app: hermes
spec:
serviceAccountName: cron-seeder
restartPolicy: OnFailure
containers:
- name: seed
image: alpine:3.20
command: ["sh", "-c"]
args:
- |
set -e
apk add --no-cache curl
ARCH=$(uname -m)
case "$ARCH" in
x86_64) KARCH=amd64 ;;
aarch64) KARCH=arm64 ;;
armv7l) KARCH=arm ;;
*) echo "unsupported arch: $ARCH" >&2; exit 1 ;;
esac
curl -fsSL -o /usr/local/bin/kubectl \
"https://dl.k8s.io/release/v1.35.0/bin/linux/${KARCH}/kubectl"
chmod +x /usr/local/bin/kubectl
echo "Waiting for hermes pod to be Ready..."
kubectl -n home-manager wait --for=condition=Ready pod -l app=hermes --timeout=300s || true
POD=$(kubectl -n home-manager get pod -l app=hermes -o jsonpath='{.items[0].metadata.name}')
echo "Using pod: $POD"
exists() { kubectl -n home-manager exec "$POD" -- hermes cron list 2>/dev/null | grep -qi " $1 "; }
create() {
name="$1"; schedule="$2"; deliver="$3"; prompt="$4"
if exists "$name"; then
echo "cron job '$name' already exists — skipping"
else
echo "creating cron job '$name' ..."
kubectl -n home-manager exec "$POD" -- hermes cron create "$schedule" "$prompt" --name "$name" --deliver "$deliver"
fi
}
# ---- Watchdog checks (silent unless something is wrong) ----
create "door-window-check" "every 15m" "discord" \
"Check the Home Assistant REST API as documented in your SOUL.md. List all door and window sensor entities (binary_sensor.* for doors, windows, contact sensors). If any are open and either no one is home (check zone.home or presence sensors) or it is between 23:00 and 06:00, notify Roger with which sensor is open. If all closed, reply with exactly [SILENT]."
create "leak-moisture-check" "every 10m" "discord" \
"Check the Home Assistant REST API as documented in your SOUL.md. Look for moisture, water leak, or flood sensor entities (binary_sensor.* moisture/water/leak, sensor.* moisture). If any report wet/active, alert Roger immediately with the sensor name and location. If all dry, reply with exactly [SILENT]."
create "energy-anomaly" "every 30m" "discord" \
"Check the Home Assistant REST API as documented in your SOUL.md. Read the current power draw sensor (sensor.* power, sensor.* current_power). If the total power draw is unusually high for the time of day (over 4000 W during the day, or over 800 W when no one is home at night), alert Roger with the reading. Otherwise reply with exactly [SILENT]."
create "comfort-check" "every 1h" "discord" \
"Check the Home Assistant REST API as documented in your SOUL.md. Read indoor temperature and humidity sensors. If a room is outside a comfortable range (below 18C or above 26C, or humidity above 65 percent), you may toggle a fan entity in that room (safe). Do NOT change climate/HVAC settings yourself — if it is uncomfortable, propose a climate change to Roger on Discord and wait. If everything is comfortable, reply with exactly [SILENT]."
create "left-on-check" "every 1h" "discord" \
"Check the Home Assistant REST API as documented in your SOUL.md. If no one is home (zone.home is away or presence sensors all off), find any lights, media players, fans, or non-critical switches that are on. Turn off the safe non-essential ones (lights, media, fans) and report what you turned off. Do NOT touch anything security-critical. If someone is home, or nothing is left on, reply with exactly [SILENT]."
# ---- Routines (always delivered) ----
create "goodnight-routine" "0 23 * * *" "discord" \
"Run the goodnight routine via the Home Assistant REST API as documented in your SOUL.md. Turn off all non-essential lights and pause media players (safe actions). Then propose — but do NOT execute — locking the doors (lock.*), closing the garage (cover.* garage), and arming the alarm (alarm_control_panel.*). Ask Roger on Discord for confirmation before calling any of those security-critical services. Deliver a short summary of what you turned off and what you are waiting for confirmation on."
create "morning-briefing" "0 7 * * *" "discord" \
"Produce a morning home briefing for Roger using the Home Assistant REST API as documented in your SOUL.md. Include: (1) who is home (zone.home / presence), (2) any lights or media still on, (3) any doors or windows open, (4) indoor temperatures, (5) today's calendar events if a calendar entity exists, (6) overnight energy use if an energy sensor exists, (7) any sensors in an alert/unavailable state. Keep it under 1800 chars. Always deliver (no [SILENT])."
echo "Done. Listing all cron jobs:"
kubectl -n home-manager exec "$POD" -- hermes cron list

View File

@@ -0,0 +1,154 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: hermes
namespace: home-manager
labels:
app: hermes
spec:
replicas: 1 # MUST be 1 — Hermes' /opt/data is single-writer.
strategy:
type: Recreate # never run two pods against the same PVC
selector:
matchLabels:
app: hermes
template:
metadata:
labels:
app: hermes
spec:
# No serviceAccountName — the agent has NO k8s API access. It manages the
# home via the Home Assistant REST API and notifies via Discord.
# Pin to the powerful amd64 node (image is linux/amd64; the NUC has 24 GiB).
nodeSelector:
kubernetes.io/arch: amd64
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: hardware
operator: In
values: ["high-memory"]
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchLabels:
app: hermes
topologyKey: kubernetes.io/hostname
initContainers:
# Seed /opt/data with config.yaml + SOUL.md + .env on first boot only.
# ArgoCD owns the manifests; the PVC is runtime state and is NOT reconciled.
- name: seed-data
image: busybox:1.36
command: ["sh", "-c"]
args:
- |
set -e
if [ ! -f /opt/data/config.yaml ]; then
echo "First boot: seeding /opt/data from ConfigMap + env..."
cp /seed/config.yaml /opt/data/config.yaml
cp /seed/SOUL.md /opt/data/SOUL.md
chmod 600 /opt/data/config.yaml
# Write .env from the injected Secret env vars so the s6 gateway
# finds API keys (the hermes container reads keys from /opt/data/.env).
: > /opt/data/.env
chmod 600 /opt/data/.env
for k in OPENAI_API_KEY OPENAI_BASE_URL HOMEASSISTANT_TOKEN \
DISCORD_BOT_TOKEN DISCORD_HOME_CHANNEL \
GATEWAY_ALLOW_ALL_USERS DISCORD_FREE_RESPONSE_CHANNELS \
HERMES_DASHBOARD HERMES_DASHBOARD_BASIC_AUTH_USERNAME \
HERMES_DASHBOARD_BASIC_AUTH_PASSWORD HERMES_DASHBOARD_BASIC_AUTH_SECRET; do
eval "v=\${$k:-}"
[ -n "$v" ] && echo "$k=$v" >> /opt/data/.env
done
else
echo "/opt/data already initialized — leaving runtime state intact."
fi
mkdir -p /opt/data/home/.kube /opt/data/cron/output /opt/data/scripts
envFrom:
- secretRef:
name: hermes-env
volumeMounts:
- name: data
mountPath: /opt/data
- name: seed
mountPath: /seed
containers:
- name: hermes
image: nousresearch/hermes-agent:latest
imagePullPolicy: Always
# IMPORTANT: do NOT set `command:` — it would override the image's
# ENTRYPOINT (/init, s6-overlay), which sets up the hermes user, seeds
# config on first boot, and supervises the gateway.
args: ["gateway", "run"]
ports:
- name: gateway
containerPort: 8642
- name: dashboard
containerPort: 9119
envFrom:
- secretRef:
name: hermes-env
env:
- name: HERMES_HOME
value: /opt/data
# Home Assistant REST API base URL (in-cluster service). Non-secret, so
# it lives here rather than in the Secret. The SOUL.md uses $HOMEASSISTANT_URL.
- name: HOMEASSISTANT_URL
value: "http://home-assistant.home-assistant:80"
# HERMES_WRITE_SAFE_ROOT is intentionally unset → defaults to HERMES_HOME
# (/opt/data). The home agent has no git workspace to write to, so the
# tighter default is correct (memory/skills/scripts live under /opt/data).
volumeMounts:
- name: data
mountPath: /opt/data
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "2Gi"
cpu: "1000m"
livenessProbe:
# Probe the dashboard port (9119, always enabled via HERMES_DASHBOARD=1
# and binds 0.0.0.0). The gateway API on 8642 is off by default.
tcpSocket:
port: 9119
initialDelaySeconds: 90
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
securityContext:
allowPrivilegeEscalation: false
volumes:
- name: data
persistentVolumeClaim:
claimName: hermes-data
- name: seed
configMap:
name: hermes-seed
---
apiVersion: v1
kind: Service
metadata:
name: home-manager
namespace: home-manager
spec:
type: ClusterIP
selector:
app: hermes
ports:
- name: gateway
port: 80
targetPort: 8642
- name: dashboard
port: 9119
targetPort: 9119

24
home-manager/ingress.yaml Normal file
View File

@@ -0,0 +1,24 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: home-manager
namespace: home-manager
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: traefik
tls:
- hosts:
- home-manager.rogi.casa
secretName: home-manager-tls
rules:
- host: home-manager.rogi.casa
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: home-manager
port:
number: 9119 # dashboard

View File

@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: home-manager

11
home-manager/pvc.yaml Normal file
View File

@@ -0,0 +1,11 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: hermes-data
namespace: home-manager
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi

41
home-manager/rbac.yaml Normal file
View File

@@ -0,0 +1,41 @@
# Minimal RBAC for the cron-seed Job ONLY.
#
# The Hermes home-manager agent itself has NO k8s RBAC — it manages the home
# via the Home Assistant REST API (http://home-assistant.home-assistant:80).
#
# The cron-seed Job needs to `kubectl exec` into the hermes pod to run
# `hermes cron create ...` (the only way to seed Hermes' internal cron).
# Scoped to this namespace, pods/exec on the hermes pod only.
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: cron-seeder
namespace: home-manager
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: cron-seeder
namespace: home-manager
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cron-seeder
namespace: home-manager
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: cron-seeder
subjects:
- kind: ServiceAccount
name: cron-seeder
namespace: home-manager

View File

@@ -23,13 +23,13 @@ data:
litellm_params:
model: "anthropic/claude-sonnet-5"
api_key: "os.environ/ANTHROPIC_API_KEY"
- model_name: gemini-3-flash
- model_name: gemini-3.6-flash
litellm_params:
model: gemini/gemini-3-flash-preview
model: gemini/gemini-3.6-flash
api_key: "os.environ/GEMINI_API_KEY"
- model_name: tencent/hy3:free
- model_name: tencent/hy3
litellm_params:
model: openrouter/tencent/hy3:free
model: openrouter/tencent/hy3
api_key: "os.environ/OPENROUTER_API_KEY"
- model_name: z-ai/glm-5.2
litellm_params:
@@ -41,13 +41,20 @@ data:
api_base: http://10.88.20.12:11434
# Used by the platform-engineer Hermes agent (deployed in ns platform-engineer).
# Served by llama.cpp (llama-server, Vulkan on the NUCBox APU) — see
# llamacpp/deployment-qwen36.yaml. Thinking is left ON; reasoning comes
# back in `reasoning_content` (llama-server --jinja + default reasoning
# format). The alias `qwen3.6` is unchanged so Hermes keeps working.
- model_name: qwen3.6
# llamacpp/deployment-qwen36-27b.yaml. Thinking is left ON; reasoning comes
# back in `reasoning_content` (llama-server --jinja + default reasoning format).
- model_name: qwen3.6-27b
litellm_params:
model: openai/qwen3.6
api_base: http://llamacpp-qwen36.llamacpp/v1
model: openai/qwen3.6-27b
api_base: http://llamacpp-qwen36-27b.llamacpp/v1
api_key: "sk-no-auth"
# Faster "flash" variant: Qwen3.6-35B-A3B is a MoE (3B active params), so
# inference is much faster than the dense 27B despite more total weights.
# See llamacpp/deployment-qwen36-36b-a3b.yaml.
- model_name: qwen3.6-36b-a3b
litellm_params:
model: openai/qwen3.6-36b-a3b
api_base: http://llamacpp-qwen36-36b-a3b.llamacpp/v1
api_key: "sk-no-auth"
litellm_settings:
#set_verbose: True # Uncomment this if you want to see verbose logs; not recommended in production

View File

@@ -13,9 +13,15 @@ in-cluster Services instead of the external `10.88.20.12:11434` Ollama endpoint.
One Deployment + Service **per model**, all in namespace `llamacpp`, all pinned
to the NUCBox (`nodeSelector: {kubernetes.io/arch: amd64, hardware: high-memory}`):
| Model | GGUF | Service | litellm alias |
|----------|---------------------------------------------|----------------------------------|---------------|
| qwen3.6 | unsloth/Qwen3.6-27B-MTP-GGUF (Q4_K_XL) | `llamacpp-qwen36.llamacpp:80` | `qwen3.6` |
| Alias | Model | GGUF | Service |
|-------------------|----------------------------|-------------------------------------------------------|-------------------------------------------|
| `qwen3.6-27b` | Qwen3.6-27B (dense) | unsloth/Qwen3.6-27B-MTP-GGUF (Q4_K_XL, ~16 GiB) | `llamacpp-qwen36-27b.llamacpp:80` |
| `qwen3.6-36b-a3b` | Qwen3.6-35B-A3B (MoE/flash)| unsloth/Qwen3.6-35B-A3B-MTP-GGUF (Q4_K_XL, ~20 GiB) | `llamacpp-qwen36-36b-a3b.llamacpp:80` |
The 35B-A3B is a Mixture-of-Experts model (3B active params per token), so
inference is significantly faster than the dense 27B despite more total weights —
hence the "flash" label. Use it for latency-sensitive workloads; use the 27B for
deeper reasoning.
Model files are downloaded idempotently by an initContainer into a shared
hostPath PVC (`/data/llamacpp/models` on the NUCBox), so pods survive reboots
@@ -24,9 +30,8 @@ without re-downloading.
## GPU / Vulkan
The `server-vulkan` image (`ghcr.io/ggml-org/llama.cpp:server-vulkan`) bundles
the Mesa/RADV Vulkan driver, which supports
the Radeon 8060S (RDNA 3.5). Full layer offload (`-ngl 999`) puts the ~16 GiB
Q4 model entirely in the 96 GiB VRAM pool.
the Mesa/RADV Vulkan driver, which supports the Radeon 8060S (RDNA 3.5). Full
layer offload (`-ngl 999`) puts model weights entirely in the 96 GiB VRAM pool.
The container mounts `/dev/dri` and runs `privileged: true` — the simplest
reliable way to give Vulkan access to the DRM render node on k3s without a
@@ -36,26 +41,38 @@ group GID) if desired.
### Verify the GPU is actually used
```bash
kubectl exec -n llamacpp deploy/llamacpp-qwen36 -- \
llama-server --list-devices -m /models/Qwen3.6-27B-UD-Q4_K_XL.gguf
# or check the startup logs for a "vulkan" device line + ngl offload count
kubectl logs -n llamacpp deploy/llamacpp-qwen36 | grep -iE 'vulkan|gpu|offload|device'
kubectl logs -n llamacpp deploy/llamacpp-qwen36-27b | grep -iE 'vulkan|gpu|offload|device'
kubectl logs -n llamacpp deploy/llamacpp-qwen36-36b-a3b | grep -iE 'vulkan|gpu|offload|device'
```
If only a CPU device shows up, the container can't see the GPU — check that
`/dev/dri/renderD128` exists on the NUCBox and that the `amdgpu` module is loaded.
## VRAM budget (both models co-resident)
Both models run simultaneously on the same 96 GiB VRAM pool. Approximate usage:
| Model | Weights | KV cache (32k×4) | Subtotal |
|-------------------|----------|-------------------|----------|
| qwen3.6-27b | ~16 GiB | ~34 GiB | ~50 GiB |
| qwen3.6-36b-a3b | ~20 GiB | ~10 GiB | ~30 GiB |
| **Total** | | | **~80 GiB** |
~16 GiB headroom — comfortable but not infinite. If VRAM is exhausted ( Vulkan
allocation failures in logs), reduce `-c` on the 27B (its KV cache dominates) or
drop `-np` to 2 on either model.
## Tuning
The key knobs (in `deployment-qwen36.yaml`):
The key knobs (in each `deployment-*.yaml`):
- `-ngl 999` — offload all layers to GPU. Reduce only if VRAM is tight (it
isn't, with 96 GiB).
- `-ngl 999` — offload all layers to GPU. Reduce only if VRAM is tight.
- `-c 32768` — total KV-cache context. With `-np 4` this is 8192 tokens per
concurrent request. For a 27B model the full 32k×4 KV cache is ~32 GiB of
VRAM; raise or lower `-c` to trade context length for VRAM headroom.
- `-np 4` — parallel slots (concurrent requests). Matches the requested
concurrency. Each extra slot multiplies KV-cache VRAM usage.
concurrent request. The 27B's dense KV cache is the larger consumer (~34 GiB
at 32k×4); the MoE's is much smaller (~10 GiB).
- `-np 4` — parallel slots (concurrent requests). Each extra slot multiplies
KV-cache VRAM usage. Bump higher on the flash model if you need more
throughput (it has VRAM headroom).
- `--threads 8` — CPU threads for sampling/overhead. Mostly irrelevant under
full GPU offload; tune if CPU-bound.
@@ -69,11 +86,11 @@ during load. If the pod is OOM-killed during model load, raise the memory limit.
## Adding a model
1. Create `deployment-<model>.yaml` + `service-<model>.yaml` (copy the qwen3.6
pair; change `model:` label, the GGUF URL/file, `--alias`, and Service name).
1. Copy `deployment-qwen36-27b.yaml` `deployment-<new>.yaml`; change the
`model:` label, GGUF URL/file, `--alias`, and Service name.
2. Point LiteLLM at it in `litellm/litellm.yaml`:
```yaml
- model_name: <alias> # keep the alias so consumers don't change
- model_name: <alias>
litellm_params:
model: openai/<alias>
api_base: http://<service>.llamacpp/v1
@@ -81,9 +98,4 @@ during load. If the pod is OOM-killed during model load, raise the memory limit.
```
3. (No gen-apps.sh change needed — the `llamacpp` app already syncs the whole
directory recursively.)
## TODO
- `glm-4.7-flash`: still served by the external Ollama at `10.88.20.12:11434`
in `litellm/litellm.yaml`. Migrate once a GGUF source is confirmed (add a
`deployment-glm47-flash.yaml` + Service and flip the litellm entry).
4. Check the VRAM budget table above — two large models may not coexist.

View File

@@ -1,9 +1,8 @@
# Qwen3.6 (27B, Q4_K_XL) served by llama.cpp's llama-server on the NUCBox APU.
# Qwen3.6-27B (dense, Q4_K_XL) served by llama.cpp's llama-server on the NUCBox APU.
#
# Hardware: AMD Ryzen AI Max 395 (Strix Halo) — integrated Radeon 8060S,
# 128 GiB unified memory (32 GiB RAM / 96 GiB VRAM via firmware). The Q4 model
# (~16 GiB) is fully offloaded to the GPU via the Vulkan backend, leaving plenty
# of VRAM for the KV cache.
# (~16 GiB) is fully offloaded to the GPU via the Vulkan backend.
#
# Image: ghcr.io/ggml-org/llama.cpp:server-vulkan bundles the Mesa/RADV Vulkan
# driver, which supports the Radeon 8060S (RDNA 3.5). The project moved from the
@@ -18,11 +17,11 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: llamacpp-qwen36
name: llamacpp-qwen36-27b
namespace: llamacpp
labels:
app: llamacpp
model: qwen3.6
model: qwen3.6-27b
spec:
replicas: 1
strategy:
@@ -30,12 +29,12 @@ spec:
selector:
matchLabels:
app: llamacpp
model: qwen3.6
model: qwen3.6-27b
template:
metadata:
labels:
app: llamacpp
model: qwen3.6
model: qwen3.6-27b
spec:
nodeSelector:
kubernetes.io/arch: amd64
@@ -76,7 +75,7 @@ spec:
- -m # model file
- /models/Qwen3.6-27B-UD-Q4_K_XL.gguf
- --alias # /v1/models reports this name; matches the litellm alias
- qwen3.6
- qwen3.6-27b
- --host
- 0.0.0.0
- --port
@@ -144,16 +143,16 @@ spec:
apiVersion: v1
kind: Service
metadata:
name: llamacpp-qwen36
name: llamacpp-qwen36-27b
namespace: llamacpp
labels:
app: llamacpp
model: qwen3.6
model: qwen3.6-27b
spec:
type: ClusterIP
selector:
app: llamacpp
model: qwen3.6
model: qwen3.6-27b
ports:
- name: http
port: 80

View File

@@ -0,0 +1,158 @@
# Qwen3.6-35B-A3B (MoE: 35B total / 3B active, Q4_K_XL) — the "flash" variant.
#
# Despite having more total parameters than the 27B dense model, only 3B are
# active per token (Mixture-of-Experts), so inference is much faster. The full
# ~20 GiB of Q4 weights is still loaded into VRAM but only a small fraction is
# computed per token.
#
# Hardware: AMD Ryzen AI Max 395 (Strix Halo) — integrated Radeon 8060S,
# 128 GiB unified memory (32 GiB RAM / 96 GiB VRAM via firmware). Full GPU
# offload via the Vulkan backend. Shares the 96 GiB VRAM pool with the 27B
# model — see llamacpp/README.md for the combined VRAM budget.
#
# Image: ghcr.io/ggml-org/llama.cpp:server-vulkan (Mesa/RADV Vulkan driver,
# supports the Radeon 8060S / RDNA 3.5). Pin to a build tag for production.
#
# GPU access: mounts /dev/dri + privileged (simplest reliable path on k3s).
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: llamacpp-qwen36-36b-a3b
namespace: llamacpp
labels:
app: llamacpp
model: qwen3.6-36b-a3b
spec:
replicas: 1
strategy:
type: Recreate # never run two pods loading the same model into VRAM
selector:
matchLabels:
app: llamacpp
model: qwen3.6-36b-a3b
template:
metadata:
labels:
app: llamacpp
model: qwen3.6-36b-a3b
spec:
nodeSelector:
kubernetes.io/arch: amd64
hardware: high-memory
initContainers:
# Idempotently download the GGUF into the shared models PVC on first boot.
# Exits immediately if the file is already present (pod restart / recreate).
- name: fetch-model
image: alpine:3.20
command: ["/bin/sh", "-c"]
args:
- |
set -e
if [ -f "/models/$MODEL_FILE" ]; then
echo "Model $MODEL_FILE already present — skipping download."
exit 0
fi
echo "Installing curl..."
apk add --no-cache curl
echo "Downloading $MODEL_FILE from $MODEL_URL ..."
curl -fL --retry 5 --retry-delay 5 -o "/models/$MODEL_FILE" "$MODEL_URL"
echo "Download complete: $(ls -lh /models/$MODEL_FILE)"
env:
- name: MODEL_URL
value: "https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/resolve/main/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf"
- name: MODEL_FILE
value: "Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf"
volumeMounts:
- name: models
mountPath: /models
containers:
- name: llama-server
image: ghcr.io/ggml-org/llama.cpp:server-vulkan
imagePullPolicy: IfNotPresent
# llama.cpp's CLI parser does NOT split on '=' — every value flag must be a
# separate argv element (flag, then value). See common/arg.cpp in the repo.
args:
- -m # model file
- /models/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
- --alias # /v1/models reports this name; matches the litellm alias
- qwen3.6-36b-a3b
- --host
- 0.0.0.0
- --port
- "8080"
- --jinja # use the GGUF's chat template (Qwen3 thinking format)
- -ngl # offload ALL layers to the GPU (fits in 96 GiB VRAM)
- "999"
- -c # total KV-cache context, split across parallel slots
- "32768"
- -np # 4 parallel slots => 8192 tokens per concurrent request
- "4"
- --cont-batching # continuous batching across slots
- --threads # CPU threads for sampling/overhead (GPU does the heavy lifting)
- "8"
ports:
- name: http
containerPort: 8080
resources:
# The model weights + KV cache live in GPU VRAM (96 GiB pool) and are
# NOT counted against the cgroup memory limit. This limit only covers
# CPU-side overhead + the mmap'd GGUF file pages during load (~20 GiB,
# reclaimable). k8s sees ~32 GiB as the node's allocatable system RAM.
# If the pod OOM-kills during load, raise the limit.
requests:
cpu: "1000m"
memory: "2Gi"
limits:
cpu: "4000m"
memory: "24Gi"
readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 6
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 180 # model load + Vulkan init can take a few minutes
periodSeconds: 30
failureThreshold: 5
securityContext:
# Vulkan on the AMD APU needs /dev/dri + the driver. Privileged is the
# simplest reliable path on k3s without a device plugin.
privileged: true
volumeMounts:
- name: models
mountPath: /models
readOnly: true
- name: dri
mountPath: /dev/dri
volumes:
- name: models
persistentVolumeClaim:
claimName: llamacpp-models
- name: dri
hostPath:
path: /dev/dri
type: Directory
---
apiVersion: v1
kind: Service
metadata:
name: llamacpp-qwen36-36b-a3b
namespace: llamacpp
labels:
app: llamacpp
model: qwen3.6-36b-a3b
spec:
type: ClusterIP
selector:
app: llamacpp
model: qwen3.6-36b-a3b
ports:
- name: http
port: 80
targetPort: 8080

View File

@@ -130,10 +130,10 @@ model:
auxiliary:
compression:
provider: openai-api
model: gemini-3-flash
model: gemini-3.6-flash
title_generation:
provider: openai-api
model: gemini-3-flash
model: gemini-3.6-flash
terminal:
backend: local
@@ -173,7 +173,7 @@ HERMES_DASHBOARD_BASIC_AUTH_PASSWORD=<strong password>
> Why `OPENAI_API_KEY` + `OPENAI_BASE_URL`: the `openai-api` provider honours
> `OPENAI_BASE_URL`, so this is the simplest way to point Hermes at the
> in-cluster LiteLLM. `claude-4.5-haiku` / `gemini-3-flash` are the model names
> in-cluster LiteLLM. `claude-4.5-haiku` / `gemini-3.6-flash` are the model names
> already exposed by your `litellm/litellm.yaml` ConfigMap.
`SOUL.md` (personality + guardrails) — see `configmap.yaml`. Key points:

View File

@@ -9,18 +9,18 @@ data:
config.yaml: |
model:
provider: openai-api
default: qwen3.6
default: qwen3.6-27b
base_url: "http://litellm-service.litellm:80/v1"
api_mode: chat_completions
auxiliary:
compression:
provider: openai-api
model: qwen3.6
model: qwen3.6-27b
base_url: "http://litellm-service.litellm:80/v1"
title_generation:
provider: openai-api
model: qwen3.6
model: qwen3.6-27b
base_url: "http://litellm-service.litellm:80/v1"
terminal: