platform-engineer-agent/k3s-cluster
1
0
Fork 0
forked from roger/k3s-cluster
Code Pull Requests Activity

set qwen models parameters to allow for a higher context window

Browse Source
This commit is contained in:
Roger Oriol
2026-07-24 21:30:01 +02:00
parent bc165d3cf1
commit 4694034a04
4 changed files with 50 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
platform-engineer/cron-seed.yaml
View File

@@ -71,8 +71,23 @@ spec:
}
# ---- Watchdog checks (silent unless something is wrong) ----
create "cluster-health-check" "every 6h" "discord" \
"Check cluster health using the HTTP APIs documented in your SOUL.md. Check: (1) any node that is NotReady, (2) any pod not in Running phase, (3) any recent error/panic/crashloop/backoff log lines in Loki across all namespaces in the last 20 minutes, (4) any ArgoCD app that is not Synced plus Healthy. If everything is healthy, reply with exactly [SILENT]. Otherwise give a concise per-resource summary of what is wrong."
# NOTE: the old single "cluster-health-check" combined 4 heavy HTTP
# gathers (Prometheus + a cluster-wide raw Loki log dump) into one
# cron session and exceeded the flash model's per-request context.
# The 35b-a3b llama-server runs -c 131072 -np 4, so each request only
# gets ~32k tokens of KV cache (NOT 132k). It has been split into
# focused sub-checks so each run stays small, and the log check now
# uses LogQL aggregation instead of dumping raw lines cluster-wide.
create "node-health-check" "every 6h" "discord" \
"Check node health using the Prometheus API documented in your SOUL.md. Query kube_node_status_condition{condition=\"Ready\",status!=\"true\"}. If all nodes are Ready, reply with exactly [SILENT]. Otherwise list each NotReady node by name. Keep it short; do not paste raw JSON — summarize with jq."
create "pod-phase-check" "every 6h" "discord" \
"Check pod health using the Prometheus API documented in your SOUL.md. Query kube_pod_status_phase{phase!=\"Running\"} and exclude Completed/Succeeded pods (jobs). If all Running, reply with exactly [SILENT]. Otherwise list each non-Running pod as namespace/pod:phase, grouped by namespace. Summarize with jq; do not dump raw JSON."
create "pod-error-log-check" "every 6h" "discord" \
"Check for recent crash-loop/error logs using the Loki API documented in your SOUL.md. IMPORTANT context discipline: do NOT fetch raw log lines across all namespaces — that overflows your context window. First run a LogQL aggregation that returns only counts: sum by (namespace, pod) (count_over_time({namespace=~\".+\"} |~ \"(?i)backoff|crashloop|panic\" [20m])). Parse the counts with jq. If every count is zero, reply with exactly [SILENT]. If any namespace/pod has a non-zero count, fetch at most 10 sample lines for THAT pod only (limit=10) to identify the cause. Summarize concisely; never paste more than a handful of lines."
# (ArgoCD health is already covered by the argocd-sync-health cron below.)
create "pod-restart-loop" "every 1h" "discord" \
"Find pods with high restart rates using the Prometheus API documented in your SOUL.md. If any pod has more than 3 restarts in the last 15 minutes, fetch its logs from Loki to diagnose the cause. If the cause is clearly fixable via a manifest change such as bumping a memory limit, fixing a config value, or bumping the restartedAt annotation, make the edit in /workspace/k3s-cluster, commit and push, then trigger an ArgoCD sync via the API. Report what you did in one line. If not clearly fixable, post the log excerpt and proposed fix, and wait for Roger. If no high-restart pods, reply [SILENT]."