roger/k3s-cluster
1
0
Fork 1
Code Issues Pull Requests 6 Actions Packages Projects Releases Wiki Activity

fix pod restart cron

Browse Source
This commit is contained in:
Roger Oriol
2026-07-25 20:19:27 +02:00
parent 8caf49fdf6
commit a7e78c18c3
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
platform-engineer/cron-seed.yaml
View File

@@ -90,7 +90,7 @@ spec:
# (ArgoCD health is already covered by the argocd-sync-health cron below.)
create "pod-restart-loop" "every 1h" "discord" \
"Find pods with high restart rates using the Prometheus API documented in your SOUL.md. If any pod has more than 3 restarts in the last 15 minutes, fetch its logs from Loki to diagnose the cause. If the cause is clearly fixable via a manifest change such as bumping a memory limit, fixing a config value, or bumping the restartedAt annotation, make the edit in /workspace/k3s-cluster, commit and push, then trigger an ArgoCD sync via the API. Report what you did in one line. If not clearly fixable, post the log excerpt and proposed fix, and wait for Roger. If no high-restart pods, reply [SILENT]."
"Find pods with high restart rates using the Prometheus API documented in your SOUL.md. Run increase(kube_pod_container_status_restarts_total[15m]) and list only pods whose value is greater than 3. If none, reply [SILENT]. For any pod that qualifies, IMPORTANT context discipline: do NOT fetch raw logs across the cluster or over a wide time window — that overflows your context window (a raw cluster-wide Loki dump can exceed 60k tokens). Fetch at most 10 sample lines for THAT specific pod only, using Loki query_range with limit=10 over the last 15 minutes, filtered to error/crash patterns (e.g. {namespace=\"<ns>\",pod=\"<pod>\"} |~ \"(?i)error|panic|crash|oom|backoff\"). Summarize the root cause in one line; never paste more than a handful of log lines. If the cause is clearly fixable via a manifest change such as bumping a memory limit, fixing a config value, or bumping the restartedAt annotation, make the edit in /workspace/k3s-cluster, commit and push, then trigger an ArgoCD sync via the API. Report what you did in one line. If not clearly fixable, post the log excerpt and proposed fix, and wait for Roger."
create "pvc-pressure" "every 1d" "discord" \
"Check storage health using the Prometheus API documented in your SOUL.md. Alert if any PVC has less than 15 percent free space, or if any node filesystem is over 85 percent full. If all healthy, reply [SILENT]."