Commit Graph

176 Commits

Author SHA1 Message Date
Roger Oriol
cf6e2784fe configure litellm models 2026-07-14 21:36:38 +02:00
Roger Oriol
dad38347e7 upgrade n8n memory requirements 2026-07-14 18:50:28 +02:00
Roger Oriol
a5b90994a4 fix myorg assistant 2026-07-11 19:22:57 +02:00
Roger Oriol
04b736287b fix argocd redirect 2026-07-09 23:48:47 +02:00
Roger Oriol
8c6950fd43 Merge branch 'main' of https://git.rogi.casa/roger/k3s-cluster 2026-07-09 00:45:40 +02:00
Roger Oriol
81dfe6fd60 fix argocd rbac and create agents.md 2026-07-09 00:45:10 +02:00
5d80abf3e8 Merge pull request 'fix: myorg-assistant cronjobs ImagePullBackOff + argocd root drift loop' (#18) from platform-engineer-agent/k3s-cluster:fix-merge-to-main into main
Reviewed-on: roger/k3s-cluster#18
2026-07-09 00:22:54 +02:00
48f18d2a3e Merge branch 'main' into fix-merge-to-main 2026-07-09 00:22:22 +02:00
Roger Oriol
ce08365e06 revert to qwen3.6 for platform engineer 2026-07-07 23:52:37 +02:00
Platform Engineer
0794153e56 fix(myorg-assistant): point cronjobs at registry image + add imagePullSecrets
CronJob pods were stuck in ImagePullBackOff because they referenced
the local-only image 'myorg-assistant:latest' which is not present
on the node. Switch all 5 cronjobs to the Gitea registry image
git.rogi.casa/roger/myorg-assistant/myorg-assistant:fcf79bf
(matching the Deployment), set imagePullPolicy: Always, and add
imagePullSecrets: gitea-registry so they can authenticate to the
private registry.
2026-07-06 14:11:44 +00:00
Platform Engineer
fc1b4383c1 fix(argocd): add ignoreDifferences to root app to stop Application CRD drift loop
The k3s-cluster-root app-of-apps has been in a continuous Synced<->OutOfSync
oscillation (489 transitions in 24h). Each cycle, ArgoCD syncs the 'argocd'
Application CRD, succeeds, then immediately detects drift because ArgoCD
adds status/operation fields to Application resources at runtime.

Adding ignoreDifferences for /status and /operation on Application resources
stops the drift loop while keeping the app-of-apps functional.
2026-07-06 14:11:44 +00:00
Roger Oriol
6b697c9665 configure higher cron schedules 2026-07-05 23:30:51 +02:00
Roger Oriol
08bb4de278 fix platform-engineer agent 2026-07-05 21:01:28 +02:00
Roger Oriol
2cccbc019f fix rbac role for platform engineer cron seed 2026-07-05 18:15:10 +02:00
Roger Oriol
3f29b77e55 remove k8s event exporter with bad sink 2026-07-05 17:48:38 +02:00
Roger Oriol
440fcf858f new argocd user platform-engineer 2026-07-05 17:39:54 +02:00
Roger Oriol
9fd7d02c7c loki + promtail new monitoring services 2026-07-05 17:32:15 +02:00
Roger Oriol
85c8cbfc31 fix searxng 2026-07-04 00:53:17 +02:00
Roger Oriol
9de2897f46 fix searxng 2026-07-04 00:51:11 +02:00
Roger Oriol
c3a07f75ab fix searxng 2026-07-04 00:44:49 +02:00
Roger Oriol
a567184347 fix searxng 2026-07-04 00:35:17 +02:00
Roger Oriol
54059cdb72 searxng 2026-07-04 00:19:48 +02:00
Roger Oriol
7faaa53855 searxng 2026-07-04 00:10:23 +02:00
Roger Oriol
6e689accd0 fix hermes cron 2026-06-29 00:48:23 +02:00
Roger Oriol
1145214e24 fix hermes qwen model 2026-06-29 00:21:57 +02:00
Roger Oriol
9eb8d344fa fix pihole liveness probe 2026-06-28 19:08:04 +02:00
Roger Oriol
22ef2a38b2 pihole unbound 2026-06-28 09:34:43 +02:00
Roger Oriol
d00c6fb63d fix qwen model name in hermes 2026-06-28 09:28:35 +02:00
Roger Oriol
734962d198 fix hermes liveness probe 2026-06-28 00:43:09 +02:00
Roger Oriol
4d9195b32d use hermes stock image for platform engineer 2026-06-27 21:00:03 +02:00
Roger Oriol
54579df4b3 use hermes stock image for platform engineer 2026-06-27 20:40:42 +02:00
Roger Oriol
3f3467cb13 gitea registry ingress 2026-06-27 11:46:53 +02:00
Roger Oriol
6e02d9a885 new platform engineer agent 2026-06-27 00:09:39 +02:00
Roger Oriol
d8012dfb6c monitoring: add dashboard ideas doc
Survey of dashboards that could be built from existing and not-yet-enabled
metrics across the cluster's services (traefik, coredns, metallb, cert-manager,
phoenix, litellm, gitea, postgres, etc.), with per-service enable steps and
a recommended priority order.
2026-06-26 20:22:54 +02:00
Roger Oriol
bf1387dc3e monitoring: add Grafana dashboards + kube-state-metrics & node-exporter
Dashboards (provisioned via ConfigMaps into Grafana pod, 'K3s Cluster' folder):
- Cluster Overview: per-namespace CPU/mem/net/fs, pod counts, pod health (KSM)
- Pods & Services: per-pod CPU/mem/net/fs, throttling, pod status, restarts, PVCs
- Nodes: per-node CPU%/mem%, load average, disk usage, network (node-exporter)
- Control Plane & API Server: request rate, latency p95, 5xx, kubelet/PLEG
- Prometheus Self-Monitoring: ingestion, series, scrape duration, memory

Exporters (auto-scraped via existing kubernetes-service-endpoints job):
- kube-state-metrics: pod/deployment/PVC/replica state (kube_pod_status_phase,
  kube_pod_container_status_restarts_total, kube_persistentvolumeclaim_*)
- node-exporter (DaemonSet, hostNetwork): node_cpu_seconds_total,
  node_memory_*, node_filesystem_*, node_load*, node_network_*
2026-06-26 19:48:17 +02:00
Roger Oriol
2eab82b430 fix nas ingress 2026-06-26 19:01:08 +02:00
Roger Oriol
3cdd40153f fix nas ingress 2026-06-26 18:54:17 +02:00
Roger Oriol
9f74a88be7 fix nas ingress 2026-06-26 18:40:41 +02:00
Roger Oriol
586e95a57d fix nas ingress 2026-06-26 18:25:29 +02:00
Roger Oriol
9f7e34ef78 fix prometheus ingress 2026-06-26 18:06:01 +02:00
Roger Oriol
b43874bdcd Expose minecraft server over TCP via MetalLB
Minecraft Java Edition uses raw TCP on port 25565, not HTTP. The previous
ClusterIP Service + HTTP Ingress (Traefik 80/443) could not carry TCP 25565
traffic, so minecraft.rogi.casa:25565 was unreachable.

- Change Service to LoadBalancer with fixed IP 10.88.20.103 (dmz-pool),
  matching the pihole-dns pattern, so port 25565 is exposed directly.
- Remove the dead HTTP ingress (it routed HTTP to a TCP game port).
2026-06-26 13:38:43 +02:00
Roger Oriol
da2bae6fa5 Merge branch 'main' of https://git.rogi.casa/roger/k3s-cluster 2026-06-26 12:01:29 +02:00
Roger Oriol
e77e170421 fix(homeassistant): trust k3s pod/service CIDRs as X-Forwarded-For proxies
HA runs with hostNetwork on roger-nucbox-evo-x2 while Traefik runs on the
raspberrypi node, so requests arrive at HA from 10.88.20.11. The previous
trusted_proxies entry (10.88.88.0/24) did not include this address, causing
HA to reject X-Forwarded-For and return 400 on every ingress request.
2026-06-26 11:58:46 +02:00
Gitea Actions
ec947bd58a Update gym-tracker image to 945910a 2026-06-25 19:03:07 +00:00
Gitea Actions
3e57da467d Update myorg-assistant image to fcf79bf 2026-06-25 11:15:19 +00:00
Gitea Actions
9eecedc396 Update gym-tracker image to 0411783 2026-06-25 10:13:21 +00:00
Roger Oriol
ab6b5dc407 fix container registry url 2026-06-25 11:55:35 +02:00
Roger Oriol
723693eb07 take out gitea runner secret 2026-06-24 22:30:44 +02:00
Roger Oriol
3ed4acd7ec change gitea runner token 2026-06-24 22:20:53 +02:00
Roger Oriol
1bcfc13047 fix gitea config 2026-06-24 20:38:28 +02:00