Root cause of the git-sync failure: the cron pod only ran git pull/push
against /data/myorg but never ensured the repo existed, and the GIT_REPO_URL
secret historically pointed at the wrong Gitea subdomain (gitea.rogi.casa),
which 526s through Cloudflare -> 'Not a git repository: /data/myorg'.
Fixes:
1. Add an idempotent git-clone initContainer (guarded by [ ! -d /data/myorg/.git ])
so the cron job self-heals and actually clones on first run.
2. Normalize GIT_REPO_URL host to git.rogi.casa at runtime (sed), so the clone
works even if the cluster secret still contains the stale gitea.rogi.casa host.
The working subdomain is git.rogi.casa.
Evidence (Loki, last 24h+):
{namespace="myorg-assistant",pod=~"myorg-git-sync.+"}
-> Pull/Push: Error: Not a git repository: /data/myorg (every run)
-> job swallowed error, exited 0, so looked successful
Risk: low. initContainer mirrors the Deployment's; host-normalization is
idempotent. No RBAC/CRD/ArgoCD/volume changes.
CronJob pods were stuck in ImagePullBackOff because they referenced
the local-only image 'myorg-assistant:latest' which is not present
on the node. Switch all 5 cronjobs to the Gitea registry image
git.rogi.casa/roger/myorg-assistant/myorg-assistant:fcf79bf
(matching the Deployment), set imagePullPolicy: Always, and add
imagePullSecrets: gitea-registry so they can authenticate to the
private registry.
The k3s-cluster-root app-of-apps has been in a continuous Synced<->OutOfSync
oscillation (489 transitions in 24h). Each cycle, ArgoCD syncs the 'argocd'
Application CRD, succeeds, then immediately detects drift because ArgoCD
adds status/operation fields to Application resources at runtime.
Adding ignoreDifferences for /status and /operation on Application resources
stops the drift loop while keeping the app-of-apps functional.
Survey of dashboards that could be built from existing and not-yet-enabled
metrics across the cluster's services (traefik, coredns, metallb, cert-manager,
phoenix, litellm, gitea, postgres, etc.), with per-service enable steps and
a recommended priority order.
Minecraft Java Edition uses raw TCP on port 25565, not HTTP. The previous
ClusterIP Service + HTTP Ingress (Traefik 80/443) could not carry TCP 25565
traffic, so minecraft.rogi.casa:25565 was unreachable.
- Change Service to LoadBalancer with fixed IP 10.88.20.103 (dmz-pool),
matching the pihole-dns pattern, so port 25565 is exposed directly.
- Remove the dead HTTP ingress (it routed HTTP to a TCP game port).
HA runs with hostNetwork on roger-nucbox-evo-x2 while Traefik runs on the
raspberrypi node, so requests arrive at HA from 10.88.20.11. The previous
trusted_proxies entry (10.88.88.0/24) did not include this address, causing
HA to reject X-Forwarded-For and return 400 on every ingress request.
