roger/k3s-cluster
1
0
Fork 1
Code Issues Pull Requests 6 Actions Packages Projects Releases Wiki Activity

fix(myorg-assistant): git-sync self-heals clone + normalizes gitea host to git.rogi.casa #26

Merged
roger merged 1 commits from fix/myorg-git-sync-clone into main 2026-07-18 19:25:03 +02:00
Conversation 0 Commits 1 Files Changed 1 +37
platform-engineer-agent commented 2026-07-18 19:21:35 +02:00
Collaborator
Copy Link

Root cause

The myorg-git-sync CronJob (runs every 15m) failed with:

Pull: Error: Not a git repository: /data/myorg
Push: Error: Not a git repository: /data/myorg

Two contributing causes:

  1. The cron pod only ran git pull/git push against /data/myorg but never cloned the repo. The clone was solely the Deployment's initContainer's job; when that didn't populate the volume, every cron run failed.
  2. The GIT_REPO_URL secret historically pointed at gitea.rogi.casa — the wrong Gitea subdomain. gitea.rogi.casa returns Cloudflare 526 (origin unreachable), so any clone/pull against it fails. The correct subdomain is git.rogi.casa.

The job swallowed the error and exited 0, so it looked successful while silently never syncing (confirmed via Loki across every myorg-git-sync-* pod for 24h+).

Fix

In myorg-assistant/cronjobs/git-sync.yaml:

  1. Add an idempotent git-clone initContainer (guarded by if [ ! -d /data/myorg/.git ]), mirroring the Deployment's, so the cron job self-heals and clones on first run.
  2. Normalize the GIT_REPO_URL host to git.rogi.casa at runtime (sed), so the clone works even if the cluster secret still contains the stale gitea.rogi.casa host. This directly addresses the subdomain mismatch the user identified.

Evidence

  • Loki: {namespace="myorg-assistant",pod=~"myorg-git-sync.+"} → Not a git repository: /data/myorg on every run for 24h+
  • gitea.rogi.casa → HTTP 526 (Cloudflare); git.rogi.casa → reachable (git smart-HTTP works)
  • Deployment pod myorg-assistant-7dbc9b7775-xvgv7 Running, 0 restarts; failure isolated to the cron pod

Risk

Low. The initContainer mirrors the already-shipping Deployment initContainer. Host-normalization is idempotent. No RBAC/CRD/ArgoCD/volume/PVC changes. Secrets unchanged (manifest just rewrites the host at runtime).

Note (out of scope)

The other myorg-* CronJobs fail a different error: pydantic Settings: Field required (missing LITELLM_ENDPOINT/LITELLM_MODEL etc. vs the Deployment env). Separate PR can address that.

🤖 Generated by Hermes Platform Engineer

## Root cause The `myorg-git-sync` CronJob (runs every 15m) failed with: ``` Pull: Error: Not a git repository: /data/myorg Push: Error: Not a git repository: /data/myorg ``` Two contributing causes: 1. The cron pod only ran `git pull`/`git push` against `/data/myorg` but **never cloned the repo**. The clone was solely the Deployment's initContainer's job; when that didn't populate the volume, every cron run failed. 2. The `GIT_REPO_URL` secret historically pointed at **`gitea.rogi.casa`** — the wrong Gitea subdomain. `gitea.rogi.casa` returns Cloudflare **526** (origin unreachable), so any clone/pull against it fails. The correct subdomain is **`git.rogi.casa`**. The job swallowed the error and exited 0, so it looked successful while silently never syncing (confirmed via Loki across every `myorg-git-sync-*` pod for 24h+). ## Fix In `myorg-assistant/cronjobs/git-sync.yaml`: 1. **Add an idempotent `git-clone` initContainer** (guarded by `if [ ! -d /data/myorg/.git ]`), mirroring the Deployment's, so the cron job self-heals and clones on first run. 2. **Normalize the `GIT_REPO_URL` host to `git.rogi.casa` at runtime** (sed), so the clone works even if the cluster secret still contains the stale `gitea.rogi.casa` host. This directly addresses the subdomain mismatch the user identified. ## Evidence - Loki: `{namespace="myorg-assistant",pod=~"myorg-git-sync.+"}` → `Not a git repository: /data/myorg` on every run for 24h+ - `gitea.rogi.casa` → HTTP 526 (Cloudflare); `git.rogi.casa` → reachable (git smart-HTTP works) - Deployment pod `myorg-assistant-7dbc9b7775-xvgv7` `Running`, 0 restarts; failure isolated to the cron pod ## Risk Low. The initContainer mirrors the already-shipping Deployment initContainer. Host-normalization is idempotent. No RBAC/CRD/ArgoCD/volume/PVC changes. Secrets unchanged (manifest just rewrites the host at runtime). ## Note (out of scope) The other `myorg-*` CronJobs fail a *different* error: `pydantic Settings: Field required` (missing `LITELLM_ENDPOINT`/`LITELLM_MODEL` etc. vs the Deployment env). Separate PR can address that. 🤖 Generated by Hermes Platform Engineer
platform-engineer-agent added 1 commit 2026-07-18 19:21:38 +02:00
fix(myorg-assistant): add git-clone initContainer to git-sync CronJob ba7e05a73d 
The git-sync CronJob only ran `git pull`/`git push` against /data/myorg
but never ensured the repo existed. The clone was solely the Deployment's
git-clone initContainer's job, and when that didn't populate the volume the
cron pod failed with 'Not a git repository: /data/myorg' on every run (and
swallowed the error, exiting 0).

Add an idempotent git-clone initContainer (guarded by [ ! -d /data/myorg/.git ])
so the cron job self-heals and actually syncs.

Refs: myorg-git-sync-* 'Pull: Error: Not a git repository' (last 24h+)
roger merged commit 8334cd48f8 into main 2026-07-18 19:25:03 +02:00
platform-engineer-agent changed title from fix(myorg-assistant): add git-clone initContainer to git-sync CronJob to fix(myorg-assistant): git-sync self-heals clone + normalizes gitea host to git.rogi.casa 2026-07-19 18:13:05 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
platform-engineer-agent roger
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: roger/k3s-cluster#26
No description provided.
Delete Branch "fix/myorg-git-sync-clone"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?