Compare commits

...

24 Commits

Author SHA1 Message Date
Hermes Platform Engineer
45a9948682 fix(myorg-assistant): normalize GIT_REPO_URL host to git.rogi.casa in git-sync
PR #26 added the git-clone initContainer but still trusted the GIT_REPO_URL
secret verbatim. That secret historically contains gitea.rogi.casa, which
526s through Cloudflare, so the clone still fails with 'Not a git repository'.

Normalize the host to git.rogi.casa at runtime (sed) so the clone works
regardless of the secret's stale host. Directly addresses the subdomain bug.
2026-07-19 16:13:45 +00:00
Roger Oriol
e44d7ba1fc Merge branch 'main' of https://git.rogi.casa/roger/k3s-cluster 2026-07-19 12:01:21 +02:00
Roger Oriol
f1005cd426 change openwebui url to ai.rogi.casa 2026-07-19 12:00:52 +02:00
Roger Oriol
43c0c2561e fix hermes write folder 2026-07-18 19:29:52 +02:00
8334cd48f8 Merge pull request 'fix(myorg-assistant): add git-clone initContainer to git-sync CronJob' (#26) from fix/myorg-git-sync-clone into main
Reviewed-on: roger/k3s-cluster#26
2026-07-18 19:25:00 +02:00
Hermes Platform Engineer
ba7e05a73d fix(myorg-assistant): add git-clone initContainer to git-sync CronJob
The git-sync CronJob only ran `git pull`/`git push` against /data/myorg
but never ensured the repo existed. The clone was solely the Deployment's
git-clone initContainer's job, and when that didn't populate the volume the
cron pod failed with 'Not a git repository: /data/myorg' on every run (and
swallowed the error, exiting 0).

Add an idempotent git-clone initContainer (guarded by [ ! -d /data/myorg/.git ])
so the cron job self-heals and actually syncs.

Refs: myorg-git-sync-* 'Pull: Error: Not a git repository' (last 24h+)
2026-07-18 17:21:26 +00:00
Roger Oriol
8bc3025296 fix platform engineer not allowed to respond to discord messages 2026-07-18 19:20:23 +02:00
Roger Oriol
7a7d67bedc configure git token env variable in myorg assistant cronjobs 2026-07-18 19:03:46 +02:00
Roger Oriol
19cdc77880 add git token env var to deadline checker 2026-07-18 18:59:36 +02:00
Roger Oriol
8983f482d0 give more memory to homeassistant 2026-07-17 23:59:50 +02:00
Roger Oriol
0b27cefd13 myorg assistant cron jobs env variables 2026-07-15 00:32:23 +02:00
Roger Oriol
279cc1f235 configure litellm models 2026-07-14 21:45:18 +02:00
Roger Oriol
cf6e2784fe configure litellm models 2026-07-14 21:36:38 +02:00
Roger Oriol
dad38347e7 upgrade n8n memory requirements 2026-07-14 18:50:28 +02:00
Roger Oriol
a5b90994a4 fix myorg assistant 2026-07-11 19:22:57 +02:00
Roger Oriol
04b736287b fix argocd redirect 2026-07-09 23:48:47 +02:00
Roger Oriol
8c6950fd43 Merge branch 'main' of https://git.rogi.casa/roger/k3s-cluster 2026-07-09 00:45:40 +02:00
Roger Oriol
81dfe6fd60 fix argocd rbac and create agents.md 2026-07-09 00:45:10 +02:00
5d80abf3e8 Merge pull request 'fix: myorg-assistant cronjobs ImagePullBackOff + argocd root drift loop' (#18) from platform-engineer-agent/k3s-cluster:fix-merge-to-main into main
Reviewed-on: roger/k3s-cluster#18
2026-07-09 00:22:54 +02:00
48f18d2a3e Merge branch 'main' into fix-merge-to-main 2026-07-09 00:22:22 +02:00
Roger Oriol
ce08365e06 revert to qwen3.6 for platform engineer 2026-07-07 23:52:37 +02:00
Platform Engineer
0794153e56 fix(myorg-assistant): point cronjobs at registry image + add imagePullSecrets
CronJob pods were stuck in ImagePullBackOff because they referenced
the local-only image 'myorg-assistant:latest' which is not present
on the node. Switch all 5 cronjobs to the Gitea registry image
git.rogi.casa/roger/myorg-assistant/myorg-assistant:fcf79bf
(matching the Deployment), set imagePullPolicy: Always, and add
imagePullSecrets: gitea-registry so they can authenticate to the
private registry.
2026-07-06 14:11:44 +00:00
Platform Engineer
fc1b4383c1 fix(argocd): add ignoreDifferences to root app to stop Application CRD drift loop
The k3s-cluster-root app-of-apps has been in a continuous Synced<->OutOfSync
oscillation (489 transitions in 24h). Each cycle, ArgoCD syncs the 'argocd'
Application CRD, succeeds, then immediately detects drift because ArgoCD
adds status/operation fields to Application resources at runtime.

Adding ignoreDifferences for /status and /operation on Application resources
stops the drift loop while keeping the app-of-apps functional.
2026-07-06 14:11:44 +00:00
Roger Oriol
6b697c9665 configure higher cron schedules 2026-07-05 23:30:51 +02:00
17 changed files with 453 additions and 34 deletions

246
AGENTS.md Normal file
View File

@@ -0,0 +1,246 @@
# AGENTS.md - Guide for Coding Agents
This file provides essential information for AI coding agents working with this Kubernetes cluster project.
## Project Overview
This repository contains Kubernetes manifests for a K3s cluster running self-hosted services on the `rogi.casa` domain. The cluster is managed via **GitOps using ArgoCD** - all changes to the cluster are deployed automatically from this Git repository.
**⚠️ CRITICAL: Permission Model**
You **DO NOT** have permission to push changes to this repository. Before applying any changes to the cluster:
1. Make the necessary code changes to the manifests
2. Clearly present the changes to the user
3. Ask the user to review and push the changes
4. Wait for confirmation that changes have been pushed
5. Only then will ArgoCD automatically deploy the changes to the cluster
## Architecture & GitOps Workflow
### ArgoCD App-of-Apps Pattern
This project uses ArgoCD's "app-of-apps" pattern:
```
argocd-bootstrap.yaml (root Application)
argocd/apps/ (directory containing all Application manifests)
Individual Applications (one per service directory)
Kubernetes manifests in each service directory (e.g., pihole/, homeassistant/)
```
### Deployment Flow
1. You make changes to Kubernetes manifests in the repository
2. User reviews and pushes changes to the `main` branch
3. ArgoCD detects changes (automatically or on sync)
4. ArgoCD applies changes to the cluster with `prune: true` and `selfHeal: true`
5. Cluster state converges to match the Git state
### Key Files
- **`argocd-bootstrap.yaml`**: The root Application that bootstraps ArgoCD. Points to `argocd/apps/` directory. This is the only file that needs manual `kubectl apply` during initial setup.
- **`argocd/apps/project.yaml`**: ArgoCD AppProject defining permissions for all applications
- **`argocd/apps/*.yaml`**: Individual ArgoCD Application manifests (one per service)
- **`argocd/gen-apps.sh`**: Script to regenerate all ArgoCD manifests from the `APPS` array
## Repository Structure
```
k3s-cluster/
├── argocd-bootstrap.yaml # Root ArgoCD Application (app-of-apps)
├── argocd/
│ ├── apps/ # Individual ArgoCD Application manifests
│ │ ├── project.yaml # AppProject definition
│ │ ├── pihole.yaml # Application for pihole/
│ │ ├── homeassistant.yaml # Application for homeassistant/
│ │ └── ... # One per service
│ ├── gen-apps.sh # Generates argocd/apps/* manifests
│ └── ingress.yaml # ArgoCD's own ingress
├── <service-name>/ # Each service has its own directory
│ ├── namespace.yaml # (Optional) Namespace definition
│ ├── deployment.yaml # Main deployment/statefulset
│ ├── service.yaml # Service definition
│ ├── ingress.yaml # Ingress configuration
│ ├── configmap.yaml # (Optional) ConfigMaps
│ ├── pvc.yaml # (Optional) PersistentVolumeClaims
│ └── secret.yaml # (Optional) Secrets (rarely committed)
├── cert-manager/ # cert-manager installation manifests
├── nas/ # External NAS service configuration
├── monitoring/ # Prometheus + Grafana stack
└── README.md # Comprehensive project documentation
```
## Current Services
The cluster runs these services (each in its own directory):
- **argocd** - GitOps continuous delivery platform
- **cert-manager** - SSL certificate management (Let's Encrypt)
- **fava** - Beancount accounting web interface
- **gitea** - Self-hosted Git server
- **glance** - Personal dashboard
- **gym-tracker** - Workout tracking application
- **homeassistant** - Home automation
- **jellyfin** - Media server
- **litellm** - LLM proxy
- **minecraft-server** - Minecraft server
- **monitoring** - Prometheus + Grafana
- **myorg-assistant** - Organization assistant
- **n8n** - Workflow automation
- **nas** - External NAS proxy
- **openwebui** - Web UI for LLMs
- **phoenix** - AI observability platform
- **pihole** - Network-wide ad blocking
- **platform-engineer** - Platform engineering tools
- **qbittorrent** - Torrent client
- **searxng** - Meta search engine
- **vaultwarden** - Password manager (Bitwarden compatible)
## How to Make Changes
### Adding a New Service
1. Create a new directory: `mkdir new-service`
2. Create Kubernetes manifests in `new-service/`:
- `namespace.yaml` (if dedicated namespace needed)
- `deployment.yaml` or `statefulset.yaml`
- `service.yaml`
- `ingress.yaml`
- Any ConfigMaps, Secrets, PVCs needed
3. Add the service to `argocd/gen-apps.sh`:
- Add a line to the `APPS` array: `"new-service|namespace|new-service|true|true"`
- Format: `name|namespace|path|recurse|validate`
4. Run `./argocd/gen-apps.sh` to regenerate ArgoCD manifests
5. **Present changes to user for review and push**
### Modifying an Existing Service
1. Edit the relevant manifest(s) in the service directory
2. If changing ArgoCD configuration, also update `argocd/gen-apps.sh` and regenerate
3. **Present changes to user for review and push**
### Removing a Service
1. Remove the service directory: `rm -rf service-name/`
2. Remove from `APPS` array in `argocd/gen-apps.sh`
3. Run `./argocd/gen-apps.sh` to regenerate
4. **Present changes to user for review and push**
5. ArgoCD will automatically prune the resources from the cluster
## Common Patterns
### Ingress Configuration
Each service has its own `ingress.yaml` with:
- `ingressClassName: traefik` (K3s default)
- TLS configured with `cert-manager.io/cluster-issuer: letsencrypt-prod`
- Host-based routing (e.g., `pihole.rogi.casa`)
Example:
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: pihole
namespace: pihole
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: traefik
tls:
- hosts:
- pihole.rogi.casa
secretName: pihole-tls
rules:
- host: pihole.rogi.casa
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: pihole-web
port:
number: 80
```
### Resource Management
- Each service typically has its own namespace
- Use ResourceRequests and Limits for all containers
- PVCs for persistent data
- ConfigMaps for configuration files
## Important Notes
### What You CAN Do
- Read and understand all manifests
- Create new manifest files
- Modify existing manifest files
- Run `./argocd/gen-apps.sh` to regenerate ArgoCD manifests
- Explain how the cluster works
- Troubleshoot issues by reading manifests
### What You CANNOT Do
- Push changes to the Git repository (no push permissions)
- Directly apply manifests with `kubectl apply` (unless explicitly asked)
- Access the Kubernetes cluster directly (unless explicitly configured)
- Create secrets that should remain private (those are managed manually)
### Secrets Management
Secrets are generally **not committed to the repository**. They must be created manually in the cluster:
```bash
kubectl create secret docker-registry gitea-registry \
--docker-server=gitea.rogi.casa \
--docker-username=<user> \
--docker-password=<token> \
-n <namespace>
```
## Workflow Summary
When asked to make changes:
1. **Understand** the current state by reading relevant files
2. **Modify** the manifests (create/edit files)
3. **Regenerate** ArgoCD manifests if needed (`./argocd/gen-apps.sh`)
4. **Present** the changes clearly to the user:
```
I've made the following changes:
- Modified pihole/deployment.yaml to update image version
- Regenerated argocd/apps/pihole.yaml
Please review and push these changes to deploy them.
```
5. **Wait** for user confirmation that changes are pushed
6. **Verify** (if possible) that ArgoCD has synced the changes
## Useful Commands (for reference)
```bash
# Regenerate ArgoCD manifests after modifying gen-apps.sh
./argocd/gen-apps.sh
# Check ArgoCD applications status (requires kubectl access)
kubectl get applications -n argocd
# View logs of a pod (requires kubectl access)
kubectl logs -n <namespace> <pod-name>
# Check ingress status (requires kubectl access)
kubectl get ingress -n <namespace>
```
## Questions?
If you're unsure about anything:
1. Read the comprehensive `README.md` in the repository root
2. Check existing service directories for examples
3. Ask the user for clarification before making changes
4. Remember: **never push without explicit user review and approval**

View File

@@ -22,3 +22,9 @@ spec:
selfHeal: true
syncOptions:
- CreateNamespace=false
ignoreDifferences:
- group: argoproj.io
kind: Application
jsonPointers:
- /status
- /operation

View File

@@ -7,6 +7,11 @@ metadata:
app.kubernetes.io/name: argocd-cm
app.kubernetes.io/part-of: argocd
data:
# Serve HTTP (no redirect to HTTPS) so the TLS-terminating Traefik ingress works.
# Without this, argocd-server redirects HTTP->HTTPS, causing an infinite
# redirect loop behind the ingress (argocd.rogi.casa unreachable).
server.insecure: "true"
# add an additional local user with apiKey and login capabilities
# apiKey - allows generating API keys
# login - allows to login using UI

View File

@@ -0,0 +1,29 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-rbac-cm
namespace: argocd
labels:
app.kubernetes.io/name: argocd-rbac-cm
app.kubernetes.io/part-of: argocd
data:
policy.csv: |
# Grant platform-engineer read-only access to applications
g, platform-engineer, role:readonly
# Custom policy for platform-engineer with application read permissions
p, role:platform-engineer, applications, get, *, allow
p, role:platform-engineer, applications, list, *, allow
p, role:platform-engineer, clusters, get, *, allow
p, role:platform-engineer, clusters, list, *, allow
p, role:platform-engineer, repositories, get, *, allow
p, role:platform-engineer, repositories, list, *, allow
p, role:platform-engineer, projects, get, *, allow
p, role:platform-engineer, projects, list, *, allow
g, platform-engineer, role:platform-engineer
# Default policy - deny by default (ArgoCD default)
policy.default: role:readonly
# Enable RBAC
rbac.enabled: "true"

View File

@@ -58,9 +58,9 @@ spec:
image: ghcr.io/home-assistant/home-assistant:stable
resources:
requests:
memory: "256Mi"
limits:
memory: "512Mi"
limits:
memory: "1Gi"
ports:
- containerPort: 8123
volumeMounts:

View File

@@ -11,18 +11,30 @@ metadata:
data:
config.yaml: |
model_list:
- model_name: gpt-5-mini
- model_name: gpt-5.6-luna
litellm_params:
model: openai/gpt-5-mini-2025-08-07
model: openai/gpt-5.6-luna
api_key: "os.environ/OPENAI_API_KEY"
- model_name: claude-4.5-haiku
- model_name: claude-haiku-4.5
litellm_params:
model: "anthropic/claude-haiku-4-5-20251001"
api_key: "os.environ/ANTHROPIC_API_KEY"
- model_name: claude-sonnet-5
litellm_params:
model: "anthropic/claude-sonnet-5"
api_key: "os.environ/ANTHROPIC_API_KEY"
- model_name: gemini-3-flash
litellm_params:
model: gemini/gemini-3-flash-preview
api_key: "os.environ/GEMINI_API_KEY"
- model_name: tencent/hy3:free
litellm_params:
model: openrouter/tencent/hy3:free
api_key: "os.environ/OPENROUTER_API_KEY"
- model_name: z-ai/glm-5.2
litellm_params:
model: openrouter/z-ai/glm-5.2
api_key: "os.environ/OPENROUTER_API_KEY"
- model_name: glm-4.7-flash
litellm_params:
model: ollama/glm-4.7-flash

View File

@@ -22,13 +22,16 @@ spec:
job: deadline-checker
spec:
restartPolicy: OnFailure
imagePullSecrets:
- name: gitea-registry
containers:
- name: deadline-checker
image: myorg-assistant:latest
imagePullPolicy: IfNotPresent
image: git.rogi.casa/roger/myorg-assistant/myorg-assistant:fcf79bf
imagePullPolicy: Always
command:
- python
- run_job.py
- -c
- "from src.scheduler.jobs import run_job; import sys; run_job(sys.argv[1])"
- deadline-checker
env:
- name: MYORG_REPO_PATH
@@ -51,6 +54,16 @@ spec:
secretKeyRef:
name: myorg-assistant-secret
key: LITELLM_API_KEY
- name: WEB_SECRET_KEY
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: WEB_SECRET_KEY
- name: GIT_TOKEN
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: GIT_TOKEN
volumeMounts:
- name: myorg-data
mountPath: /data/myorg

View File

@@ -22,13 +22,16 @@ spec:
job: evening-summary
spec:
restartPolicy: OnFailure
imagePullSecrets:
- name: gitea-registry
containers:
- name: evening-summary
image: myorg-assistant:latest
imagePullPolicy: IfNotPresent
image: git.rogi.casa/roger/myorg-assistant/myorg-assistant:fcf79bf
imagePullPolicy: Always
command:
- python
- run_job.py
- -c
- "from src.scheduler.jobs import run_job; import sys; run_job(sys.argv[1])"
- evening-summary
env:
- name: MYORG_REPO_PATH
@@ -51,6 +54,16 @@ spec:
secretKeyRef:
name: myorg-assistant-secret
key: LITELLM_API_KEY
- name: WEB_SECRET_KEY
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: WEB_SECRET_KEY
- name: GIT_TOKEN
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: GIT_TOKEN
volumeMounts:
- name: myorg-data
mountPath: /data/myorg

View File

@@ -22,13 +22,57 @@ spec:
job: git-sync
spec:
restartPolicy: OnFailure
imagePullSecrets:
- name: gitea-registry
initContainers:
- name: git-clone
image: alpine/git:latest
command:
- sh
- -c
- |
# Normalize the repo URL host to the correct Gitea subdomain.
# The GIT_REPO_URL secret historically contained gitea.rogi.casa,
# which 526s through Cloudflare; the working subdomain is git.rogi.casa.
export GIT_REPO_URL="$(echo "${GIT_REPO_URL}" | sed -E 's#https?://[^/@]+@?gitea\.rogi\.casa#https://'"${GIT_USERNAME}"':'"${GIT_TOKEN}"'@git.rogi.casa#')"
if [ ! -d /data/myorg/.git ]; then
echo "Cloning repository..."
git clone ${GIT_REPO_URL} /data/myorg
cd /data/myorg
git config user.name "${GIT_USERNAME}"
git config user.email "${GIT_USERNAME}@rogi.casa"
git config credential.helper store
echo "https://${GIT_USERNAME}:${GIT_TOKEN}@git.rogi.casa" > ~/.git-credentials
else
echo "Repository already exists, skipping clone."
fi
env:
- name: GIT_REPO_URL
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: GIT_REPO_URL
- name: GIT_USERNAME
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: GIT_USERNAME
- name: GIT_TOKEN
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: GIT_TOKEN
volumeMounts:
- name: myorg-data
mountPath: /data/myorg
containers:
- name: git-sync
image: myorg-assistant:latest
imagePullPolicy: IfNotPresent
image: git.rogi.casa/roger/myorg-assistant/myorg-assistant:fcf79bf
imagePullPolicy: Always
command:
- python
- run_job.py
- -c
- "from src.scheduler.jobs import run_job; import sys; run_job(sys.argv[1])"
- git-sync
env:
- name: MYORG_REPO_PATH
@@ -66,6 +110,11 @@ spec:
secretKeyRef:
name: myorg-assistant-secret
key: LITELLM_API_KEY
- name: WEB_SECRET_KEY
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: WEB_SECRET_KEY
volumeMounts:
- name: myorg-data
mountPath: /data/myorg

View File

@@ -22,13 +22,16 @@ spec:
job: morning-briefing
spec:
restartPolicy: OnFailure
imagePullSecrets:
- name: gitea-registry
containers:
- name: morning-briefing
image: myorg-assistant:latest
imagePullPolicy: IfNotPresent
image: git.rogi.casa/roger/myorg-assistant/myorg-assistant:fcf79bf
imagePullPolicy: Always
command:
- python
- run_job.py
- -c
- "from src.scheduler.jobs import run_job; import sys; run_job(sys.argv[1])"
- morning-briefing
env:
# From ConfigMap
@@ -58,6 +61,16 @@ spec:
secretKeyRef:
name: myorg-assistant-secret
key: LITELLM_API_KEY
- name: WEB_SECRET_KEY
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: WEB_SECRET_KEY
- name: GIT_TOKEN
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: GIT_TOKEN
volumeMounts:
- name: myorg-data
mountPath: /data/myorg

View File

@@ -22,13 +22,16 @@ spec:
job: waiting-followup
spec:
restartPolicy: OnFailure
imagePullSecrets:
- name: gitea-registry
containers:
- name: waiting-followup
image: myorg-assistant:latest
imagePullPolicy: IfNotPresent
image: git.rogi.casa/roger/myorg-assistant/myorg-assistant:fcf79bf
imagePullPolicy: Always
command:
- python
- run_job.py
- -c
- "from src.scheduler.jobs import run_job; import sys; run_job(sys.argv[1])"
- waiting-followup
env:
- name: MYORG_REPO_PATH
@@ -51,6 +54,16 @@ spec:
secretKeyRef:
name: myorg-assistant-secret
key: LITELLM_API_KEY
- name: WEB_SECRET_KEY
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: WEB_SECRET_KEY
- name: GIT_TOKEN
valueFrom:
secretKeyRef:
name: myorg-assistant-secret
key: GIT_TOKEN
volumeMounts:
- name: myorg-data
mountPath: /data/myorg

View File

@@ -34,7 +34,7 @@ spec:
git config user.name "${GIT_USERNAME}"
git config user.email "${GIT_USERNAME}@rogi.casa"
git config credential.helper store
echo "https://${GIT_USERNAME}:${GIT_TOKEN}@gitea.rogi.casa" > ~/.git-credentials
echo "https://${GIT_USERNAME}:${GIT_TOKEN}@git.rogi.casa" > ~/.git-credentials
else
echo "Repository already exists, pulling latest changes..."
cd /data/myorg

View File

@@ -53,15 +53,17 @@ spec:
value: http
- name: N8N_PORT
value: "5678"
- name: NODE_OPTIONS
value: "--max-old-space-size=768"
image: n8nio/n8n
name: n8n
ports:
- containerPort: 5678
resources:
requests:
memory: "250Mi"
memory: "512Mi"
limits:
memory: "500Mi"
memory: "1Gi"
volumeMounts:
- mountPath: /home/node/.n8n
name: n8n-claim0

View File

@@ -9,10 +9,10 @@ spec:
ingressClassName: traefik
tls:
- hosts:
- openai.rogi.casa
- ai.rogi.casa
secretName: openwebui-tls
rules:
- host: openai.rogi.casa
- host: ai.rogi.casa
http:
paths:
- path: /

View File

@@ -9,18 +9,18 @@ data:
config.yaml: |
model:
provider: openai-api
default: z-ai/glm-5.2
default: qwen3.6
base_url: "http://litellm-service.litellm:80/v1"
api_mode: chat_completions
auxiliary:
compression:
provider: openai-api
model: z-ai/glm-5.2
model: qwen3.6
base_url: "http://litellm-service.litellm:80/v1"
title_generation:
provider: openai-api
model: z-ai/glm-5.2
model: qwen3.6
base_url: "http://litellm-service.litellm:80/v1"
terminal:
@@ -59,6 +59,16 @@ data:
cron:
wrap_response: false
discord:
allowed_channels: '1470909384162017444' # DISCORD_HOME_CHANNEL
free_response_channels: '1470909384162017444' # no @mention needed here
# Per-platform gateway auth. Paired with GATEWAY_ALLOW_ALL_USERS=true in
# the env (secret.yaml), this lets the bot reply to inbound DMs and
# group messages from anyone. Tighten later by switching to
# DISCORD_ALLOWED_USERS=<id> in the secret and dropping these two lines.
dm_policy: open
group_policy: open
memory:
memory_enabled: true
user_profile_enabled: true

View File

@@ -65,22 +65,22 @@ spec:
}
# ---- Watchdog checks (silent unless something is wrong) ----
create "cluster-health-check" "every 15m" "discord" \
create "cluster-health-check" "every 6h" "discord" \
"Check cluster health using the HTTP APIs documented in your SOUL.md. Check: (1) any node that is NotReady, (2) any pod not in Running phase, (3) any recent error/panic/crashloop/backoff log lines in Loki across all namespaces in the last 20 minutes, (4) any ArgoCD app that is not Synced plus Healthy. If everything is healthy, reply with exactly [SILENT]. Otherwise give a concise per-resource summary of what is wrong."
create "pod-restart-loop" "every 10m" "discord" \
create "pod-restart-loop" "every 1h" "discord" \
"Find pods with high restart rates using the Prometheus API documented in your SOUL.md. If any pod has more than 3 restarts in the last 15 minutes, fetch its logs from Loki to diagnose the cause. If the cause is clearly fixable via a manifest change such as bumping a memory limit, fixing a config value, or bumping the restartedAt annotation, make the edit in /workspace/k3s-cluster, commit and push, then trigger an ArgoCD sync via the API. Report what you did in one line. If not clearly fixable, post the log excerpt and proposed fix, and wait for Roger. If no high-restart pods, reply [SILENT]."
create "pvc-pressure" "every 30m" "discord" \
create "pvc-pressure" "every 1d" "discord" \
"Check storage health using the Prometheus API documented in your SOUL.md. Alert if any PVC has less than 15 percent free space, or if any node filesystem is over 85 percent full. If all healthy, reply [SILENT]."
create "argocd-sync-health" "every 1h" "discord" \
create "argocd-sync-health" "every 6h" "discord" \
"Check ArgoCD app health using the API documented in your SOUL.md. If every app is Synced and Healthy, reply [SILENT]. Otherwise list the OutOfSync or Degraded apps with their status. If an app is OutOfSync and you believe a recent git push caused it, you may trigger a sync via the API. Do NOT hand-edit resources to fix them — fix the source repo."
create "cert-expiry" "0 9 * * *" "discord" \
"Check certificate expiry using the Prometheus API documented in your SOUL.md. Alert on any certificate expiring in under 21 days, with its name and namespace. If none, reply [SILENT]."
create "node-resource-drift" "every 30m" "discord" \
create "node-resource-drift" "every 1d" "discord" \
"Check node resources using the Prometheus API documented in your SOUL.md. Alert if any node is NotReady, or if any node has CPU over 90 percent or memory over 90 percent. Otherwise reply [SILENT]."
# ---- Daily report (always delivered) ----

View File

@@ -86,6 +86,7 @@ spec:
: > /opt/data/.env
chmod 600 /opt/data/.env
for k in OPENAI_API_KEY OPENAI_BASE_URL DISCORD_BOT_TOKEN DISCORD_HOME_CHANNEL \
DISCORD_ALLOW_ALL_USERS DISCORD_FREE_RESPONSE_CHANNELS \
GITEA_TOKEN GITEA_REPO_URL ARGOCD_API_TOKEN ARGOCD_SERVER \
HERMES_DASHBOARD HERMES_DASHBOARD_BASIC_AUTH_USERNAME \
HERMES_DASHBOARD_BASIC_AUTH_PASSWORD HERMES_DASHBOARD_BASIC_AUTH_SECRET; do
@@ -124,6 +125,13 @@ spec:
env:
- name: HERMES_HOME
value: /opt/data
# Hermes' file-write tool refuses any path outside HERMES_WRITE_SAFE_ROOT.
# When unset it defaults to HERMES_HOME (/opt/data), which blocks the
# agent's only GitOps remediation path (editing manifests under
# /workspace/k3s-cluster). Whitelist the whole filesystem — consistent
# with yolo:true, approvals.mode:off, and the agent having no k8s RBAC.
- name: HERMES_WRITE_SAFE_ROOT
value: "/"
volumeMounts:
- name: data
mountPath: /opt/data