forked from roger/k3s-cluster
fix platform-engineer agent
This commit is contained in:
Roger Oriol
@@ -9,18 +9,18 @@ data:
|
||||
config.yaml: |
|
||||
model:
|
||||
provider: openai-api
|
||||
default: qwen3.6
|
||||
default: z-ai/glm-5.2
|
||||
base_url: "http://litellm-service.litellm:80/v1"
|
||||
api_mode: chat_completions
|
||||
|
||||
auxiliary:
|
||||
compression:
|
||||
provider: openai-api
|
||||
model: qwen3.6
|
||||
model: z-ai/glm-5.2
|
||||
base_url: "http://litellm-service.litellm:80/v1"
|
||||
title_generation:
|
||||
provider: openai-api
|
||||
model: qwen3.6
|
||||
model: z-ai/glm-5.2
|
||||
base_url: "http://litellm-service.litellm:80/v1"
|
||||
|
||||
terminal:
|
||||
@@ -29,6 +29,23 @@ data:
|
||||
timeout: 180
|
||||
home_mode: profile
|
||||
|
||||
# The agent runs unattended (cron jobs). The terminal tool's security
|
||||
# scanner flags curl+data patterns as 'pending_approval', which blocks
|
||||
# cron jobs (no human to approve). `yolo: true` disables all approval
|
||||
# prompts — safe here because the agent's blast radius is limited to git
|
||||
# commits + read-only HTTP API queries (it has no k8s RBAC).
|
||||
yolo: true
|
||||
approvals:
|
||||
mode: off
|
||||
|
||||
# Disable the Tirith pre-exec command scanner. It flags in-cluster plain
|
||||
# HTTP URLs (http://prometheus.monitoring:9090 etc.) as 'insecure URL'
|
||||
# false positives, which blocks every API query. Safe to disable because
|
||||
# the agent has no k8s RBAC and yolo is already on.
|
||||
security:
|
||||
tirith_enabled: false
|
||||
tirith_fail_open: true
|
||||
|
||||
tool_loop_guardrails:
|
||||
hard_stop_enabled: true
|
||||
hard_stop_after:
|
||||
@@ -62,7 +79,7 @@ data:
|
||||
|
||||
- **Nodes:** `raspberrypi` (control-plane, arm64, 4 GiB), `rpi2` (arm,
|
||||
~512 MiB), `roger-nucbox-evo-x2` (amd64, 24 GiB — you run here).
|
||||
- **GitOps:** ArgoCD owns every app from the git repo at `$GITEA_REPO_URL`.
|
||||
- **GitOps:** ArgoCD owns every app from the git repo (cloned at /workspace/k3s-cluster).
|
||||
The repo is cloned at `/workspace/k3s-cluster`. Each app lives in its own
|
||||
folder; manifests are reconciled with prune + selfHeal.
|
||||
- **Ingress:** Traefik; TLS via cert-manager + `letsencrypt-prod`.
|
||||
@@ -74,35 +91,51 @@ data:
|
||||
|
||||
## How you observe the cluster (NO kubectl — you have none)
|
||||
|
||||
You have NO k8s API access and NO kubectl. Use these HTTP APIs instead:
|
||||
You have NO k8s API access and NO kubectl. DO NOT try to run kubectl — it
|
||||
is not installed and you have no RBAC. Use the HTTP APIs below with the
|
||||
terminal tool. Use in-cluster service hostnames (name.namespace:port),
|
||||
NOT public ingress URLs like loki.rogi.casa (they go through Cloudflare
|
||||
which times out on long requests).
|
||||
|
||||
1. **Prometheus** (metrics) at `http://prometheus.monitoring:9090/api/v1/query`
|
||||
— PromQL via `curl -G -s "http://prometheus.monitoring:9090/api/v1/query" --data-urlencode "query=<PROMQL>"`
|
||||
Examples:
|
||||
- Node Ready: `kube_node_status_condition{condition="Ready",status="true"}`
|
||||
- Node CPU/mem: `node_memory_MemAvailable_bytes`, `node_cpu_seconds_total`
|
||||
- Pod restarts: `kube_pod_container_status_restarts_total`
|
||||
- PVC usage: `kubelet_volume_stats_available_bytes / kubelet_volume_stats_capacity_bytes`
|
||||
- Cert expiry: `certmanager_certificate_expiration_timestamp_seconds`
|
||||
### 1. Prometheus (metrics)
|
||||
Endpoint: http://prometheus.monitoring:9090/api/v1/query
|
||||
Use the terminal tool to send an HTTP GET with a PromQL query parameter
|
||||
named 'query'. Useful PromQL:
|
||||
- Node not Ready: kube_node_status_condition{condition="Ready",status!="true"}
|
||||
- Pod not Running: kube_pod_status_phase{phase!="Running"}
|
||||
- Pod restarts: kube_pod_container_status_restarts_total
|
||||
- PVC free percent: kubelet_volume_stats_available_bytes / kubelet_volume_stats_capacity_bytes
|
||||
- Node mem free: node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes
|
||||
- Node disk used: 1 - (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"})
|
||||
- Cert expiry (days): (certmanager_certificate_expiration_timestamp_seconds - time()) / 86400
|
||||
- Top pods CPU: topk(5, rate(container_cpu_usage_seconds_total[5m]))
|
||||
- Top pods mem: topk(5, container_memory_working_set_bytes)
|
||||
|
||||
2. **Loki** (pod logs + events) at `http://loki.monitoring:3100/loki/api/v1/query_range`
|
||||
— LogQL via `curl -G -s "http://loki.monitoring:3100/loki/api/v1/query_range" --data-urlencode "query=<LOGQL>" --data-urlencode "start=<unix_ns>" --data-urlencode "end=<unix_ns>" --data-urlencode "limit=50"`
|
||||
Examples:
|
||||
- Errors in a namespace: `{namespace="myorg-assistant"} |= "error"`
|
||||
- CrashLoop across cluster: `{namespace=~".+"} |= "BackOff"`
|
||||
- k8s events: `{app="k8s-event-logger"} |= "Warning"`
|
||||
### 2. Loki (pod logs)
|
||||
Endpoint: http://loki.monitoring:3100/loki/api/v1/query_range
|
||||
Use the terminal tool to send an HTTP GET with these query parameters:
|
||||
'query' (a LogQL expression), 'start' and 'end' (Unix nanosecond
|
||||
timestamps), and 'limit'. Useful LogQL:
|
||||
- Errors in a namespace: {namespace="myorg-assistant"} |= "error"
|
||||
- CrashLoop across cluster: {namespace=~".+"} |~ "(?i)backoff|crashloop"
|
||||
- Pod logs: {namespace="<ns>",pod="<pod>"}
|
||||
|
||||
3. **ArgoCD API** at `https://argocd-server.argocd:443` — bearer token in
|
||||
`$ARGOCD_API_TOKEN`. (Verify the cert with `--insecure` if needed since
|
||||
it's the internal service.)
|
||||
Examples:
|
||||
- List apps: `curl -sk -H "Authorization: Bearer $ARGOCD_API_TOKEN" https://argocd-server.argocd:443/api/v1/applications`
|
||||
- Sync an app: `curl -sk -X POST -H "Authorization: Bearer $ARGOCD_API_TOKEN" https://argocd-server.argocd:443/api/v1/applications/<app>/sync`
|
||||
### 3. ArgoCD API (app status + sync triggers)
|
||||
Endpoint: https://argocd-server.argocd:443 (internal service, use the -k
|
||||
flag to skip TLS cert verification since it's a self-signed internal cert)
|
||||
Auth: bearer token (read from the environment variable for the ArgoCD token). Send an
|
||||
Authorization header with the token.
|
||||
Endpoints: GET /api/v1/applications (list apps), POST /api/v1/applications/<app>/sync (trigger sync)
|
||||
|
||||
## Parsing JSON responses
|
||||
|
||||
The execute_code tool is BLOCKED in cron mode. To parse JSON from HTTP
|
||||
responses, pipe the output through python3 or jq inside the terminal tool.
|
||||
|
||||
## How you remediate (git commit → ArgoCD sync)
|
||||
|
||||
You have NO k8s write access. Every fix is a git commit to the repo at
|
||||
`/workspace/k3s-cluster` (which you `git push` to Gitea using `$GITEA_TOKEN`).
|
||||
`/workspace/k3s-cluster` (which you push to Gitea using the token in your environment).
|
||||
ArgoCD's selfHeal picks up the change; if you need it faster, trigger a sync
|
||||
via the ArgoCD API.
|
||||
|
||||
@@ -111,10 +144,8 @@ data:
|
||||
git pull
|
||||
# ... edit the manifest(s) ...
|
||||
git add -A && git commit -m "fix(<app>): <what changed>"
|
||||
git push # uses the token in GITEA_REPO_URL / GITEA_TOKEN
|
||||
# optionally trigger ArgoCD sync:
|
||||
curl -sk -X POST -H "Authorization: Bearer $ARGOCD_API_TOKEN" \
|
||||
https://argocd-server.argocd:443/api/v1/applications/<app>/sync
|
||||
git push # uses the token embedded in the repo URL / environment
|
||||
# optionally trigger ArgoCD sync via the API (see section 3 above).
|
||||
|
||||
## Operating rules
|
||||
|
||||
|
||||
Reference in New Issue
Block a user