Files
k3s-cluster/monitoring/prometheus-deployment.yaml
Platform Engineer 6df0be81c9 fix(platform): fix high-restart pods - prometheus nodeSelector, litellm resources, gitea-runner limits
- Prometheus: change nodeSelector from hardware=high-memory (nonexistent label) to kubernetes.io/arch: amd64
  Fixes OOMKilled cause - pod was being scheduled on wrong nodes.
- litellm-deployment: add resource limits (2Gi/500m requests, 4Gi/2000m limits) + pin image to v1.34.0
  Fixes OOMKilled cause - no resource limits were set.
- gitea-runner: pin image to v0.12.0 from :latest + add resource limits (512Mi/250m requests, 1Gi/500m limits)
  Addresses extreme restart count (281) caused by unbounded memory usage and rolling image updates.
2026-07-21 20:28:12 +00:00

55 lines
1.5 KiB
YAML

apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus
namespace: monitoring
labels:
app: prometheus
spec:
replicas: 1
selector:
matchLabels:
app: prometheus
template:
metadata:
labels:
app: prometheus
spec:
# Target the nucbox (amd64, 24Gi RAM) which is the only node with enough memory for Prometheus.
nodeSelector:
kubernetes.io/os: linux
kubernetes.io/arch: amd64
serviceAccountName: prometheus
containers:
- name: prometheus
image: prom/prometheus:v2.48.1
args:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/etc/prometheus/console_libraries'
- '--web.console.templates=/etc/prometheus/consoles'
- '--storage.tsdb.retention.time=15d'
- '--web.enable-lifecycle'
ports:
- containerPort: 9090
name: web
volumeMounts:
- name: prometheus-config
mountPath: /etc/prometheus
- name: prometheus-storage
mountPath: /prometheus
resources:
requests:
memory: "512Mi"
cpu: "500m"
limits:
memory: "1Gi"
cpu: "1000m"
volumes:
- name: prometheus-config
configMap:
name: prometheus-config
- name: prometheus-storage
persistentVolumeClaim:
claimName: prometheus-storage