Skip to main content

Grafana

Visualization layer for Prometheus metrics and Loki logs. Everything is provisioned from code — no manual setup needed.

Grafana

What's provisioned​

ResourceFilePurpose
Prometheus datasourceprovisioning/datasources/prometheus.yamlDefault datasource, 15s scrape interval
Loki datasourceprovisioning/datasources/loki.yamlLog queries, max 1000 lines
Decree Overview dashboardprovisioning/dashboards/decree-overview.jsonPre-built Decree monitoring

Grafana re-reads the provisioning directory every 30 seconds, so changes to files take effect without a restart.

Decree Overview dashboard​

Covers the Decree automation engine with panels for:

  • Current Status — last run success/failure per routine (Prometheus gauges)
  • Run logs — live log stream from Loki, filterable by routine, trigger type, and exit code

This is the main place to check when a Decree automation fails or behaves unexpectedly.

Extending​

  • New dashboard: drop a JSON export into hosting/grafana/provisioning/dashboards/. Grafana auto-loads it.
  • New datasource: add a YAML file to hosting/grafana/provisioning/datasources/.
  • Alerts: configure in the Grafana UI or add alert rules to dashboard JSON — they persist in the grafana_data volume.

Data flow summary​

Decree run
  └─ afterEach.sh
       ├─ Pushgateway → Prometheus (metrics: success, duration, attempts)
       └─ Loki push API (log: structured summary line)

automations/runs/**/routine.log
  └─ Promtail → Loki (full routine output, labeled by run_id/chain/seq)

Grafana
  ├─ queries Prometheus for metric panels
  └─ queries Loki for log panels