fester/CHEATSHEET.md

Fester Operator Cheatsheet

Distributed Build + Scheduler + Observability System

---

Starting a Build

fester build ./project.yaml

Key Concepts

Nodes — Machines participating in builds (physical, VM, or container).

Targets — Compilation environments (x86_64-linux-gnu, aarch64-linux-gnu, riscv64, embedded/toolchain targets).

Actions — Graph nodes representing individual build steps.

Scheduler — Chooses the best node based on load, temperature, policy rules, and historical performance.

Build Systems View

Cockpit Module: Fester
Live DAG:       /ui/live_dag.html
Replay View:    /ui/replay.html

Debugging

# Failure Autopsy
GET /api/autopsy/<build_id>

# Timeline Replay
GET /api/replay/<session_id>

Cache System

Supports multiple backends:

• ccache (local compiler cache)
• MinIO (distributed cache)
• Btrfs CoW snapshots (reflink)
• tmpfs (in-memory acceleration)
• Shared CAS API (/api/cas/)

Snapshots

Freeze build state using:

• QCOW2 image snapshots
• Btrfs copy-on-write states

Node Control

fester node list
fester node set-policy <node> preferred
fester node set-policy <node> avoid

Scheduler Modes

Mode	Description
unified	Default weighted scoring
weighted	Thermal-aware throttling above 85 C
cache-first	Prioritize nodes with cached artifacts
target-isolated	Each target arch runs on designated nodes
intelligence	Experimental ML-driven scheduling

Safety Notes

• Do not run unrestricted builds on production nodes.
• Monitor thermal load in Grafana.
• Ensure cache integrity for distributed builds.
• Validate cross-compile toolchains before enabling targets.
• Use a reverse proxy with authentication for public exposure.

Design Principle

Every build is reproducible, explainable, and replayable.

System Maturity

Level	Capability
1	Distributed compiler system (distcc-like)
2	Smart scheduler (load/thermal/cache aware)
3	Observability system (metrics + Grafana + Cockpit)
4	Causal execution graph (why decisions happen)
5	Replayable build brain (session + timeline + autopsy)