# Fester Cluster Build System — Cheatsheet

## 🧠 Core Concept

A distributed, DAG-driven build execution system with:

- real-time scheduling
- node-aware load balancing
- thermal + policy constraints
- cache-aware execution
- deterministic replay/debugging

---

## 🧩 System Components

### 1. Scheduler
Chooses best node per action using:

- CPU load
- temperature
- policy rules
- historical intelligence feedback

---

### 2. Pipeline Engine
Executes DAG actions sequentially or step-debugged.

Supports:
- live execution
- interactive stepping
- replay mode

---

### 3. Timeline Store
Immutable event log of entire system.

Used for:
- replay
- debugging
- autopsy analysis

---

### 4. Failure System
Detects and propagates failure:

- backward → root cause candidates
- forward → downstream impact

---

### 5. Critical Path Analyzer
Identifies bottleneck chain in DAG execution.

---

### 6. Cache Layer
Supports:
- distributed artifact reuse
- MinIO backend
- future: Btrfs/QCOW2 snapshot acceleration

---

### 7. Debugger Mode
Interactive execution control:

- pause
- step
- resume
- scheduler preview before execution

---

## 🧭 Execution Flow

1. Build graph generated from spec
2. Scheduler selects node per action
3. Timeline records decision
4. Executor runs action on node
5. Events streamed to UI
6. Failures propagate through DAG
7. Optional debugger intercepts execution

---

## 📡 Event Types

- `node`
- `pipeline`
- `schedule_decision`
- `failure_propagation`
- `debugger_preview`

---

## 🖥 UI Modes

### Live Mode
- real-time DAG rendering
- node state updates

### Debug Mode
- step execution
- scheduler inspection
- manual control

### Autopsy Mode
- failure root cause analysis
- dependency tracing
- critical path overlay

---

## ⚙️ Node Selection Model

Score-based weighted system:

- CPU availability
- thermal headroom
- policy constraints
- historical instability penalties

---

## 🧪 Execution Backends (planned)

- distcc
- LXC
- libvirt
- native execution
- cross-compilation toolchains

---

## 📦 Cache Strategy

- local ccache
- shared MinIO cache
- future: Btrfs/QCOW2 snapshot acceleration

---

## 🔐 Design Principle

> The system must always be replayable, explainable, and deterministic.

No hidden state.
No opaque scheduling.
Everything is observable.