# Changelog

All notable changes to Fester are documented here.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
- **BTC 0.4.0 multi-arch cross-compilation support** — `backend/toolchain/btc.py`
  - `probe_btc()` now loads BTC manifest JSON sidecars for structured metadata
    (target_id, arch, clib, triple, cross_mode, target_march)
  - `btc_build_env()` supports `target` parameter to select specific cross-compiler
    golden images when multiple are installed on a node
  - `list_btc_targets()` enumerates all available BTC golden images with
    parsed SYS_LABEL and manifest data — usable by the Fester API
  - ISA flag table (`_ISA_FLAGS`) for AVX512, AVX2, SSE4_2, NEON, MIPS32, TILE
  - SSE4_2 ISA tier added for Intel Atom and AMD APU targets that lack AVX support
  - 19 cross-compilation targets: Intel HEDT/Server (5), AMD Ryzen/EPYC (4),
    AMD APU (4: apu-zn1 through apu-zn4), Intel Atom (4: silvermont, goldmont,
    tremont, sierraforest), embedded (mipselr2, armv7, tilegx)
  - SYS_LABEL parser updated for BTC 0.4.0 cross format:
    `DCOSNET-{FAMILY}-{TARGET_ID}-{ISA}-CROSS`
  - Per-node BTC target selection via `node_config.btc.target`
  - New env vars: `BTC_TARGET_ID`, `BTC_CROSS`, `BTC_CLIB`, `BTC_TARGET_TRIPLE`
- **Shared Content-Addressable Store (CAS) API** — `backend/storage/cas_api.py`
  - `PUT /api/cas/{sha256}` — store an artifact (SHA-256 verified server-side)
  - `GET /api/cas/{sha256}` — retrieve an artifact (streamed file download)
  - `HEAD /api/cas/{sha256}` — check existence (returns metadata JSON)
  - `DELETE /api/cas/{sha256}` — remove an artifact
  - `GET /api/cas/` — list all artifacts (paginated, filterable by target)
  - `GET /api/cas/stats` — cache statistics (hits, misses, utilization %)
  - SQLite-backed index for fast listing, LRU eviction, and cross-query filtering
  - btrfs reflink support for zero-copy artifact storage
  - DAG-aware cache integration in PipelineEngine — checks CAS before each action
- **BTC.sh toolchain integration** — `backend/toolchain/btc.py`
  - `probe_btc()` — detect BTC golden images, parse SYS_LABEL
  - `btc_build_env()` — return BTC-aware CC/CXX/CFLAGS/LDFLAGS
  - `verify_btc_stamp()` — read .note.BTC ELF note + xattr stamps
  - `apply_btc_stamp()` — inject forensic stamps via assembly + objcopy + xattr
  - `merge_compiler_env()` — overlay BTC on ccache/distcc env
  - Cross-project compatible: pipe-delimited .note.BTC format, type=1 (NT_VERSION), bare hex hash
- **Podman integration** — `backend/integrations/podman.py` (full PodmanManager)
- **Firecracker integration** — `backend/integrations/firecracker.py` (full FirecrackerManager)
- **Podman/Firecracker executor adapters** — `backend/executor/adapters.py` (run_podman, run_firecracker)
- **Runtime router** updated — `backend/executor/runtime_router.py` routes podman and firecracker runtimes
- Documentation: README.md, quickstart.md, CONTRIBUTING.md, CHANGELOG.md
- Tooling: bootstrap.sh, run.sh, pyproject.toml, .gitignore
- Docker support (Dockerfile)
- Screenshots for all 8 UI pages

## [0.3.0] — 2026-06-28

### Added — Storage layer
- SQLite-backed persistence for builds, sessions, events, and node states
- Btrfs CoW reflink snapshots (`backend/storage/btrfs_cas.py`)
- QCOW2 workspace freezing with real mount + rsync (`backend/storage/qcow2_freeze.py`)
- tmpfs workspace acceleration
- Storage status + config + freeze + reflink + snapshots endpoints

### Added — tmux integration (E9)
- TmuxManager with session create/list/inspect/capture-output/kill
- `runtime: "tmux"` action runtime — runs actions in detached tmux sessions
- Live output viewer in Sessions page UI
- `/api/actions/active`, `/api/actions/{name}/tmux/output`, `/tmux/kill` endpoints

### Added — mosh/ssh shell attach (E10)
- MoshManager with command builder (mosh-via-SSH-bootstrap)
- `/api/nodes/{name}/shell` GET + POST endpoints
- "⌘ Shell" button on Dashboard node rows (copies command to clipboard)

### Added — Real engine wiring
- BuildRunner wraps PipelineEngine with build lifecycle + persistence
- Real DAG execution with proper cwd + env passing
- Failure propagation: skipped actions on failed deps
- Critical path computation + tagging in events
- Debugger pause/resume/step actually controls running engines

### Added — Cause graph + blast radius
- `/api/propagation/{action}` — forward blast radius computation
- `/api/cause/explain/{node}` — causal chain for a node
- Blast Radius panel in Cause Graph UI page
- Failure autopsy with proper session ID correlation

### Added — Storage + target catalog
- `/api/targets` reads from `backend/targets/catalog.py` (6 systems: Gentoo, Buildroot, OpenWrt, ALFS, SourceMage, Lunar)
- `/api/targets/arches` + `/api/targets/runtimes` endpoints

### Added — Health + metrics
- `/api/health` endpoint (version, bus_subscribers, ws_clients, builds_known, timeline_events)
- Live health indicator in topbar (all pages)
- Prometheus metrics aligned with grafana config names (`fester_node_cpu`, `fester_pipeline_actions_total`, etc.)
- Pipeline action counters + cache hit counters wired via bus subscriber

### Added — UI enhancements
- New Timeline page (per-node event drill-down)
- Node policy dropdowns (preferred/avoid/neutral) on Dashboard
- Per-node "Probe" buttons + "Probe All" button
- Build cancel + replay buttons on Sessions page
- Event filter chips on Dashboard (toggle by event type)

### Added — CLI overhaul
- 35+ subcommands (was 11)
- Nested subcommands: `node {list,set-policy,probe,probe-all}`, `cause {explain,trace,events}`, `replay {start,step,reset}`, `debugger {state,pause,resume,step}`
- New: `build`, `builds`, `build-info`, `cancel`, `release`, `targets`, `toggle-target`, `blast`, `timeline`, `rewind`, `autopsy`, `policy-list`, `health`
- `--watch` flag on `build` command (streams events via WebSocket)

### Fixed — Critical backend bugs
- `api/api.py` order bug (`bus.subscribe()` before `bus = EventBus()`)
- 7 broken `_endpoint` symbol imports
- `cause_graph.attach_bus` + `emit_debug` defined at module scope (zero indentation)
- Three missing files: `timeline_store.py`, `failure_autopsy.py`, `failure_propagation.py`
- Two Python module/package name collisions (`scheduler.py` vs `scheduler/`, `nodes.py` vs `nodes/`, `cache.py` vs `cache/`)
- `execute_action` passing empty `{}` as cwd → `TypeError`
- Build env never reaching subprocess
- Debugger/pipeline-control `_ENGINE` always `None`
- `failure_propagation` looking for `event.data.deps` (legacy shape)
- Autopsy session ID mismatch
- `/api/release` was a pure stub
- `/api/policy/set` didn't persist
- `/ws-debugger` returned canned data, ignored commands
- `forgejo.on_push_event` wrong PipelineEngine arity
- `pipeline/feedback.py` PolicyEngine() with no args

### Fixed — Standardization
- ~20 bare imports converted to `backend.*` prefix
- Two parallel event taxonomies unified (`types.py` re-exports from `schema.py`)
- `MinioCache.__init__` made lazy (was crashing on import if MinIO unreachable)
- `cache.py` graceful fallback for non-writable paths
- `roles_store.py` graceful fallback for non-writable `/etc/fester/`
- `FesterEvent` made `node`/`action`/`state` truly optional (had `Optional[str]` type but no default)

## [0.2.0] — 2026-06-27

### Added — Real FastAPI backend
- `backend/main.py` — FastAPI app with 50+ routes
- All routers wired: replay, autopsy, timeline, debugger, pipeline_control, nodes, metrics, cause
- WebSocket hub with `/ws`, `/ws-targets`, `/ws-debugger`
- Ambient loop for node probing + drift
- SQLite storage layer (`backend/storage/sqlite_db.py`)
- Node probe module (`backend/nodes/probe.py`)
- BuildRunner (`backend/pipeline/runner.py`)

### Added — UI overhaul (7 pages)
- Dashboard (`index.html`) — 3-column layout with cluster stats, live stream, quick actions
- Live DAG (`ui/live_dag.html`) — layered Sugiyama-lite layout, SVG edges, click-to-inspect
- Replay (`ui/replay.html`) — timeline scrubber, play/pause/step, structured inspector
- Sessions (`ui/sessions.html`) — build history + replay links
- Metrics (`ui/metrics.html`) — live charts + per-node cards
- Cause Graph (`ui/cause.html`) — radial graph visualization
- Debugger (`ui/debugger.html`) — step-through execution control

### Added — Shared infrastructure
- `style.css` — full design system (dark theme tokens, panels, badges, buttons, scrollbars)
- `ui/app.js` — shared shell (WS auto-reconnect, FesterTargets global, toasts, topbar nav)
- `cockpit/fester-module/targets.js` — fixed to expose `FesterTargets` global

### Fixed
- `live_dag.html` was broken (referenced undefined `FesterTargets`)
- `style.css` was 0 bytes
- `index.html` was a 30-line throwaway

## [0.1.0] — Initial commit

### Added
- Basic project structure
- Backend modules (many stubbed)
- Cockpit module
- Install scripts
- CHEATSHEET.md