Sorcery-Go is a high-concurrency, source-based infrastructure management suite. It implements a spell format compatible with Source Mage GNU/Linux grimoires for convenience, but the two projects have separate maintainers, separate codebases, and separate governance.

This commit is contained in:
Jeremy Anderson 2026-07-12 01:40:30 -04:00
parent 3ea742f4b5
commit b1e7133369
18 changed files with 578 additions and 87 deletions

View File

@ -4,7 +4,7 @@ All notable changes to Sorcery-Go are documented here. The format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased] ## [1.1.0] — 2026-07-12
### Added ### Added
- **BTC 0.4.0 multi-arch cross-compilation support** (`pkg/toolchain/btc.go`) - **BTC 0.4.0 multi-arch cross-compilation support** (`pkg/toolchain/btc.go`)
@ -32,12 +32,6 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- `RetrieveArtifact` / `RetrieveArtifactToFile` — fetch cached artifacts - `RetrieveArtifact` / `RetrieveArtifactToFile` — fetch cached artifacts
- `Stats` — cache hit rate, utilization, artifact count - `Stats` — cache hit rate, utilization, artifact count
- Auto-initialized when Fester integration is active - Auto-initialized when Fester integration is active
- **BTC.sh toolchain integration** (`pkg/toolchain/btc.go`)
- `Probe()` — detect BTC.sh golden images at /opt/BTC
- `BuildEnv()` — return BTC-aware CC/CXX/CFLAGS/LDFLAGS
- `VerifyStamp()` — read .note.BTC ELF note and xattr stamps
- `StampBinary()` — apply forensic stamps (ELF note, xattr, debug symbols)
- Pipe-delimited format matching BTC.sh and Fester
- **Cauldron.BundleAndCache** — one-call bundle + CAS push method - **Cauldron.BundleAndCache** — one-call bundle + CAS push method
- **Scheduler.Dispatch now CAS-aware** — checks shared CAS before - **Scheduler.Dispatch now CAS-aware** — checks shared CAS before
dispatching to Fester; skips builds for cached artifacts dispatching to Fester; skips builds for cached artifacts
@ -54,7 +48,6 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- **eBPF security** replaces AppArmor as primary enforcement mechanism - **eBPF security** replaces AppArmor as primary enforcement mechanism
- `tomb_guard.bpf.c` — LSM hooks for file/inode protection - `tomb_guard.bpf.c` — LSM hooks for file/inode protection
- `sorcery_filter.bpf.c` — cgroup filters for device/network whitelisting - `sorcery_filter.bpf.c` — cgroup filters for device/network whitelisting
- AppArmor profiles retained as fallback
- **License** changed from GPL-3.0 to AGPL-3.0-or-later - **License** changed from GPL-3.0 to AGPL-3.0-or-later
- **Fester cluster integration** — full delegation of distributed build - **Fester cluster integration** — full delegation of distributed build
scheduling, node telemetry, and build dispatch to Fester scheduling, node telemetry, and build dispatch to Fester

View File

@ -2,7 +2,7 @@
# "The Forge is hot. The Warding is strong. The Coven is ready." # "The Forge is hot. The Warding is strong. The Coven is ready."
BINARY = sorcery BINARY = sorcery
VERSION ?= 1.0.0 VERSION ?= 1.1.0
BUILD_DIR := build BUILD_DIR := build
GO_FLAGS := -ldflags="-s -w -X main.Version=$(VERSION)" GO_FLAGS := -ldflags="-s -w -X main.Version=$(VERSION)"
CGO ?= 0 CGO ?= 0
@ -113,7 +113,7 @@ ebpf:
# Source Mage chroot resurrection helpers # Source Mage chroot resurrection helpers
# ============================================================================= # =============================================================================
# Thin wrappers around scripts/smgl-getting-started.sh so you can drive the # Thin wrappers around scripts/smgl-getting-started.sh so you can drive the
# whole 8-phase pipeline from make. See docs/GETTING_STARTED_SMGL_CHROOT.md # whole 9-phase pipeline from make. See docs/GETTING_STARTED_SMGL_CHROOT.md
# for the full walkthrough. # for the full walkthrough.
# #
# Example: # Example:

View File

@ -40,7 +40,7 @@ allow traffic on the sorcery-go (8080) and Fester (8181) ports.
```bash ```bash
# 1. Verify network connectivity to the new Master # 1. Verify network connectivity to the new Master
sorcery coven pulse sorcery-go coven pulse
# 2. If workers were using a Fester controller, update its URL # 2. If workers were using a Fester controller, update its URL
# (set in /etc/sorcery-go/config.yaml or SORCERY_GO_FESTER_URL env var) # (set in /etc/sorcery-go/config.yaml or SORCERY_GO_FESTER_URL env var)
@ -56,11 +56,11 @@ corrupted during the failure:
```bash ```bash
# Deep Gaze — recompute the Merkle root of every file in the Tomb and # Deep Gaze — recompute the Merkle root of every file in the Tomb and
# compare against the restored Tablet entries # compare against the restored Tablet entries
sorcery tomb verify --all sorcery-go tomb verify --all
# Ghost Build — pick a core tool (like busybox) and re-forge it in a # Ghost Build — pick a core tool (like busybox) and re-forge it in a
# temporary sandbox. The new binary must match the restored Essence. # temporary sandbox. The new binary must match the restored Essence.
sorcery cast busybox --static --default sorcery-go cast busybox --static --default
``` ```
## IV. Restoring the Sanctums (LXC Containers) ## IV. Restoring the Sanctums (LXC Containers)
@ -71,7 +71,7 @@ work for containers — the links are broken. Run the Re-Hydration Ritual:
```bash ```bash
# Reads the Tablet to see which Essences belong in which containers, # Reads the Tablet to see which Essences belong in which containers,
# then re-links them from the Tomb # then re-links them from the Tomb
sorcery reanimate --all-containers sorcery-go reanimate --all-containers
``` ```
## V. Backup Strategy ## V. Backup Strategy
@ -86,9 +86,9 @@ sorcery reanimate --all-containers
After reconstruction, verify each layer: After reconstruction, verify each layer:
- [ ] `sorcery coven pulse` — every node green - [ ] `sorcery-go coven pulse` — every node green
- [ ] `sorcery tomb verify --all` — zero Merkle mismatches - [ ] `sorcery-go tomb verify --all` — zero Merkle mismatches
- [ ] `sorcery ward status` — eBPF Tomb Guard active - [ ] `sorcery-go ward status` — eBPF Tomb Guard active
- [ ] `sorcery legal audit` — zero license violations - [ ] `sorcery-go legal audit` — zero license violations
- [ ] `gaze whereis /usr/bin/bash` — owned by `coreutils` - [ ] `sorcery-go gaze whereis /usr/bin/bash` — owned by `coreutils`
- [ ] Cast a smoke-test spell: `sorcery cast busybox --static --default` - [ ] Cast a smoke-test spell: `sorcery-go cast busybox --static --default`

View File

@ -86,7 +86,7 @@ x86_64 binaries into an AArch64 container:
{ {
"arch": "aarch64", "arch": "aarch64",
"instruction_set": "ARMv8-A", "instruction_set": "ARMv8-A",
"merkle_root": "sha256:9f4e2a8b..." "merkle_root": "9f4e2a8b..."
} }
``` ```
@ -97,7 +97,7 @@ expected to be PGP-signed by the High Mage:
```bash ```bash
gpg --detach-sign -a grimoire/libs/openssl/DETAILS gpg --detach-sign -a grimoire/libs/openssl/DETAILS
sorcery cast openssl --verify-pgp sorcery-go cast openssl --verify-pgp
``` ```
The Tablet records who signed each spell — the WebUI shows a "Trusted Source" The Tablet records who signed each spell — the WebUI shows a "Trusted Source"

View File

@ -35,7 +35,7 @@ to boot.
--- ---
## The 8-phase pipeline ## The 9-phase pipeline
| Phase | What | Where | Script subcommand | | Phase | What | Where | Script subcommand |
|-------|------|-------|-------------------| |-------|------|-------|-------------------|

View File

@ -13,11 +13,11 @@ sudo make caps
| Ritual | Command | Effect | | Ritual | Command | Effect |
|----------|--------------------------------------|-------------------------------------| |----------|--------------------------------------|-------------------------------------|
| Cast | `sorcery cast <spell>` | Forge an Essence in the Cauldron | | Cast | `sorcery-go cast <spell>` | Forge an Essence in the Cauldron |
| Reanimate| `sorcery reanimate <ess> --sanctum <id>` | Deploy an Essence from the Tomb | | Reanimate| `sorcery-go reanimate <ess> --sanctum <id>` | Deploy an Essence from the Tomb |
| Gaze | `sorcery gaze install <spell>` | Inspect every file owned by an Essence | | Gaze | `sorcery-go gaze install <spell>` | Inspect every file owned by an Essence |
| Ward | `sorcery ward status` | Check firewall and alarm status | | Ward | `sorcery-go ward status` | Check firewall and alarm status |
| Banish | `sorcery dispel <spell>` | Surgical removal from a Sanctum | | Banish | `sorcery-go dispel <spell>` | Surgical removal from a Sanctum |
## 3. Cast Flags ## 3. Cast Flags
@ -32,10 +32,10 @@ sudo make caps
## 4. Coven Management ## 4. Coven Management
```bash ```bash
sorcery coven join <master-ip> # Add a new Sanctum to the Coven sorcery-go coven join <master-ip> # Add a new Sanctum to the Coven
sorcery coven pulse # Live heartbeat of every node sorcery-go coven pulse # Live heartbeat of every node
sorcery coven drain <node-id> # Migrate builds off a node for maintenance sorcery-go coven drain <node-id> # Migrate builds off a node for maintenance
sorcery coven list # Show all known Sanctums sorcery-go coven list # Show all known Sanctums
``` ```
Distributed build scheduling is delegated to Fester when a cluster is active. Distributed build scheduling is delegated to Fester when a cluster is active.
@ -43,16 +43,16 @@ Distributed build scheduling is delegated to Fester when a cluster is active.
## 5. Tomb (Binary Storage) ## 5. Tomb (Binary Storage)
```bash ```bash
sorcery tomb list # Show every Essence in the Tomb sorcery-go tomb list # Show every Essence in the Tomb
sorcery tomb verify --all # Warding bit-rot check sorcery-go tomb verify --all # Warding bit-rot check
sorcery tomb purge # Garbage-collect unreferenced blobs sorcery-go tomb purge # Garbage-collect unreferenced blobs
``` ```
## 6. Warding (Security) ## 6. Warding (Security)
```bash ```bash
sorcery ward status # Active alarms + quarantined nodes sorcery-go ward status # Active alarms + quarantined nodes
sorcery ward banish <node-id> # Freeze a Sanctum and log the banishment sorcery-go ward banish <node-id> # Freeze a Sanctum and log the banishment
``` ```
Transport security is handled at the network firewall layer (OPNsense / IPFire). Transport security is handled at the network firewall layer (OPNsense / IPFire).
@ -61,10 +61,10 @@ There are no certificates or key material to manage.
## 7. Legal Sentinel ## 7. Legal Sentinel
```bash ```bash
sorcery legal audit # Scan fleet for license violations sorcery-go legal audit # Scan fleet for license violations
sorcery legal sbom <essence_id> # CycloneDX SBOM on stdout sorcery-go legal sbom <essence_id> # CycloneDX SBOM on stdout
sorcery legal set-posture lawless # strict_copyleft | corporate_lite | lawless sorcery-go legal set-posture lawless # strict_copyleft | corporate_lite | lawless
sorcery legal credits --out ./DIR # Collect every COPYING / LICENSE file sorcery-go legal credits --out ./DIR # Collect every COPYING / LICENSE file
``` ```
## 8. Coven Mirror (WebUI) ## 8. Coven Mirror (WebUI)
@ -80,7 +80,7 @@ Tabs: Grimoire, Tomb, Pulse, Sanctum, Portable Bin, Compliance.
```bash ```bash
# Forge a static, portable version of a tool # Forge a static, portable version of a tool
sorcery cast procps --static --target aarch64 --essence-out ./portable-top.ess sorcery-go cast procps --static --target aarch64 --essence-out ./portable-top.ess
# Generate the curated Emergency Kit # Generate the curated Emergency Kit
cauldron emergency-kit --bundle ./emergency.svb cauldron emergency-kit --bundle ./emergency.svb

View File

@ -61,11 +61,11 @@ Now the Essence must be brought to life inside a Sanctum.
When a ritual fails or a node acts strangely, use the Gaze to see the truth: When a ritual fails or a node acts strangely, use the Gaze to see the truth:
```bash ```bash
gaze install <spell> # "Show me every file this spell owns." sorcery-go gaze install <spell> # "Show me every file this spell owns."
gaze tablet <spell> # "Show me exactly how this was configured." sorcery-go gaze tablet <spell> # "Show me exactly how this was configured."
gaze essence <hash> # "Which spell + flags produced this blob?" sorcery-go gaze essence <hash> # "Which spell + flags produced this blob?"
gaze whereis /usr/bin/wget # "Who owns this file?" sorcery-go gaze whereis /usr/bin/wget # "Who owns this file?"
sorcery ward status # "Is the Warding active? Any tainted Essences?" sorcery-go ward status # "Is the Warding active? Any tainted Essences?"
``` ```
## Vocabulary Cross-Reference ## Vocabulary Cross-Reference

View File

@ -68,8 +68,8 @@ the appropriate rules.
"operator": { "operator": {
"type": "list", "type": "list",
"data": [ "data": [
{ "type": "simple", "operand": "process", "data": "/usr/local/bin/sorcery-go" }, { "type": "simple", "operand": "process", "data": "/usr/local/sbin/sorcery-go" },
{ "type": "regexp", "operand": "dest_host", "data": "registry\\.local" } { "type": "regexp", "operand": "dest_host", "data": "registry\\.example\\.local" }
] ]
} }
} }

View File

@ -1,35 +1,28 @@
#include <tunables/global> # DEPRECATED — Retained for reference only.
# AppArmor profile for an LXC container running inside the Coven.
# File: /etc/apparmor.d/lxc/lxc-sorcery-essence
# #
# This profile is the "Immutable Vault". It allows containers to READ # The Sorcery-Go security model uses eBPF LSM programs (Tomb Guard +
# from the Tomb (to run their software) but strictly forbids WRITING, # cgroup filters) for in-kernel enforcement. This AppArmor profile is
# LINKING, or DELETING. A compromised container cannot poison the # superseded by pkg/warding/ebpf/c/tomb_guard.bpf.c and is no longer
# global Essence store. # loaded by the Warding. See docs/SECURITY.md for the current model.
#include <tunables/global>
profile lxc-sorcery-essence flags=(attach_disconnected, mediate_deleted) { profile lxc-sorcery-essence flags=(attach_disconnected, mediate_deleted) {
#include <abstractions/lxc/container-default> #include <abstractions/lxc/container-default>
# 1. Global Essence Store Access (READ ONLY) # 1. Global Essence Store Access (READ ONLY)
# Prevents any container from modifying the master hashes /var/lib/sorcery-go/tomb/ r,
/var/lib/sorcery/tomb/ r, /var/lib/sorcery-go/tomb/** r,
/var/lib/sorcery/tomb/** r,
/var/lib/sorcery/essences/ r,
/var/lib/sorcery/essences/** r,
# 2. Deny all write/append/link/rename/delete attempts # 2. Deny all write/append/link/rename/delete attempts
deny /var/lib/sorcery/tomb/** wklx, deny /var/lib/sorcery-go/tomb/** wklx,
deny /var/lib/sorcery/essences/** wklx,
# 3. Allow BoltDB read-only access for local manifest verification # 3. Allow BoltDB read-only access for local manifest verification
/var/lib/sorcery/state/state.db r, /var/lib/sorcery-go/state/state.db r,
# 4. Allow the Sorcery-Go engine binary to execute # 4. Allow the Sorcery-Go engine binary to execute
/usr/local/bin/sorcery rix, /usr/local/sbin/sorcery-go rix,
/usr/local/bin/warding rix,
/usr/local/bin/gaze rix,
# 5. Allow writes only inside the container's own rootfs # 5. Allow writes only inside the container's own rootfs
/var/lib/lxc/*/rootfs/** rwkl, /var/lib/lxc/*/rootfs/** rwkl,
} }

View File

@ -11,12 +11,12 @@
{ {
"type": "simple", "type": "simple",
"operand": "process", "operand": "process",
"data": "/usr/local/bin/sorcery-go" "data": "/usr/local/sbin/sorcery-go"
}, },
{ {
"type": "regexp", "type": "regexp",
"operand": "dest_host", "operand": "dest_host",
"data": "registry\\.local" "data": "registry\\.example\\.local"
}, },
{ {
"type": "simple", "type": "simple",

View File

@ -3,21 +3,22 @@
# File: /etc/init.d/sorcery-go # File: /etc/init.d/sorcery-go
# #
# Install with: # Install with:
# sudo cp sorcery-go /etc/init.d/ # sudo cp sorcery-go.openrc /etc/init.d/sorcery-go
# sudo chmod +x /etc/init.d/sorcery-go # sudo chmod +x /etc/init.d/sorcery-go
# sudo rc-update add sorcery-go default # sudo rc-update add sorcery-go default
# sudo rc-service sorcery-go start # sudo rc-service sorcery-go start
description="Sorcery-Go Sovereign Coven Engine" description="Sorcery-Go Sovereign Coven Engine"
command="/usr/local/bin/sorcery" command="/usr/local/sbin/sorcery-go"
command_args="web --port 8080 --cockpit-integration" command_args="web --port 8080 --cockpit-integration"
command_background="yes" command_background="yes"
pidfile="/run/${RC_SVCNAME}.pid" pidfile="/run/${RC_SVCNAME}.pid"
output_log="/var/log/sorcery/sorcery-go.log" output_log="/var/log/sorcery-go/sorcery-go.log"
error_log="/var/log/sorcery/sorcery-go.err" error_log="/var/log/sorcery-go/sorcery-go.err"
# OpenRC capability management — grant the powers the Warding needs # OpenRC capability management — grant the powers the Warding needs
capabilities="cap_sys_admin,cap_chown,cap_dac_override+ep" # CAP_BPF is required for loading eBPF Tomb Guard programs
capabilities="cap_sys_admin,cap_chown,cap_dac_override,cap_bpf+ep"
depend() { depend() {
need localmount need localmount
@ -25,7 +26,7 @@ depend() {
} }
start_pre() { start_pre() {
# Ensure the BoltDB directory exists before starting # Ensure the state and log directories exist before starting
checkpath --directory --owner root:root --mode 0700 /var/lib/sorcery/state checkpath --directory --owner root:root --mode 0700 /var/lib/sorcery-go/state
checkpath --directory --owner root:root --mode 0755 /var/log/sorcery checkpath --directory --owner root:root --mode 0755 /var/log/sorcery-go
} }

92
pkg/tomb/reanimate.go Executable file
View File

@ -0,0 +1,92 @@
// Reanimation: binding an Essence from the Tomb into a Sanctum.
//
// In the Coven mythos, Reanimation is the act of bringing a sealed binary
// back to life inside a Sanctum. A Sanctum can be an LXC container, Podman
// container, Firecracker microVM, or bare-metal chroot.
//
// Technically this means creating reflinks (preferred — zero-copy on
// btrfs/xfs) or hardlinks from the Tomb's blob store into the Sanctum's
// rootfs. If neither is possible (cross-filesystem), we fall back to a
// plain copy.
package tomb
import (
"fmt"
"io"
"os"
"path/filepath"
"syscall"
)
// Reanimate links every file of `essenceID` into `sanctumRoot` so the
// Sanctum can execute them. The caller MUST have already called
// VerifyRoot() (or VerifyBlobs()) on the essence — Reanimate does not
// re-check integrity for performance reasons.
func (t *Tomb) Reanimate(essenceID, sanctumRoot string) error {
s, err := t.GetSarcophagus(essenceID)
if err != nil {
return err
}
for path, hash := range s.Files {
src := t.blobPath(hash)
dst := filepath.Join(sanctumRoot, path)
if err := os.MkdirAll(filepath.Dir(dst), 0755); err != nil {
return fmt.Errorf("reanimate: mkdir %s: %w", filepath.Dir(dst), err)
}
if err := linkOrCopy(src, dst); err != nil {
return fmt.Errorf("reanimate: link %s: %w", dst, err)
}
}
return nil
}
// linkOrCopy tries reflink (btrfs/xfs FICLONE), then hardlink (same
// filesystem), then plain copy (cross-filesystem).
func linkOrCopy(src, dst string) error {
// 1. Reflink (zero-copy on btrfs/xfs).
if err := reflinkFile(src, dst); err == nil {
return nil
}
// 2. Hardlink (fast, same-filesystem only).
if err := os.Link(src, dst); err == nil {
return nil
}
// 3. Plain copy.
return copyFile(src, dst)
}
// reflinkFile wraps the FICLONE ioctl. Returns nil on success.
func reflinkFile(src, dst string) error {
srcF, err := os.Open(src)
if err != nil {
return err
}
defer srcF.Close()
dstF, err := os.Create(dst)
if err != nil {
return err
}
defer dstF.Close()
const FICLONE = 0x40049409
_, _, errno := syscall.Syscall(syscall.SYS_IOCTL,
dstF.Fd(), uintptr(FICLONE), srcF.Fd())
if errno != 0 {
return fmt.Errorf("FICLONE: %w", errno)
}
return nil
}
func copyFile(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return err
}
defer in.Close()
out, err := os.Create(dst)
if err != nil {
return err
}
defer out.Close()
_, err = io.Copy(out, in)
return err
}

302
pkg/tomb/storage.go Executable file
View File

@ -0,0 +1,302 @@
// Package tomb is the content-addressable storage (CAS) layer of the Coven.
//
// "In the Coven, we forge the living. In the Tomb, we preserve the eternal."
//
// A Sarcophagus is the on-disk record of one Essence — the binaries,
// libraries and metadata produced by one cast. The Epitaph is the metadata
// sidecar (JSON): name, version, y/n flags, toolchain id, Merkle root.
// Reanimation is the act of binding an Essence into a Sanctum.
// Technically this means creating reflinks (preferred — zero-copy on
// btrfs/xfs) or hardlinks from the Tomb's blob store into the Sanctum's
// rootfs. If neither is possible (cross-filesystem), we fall back to a
// plain copy.
//
// Works identically across all runtimes: LXC, Podman, Firecracker, baremetal.
//
// Files are stored by their SHA-256 hash under
// /var/lib/sorcery-go/tomb/blobs/<ab>/<hash> (sharded by first two hex
// chars) so a single directory never grows past ~65k entries — keeping
// `ls` fast on cold disks.
package tomb
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"log"
"os"
"path/filepath"
"sort"
"strings"
)
// Sarcophagus is the on-disk record of one Essence variant.
type Sarcophagus struct {
EssenceID string `json:"essence_id"` // Merkle root
SpellName string `json:"spell_name"`
Version string `json:"version"`
VariantHash string `json:"variant_hash"` // sha256(version + flags + arch + toolchain)
Arch string `json:"arch"`
Linkage string `json:"linkage"` // "dynamic" | "static" | "hermetic"
Config map[string]bool `json:"config"` // y/n answers
Files map[string]string `json:"files"` // path -> sha256
CreatedAt string `json:"created_at"`
Toolchain string `json:"toolchain"`
License string `json:"license"`
SignedBy string `json:"signed_by,omitempty"` // PGP key fingerprint
}
// Tomb is the storage manager.
type Tomb struct {
Root string // /var/lib/sorcery-go/tomb
}
// New returns a Tomb rooted at `root`.
func New(root string) *Tomb {
return &Tomb{Root: root}
}
// IngestBlob copies a file from `srcPath` into the Tomb's blob store,
// returning the SHA-256 hash. Content-addressed storage means duplicate
// blobs are harmlessly re-written with identical content.
func (t *Tomb) IngestBlob(srcPath string) (string, error) {
hash, err := HashFile(srcPath)
if err != nil {
return "", err
}
dst := t.blobPath(hash)
if err := os.MkdirAll(filepath.Dir(dst), 0700); err != nil {
return "", err
}
in, err := os.Open(srcPath)
if err != nil {
return "", err
}
defer in.Close()
// Use O_CREATE|O_TRUNC directly — no TOCTOU race from a prior Stat.
// Content-addressed means re-writing the same content is safe.
out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
return "", err
}
defer out.Close()
if _, err := io.Copy(out, in); err != nil {
return "", err
}
return hash, nil
}
// Store writes a Sarcophagus's epitaph to disk. Callers must have already
// ingested every blob via IngestBlob.
func (t *Tomb) Store(sarc *Sarcophagus) error {
if err := os.MkdirAll(filepath.Join(t.Root, "epitaphs"), 0700); err != nil {
return err
}
// Recompute the Merkle root from the file map — this is the canonical
// EssenceID. If the caller pre-set EssenceID we verify it matches.
computed := ComputeMerkleRoot(sarc.Files)
if sarc.EssenceID == "" {
sarc.EssenceID = computed
} else if sarc.EssenceID != computed {
return fmt.Errorf("tomb: essence_id mismatch (stored=%s, computed=%s) — refusing to write taint",
sarc.EssenceID, computed)
}
path := filepath.Join(t.Root, "epitaphs", sarc.EssenceID+".json")
data, err := json.MarshalIndent(sarc, "", " ")
if err != nil {
return err
}
return os.WriteFile(path, data, 0600)
}
// GetSarcophagus loads the epitaph for an Essence ID.
func (t *Tomb) GetSarcophagus(essenceID string) (*Sarcophagus, error) {
path := filepath.Join(t.Root, "epitaphs", essenceID+".json")
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("tomb: sarcophagus %s not found: %w", essenceID, err)
}
var s Sarcophagus
if err := json.Unmarshal(data, &s); err != nil {
return nil, err
}
return &s, nil
}
// VerifyRoot recomputes the Merkle root of a Sarcophagus from its file
// hashes and compares it against the stored EssenceID. Any mismatch
// means "Taint" — the Warding must reject Reanimation.
//
// Note: this only verifies the metadata. To verify the blobs themselves
// are not bit-rotted, use VerifyBlobs which re-hashes every blob on disk.
func (t *Tomb) VerifyRoot(essenceID string) error {
s, err := t.GetSarcophagus(essenceID)
if err != nil {
return err
}
computed := ComputeMerkleRoot(s.Files)
if computed != essenceID {
return fmt.Errorf("tomb: merkle mismatch for %s (expected %s, recomputed %s)",
s.SpellName, essenceID, computed)
}
return nil
}
// VerifyBlobs re-hashes every blob referenced by the Sarcophagus and
// confirms the bytes on disk still match. This is the slow bit-rot check
// run by `sorcery tomb verify --all`.
func (t *Tomb) VerifyBlobs(essenceID string) error {
s, err := t.GetSarcophagus(essenceID)
if err != nil {
return err
}
for path, expectedHash := range s.Files {
blobPath := t.blobPath(expectedHash)
actualHash, err := HashFile(blobPath)
if err != nil {
return fmt.Errorf("tomb: blob %s missing for %s: %w", expectedHash, path, err)
}
if actualHash != expectedHash {
return fmt.Errorf("tomb: blob %s bit-rotted (path %s)", expectedHash, path)
}
}
return nil
}
// List returns every Essence currently resting in the Tomb.
func (t *Tomb) List() ([]*Sarcophagus, error) {
dir := filepath.Join(t.Root, "epitaphs")
var out []*Sarcophagus
err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return nil
}
if !strings.HasSuffix(path, ".json") {
return nil
}
data, e := os.ReadFile(path)
if e != nil {
log.Printf("tomb: skipping unreadable file %s: %v", path, e)
return nil
}
var s Sarcophagus
if e := json.Unmarshal(data, &s); e == nil {
out = append(out, &s)
}
return nil
})
return out, err
}
// FindByVariant returns the EssenceID of the Sarcophagus whose VariantHash
// matches, or "" when not found. This is the cache-hit check in the Cast
// pipeline.
func (t *Tomb) FindByVariant(variantHash string) (string, error) {
all, err := t.List()
if err != nil {
return "", err
}
for _, s := range all {
if s.VariantHash == variantHash {
return s.EssenceID, nil
}
}
return "", nil
}
// FindBySpell returns the most recently created Sarcophagus for a spell
// (any variant). Used by `cauldron` to pick the latest essence for an ISO.
func (t *Tomb) FindBySpell(spell, arch string) (*Sarcophagus, error) {
all, err := t.List()
if err != nil {
return nil, err
}
var best *Sarcophagus
for _, s := range all {
if s.SpellName != spell {
continue
}
if arch != "" && s.Arch != arch {
continue
}
if best == nil || s.CreatedAt > best.CreatedAt {
best = s
}
}
if best == nil {
return nil, fmt.Errorf("tomb: no essence for %s on %s", spell, arch)
}
return best, nil
}
// Prune (Garbage Collection) walks the blob store and removes any hash
// that is not referenced by an active epitaph. Returns the bytes reclaimed.
func (t *Tomb) Prune() (int64, error) {
active := make(map[string]bool)
epitaphs, err := t.List()
if err != nil {
return 0, fmt.Errorf("tomb: prune: list epitaphs: %w", err)
}
for _, s := range epitaphs {
for _, h := range s.Files {
active[h] = true
}
}
var reclaimed int64
blobDir := filepath.Join(t.Root, "blobs")
err := filepath.Walk(blobDir, func(path string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return nil
}
name := info.Name()
if !active[name] {
reclaimed += info.Size()
_ = os.Remove(path)
}
return nil
})
return reclaimed, err
}
// blobPath returns /var/lib/sorcery-go/tomb/blobs/<ab>/<hash>.
func (t *Tomb) blobPath(hash string) string {
if len(hash) < 2 {
return filepath.Join(t.Root, "blobs", hash)
}
return filepath.Join(t.Root, "blobs", hash[:2], hash)
}
// HashFile computes the sha256 of a file on disk — used by the Committer
// and the Cast pipeline before storing a Sarcophagus.
func HashFile(path string) (string, error) {
f, err := os.Open(path)
if err != nil {
return "", err
}
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
// ComputeMerkleRoot concatenates every file path + hash (sorted by path)
// and hashes the result. This catches a single-bit flip anywhere in the
// file set. A real binary Merkle tree would let us localise the failure;
// for now we trade precision for simplicity and speed.
func ComputeMerkleRoot(files map[string]string) string {
keys := make([]string, 0, len(files))
for k := range files {
keys = append(keys, k)
}
sort.Strings(keys)
h := sha256.New()
for _, k := range keys {
h.Write([]byte(k))
h.Write([]byte(files[k]))
}
return hex.EncodeToString(h.Sum(nil))
}

102
pkg/tomb/storage_test.go Executable file
View File

@ -0,0 +1,102 @@
package tomb
import (
"os"
"path/filepath"
"testing"
)
func TestIngestStoreVerify(t *testing.T) {
tmp := t.TempDir()
tomb := New(tmp)
// Create a fake source file to ingest.
srcBlob := filepath.Join(tmp, "src.txt")
if err := os.WriteFile(srcBlob, []byte("hello world"), 0644); err != nil {
t.Fatal(err)
}
hash, err := tomb.IngestBlob(srcBlob)
if err != nil {
t.Fatalf("IngestBlob: %v", err)
}
if hash == "" {
t.Fatal("expected non-empty hash")
}
sarc := &Sarcophagus{
SpellName: "test",
Version: "1.0",
Arch: "x86_64",
Files: map[string]string{"/usr/bin/test": hash},
}
if err := tomb.Store(sarc); err != nil {
t.Fatalf("Store: %v", err)
}
if sarc.EssenceID == "" {
t.Fatal("Store should set EssenceID")
}
// VerifyRoot must pass.
if err := tomb.VerifyRoot(sarc.EssenceID); err != nil {
t.Fatalf("VerifyRoot: %v", err)
}
// VerifyBlobs must pass.
if err := tomb.VerifyBlobs(sarc.EssenceID); err != nil {
t.Fatalf("VerifyBlobs: %v", err)
}
// Tamper with the blob — VerifyBlobs must fail.
blobPath := tomb.blobPath(hash)
if err := os.WriteFile(blobPath, []byte("tampered"), 0644); err != nil {
t.Fatal(err)
}
if err := tomb.VerifyBlobs(sarc.EssenceID); err == nil {
t.Fatal("VerifyBlobs must detect tampering")
} else {
t.Logf("✓ correctly detected bit-rot: %v", err)
}
}
func TestDedup(t *testing.T) {
tmp := t.TempDir()
tomb := New(tmp)
// Two identical source files.
src1 := filepath.Join(tmp, "a.txt")
src2 := filepath.Join(tmp, "b.txt")
_ = os.WriteFile(src1, []byte("same content"), 0644)
_ = os.WriteFile(src2, []byte("same content"), 0644)
h1, _ := tomb.IngestBlob(src1)
h2, _ := tomb.IngestBlob(src2)
if h1 != h2 {
t.Fatalf("dedup failed: %s != %s", h1, h2)
}
t.Logf("✓ dedup hit: %s", h1)
}
func TestFindByVariant(t *testing.T) {
tmp := t.TempDir()
tomb := New(tmp)
sarc := &Sarcophagus{
VariantHash: "v-1",
SpellName: "demo",
Version: "1.0",
Files: map[string]string{"/usr/bin/demo": "h1"},
}
if err := tomb.Store(sarc); err != nil {
t.Fatal(err)
}
if id, err := tomb.FindByVariant("v-1"); err != nil {
t.Fatalf("FindByVariant: %v", err)
} else if id != sarc.EssenceID {
t.Fatalf("FindByVariant returned %s, want %s", id, sarc.EssenceID)
}
if id, err := tomb.FindByVariant("missing"); err != nil {
t.Fatalf("FindByVariant: %v", err)
} else if id != "" {
t.Fatalf("FindByVariant should return empty for unknown variant, got %s", id)
}
}

View File

@ -13,6 +13,7 @@ echo " Runtime: $RUNTIME"
# 1. Setup Directory Structure # 1. Setup Directory Structure
mkdir -p bin pkg grimoire tablet tomb docs manifests mkdir -p bin pkg grimoire tablet tomb docs manifests
mkdir -p /var/lib/sorcery-go/ebpf/maps mkdir -p /var/lib/sorcery-go/ebpf/maps
mkdir -p /var/log/sorcery-go /run/sorcery-go
# 2. Initialize the State Database # 2. Initialize the State Database
"$SORCERY" init --db ./tablet/state.db "$SORCERY" init --db ./tablet/state.db
@ -25,18 +26,18 @@ echo "→ Loading eBPF Tomb Guard..."
# 4. Start the Warding Monitor (background) # 4. Start the Warding Monitor (background)
if [ -x ./build/warding ]; then if [ -x ./build/warding ]; then
nohup ./build/warding watch > /var/log/sorcery/warding.log 2>&1 & nohup ./build/warding watch > /var/log/sorcery-go/warding.log 2>&1 &
echo $! > /run/warding.pid echo $! > /run/sorcery-go/warding.pid
echo "✓ Warding monitor started (PID $(cat /run/warding.pid))" echo "✓ Warding monitor started (PID $(cat /run/sorcery-go/warding.pid))"
fi fi
# 5. Launch the Coven Mirror WebUI # 5. Launch the Coven Mirror WebUI
# (foreground by default; use --background for systemd/OpenRC) # (foreground by default; use --background for systemd/OpenRC)
if [ "$1" = "--background" ]; then if [ "$1" = "--background" ]; then
nohup "$SORCERY" web --port 8080 --cockpit-integration \ nohup "$SORCERY" web --port 8080 --cockpit-integration \
> /var/log/sorcery/web.log 2>&1 & > /var/log/sorcery-go/web.log 2>&1 &
echo $! > /run/sorcery-web.pid echo $! > /run/sorcery-go/web.pid
echo "✓ Coven Mirror running in background (PID $(cat /run/sorcery-web.pid))" echo "✓ Coven Mirror running in background (PID $(cat /run/sorcery-go/web.pid))"
echo " Open: http://localhost:8080" echo " Open: http://localhost:8080"
else else
echo "✓ Launching Coven Mirror in foreground..." echo "✓ Launching Coven Mirror in foreground..."

View File

@ -65,7 +65,7 @@ case "$RUNTIME" in
mkdir -p "$SANCTUM_PATH" mkdir -p "$SANCTUM_PATH"
;; ;;
baremetal|*) baremetal|*)
SANCTUM_PATH="/opt/sorcery/sanctums/sanctum-alpha" SANCTUM_PATH="/var/lib/sorcery-go/sanctums/sanctum-alpha"
sudo mkdir -p "$SANCTUM_PATH" sudo mkdir -p "$SANCTUM_PATH"
;; ;;
esac esac

3
tablet/.gitkeep Executable file
View File

@ -0,0 +1,3 @@
# BoltDB state files (state.db, journal, manifests, tablet) are created
# here at runtime by `sorcery init`. This file is just a placeholder so the
# directory is preserved in version control.

4
tomb/.gitkeep Executable file
View File

@ -0,0 +1,4 @@
# Content-addressable blob store. The Cauldron writes Essence blobs here
# under blobs/<ab>/<hash>; epitaphs (JSON metadata sidecars) live under
# epitaphs/. Created at runtime by `sorcery init`. This file is just a
# placeholder so the directory is preserved in version control.