Sorcery-Go is a high-concurrency, source-based infrastructure management suite. It implements a spell format compatible with Source Mage GNU/Linux grimoires for convenience, but the two projects have separate maintainers, separate codebases, and separate governance.
This commit is contained in:
parent
3ea742f4b5
commit
b1e7133369
|
|
@ -4,7 +4,7 @@ All notable changes to Sorcery-Go are documented here. The format follows
|
|||
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project
|
||||
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
## [1.1.0] — 2026-07-12
|
||||
|
||||
### Added
|
||||
- **BTC 0.4.0 multi-arch cross-compilation support** (`pkg/toolchain/btc.go`)
|
||||
|
|
@ -32,12 +32,6 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|||
- `RetrieveArtifact` / `RetrieveArtifactToFile` — fetch cached artifacts
|
||||
- `Stats` — cache hit rate, utilization, artifact count
|
||||
- Auto-initialized when Fester integration is active
|
||||
- **BTC.sh toolchain integration** (`pkg/toolchain/btc.go`)
|
||||
- `Probe()` — detect BTC.sh golden images at /opt/BTC
|
||||
- `BuildEnv()` — return BTC-aware CC/CXX/CFLAGS/LDFLAGS
|
||||
- `VerifyStamp()` — read .note.BTC ELF note and xattr stamps
|
||||
- `StampBinary()` — apply forensic stamps (ELF note, xattr, debug symbols)
|
||||
- Pipe-delimited format matching BTC.sh and Fester
|
||||
- **Cauldron.BundleAndCache** — one-call bundle + CAS push method
|
||||
- **Scheduler.Dispatch now CAS-aware** — checks shared CAS before
|
||||
dispatching to Fester; skips builds for cached artifacts
|
||||
|
|
@ -54,7 +48,6 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|||
- **eBPF security** replaces AppArmor as primary enforcement mechanism
|
||||
- `tomb_guard.bpf.c` — LSM hooks for file/inode protection
|
||||
- `sorcery_filter.bpf.c` — cgroup filters for device/network whitelisting
|
||||
- AppArmor profiles retained as fallback
|
||||
- **License** changed from GPL-3.0 to AGPL-3.0-or-later
|
||||
- **Fester cluster integration** — full delegation of distributed build
|
||||
scheduling, node telemetry, and build dispatch to Fester
|
||||
|
|
|
|||
4
Makefile
4
Makefile
|
|
@ -2,7 +2,7 @@
|
|||
# "The Forge is hot. The Warding is strong. The Coven is ready."
|
||||
|
||||
BINARY = sorcery
|
||||
VERSION ?= 1.0.0
|
||||
VERSION ?= 1.1.0
|
||||
BUILD_DIR := build
|
||||
GO_FLAGS := -ldflags="-s -w -X main.Version=$(VERSION)"
|
||||
CGO ?= 0
|
||||
|
|
@ -113,7 +113,7 @@ ebpf:
|
|||
# Source Mage chroot resurrection helpers
|
||||
# =============================================================================
|
||||
# Thin wrappers around scripts/smgl-getting-started.sh so you can drive the
|
||||
# whole 8-phase pipeline from make. See docs/GETTING_STARTED_SMGL_CHROOT.md
|
||||
# whole 9-phase pipeline from make. See docs/GETTING_STARTED_SMGL_CHROOT.md
|
||||
# for the full walkthrough.
|
||||
#
|
||||
# Example:
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ allow traffic on the sorcery-go (8080) and Fester (8181) ports.
|
|||
|
||||
```bash
|
||||
# 1. Verify network connectivity to the new Master
|
||||
sorcery coven pulse
|
||||
sorcery-go coven pulse
|
||||
|
||||
# 2. If workers were using a Fester controller, update its URL
|
||||
# (set in /etc/sorcery-go/config.yaml or SORCERY_GO_FESTER_URL env var)
|
||||
|
|
@ -56,11 +56,11 @@ corrupted during the failure:
|
|||
```bash
|
||||
# Deep Gaze — recompute the Merkle root of every file in the Tomb and
|
||||
# compare against the restored Tablet entries
|
||||
sorcery tomb verify --all
|
||||
sorcery-go tomb verify --all
|
||||
|
||||
# Ghost Build — pick a core tool (like busybox) and re-forge it in a
|
||||
# temporary sandbox. The new binary must match the restored Essence.
|
||||
sorcery cast busybox --static --default
|
||||
sorcery-go cast busybox --static --default
|
||||
```
|
||||
|
||||
## IV. Restoring the Sanctums (LXC Containers)
|
||||
|
|
@ -71,7 +71,7 @@ work for containers — the links are broken. Run the Re-Hydration Ritual:
|
|||
```bash
|
||||
# Reads the Tablet to see which Essences belong in which containers,
|
||||
# then re-links them from the Tomb
|
||||
sorcery reanimate --all-containers
|
||||
sorcery-go reanimate --all-containers
|
||||
```
|
||||
|
||||
## V. Backup Strategy
|
||||
|
|
@ -86,9 +86,9 @@ sorcery reanimate --all-containers
|
|||
|
||||
After reconstruction, verify each layer:
|
||||
|
||||
- [ ] `sorcery coven pulse` — every node green
|
||||
- [ ] `sorcery tomb verify --all` — zero Merkle mismatches
|
||||
- [ ] `sorcery ward status` — eBPF Tomb Guard active
|
||||
- [ ] `sorcery legal audit` — zero license violations
|
||||
- [ ] `gaze whereis /usr/bin/bash` — owned by `coreutils`
|
||||
- [ ] Cast a smoke-test spell: `sorcery cast busybox --static --default`
|
||||
- [ ] `sorcery-go coven pulse` — every node green
|
||||
- [ ] `sorcery-go tomb verify --all` — zero Merkle mismatches
|
||||
- [ ] `sorcery-go ward status` — eBPF Tomb Guard active
|
||||
- [ ] `sorcery-go legal audit` — zero license violations
|
||||
- [ ] `sorcery-go gaze whereis /usr/bin/bash` — owned by `coreutils`
|
||||
- [ ] Cast a smoke-test spell: `sorcery-go cast busybox --static --default`
|
||||
|
|
@ -86,7 +86,7 @@ x86_64 binaries into an AArch64 container:
|
|||
{
|
||||
"arch": "aarch64",
|
||||
"instruction_set": "ARMv8-A",
|
||||
"merkle_root": "sha256:9f4e2a8b..."
|
||||
"merkle_root": "9f4e2a8b..."
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -97,7 +97,7 @@ expected to be PGP-signed by the High Mage:
|
|||
|
||||
```bash
|
||||
gpg --detach-sign -a grimoire/libs/openssl/DETAILS
|
||||
sorcery cast openssl --verify-pgp
|
||||
sorcery-go cast openssl --verify-pgp
|
||||
```
|
||||
|
||||
The Tablet records who signed each spell — the WebUI shows a "Trusted Source"
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ to boot.
|
|||
|
||||
---
|
||||
|
||||
## The 8-phase pipeline
|
||||
## The 9-phase pipeline
|
||||
|
||||
| Phase | What | Where | Script subcommand |
|
||||
|-------|------|-------|-------------------|
|
||||
|
|
|
|||
|
|
@ -13,11 +13,11 @@ sudo make caps
|
|||
|
||||
| Ritual | Command | Effect |
|
||||
|----------|--------------------------------------|-------------------------------------|
|
||||
| Cast | `sorcery cast <spell>` | Forge an Essence in the Cauldron |
|
||||
| Reanimate| `sorcery reanimate <ess> --sanctum <id>` | Deploy an Essence from the Tomb |
|
||||
| Gaze | `sorcery gaze install <spell>` | Inspect every file owned by an Essence |
|
||||
| Ward | `sorcery ward status` | Check firewall and alarm status |
|
||||
| Banish | `sorcery dispel <spell>` | Surgical removal from a Sanctum |
|
||||
| Cast | `sorcery-go cast <spell>` | Forge an Essence in the Cauldron |
|
||||
| Reanimate| `sorcery-go reanimate <ess> --sanctum <id>` | Deploy an Essence from the Tomb |
|
||||
| Gaze | `sorcery-go gaze install <spell>` | Inspect every file owned by an Essence |
|
||||
| Ward | `sorcery-go ward status` | Check firewall and alarm status |
|
||||
| Banish | `sorcery-go dispel <spell>` | Surgical removal from a Sanctum |
|
||||
|
||||
## 3. Cast Flags
|
||||
|
||||
|
|
@ -32,10 +32,10 @@ sudo make caps
|
|||
## 4. Coven Management
|
||||
|
||||
```bash
|
||||
sorcery coven join <master-ip> # Add a new Sanctum to the Coven
|
||||
sorcery coven pulse # Live heartbeat of every node
|
||||
sorcery coven drain <node-id> # Migrate builds off a node for maintenance
|
||||
sorcery coven list # Show all known Sanctums
|
||||
sorcery-go coven join <master-ip> # Add a new Sanctum to the Coven
|
||||
sorcery-go coven pulse # Live heartbeat of every node
|
||||
sorcery-go coven drain <node-id> # Migrate builds off a node for maintenance
|
||||
sorcery-go coven list # Show all known Sanctums
|
||||
```
|
||||
|
||||
Distributed build scheduling is delegated to Fester when a cluster is active.
|
||||
|
|
@ -43,16 +43,16 @@ Distributed build scheduling is delegated to Fester when a cluster is active.
|
|||
## 5. Tomb (Binary Storage)
|
||||
|
||||
```bash
|
||||
sorcery tomb list # Show every Essence in the Tomb
|
||||
sorcery tomb verify --all # Warding bit-rot check
|
||||
sorcery tomb purge # Garbage-collect unreferenced blobs
|
||||
sorcery-go tomb list # Show every Essence in the Tomb
|
||||
sorcery-go tomb verify --all # Warding bit-rot check
|
||||
sorcery-go tomb purge # Garbage-collect unreferenced blobs
|
||||
```
|
||||
|
||||
## 6. Warding (Security)
|
||||
|
||||
```bash
|
||||
sorcery ward status # Active alarms + quarantined nodes
|
||||
sorcery ward banish <node-id> # Freeze a Sanctum and log the banishment
|
||||
sorcery-go ward status # Active alarms + quarantined nodes
|
||||
sorcery-go ward banish <node-id> # Freeze a Sanctum and log the banishment
|
||||
```
|
||||
|
||||
Transport security is handled at the network firewall layer (OPNsense / IPFire).
|
||||
|
|
@ -61,10 +61,10 @@ There are no certificates or key material to manage.
|
|||
## 7. Legal Sentinel
|
||||
|
||||
```bash
|
||||
sorcery legal audit # Scan fleet for license violations
|
||||
sorcery legal sbom <essence_id> # CycloneDX SBOM on stdout
|
||||
sorcery legal set-posture lawless # strict_copyleft | corporate_lite | lawless
|
||||
sorcery legal credits --out ./DIR # Collect every COPYING / LICENSE file
|
||||
sorcery-go legal audit # Scan fleet for license violations
|
||||
sorcery-go legal sbom <essence_id> # CycloneDX SBOM on stdout
|
||||
sorcery-go legal set-posture lawless # strict_copyleft | corporate_lite | lawless
|
||||
sorcery-go legal credits --out ./DIR # Collect every COPYING / LICENSE file
|
||||
```
|
||||
|
||||
## 8. Coven Mirror (WebUI)
|
||||
|
|
@ -80,7 +80,7 @@ Tabs: Grimoire, Tomb, Pulse, Sanctum, Portable Bin, Compliance.
|
|||
|
||||
```bash
|
||||
# Forge a static, portable version of a tool
|
||||
sorcery cast procps --static --target aarch64 --essence-out ./portable-top.ess
|
||||
sorcery-go cast procps --static --target aarch64 --essence-out ./portable-top.ess
|
||||
|
||||
# Generate the curated Emergency Kit
|
||||
cauldron emergency-kit --bundle ./emergency.svb
|
||||
|
|
|
|||
|
|
@ -61,11 +61,11 @@ Now the Essence must be brought to life inside a Sanctum.
|
|||
When a ritual fails or a node acts strangely, use the Gaze to see the truth:
|
||||
|
||||
```bash
|
||||
gaze install <spell> # "Show me every file this spell owns."
|
||||
gaze tablet <spell> # "Show me exactly how this was configured."
|
||||
gaze essence <hash> # "Which spell + flags produced this blob?"
|
||||
gaze whereis /usr/bin/wget # "Who owns this file?"
|
||||
sorcery ward status # "Is the Warding active? Any tainted Essences?"
|
||||
sorcery-go gaze install <spell> # "Show me every file this spell owns."
|
||||
sorcery-go gaze tablet <spell> # "Show me exactly how this was configured."
|
||||
sorcery-go gaze essence <hash> # "Which spell + flags produced this blob?"
|
||||
sorcery-go gaze whereis /usr/bin/wget # "Who owns this file?"
|
||||
sorcery-go ward status # "Is the Warding active? Any tainted Essences?"
|
||||
```
|
||||
|
||||
## Vocabulary Cross-Reference
|
||||
|
|
|
|||
|
|
@ -68,8 +68,8 @@ the appropriate rules.
|
|||
"operator": {
|
||||
"type": "list",
|
||||
"data": [
|
||||
{ "type": "simple", "operand": "process", "data": "/usr/local/bin/sorcery-go" },
|
||||
{ "type": "regexp", "operand": "dest_host", "data": "registry\\.local" }
|
||||
{ "type": "simple", "operand": "process", "data": "/usr/local/sbin/sorcery-go" },
|
||||
{ "type": "regexp", "operand": "dest_host", "data": "registry\\.example\\.local" }
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,35 +1,28 @@
|
|||
#include <tunables/global>
|
||||
|
||||
# AppArmor profile for an LXC container running inside the Coven.
|
||||
# File: /etc/apparmor.d/lxc/lxc-sorcery-essence
|
||||
# DEPRECATED — Retained for reference only.
|
||||
#
|
||||
# This profile is the "Immutable Vault". It allows containers to READ
|
||||
# from the Tomb (to run their software) but strictly forbids WRITING,
|
||||
# LINKING, or DELETING. A compromised container cannot poison the
|
||||
# global Essence store.
|
||||
# The Sorcery-Go security model uses eBPF LSM programs (Tomb Guard +
|
||||
# cgroup filters) for in-kernel enforcement. This AppArmor profile is
|
||||
# superseded by pkg/warding/ebpf/c/tomb_guard.bpf.c and is no longer
|
||||
# loaded by the Warding. See docs/SECURITY.md for the current model.
|
||||
|
||||
#include <tunables/global>
|
||||
|
||||
profile lxc-sorcery-essence flags=(attach_disconnected, mediate_deleted) {
|
||||
#include <abstractions/lxc/container-default>
|
||||
|
||||
# 1. Global Essence Store Access (READ ONLY)
|
||||
# Prevents any container from modifying the master hashes
|
||||
/var/lib/sorcery/tomb/ r,
|
||||
/var/lib/sorcery/tomb/** r,
|
||||
/var/lib/sorcery/essences/ r,
|
||||
/var/lib/sorcery/essences/** r,
|
||||
/var/lib/sorcery-go/tomb/ r,
|
||||
/var/lib/sorcery-go/tomb/** r,
|
||||
|
||||
# 2. Deny all write/append/link/rename/delete attempts
|
||||
deny /var/lib/sorcery/tomb/** wklx,
|
||||
deny /var/lib/sorcery/essences/** wklx,
|
||||
deny /var/lib/sorcery-go/tomb/** wklx,
|
||||
|
||||
# 3. Allow BoltDB read-only access for local manifest verification
|
||||
/var/lib/sorcery/state/state.db r,
|
||||
/var/lib/sorcery-go/state/state.db r,
|
||||
|
||||
# 4. Allow the Sorcery-Go engine binary to execute
|
||||
/usr/local/bin/sorcery rix,
|
||||
/usr/local/bin/warding rix,
|
||||
/usr/local/bin/gaze rix,
|
||||
/usr/local/sbin/sorcery-go rix,
|
||||
|
||||
# 5. Allow writes only inside the container's own rootfs
|
||||
/var/lib/lxc/*/rootfs/** rwkl,
|
||||
}
|
||||
}
|
||||
|
|
@ -11,12 +11,12 @@
|
|||
{
|
||||
"type": "simple",
|
||||
"operand": "process",
|
||||
"data": "/usr/local/bin/sorcery-go"
|
||||
"data": "/usr/local/sbin/sorcery-go"
|
||||
},
|
||||
{
|
||||
"type": "regexp",
|
||||
"operand": "dest_host",
|
||||
"data": "registry\\.local"
|
||||
"data": "registry\\.example\\.local"
|
||||
},
|
||||
{
|
||||
"type": "simple",
|
||||
|
|
|
|||
|
|
@ -3,21 +3,22 @@
|
|||
# File: /etc/init.d/sorcery-go
|
||||
#
|
||||
# Install with:
|
||||
# sudo cp sorcery-go /etc/init.d/
|
||||
# sudo cp sorcery-go.openrc /etc/init.d/sorcery-go
|
||||
# sudo chmod +x /etc/init.d/sorcery-go
|
||||
# sudo rc-update add sorcery-go default
|
||||
# sudo rc-service sorcery-go start
|
||||
|
||||
description="Sorcery-Go Sovereign Coven Engine"
|
||||
command="/usr/local/bin/sorcery"
|
||||
command="/usr/local/sbin/sorcery-go"
|
||||
command_args="web --port 8080 --cockpit-integration"
|
||||
command_background="yes"
|
||||
pidfile="/run/${RC_SVCNAME}.pid"
|
||||
output_log="/var/log/sorcery/sorcery-go.log"
|
||||
error_log="/var/log/sorcery/sorcery-go.err"
|
||||
output_log="/var/log/sorcery-go/sorcery-go.log"
|
||||
error_log="/var/log/sorcery-go/sorcery-go.err"
|
||||
|
||||
# OpenRC capability management — grant the powers the Warding needs
|
||||
capabilities="cap_sys_admin,cap_chown,cap_dac_override+ep"
|
||||
# CAP_BPF is required for loading eBPF Tomb Guard programs
|
||||
capabilities="cap_sys_admin,cap_chown,cap_dac_override,cap_bpf+ep"
|
||||
|
||||
depend() {
|
||||
need localmount
|
||||
|
|
@ -25,7 +26,7 @@ depend() {
|
|||
}
|
||||
|
||||
start_pre() {
|
||||
# Ensure the BoltDB directory exists before starting
|
||||
checkpath --directory --owner root:root --mode 0700 /var/lib/sorcery/state
|
||||
checkpath --directory --owner root:root --mode 0755 /var/log/sorcery
|
||||
}
|
||||
# Ensure the state and log directories exist before starting
|
||||
checkpath --directory --owner root:root --mode 0700 /var/lib/sorcery-go/state
|
||||
checkpath --directory --owner root:root --mode 0755 /var/log/sorcery-go
|
||||
}
|
||||
|
|
@ -0,0 +1,92 @@
|
|||
// Reanimation: binding an Essence from the Tomb into a Sanctum.
|
||||
//
|
||||
// In the Coven mythos, Reanimation is the act of bringing a sealed binary
|
||||
// back to life inside a Sanctum. A Sanctum can be an LXC container, Podman
|
||||
// container, Firecracker microVM, or bare-metal chroot.
|
||||
//
|
||||
// Technically this means creating reflinks (preferred — zero-copy on
|
||||
// btrfs/xfs) or hardlinks from the Tomb's blob store into the Sanctum's
|
||||
// rootfs. If neither is possible (cross-filesystem), we fall back to a
|
||||
// plain copy.
|
||||
package tomb
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// Reanimate links every file of `essenceID` into `sanctumRoot` so the
|
||||
// Sanctum can execute them. The caller MUST have already called
|
||||
// VerifyRoot() (or VerifyBlobs()) on the essence — Reanimate does not
|
||||
// re-check integrity for performance reasons.
|
||||
func (t *Tomb) Reanimate(essenceID, sanctumRoot string) error {
|
||||
s, err := t.GetSarcophagus(essenceID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for path, hash := range s.Files {
|
||||
src := t.blobPath(hash)
|
||||
dst := filepath.Join(sanctumRoot, path)
|
||||
if err := os.MkdirAll(filepath.Dir(dst), 0755); err != nil {
|
||||
return fmt.Errorf("reanimate: mkdir %s: %w", filepath.Dir(dst), err)
|
||||
}
|
||||
if err := linkOrCopy(src, dst); err != nil {
|
||||
return fmt.Errorf("reanimate: link %s: %w", dst, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// linkOrCopy tries reflink (btrfs/xfs FICLONE), then hardlink (same
|
||||
// filesystem), then plain copy (cross-filesystem).
|
||||
func linkOrCopy(src, dst string) error {
|
||||
// 1. Reflink (zero-copy on btrfs/xfs).
|
||||
if err := reflinkFile(src, dst); err == nil {
|
||||
return nil
|
||||
}
|
||||
// 2. Hardlink (fast, same-filesystem only).
|
||||
if err := os.Link(src, dst); err == nil {
|
||||
return nil
|
||||
}
|
||||
// 3. Plain copy.
|
||||
return copyFile(src, dst)
|
||||
}
|
||||
|
||||
// reflinkFile wraps the FICLONE ioctl. Returns nil on success.
|
||||
func reflinkFile(src, dst string) error {
|
||||
srcF, err := os.Open(src)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer srcF.Close()
|
||||
dstF, err := os.Create(dst)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer dstF.Close()
|
||||
const FICLONE = 0x40049409
|
||||
_, _, errno := syscall.Syscall(syscall.SYS_IOCTL,
|
||||
dstF.Fd(), uintptr(FICLONE), srcF.Fd())
|
||||
if errno != 0 {
|
||||
return fmt.Errorf("FICLONE: %w", errno)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func copyFile(src, dst string) error {
|
||||
in, err := os.Open(src)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer in.Close()
|
||||
out, err := os.Create(dst)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer out.Close()
|
||||
_, err = io.Copy(out, in)
|
||||
return err
|
||||
}
|
||||
|
|
@ -0,0 +1,302 @@
|
|||
// Package tomb is the content-addressable storage (CAS) layer of the Coven.
|
||||
//
|
||||
// "In the Coven, we forge the living. In the Tomb, we preserve the eternal."
|
||||
//
|
||||
// A Sarcophagus is the on-disk record of one Essence — the binaries,
|
||||
// libraries and metadata produced by one cast. The Epitaph is the metadata
|
||||
// sidecar (JSON): name, version, y/n flags, toolchain id, Merkle root.
|
||||
// Reanimation is the act of binding an Essence into a Sanctum.
|
||||
// Technically this means creating reflinks (preferred — zero-copy on
|
||||
// btrfs/xfs) or hardlinks from the Tomb's blob store into the Sanctum's
|
||||
// rootfs. If neither is possible (cross-filesystem), we fall back to a
|
||||
// plain copy.
|
||||
//
|
||||
// Works identically across all runtimes: LXC, Podman, Firecracker, baremetal.
|
||||
//
|
||||
// Files are stored by their SHA-256 hash under
|
||||
// /var/lib/sorcery-go/tomb/blobs/<ab>/<hash> (sharded by first two hex
|
||||
// chars) so a single directory never grows past ~65k entries — keeping
|
||||
// `ls` fast on cold disks.
|
||||
package tomb
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Sarcophagus is the on-disk record of one Essence variant.
|
||||
type Sarcophagus struct {
|
||||
EssenceID string `json:"essence_id"` // Merkle root
|
||||
SpellName string `json:"spell_name"`
|
||||
Version string `json:"version"`
|
||||
VariantHash string `json:"variant_hash"` // sha256(version + flags + arch + toolchain)
|
||||
Arch string `json:"arch"`
|
||||
Linkage string `json:"linkage"` // "dynamic" | "static" | "hermetic"
|
||||
Config map[string]bool `json:"config"` // y/n answers
|
||||
Files map[string]string `json:"files"` // path -> sha256
|
||||
CreatedAt string `json:"created_at"`
|
||||
Toolchain string `json:"toolchain"`
|
||||
License string `json:"license"`
|
||||
SignedBy string `json:"signed_by,omitempty"` // PGP key fingerprint
|
||||
}
|
||||
|
||||
// Tomb is the storage manager.
|
||||
type Tomb struct {
|
||||
Root string // /var/lib/sorcery-go/tomb
|
||||
}
|
||||
|
||||
// New returns a Tomb rooted at `root`.
|
||||
func New(root string) *Tomb {
|
||||
return &Tomb{Root: root}
|
||||
}
|
||||
|
||||
// IngestBlob copies a file from `srcPath` into the Tomb's blob store,
|
||||
// returning the SHA-256 hash. Content-addressed storage means duplicate
|
||||
// blobs are harmlessly re-written with identical content.
|
||||
func (t *Tomb) IngestBlob(srcPath string) (string, error) {
|
||||
hash, err := HashFile(srcPath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
dst := t.blobPath(hash)
|
||||
if err := os.MkdirAll(filepath.Dir(dst), 0700); err != nil {
|
||||
return "", err
|
||||
}
|
||||
in, err := os.Open(srcPath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer in.Close()
|
||||
// Use O_CREATE|O_TRUNC directly — no TOCTOU race from a prior Stat.
|
||||
// Content-addressed means re-writing the same content is safe.
|
||||
out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer out.Close()
|
||||
if _, err := io.Copy(out, in); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hash, nil
|
||||
}
|
||||
|
||||
// Store writes a Sarcophagus's epitaph to disk. Callers must have already
|
||||
// ingested every blob via IngestBlob.
|
||||
func (t *Tomb) Store(sarc *Sarcophagus) error {
|
||||
if err := os.MkdirAll(filepath.Join(t.Root, "epitaphs"), 0700); err != nil {
|
||||
return err
|
||||
}
|
||||
// Recompute the Merkle root from the file map — this is the canonical
|
||||
// EssenceID. If the caller pre-set EssenceID we verify it matches.
|
||||
computed := ComputeMerkleRoot(sarc.Files)
|
||||
if sarc.EssenceID == "" {
|
||||
sarc.EssenceID = computed
|
||||
} else if sarc.EssenceID != computed {
|
||||
return fmt.Errorf("tomb: essence_id mismatch (stored=%s, computed=%s) — refusing to write taint",
|
||||
sarc.EssenceID, computed)
|
||||
}
|
||||
path := filepath.Join(t.Root, "epitaphs", sarc.EssenceID+".json")
|
||||
data, err := json.MarshalIndent(sarc, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.WriteFile(path, data, 0600)
|
||||
}
|
||||
|
||||
// GetSarcophagus loads the epitaph for an Essence ID.
|
||||
func (t *Tomb) GetSarcophagus(essenceID string) (*Sarcophagus, error) {
|
||||
path := filepath.Join(t.Root, "epitaphs", essenceID+".json")
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("tomb: sarcophagus %s not found: %w", essenceID, err)
|
||||
}
|
||||
var s Sarcophagus
|
||||
if err := json.Unmarshal(data, &s); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &s, nil
|
||||
}
|
||||
|
||||
// VerifyRoot recomputes the Merkle root of a Sarcophagus from its file
|
||||
// hashes and compares it against the stored EssenceID. Any mismatch
|
||||
// means "Taint" — the Warding must reject Reanimation.
|
||||
//
|
||||
// Note: this only verifies the metadata. To verify the blobs themselves
|
||||
// are not bit-rotted, use VerifyBlobs which re-hashes every blob on disk.
|
||||
func (t *Tomb) VerifyRoot(essenceID string) error {
|
||||
s, err := t.GetSarcophagus(essenceID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
computed := ComputeMerkleRoot(s.Files)
|
||||
if computed != essenceID {
|
||||
return fmt.Errorf("tomb: merkle mismatch for %s (expected %s, recomputed %s)",
|
||||
s.SpellName, essenceID, computed)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// VerifyBlobs re-hashes every blob referenced by the Sarcophagus and
|
||||
// confirms the bytes on disk still match. This is the slow bit-rot check
|
||||
// run by `sorcery tomb verify --all`.
|
||||
func (t *Tomb) VerifyBlobs(essenceID string) error {
|
||||
s, err := t.GetSarcophagus(essenceID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for path, expectedHash := range s.Files {
|
||||
blobPath := t.blobPath(expectedHash)
|
||||
actualHash, err := HashFile(blobPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("tomb: blob %s missing for %s: %w", expectedHash, path, err)
|
||||
}
|
||||
if actualHash != expectedHash {
|
||||
return fmt.Errorf("tomb: blob %s bit-rotted (path %s)", expectedHash, path)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// List returns every Essence currently resting in the Tomb.
|
||||
func (t *Tomb) List() ([]*Sarcophagus, error) {
|
||||
dir := filepath.Join(t.Root, "epitaphs")
|
||||
var out []*Sarcophagus
|
||||
err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil || info.IsDir() {
|
||||
return nil
|
||||
}
|
||||
if !strings.HasSuffix(path, ".json") {
|
||||
return nil
|
||||
}
|
||||
data, e := os.ReadFile(path)
|
||||
if e != nil {
|
||||
log.Printf("tomb: skipping unreadable file %s: %v", path, e)
|
||||
return nil
|
||||
}
|
||||
var s Sarcophagus
|
||||
if e := json.Unmarshal(data, &s); e == nil {
|
||||
out = append(out, &s)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return out, err
|
||||
}
|
||||
|
||||
// FindByVariant returns the EssenceID of the Sarcophagus whose VariantHash
|
||||
// matches, or "" when not found. This is the cache-hit check in the Cast
|
||||
// pipeline.
|
||||
func (t *Tomb) FindByVariant(variantHash string) (string, error) {
|
||||
all, err := t.List()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
for _, s := range all {
|
||||
if s.VariantHash == variantHash {
|
||||
return s.EssenceID, nil
|
||||
}
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// FindBySpell returns the most recently created Sarcophagus for a spell
|
||||
// (any variant). Used by `cauldron` to pick the latest essence for an ISO.
|
||||
func (t *Tomb) FindBySpell(spell, arch string) (*Sarcophagus, error) {
|
||||
all, err := t.List()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var best *Sarcophagus
|
||||
for _, s := range all {
|
||||
if s.SpellName != spell {
|
||||
continue
|
||||
}
|
||||
if arch != "" && s.Arch != arch {
|
||||
continue
|
||||
}
|
||||
if best == nil || s.CreatedAt > best.CreatedAt {
|
||||
best = s
|
||||
}
|
||||
}
|
||||
if best == nil {
|
||||
return nil, fmt.Errorf("tomb: no essence for %s on %s", spell, arch)
|
||||
}
|
||||
return best, nil
|
||||
}
|
||||
|
||||
// Prune (Garbage Collection) walks the blob store and removes any hash
|
||||
// that is not referenced by an active epitaph. Returns the bytes reclaimed.
|
||||
func (t *Tomb) Prune() (int64, error) {
|
||||
active := make(map[string]bool)
|
||||
epitaphs, err := t.List()
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("tomb: prune: list epitaphs: %w", err)
|
||||
}
|
||||
for _, s := range epitaphs {
|
||||
for _, h := range s.Files {
|
||||
active[h] = true
|
||||
}
|
||||
}
|
||||
var reclaimed int64
|
||||
blobDir := filepath.Join(t.Root, "blobs")
|
||||
err := filepath.Walk(blobDir, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil || info.IsDir() {
|
||||
return nil
|
||||
}
|
||||
name := info.Name()
|
||||
if !active[name] {
|
||||
reclaimed += info.Size()
|
||||
_ = os.Remove(path)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return reclaimed, err
|
||||
}
|
||||
|
||||
// blobPath returns /var/lib/sorcery-go/tomb/blobs/<ab>/<hash>.
|
||||
func (t *Tomb) blobPath(hash string) string {
|
||||
if len(hash) < 2 {
|
||||
return filepath.Join(t.Root, "blobs", hash)
|
||||
}
|
||||
return filepath.Join(t.Root, "blobs", hash[:2], hash)
|
||||
}
|
||||
|
||||
// HashFile computes the sha256 of a file on disk — used by the Committer
|
||||
// and the Cast pipeline before storing a Sarcophagus.
|
||||
func HashFile(path string) (string, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer f.Close()
|
||||
h := sha256.New()
|
||||
if _, err := io.Copy(h, f); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(h.Sum(nil)), nil
|
||||
}
|
||||
|
||||
// ComputeMerkleRoot concatenates every file path + hash (sorted by path)
|
||||
// and hashes the result. This catches a single-bit flip anywhere in the
|
||||
// file set. A real binary Merkle tree would let us localise the failure;
|
||||
// for now we trade precision for simplicity and speed.
|
||||
func ComputeMerkleRoot(files map[string]string) string {
|
||||
keys := make([]string, 0, len(files))
|
||||
for k := range files {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
h := sha256.New()
|
||||
for _, k := range keys {
|
||||
h.Write([]byte(k))
|
||||
h.Write([]byte(files[k]))
|
||||
}
|
||||
return hex.EncodeToString(h.Sum(nil))
|
||||
}
|
||||
|
|
@ -0,0 +1,102 @@
|
|||
package tomb
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIngestStoreVerify(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
tomb := New(tmp)
|
||||
|
||||
// Create a fake source file to ingest.
|
||||
srcBlob := filepath.Join(tmp, "src.txt")
|
||||
if err := os.WriteFile(srcBlob, []byte("hello world"), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
hash, err := tomb.IngestBlob(srcBlob)
|
||||
if err != nil {
|
||||
t.Fatalf("IngestBlob: %v", err)
|
||||
}
|
||||
if hash == "" {
|
||||
t.Fatal("expected non-empty hash")
|
||||
}
|
||||
|
||||
sarc := &Sarcophagus{
|
||||
SpellName: "test",
|
||||
Version: "1.0",
|
||||
Arch: "x86_64",
|
||||
Files: map[string]string{"/usr/bin/test": hash},
|
||||
}
|
||||
if err := tomb.Store(sarc); err != nil {
|
||||
t.Fatalf("Store: %v", err)
|
||||
}
|
||||
if sarc.EssenceID == "" {
|
||||
t.Fatal("Store should set EssenceID")
|
||||
}
|
||||
|
||||
// VerifyRoot must pass.
|
||||
if err := tomb.VerifyRoot(sarc.EssenceID); err != nil {
|
||||
t.Fatalf("VerifyRoot: %v", err)
|
||||
}
|
||||
|
||||
// VerifyBlobs must pass.
|
||||
if err := tomb.VerifyBlobs(sarc.EssenceID); err != nil {
|
||||
t.Fatalf("VerifyBlobs: %v", err)
|
||||
}
|
||||
|
||||
// Tamper with the blob — VerifyBlobs must fail.
|
||||
blobPath := tomb.blobPath(hash)
|
||||
if err := os.WriteFile(blobPath, []byte("tampered"), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := tomb.VerifyBlobs(sarc.EssenceID); err == nil {
|
||||
t.Fatal("VerifyBlobs must detect tampering")
|
||||
} else {
|
||||
t.Logf("✓ correctly detected bit-rot: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDedup(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
tomb := New(tmp)
|
||||
|
||||
// Two identical source files.
|
||||
src1 := filepath.Join(tmp, "a.txt")
|
||||
src2 := filepath.Join(tmp, "b.txt")
|
||||
_ = os.WriteFile(src1, []byte("same content"), 0644)
|
||||
_ = os.WriteFile(src2, []byte("same content"), 0644)
|
||||
|
||||
h1, _ := tomb.IngestBlob(src1)
|
||||
h2, _ := tomb.IngestBlob(src2)
|
||||
if h1 != h2 {
|
||||
t.Fatalf("dedup failed: %s != %s", h1, h2)
|
||||
}
|
||||
t.Logf("✓ dedup hit: %s", h1)
|
||||
}
|
||||
|
||||
func TestFindByVariant(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
tomb := New(tmp)
|
||||
sarc := &Sarcophagus{
|
||||
VariantHash: "v-1",
|
||||
SpellName: "demo",
|
||||
Version: "1.0",
|
||||
Files: map[string]string{"/usr/bin/demo": "h1"},
|
||||
}
|
||||
if err := tomb.Store(sarc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if id, err := tomb.FindByVariant("v-1"); err != nil {
|
||||
t.Fatalf("FindByVariant: %v", err)
|
||||
} else if id != sarc.EssenceID {
|
||||
t.Fatalf("FindByVariant returned %s, want %s", id, sarc.EssenceID)
|
||||
}
|
||||
if id, err := tomb.FindByVariant("missing"); err != nil {
|
||||
t.Fatalf("FindByVariant: %v", err)
|
||||
} else if id != "" {
|
||||
t.Fatalf("FindByVariant should return empty for unknown variant, got %s", id)
|
||||
}
|
||||
}
|
||||
|
|
@ -13,6 +13,7 @@ echo " Runtime: $RUNTIME"
|
|||
# 1. Setup Directory Structure
|
||||
mkdir -p bin pkg grimoire tablet tomb docs manifests
|
||||
mkdir -p /var/lib/sorcery-go/ebpf/maps
|
||||
mkdir -p /var/log/sorcery-go /run/sorcery-go
|
||||
|
||||
# 2. Initialize the State Database
|
||||
"$SORCERY" init --db ./tablet/state.db
|
||||
|
|
@ -25,18 +26,18 @@ echo "→ Loading eBPF Tomb Guard..."
|
|||
|
||||
# 4. Start the Warding Monitor (background)
|
||||
if [ -x ./build/warding ]; then
|
||||
nohup ./build/warding watch > /var/log/sorcery/warding.log 2>&1 &
|
||||
echo $! > /run/warding.pid
|
||||
echo "✓ Warding monitor started (PID $(cat /run/warding.pid))"
|
||||
nohup ./build/warding watch > /var/log/sorcery-go/warding.log 2>&1 &
|
||||
echo $! > /run/sorcery-go/warding.pid
|
||||
echo "✓ Warding monitor started (PID $(cat /run/sorcery-go/warding.pid))"
|
||||
fi
|
||||
|
||||
# 5. Launch the Coven Mirror WebUI
|
||||
# (foreground by default; use --background for systemd/OpenRC)
|
||||
if [ "$1" = "--background" ]; then
|
||||
nohup "$SORCERY" web --port 8080 --cockpit-integration \
|
||||
> /var/log/sorcery/web.log 2>&1 &
|
||||
echo $! > /run/sorcery-web.pid
|
||||
echo "✓ Coven Mirror running in background (PID $(cat /run/sorcery-web.pid))"
|
||||
> /var/log/sorcery-go/web.log 2>&1 &
|
||||
echo $! > /run/sorcery-go/web.pid
|
||||
echo "✓ Coven Mirror running in background (PID $(cat /run/sorcery-go/web.pid))"
|
||||
echo " Open: http://localhost:8080"
|
||||
else
|
||||
echo "✓ Launching Coven Mirror in foreground..."
|
||||
|
|
|
|||
|
|
@ -65,7 +65,7 @@ case "$RUNTIME" in
|
|||
mkdir -p "$SANCTUM_PATH"
|
||||
;;
|
||||
baremetal|*)
|
||||
SANCTUM_PATH="/opt/sorcery/sanctums/sanctum-alpha"
|
||||
SANCTUM_PATH="/var/lib/sorcery-go/sanctums/sanctum-alpha"
|
||||
sudo mkdir -p "$SANCTUM_PATH"
|
||||
;;
|
||||
esac
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
# BoltDB state files (state.db, journal, manifests, tablet) are created
|
||||
# here at runtime by `sorcery init`. This file is just a placeholder so the
|
||||
# directory is preserved in version control.
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
# Content-addressable blob store. The Cauldron writes Essence blobs here
|
||||
# under blobs/<ab>/<hash>; epitaphs (JSON metadata sidecars) live under
|
||||
# epitaphs/. Created at runtime by `sorcery init`. This file is just a
|
||||
# placeholder so the directory is preserved in version control.
|
||||
Loading…
Reference in New Issue