sorcery-go/docs/GETTING_STARTED_SMGL_CHROOT.md

# Getting Started with an existing Source Mage chroot

> *"Resurrect the ancient tarball. Strip the relics. Drop in the new engine."*

**Note:** Sorcery-Go is developed by dcos.net and is not affiliated with Source
Mage GNU/Linux or sourcemage.org. This guide explains how to use Sorcery-Go
with Source Mage chroots (which are produced by the Source Mage project). The
two projects are independent.

This guide walks you through bringing a frozen Source Mage chroot tarball
(the classic 0.62-11 release or a 0.63 test branch) into the modern era,
then dropping in `sorcery-go` as the new engine — coexisting with the
legacy Bash `sorcery` so you can migrate at your own pace.

The whole flow is automated by **`scripts/smgl-getting-started.sh`**. This
document is the long-form companion: it explains *why* each phase exists,
what to do when something breaks, and how the host-side and chroot-side
subcommands fit together.

---

## Why this guide exists

Source Mage hasn't seen an official stable ISO update in years. If you
boot a raw 0.62-11 tarball on modern hardware it will crash on two fronts:

1. **Ancient kernel panic** — no support for modern storage / NVMe controllers.
2. **GRUB 1 (Legacy)** — can't read GPT partition tables or modern ext4 metadata.

On top of that, the toolchain inside the tarball is frozen at ~2017-era
GCC/glibc, which can't compile modern spell recipes (GCC 15+, 6.x kernels,
LLVM 22). So we stage everything inside a chroot from a modern host, rip
out the broken pieces, and inject modern scaffolding before we ever try
to boot.

---

## The 8-phase pipeline

| Phase | What | Where | Script subcommand |
|-------|------|-------|-------------------|
| 1 | Extract + mount API filesystems | host | `extract`, `mount` |
| 2 | Purge GRUB 1, cast GRUB 2 | chroot | `chroot-purge-grub1`, `chroot-cast-grub2` |
| 3 | Strip old kernel, inject modern host-built kernel | host | `inject-kernel` |
| 4 | Rewrite fstab with UUID identifiers | host | `fix-fstab` |
| 5 | Inject modern sorcery engine (Bash) | host | `inject-sorcery` |
| 6 | Set conservative CFLAGS for the step-upgrade | chroot | (part of `chroot-stepupgrade`) |
| 7 | Cut over from dead stable grimoire to live test branch | chroot | `chroot-scribe-test` |
| 8 | Step-upgrade ladder: make → binutils → gcc → glibc | chroot | `chroot-stepupgrade` |
| 9 | Drop in sorcery-go and init | both | `inject-sorcery-go` (host) + `chroot-init-sorcery-go` (chroot) |

---

## Prerequisites

On your **modern host OS** (the machine doing the surgery):

- root (sudo) — required for mount, chroot, setcap
- `tar`, `wget`, `bash` ≥ 4
- A modern kernel build at `/usr/src/linux/arch/x86/boot/bzImage` (or wherever your custom 6.x kernel lives)
- The matching module tree at `/lib/modules/<version>`
- Go ≥ 1.21 (only if you want to build sorcery-go yourself; you can also use a pre-built binary)
- `setcap` (from `libcap`) — optional, for non-root OverlayFS

On the **chroot tarball**:

- `source-mage-x86_64-0.62-11.tar.xz` (or similar test-branch tarball)
- ~2 GB free disk for the extracted rootfs + state

---

## Walkthrough

### Phase 1 — Extract + mount (host)

```bash
# From the sorcery-go project root:
sudo ./scripts/smgl-getting-started.sh extract \
    ~/Downloads/source-mage-x86_64-0.62-11.tar.xz \
    /mnt/AI/sourcemage_root

sudo ./scripts/smgl-getting-started.sh mount /mnt/AI/sourcemage_root
```

This bind-mounts `/dev`, `/proc`, `/sys` into the chroot and copies your
host's `/etc/resolv.conf` so the chroot has working DNS. Don't skip the
resolv.conf copy — without it every `wget` inside the chroot fails with
"connection timed out" and you'll waste an hour debugging.

### Phase 5 — Inject the modern Bash sorcery engine (host)

Do this *before* entering the chroot so the modern `cast` / `scribe` /
`dispel` scripts are in place when you arrive:

```bash
sudo ./scripts/smgl-getting-started.sh inject-sorcery /mnt/AI/sourcemage_root
```

This downloads `https://sourcemage.org/codex/sorcery-stable.tar.bz2` and
overlays its `usr/` and `etc/sorcery/` into the chroot. If the download
fails (firewall, dead mirror), pass a local copy:

```bash
sudo ./scripts/smgl-getting-started.sh inject-sorcery \
    /mnt/AI/sourcemage_root ~/Downloads/sorcery-stable.tar.bz2
```

### Phase 3 — Inject a modern kernel (host)

You cannot compile a 6.x kernel with the 2017-era GCC inside the chroot —
it'll segfault. Build the kernel on your modern host first, then slide it
in:

```bash
sudo ./scripts/smgl-getting-started.sh inject-kernel \
    /mnt/AI/sourcemage_root \
    /usr/src/linux/arch/x86/boot/bzImage \
    /lib/modules/6.12.4-custom
```

The script purges `/boot/vmlinuz*`, `/boot/initrd*`, and `/lib/modules/*`
before copying the new files in, so there's no risk of the bootloader
finding a stale kernel.

### Drop in sorcery-go (host)

Now is the time to drop the new engine in — but **do not init it yet**.
The chroot's glibc is still ancient; `sorcery-go init` would index a
grimoire it can't actually cast against.

```bash
# Build it first if you haven't:
make build

sudo ./scripts/smgl-getting-started.sh inject-sorcery-go \
    /mnt/AI/sourcemage_root
```

This installs to `/usr/local/sbin/sorcery-go` (NOT `/usr/sbin/sorcery` —
the legacy Bash binary stays put). State goes to `/var/lib/sorcery-go/`,
completely separate from `/var/lib/sorcery/`.

### Phase 4 — Fix fstab (host)

```bash
sudo blkid /dev/sdX1   # find your root partition's UUID
sudo blkid /dev/sdX2   # and boot, if separate

sudo ./scripts/smgl-getting-started.sh fix-fstab \
    /mnt/AI/sourcemage_root \
    XXXX-XXXX-XXXX-XXXX \
    YYYY-YYYY
```

The original fstab is backed up to `etc/fstab.pre-sorcery-go.bak`.

### Enter the chroot

```bash
sudo ./scripts/smgl-getting-started.sh enter /mnt/AI/sourcemage_root
```

This `chroot`s in with a login shell and copies the script itself into
`/usr/local/sbin/smgl-getting-started.sh` so the `chroot-*` subcommands
work from inside.

### Phase 2 — Purge GRUB 1, cast GRUB 2 (chroot)

```bash
smgl-getting-started.sh chroot-purge-grub1
smgl-getting-started.sh chroot-cast-grub2
```

`chroot-cast-grub2` will likely **fail** at this point — the 2017-era
toolchain can't compile modern GRUB. That's fine; skip it. After Phase 8
completes you can run `grub-install` from your modern host instead:

```bash
# From the host, after the step-upgrade:
sudo grub-install --boot-directory=/mnt/AI/sourcemage_root/boot /dev/sdX
```

### Phase 7 — Cut over to the test grimoire (chroot)

```bash
smgl-getting-started.sh chroot-scribe-test
```

This drops the dead `stable` codex and anchors `scribe` to the live `test`
branch:

```
scribe remove stable
scribe add test from git://download.sourcemage.org/smgl/grimoire.git
```

The `test` branch is where the active development happens — modern spells
for GCC 15/16, LLVM 22, Firefox 151, and 6.x kernel configs land there
daily. The `stable` grimoire is a museum piece; don't waste time on it.

If `git://` is firewalled, the script falls back to `https://` automatically.

### Phase 8 — Step-upgrade the toolchain (chroot)

This is the most fragile phase. The script walks the ladder in the
correct order:

```bash
smgl-getting-started.sh chroot-stepupgrade
```

What it does, in order:

1. **Phase 6 (prep)** — writes conservative `OPTIMIZATION_FLAGS="-O2 -march=native"`
   to `/etc/sorcery/config` and disables experimental GCC flags. This
   prevents the ancient compiler from choking on modern syntax variations.

2. **8.1 `cast make`** — modern make first, so the rest of the ladder has
   a working build system.

3. **8.2 `cast binutils`** — modern assembler/linker so modern binary
   headers parse correctly.

4. **8.3 `cast gcc`** — an intermediate GCC that can parse modern syntax
   but can still be compiled by the ancient root compiler. If `cast gcc`
   tries to jump straight to GCC 15 and fails, edit the spell version
   downward (GCC 9 or 10 is a safe stepping stone).

5. **8.4 `cast glibc`** — the cutover. Once glibc builds, the runtime
   shifts under your feet: modern syscall wrappers (`statx`, `clone3`)
   are now bound to the chroot.

**If any step segfaults**, don't panic. Use the escape hatch from the
path document: build the offending package statically on your modern
host and copy the binary into the chroot's `/usr/local/bin/`:

```bash
# On the host:
gcc -static -o /tmp/modern-gcc-wrapper ...
cp /tmp/modern-gcc-wrapper /mnt/AI/sourcemage_root/usr/local/bin/

# Or copy a whole modern compiler:
cp -a /usr/bin/gcc-something /mnt/AI/sourcemage_root/usr/local/bin/gcc-host
```

### Phase 9 — Init sorcery-go (chroot)

Now that the toolchain is modern, init the Go engine:

```bash
smgl-getting-started.sh chroot-init-sorcery-go
```

This runs `sorcery-go init --force` against the test grimoire, indexes
every spell into memory, and prints the spell count. You should see
thousands of spells indexed on a real test-branch grimoire.

Try your first Go-powered cast:

```bash
sorcery-go gaze depends wget
sudo sorcery-go cast busybox --static --default
sorcery-go tomb list
```

Launch the Coven Mirror WebUI:

```bash
sudo sorcery-go web --port 8080
```

### Leave + unmount (host)

```bash
# Inside the chroot:
exit

# Back on the host:
sudo ./scripts/smgl-getting-started.sh unmount /mnt/AI/sourcemage_root
```

The rootfs is now safe to tar up, dd onto a partition, or NFS-export.

---

## Troubleshooting

### "dispel: command not found" inside the chroot
You skipped Phase 5 (`inject-sorcery`). Leave the chroot, run
`inject-sorcery` from the host, then re-enter.

### "scribe add test" fails with SSL errors
The chroot's CA certificates are 9 years old. Two options:

1. Use the https fallback (the script tries it automatically).
2. Copy modern certs from the host:
   ```bash
   sudo cp /etc/ssl/certs/ca-certificates.crt \
       /mnt/AI/sourcemage_root/etc/ssl/certs/ca-certificates.crt
   ```

### "cast gcc" segfaults
The 2017-era compiler can't bootstrap a modern GCC. Use the escape hatch:
copy a host-built intermediate GCC (9 or 10) into the chroot's
`/usr/local/bin/`, then re-run `cast gcc` — it'll use the host binary as
the bootstrap compiler.

### "cast glibc" fails with "kernel headers too old"
The chroot's `linux-headers` are ancient. Cast a modern linux-headers
spell first:

```bash
cast linux-headers
cast glibc
```

### `sorcery-go init` reports 0 spells indexed
You're pointing at the wrong grimoire path. Check what scribe indexed:

```bash
ls /var/lib/sorcery/codex/
# Should show: test/  (and possibly stable/ if you didn't remove it)
```

Then set `SORCERY_GO_GRIMOIRE=/var/lib/sorcery/codex/test` and re-run
`sorcery-go init --force`.

### The chroot's network is dead
You probably skipped the `resolv.conf` copy in Phase 1. From the host:

```bash
sudo cp /etc/resolv.conf /mnt/AI/sourcemage_root/etc/resolv.conf
```

If your host uses `systemd-resolved`, the actual resolv.conf is at
`/run/systemd/resolve/resolv.conf` — copy that instead.

---

## Coexistence with legacy Bash sorcery

After the full pipeline completes, both engines live in the chroot:

| Tool | Binary | State | Reads grimoire |
|------|--------|-------|----------------|
| Legacy Bash sorcery | `/usr/sbin/cast` | `/var/lib/sorcery/` | `/var/lib/sorcery/codex/test/` |
| Sorcery-Go | `/usr/local/sbin/sorcery-go` | `/var/lib/sorcery-go/` | same (read-only) |

They never touch each other's state. You can switch freely:

```bash
sudo cast wget              # legacy Bash cast
sudo sorcery-go cast wget   # new Go cast
```

A broken `sorcery-go cast` never affects the Bash install, and vice versa.

---

## Next steps

- **Disaster recovery**: see `docs/DISASTER_RECOVERY.md` for how to back
  up the Tomb and Tablet once you've cast real spells.
- **Coven Mirror WebUI**: see `docs/QUICKSTART.md` for the dashboard tabs.
- **Warding setup**: see `docs/SECURITY.md` for eBPF Tomb Guard and
  firewall configuration once the chroot is booted on bare metal.
- **Custom toolchains**: see `docs/TOOLCHAIN_SPEC.md` if you want
  sorcery-go to use your own GCC/LLVM instead of the chroot's defaults.

---

*"The Ley-Lines are humming. The Tomb is secure. The Coven is active."*