38 KiB

Executable File

Raw Permalink Blame History

BTC.sh and the Clean Toolchain Build

Fingerprinting, Forensic Identity, and Integration with Source Build Systems

dcos.net | July 2026

There is a particular kind of systems engineering that refuses to delegate its compiler to anyone else.

BTC.sh (Build Tool Chain) is a single Bash script, version 0.4.1, that produces cross-compilation toolchains entirely from upstream source. It builds binutils, a two-stage GCC, a C library (glibc or musl, depending on the target architecture), Linux kernel headers, a monolithic kernel binary, and libxcrypt for glibc-based targets, all inside a ramfs mount at /usr/src. The entire pipeline runs from a single invocation: BTC.sh haswell, BTC.sh armv7, BTC.sh --native, or any of the 19 registered cross-compilation targets. No pre-built binaries, no binary bootstrap packages, no distribution toolchain metapackages. Every object file on the target system traces back to source code you can audit, compiled by a compiler built on your hardware, producing binaries stamped with forensic provenance that identifies the exact machine, microarchitecture, toolchain version, and signature tier that produced them.

BTC.sh does not exist in isolation. It was designed as the compiler layer of a vertically integrated build stack. Source Mage GNU/Linux provides the spell format that describes how to build each package from upstream source. Sorcery-go reads those spell files, resolves dependency graphs, and manages the package lifecycle. Fester is a distributed, DAG-driven build execution system that decides where each build runs across a cluster of machines. And a shared Content-Addressable Store ensures no artifact is ever built twice. This post is a technical walkthrough of how BTC.sh works and how it integrates with each layer of that stack, grounded in the source code rather than marketing claims.

The Build Architecture

Target Registry and ISA Tiers

BTC.sh organizes its 19 cross-compilation targets through an associative array called BTC_TARGETS. Each entry is a pipe-delimited property string containing eleven fields: architecture, build CPU, microarchitecture march, ISA tier, ABI, C library choice, endianness, family grouping, human-readable description, minimum kernel version, and GCC-specific extra configure flags. The design follows PEP 868 and MISRA conventions for table-driven dispatch rather than ad-hoc if/else chains. When you run BTC.sh haswell, the target ID is resolved against this registry, the property string is split into named variables (BTC_T_ARCH, BTC_T_MARCH, BTC_T_ISA, BTC_T_CLIB, and so on), and those variables drive every subsequent decision in the build pipeline, from configure flags to kernel defconfig selection to the forensic stamp embedded in each binary.

The targets span five microarchitecture families. Intel HEDT and server platforms run from Haswell through Skylake-Server, covering AVX2 and AVX512 ISA tiers. AMD Ryzen and EPYC span Zen 1 through Zen 4. AMD APU mobile and embedded parts (Raven Ridge through Cezanne) map to their respective Zen generations with AVX2. Intel Atom embedded processors (Silvermont, Goldmont, Tremont, Sierra Forest) occupy the SSE4.2 tier. And three embedded architectures — MIPS32R2 little-endian (musl), ARMv7-A hard-float NEON (musl), and Tilera TILE-Gx72 (musl via GCC 10.3.0, since upstream dropped Tile-Gx after GCC 11) — round out the registry. Six ISA tiers (AVX512, AVX2, SSE4.2, NEON, MIPS32, TILE) govern which optimization flags are injected into GLOBAL_CFLAGS and GLOBAL_LDFLAGS through a case dispatch in f_set_paths.

Family	Targets	ISA Tier	C Library
Intel HEDT/Server	haswell, skylake, skylake-x, skylake-server	AVX2 / AVX512	glibc
AMD Ryzen/EPYC	znver1, znver2, znver3, znver4	AVX2 / AVX512	glibc
AMD APU	apu-zn1 through apu-zn4	AVX2	glibc
Intel Atom	silvermont, goldmont, tremont, sierraforest	SSE4.2	glibc
Embedded	mipselr2, armv7, tilegx	MIPS32 / NEON / TILE	musl

The Ramfs Cleanroom

Before any compilation begins, f_setup creates the build environment. It mounts a ramfs filesystem at /usr/src with a configurable size ceiling (default 12 GB), creates the sysroot directory structure under ${NEWROOT} (which lives at /usr/src/${SYS_LABEL}-cleanroom), and adjusts the PATH to prioritize the new toolchain's bin/ directory. The ramfs mount is the foundation of BTC.sh's build isolation strategy. Everything — source extraction, object files, intermediate build artifacts — lives in volatile memory. When the build completes, f_main unmounts the ramfs, and all build intermediates vanish. No I/O wear on host storage, no residual object files contaminating subsequent builds, no possibility of stale headers or cached configurations leaking between invocations. The only persistent outputs are the golden image tarball (compressed with xz -9 by default, though lrzip and bzip2 are available via USE_COMPRESSOR), the per-binary debug symbols, and the integration manifest JSON.

The sysroot layout is architecture-aware. x86_64 targets create a lib64 symlink pointing to lib. ARM targets create a lib32 symlink. MIPS and Tile-Gx have their own conventions. This matches the CLFS (Cross Linux From Scratch) sysroot layout and ensures that the cross-compiler installs libraries into the paths that the target C library and dynamic linker expect.

The Linear Build Sequence

BTC.sh executes eight build functions in strict linear order inside f_main. First, f_binutils builds the cross-assembler, linker, and binary utilities, configured with --target=${TARGET} and --with-sysroot=${NEWROOT}. Second, f_kernel_headers extracts the Linux kernel source, runs make mrproper and make headers, sanitizes the header tree by removing non-.h files, and copies the result into ${NEWROOT}/usr/include. Third, f_gcc_p1 extracts GCC, nests GMP, MPFR, and MPC inside the GCC source tree (GCC's internal build system prefers bundled libraries for cross-compilation), applies architecture-specific source patches (such as the x86_64 lib64 to lib multilib path fix, which in GCC 15 now falls through to t-linux since t-linux64 was removed), and builds a bootstrap compiler with --with-newlib --without-headers --disable-shared --disable-threads.

Fourth, f_clib dispatches to either f_glibc or f_musl depending on the target registry. glibc targets configure with --enable-kernel=${BTC_T_KERN_MIN} (the minimum kernel version from the target entry, which prevents glibc from using syscalls that do not exist on older kernels) and libc_cv_slibdir=/usr/lib to enforce the lib-not-lib64 convention. musl targets build a standalone cross-compiler wrapper that produces ${TARGET}-musl-gcc and installs the musl C library into the sysroot. Fifth, f_libxcrypt builds the besser82 libxcrypt library for glibc targets (skipped for musl, which has built-in crypt support). Sixth, f_gcc_p2 builds the final compiler against the now-complete sysroot, with full C and C++ support, PIE, and stack protector enabled. Seventh, f_kernel_binary cross-compiles a monolithic kernel with modules disabled, KALLSYMS and DEBUG_FS turned off for attack surface reduction, and the LOCALVERSION set to the system label. Eighth, f_package produces the golden image tarball, the SHA-256 checksum, and the integration manifest.

Integration with Source Build Systems

The output of BTC.sh is a directory tree rooted at ${SYS_LABEL}-cleanroom, compressed into a golden image tarball with a sidecar JSON manifest. This manifest contains the target triple, microarchitecture, GCC version, C library, CFLAGS, LDFLAGS, stamp field names, signature tier, and integration keys for sorcery-go and Fester. Any build system that can set CC, CXX, CFLAGS, and LDFLAGS can consume a BTC toolchain. But five source-based Linux systems have explicit, tested integration paths that go beyond simple environment variable injection.

Source Mage GNU/Linux

Source Mage GNU/Linux (SMGL) is the most natural consumer of a BTC-built toolchain because it is the build system that sorcery-go was designed to be compatible with. SMGL organizes its package recipes as "spells" — directories under a grimoire tree, each containing a DETAILS file (metadata, source URL, version), a DEPENDS file (runtime and build dependencies), and a BUILD file (the compilation sequence as Bash). Sorcery-go implements a spell format compatible with SMGL grimoires. It is not a fork of the original Source Mage sorcery tool and is not affiliated with the Source Mage project, but it reads the same spell files. The SMGL test-branch grimoire contains thousands of actively maintained spell recipes for modern software (GCC 15, LLVM 22, Firefox 151, 6.x kernels), and reimplementing all of that from scratch would be pointless.

Sorcery-go integrates with BTC.sh through pkg/toolchain/btc.go. The Go code defines a BTCStamp struct that mirrors the data BTC.sh writes into the ELF NOTE section and extended attributes. The ValidateBTC function locates the BTC golden image tarball (matching *-toolchain-golden.tar.xz), extracts just the GCC binary from it, runs readelf -n to check for the .note.BTC section, and parses the forensic stamp into the BTCStamp struct. It also runs gcc -v to verify that the toolchain supports LTO, PIE, and Stack Smashing Protection — all three must be present for the validation to pass. The ProbeGoldenImage function searches standard BTC installation paths (/opt/BTC/, /opt/btc/, configurable via the sorcery-go config), locates the golden image, and parses the manifest JSON sidecar to extract the SYS_LABEL, target triple, and ISA tier.

In practice, using a BTC toolchain with SMGL means extracting the golden image to a known path, pointing sorcery-go at it via the SORCERY_GO_BTC_PATH environment variable (or the toolchain: btc configuration key), and letting the package manager use the cross-compiler for every build. The forensic stamps in each binary are preserved through the spell build process because sorcery-go invokes the cross-compiler directly rather than wrapping it. The user.btc.identity and user.btc.hash extended attributes survive the spell's install phase because they are filesystem-level metadata attached by setfattr, not something the build system touches. This means that a Source Mage system built entirely with a BTC toolchain produces a fully auditable binary population where every file can be traced back to its build origin with readelf -n and getfattr.

Gentoo

Gentoo's cross-compilation model is built around the CBUILD, CHOST, and CTARGET triple system. The host system's triple is CBUILD, the target system's triple is CHOST (for native builds) or CTARGET (for true cross-compilation). Gentoo's crossdev tool can generate cross-compiler ebuilds, but it typically builds them against the host system's libraries and GCC configuration, which means the resulting toolchain carries the host's optimization flags and library versions rather than the target's. BTC.sh takes the opposite approach: it builds the toolchain in complete isolation inside the ramfs cleanroom, with no reference to the host system's compiler beyond using it to bootstrap the first stage.

Integration with Gentoo involves extracting the BTC golden image, creating a cross-environment profile that sets CBUILD=x86_64-pc-linux-gnu, CHOST=${TARGET} (from the manifest), and pointing Portage's ROOT and SYSROOT at the cleanroom sysroot. The BTC manifest provides the exact CFLAGS and LDFLAGS that the build used, which can be injected into make.conf for the target board. Because BTC.sh already handles --disable-multilib, --enable-default-pie, --enable-default-ssp, and microarchitecture-specific flags like -march=haswell -mavx2 -flto, the Portage configuration can be minimal. The forensic stamps are invisible to Portage, which treats the toolchain binaries as ordinary compiler executables, but they are present in every binary that the cross-compiler produces. Fester's backend/toolchain/chroot.py module implements this integration path under the gentoo provider, detecting the cross-compiler by its triple prefix and automatically setting CC, CXX, AR, RANLIB, and NM.

Lunar Linux

Lunar Linux is a source-based distribution that uses its own build tools (lunar, moonbase) to manage package compilation from source. Like Source Mage, it is a descendant of the original Sorcery package management system, though it has diverged significantly. Lunar's model is similar to SMGL's in that each package has a build script, but the implementation details, dependency resolution, and configuration engine are different.

A BTC-built toolchain integrates with Lunar at the shell environment level. The operator sources the manifest JSON to extract TARGET, GLOBAL_CFLAGS, and GLOBAL_LDFLAGS, sets CC=${TARGET}-gcc, CXX=${TARGET}-g++, and adds the golden image's bin/ directory to PATH. Lunar's build system then uses these environment variables for every package compilation, exactly as it would use the native compiler. The LTO flags in BTC.sh's GLOBAL_CFLAGS (-flto -ffat-lto-objects) are compatible with Lunar's build process because LTO is a compiler-linker concern, not a package manager concern. The forensic stamps in the toolchain's own binaries (GCC, binutils, the kernel) are set during the build, and the stamps propagate to every package Lunar builds because those packages are compiled by the stamped cross-compiler. Fester's chroot module provides the same lunar provider integration, detecting the cross-compiler by triple prefix and configuring the build environment automatically.

LEDE / OpenWrt

LEDE (Linux Embedded Development Environment) and its successor OpenWrt have a well-defined toolchain overlay mechanism. The OpenWrt SDK expects a cross-compiler toolchain at a specific path within the SDK directory structure, with a specific naming convention for the triple prefix. The CONFIG_TARGET_OPTIMIZE and CONFIG_GCC_VERSION settings in OpenWrt's .config control which optimization flags and GCC version the build system uses. BTC.sh's output maps directly onto this model.

The integration path works as follows. The BTC golden image is extracted to a staging directory. The OpenWrt .config is modified to set CONFIG_EXTERNAL_TOOLCHAIN=y, CONFIG_PACKAGE_kmod=n (since BTC.sh builds a monolithic kernel), and CONFIG_TARGET_OPTIMIZE to the BTC target's march string (for example, -O2 -pipe -march=haswell -mavx2). The STAGING_DIR and TOOLCHAIN_DIR environment variables are pointed at the golden image's sysroot. Fester's chroot module implements this under the lede provider, which detects the OpenWrt SDK layout, locates the cross-compiler by its triple prefix (x86_64-dcosnet-linux-gnu-gcc), and sets CC, CXX, CFLAGS, CXXFLAGS, and LDFLAGS accordingly. The BTC toolchain's monolithic kernel (built by f_kernel_binary) replaces OpenWrt's kernel, and the sysroot provides the C library and headers that the SDK's package builds need.

Generic Makefile Use

For projects that do not use a package manager, the integration is the simplest and most universal form. The manifest JSON provides target_triple, cflags, and ldflags as plain strings. A Makefile or build script sets three to five environment variables and calls it done. The golden image's bin/ directory contains the full cross-compiler suite: ${TARGET}-gcc, ${TARGET}-g++, ${TARGET}-ld, ${TARGET}-ar, ${TARGET}-ranlib, ${TARGET}-objcopy, ${TARGET}-strip, ${TARGET}-readelf, and the rest of the binutils toolchain. Adding the bin/ directory to PATH and setting CC and CXX is sufficient for most autotools, CMake, and Meson-based projects.

For Makefiles that respect CROSS_COMPILE (as the Linux kernel build system does), setting CROSS_COMPILE=${TARGET}- configures all toolchain binary lookups in a single variable. For projects that use CC directly, the manifest's target_triple field provides the prefix. The forensic stamps in the resulting binaries are a free side effect: any binary compiled with a BTC-built toolchain receives the .note.BTC ELF section and the extended filesystem attributes, regardless of whether the project's build system knows about BTC.sh or not. The stamps are injected at the compiler level, not the build system level, which is the entire point of the build's design.

Fester's Chroot Provider Layer

Fester's backend/toolchain/chroot.py module implements a unified abstraction over seven toolchain providers: native, buildroot, sourcemage, gentoo, lede (OpenWrt), lunar, and generic chroot. When a BTC toolchain is active on a Fester node, the chroot module detects the cross-compiler by its triple prefix — the dcosnet component in x86_64-dcosnet-linux-gnu-gcc is the fingerprint — and automatically configures CC, CXX, CFLAGS, CXXFLAGS, and LDFLAGS for the build environment. Per-node BTC target selection is configured in Fester's config.yaml under nodes[].btc.target, and when a BTC toolchain is active, Fester sets the cross-compiler environment variables automatically before dispatching any build action on that node.

This means that Fester does not need to know how BTC.sh works internally. It probes for the golden image, reads the manifest JSON sidecar, and extracts the target triple and compiler flags. The manifest is the contract. Any toolchain that produces a compatible manifest — not just BTC.sh — can be consumed by Fester's toolchain layer.

The Fingerprinting Strategy

Every binary produced by a BTC-built toolchain carries two forms of identification. The first is an ELF NOTE section named .note.BTC, injected by f_stamp_binary using an assembly stub compiled into an object file and linked into the binary via objcopy --add-section. The second is a set of extended filesystem attributes, set via setfattr, that persist the same identity data at the filesystem level. Together, these form what BTC.sh calls the "Silicon Birth Certificate" — you can take any binary on any machine and, with readelf -n and getfattr, determine exactly which hardware, toolchain version, and build environment produced it.

The ELF NOTE Section (.note.BTC)

The f_stamp_binary function generates a small assembly file, btc_stamp.s, that defines a .note.BTC section with three fields: a 4-byte namesz, a 4-byte descsz, and a 4-byte type (always 1, per the ELF specification for vendor 0xB7C notes). The name field contains the string DCOSNET. The description field is a single ASCII string containing the organization (dcos.net), kernel version (K:${v_linux}), target architecture (Arch:${BTC_T_ID}), system label (Label:${SYS_LABEL}), the build stage that produced the binary (Build:${log_base}), and the active signature tier and token (SigTier:${BTC_SIG_TIER}|Sig:${BTC_SIG_TOKEN}). This assembly is compiled with the target's assembler (the cross-assembler for cross-compiled binaries, or the native gcc for native builds), linked into an object file, and grafted into the target binary with objcopy --add-section .note.BTC=btc_stamp.o. The section is marked allocable and not-loadable, which means it appears in readelf -n output but does not affect runtime behavior.

$ readelf -n /usr/bin/gcc
Displaying notes found in: .note.BTC
  Owner                 Data size  Description
  DCOSNET               0x00000e4  vendor 0xB7C (unknown)
    Org: dcos.net|K:linux-7.1|Arch:haswell|Label:DCOSNET-HASWELL-AVX2-LTO
    Build:gcc|SigTier:poly|Sig:a3f7c2e1b8d40926...

The note payload is human-readable by design. There is no encryption, no encoding, no binary serialization format. This is a deliberate choice: forensic tooling should not require a proprietary parser. Any sysadmin with readelf can extract the full provenance chain from any binary on the system. The note's content is deterministic for a given target, kernel version, and signature tier, which means it can be used in automated compliance audits that verify every binary on a production system was produced by an authorized build.

Sorcery-go's ValidateBTC function reads this same note programmatically. It runs readelf -n on the golden image's GCC binary, parses the .note.BTC output into the BTCStamp struct, and makes the provenance data available to the rest of sorcery-go. The pkg/toolchain/validator.go file provides a broader Validate function that works with any GCC or LLVM compiler — not just BTC-built ones. It runs a smoke test: compile a Hello World with -fstack-protector-all -pie, inspect the resulting binary for SSP, PIE, and the expected target triple. The Report struct includes an IsBTC boolean and a BTCStamp pointer, so the rest of sorcery-go can make BTC-aware decisions downstream (for example, including provenance metadata in the Essence Sarcophagus when the toolchain is BTC-built).

Extended Filesystem Attributes

In addition to the ELF NOTE, f_stamp_binary sets four extended attributes on each binary using setfattr. The user.btc.identity attribute contains the full system label and organization (for example, BTC-DCOSNET-HASWELL-AVX2-LTO-clean). The user.btc.hash attribute contains the SHA-256 hash of the binary's content at stamp time. The user.btc.sig.tier attribute records which signature tier was active (cluster, tpm, or poly). The user.btc.sig.token attribute records the token itself. These attributes persist as long as the filesystem supports extended attributes (ext4, xfs, btrfs all do), and they survive cp and tar operations when the appropriate flags are used. They provide a filesystem-level parallel to the ELF NOTE: even if a binary is stripped, the xattr identity remains intact.

Poly-Signature Tiers

BTC.sh implements three signature tiers, selected at build time through a combination of CLI flags and persistent state files. The selection priority is cluster-join, then TPM-seal, then poly (the default).

Tier 1 — Cluster is for deployments that need a unified forensic identity across multiple machines. When a cluster token is provided via --join-cluster=TOKEN or inherited from a pre-existing token file at /opt/BTC/.btc-cluster-token, every binary on that machine receives the same signature token. This means that binaries from different machines in the same cluster can be cryptographically correlated; they share the same Sig: field in their .note.BTC section. The token is write-once. Once a machine has joined a cluster, subsequent builds reuse the existing token without drift.

Tier 2 — TPM binds the signature to the hardware's TPM PCR state. The f_tpm_pcr_digest function attempts to read PCRs 0 through 7 from a TPM 2.0 device using tpm2_pcrread, or from a TPM 1.2 device via /sys/class/tpm/tpm0/pcrs, and hashes the result with SHA-256. The first 32 hex characters of this digest become the signature token, prefixed with tpm:. This means that the signature is bound to the machine's firmware, bootloader, and platform configuration; if any of these change, the PCR values change, and a re-build with --tpm-seal would produce a different token. This tier is intended for singular deployments where the toolchain is bound to a specific piece of hardware.

Tier 3 — Poly (the default) generates a per-machine random salt using openssl rand -hex 16 on first invocation and writes it to /opt/BTC/.btc-salt. Subsequent builds on the same machine reuse the salt. Two independent machines building the same target produce different forensic signatures because their random salts differ, even though the target architecture, kernel version, and toolchain components are identical. This is the "poly" in poly-signature: the same source matrix produces polymorphic forensic identities depending on the machine that executes the build.

The tier system is not a security mechanism in the cryptographic sense; it is a provenance mechanism. It answers the question "which machine produced this binary?" rather than "is this binary authentic?"

Debug Symbol Extraction

As a final step in the stamping process, f_stamp_binary optionally extracts debug symbols from each binary (controlled by BTC_STRIP_MODE, defaulting to 1). It uses objcopy --only-keep-debug to write the debug info to a separate .debug file under /opt/BTC/symbols/${SYS_LABEL}/, then strips the binary with strip --strip-unneeded, and links the debug file back to the binary with objcopy --add-gnu-debuglink. This means that the production binaries are stripped for size, but the full debug information is preserved separately and can be retrieved with gdb when needed. The debug symbol files themselves are not stamped, but their filenames include the log_base (for example, gcc.debug, ld.debug), which maps them to the specific build stage that produced them.

Security Methods

BTC.sh's security model operates at three levels: supply chain control, build environment isolation, and runtime hardware monitoring. None of these are cryptographic protocols in the traditional sense. There is no TLS, no code signing, no GPG verification of downloads (intentionally, as explained below). The security model is instead structural: it is designed to make certain classes of attacks difficult or impossible by controlling the build environment rather than by verifying cryptographic signatures.

Supply Chain Control: Single Source of Truth

All upstream source URLs are defined in a single associative array, A_SRC_URL, at the top of the script. This array maps package stems (binutils, linux, gcc, glibc, musl, gmp, mpfr, mpc, libxcrypt) to their official upstream URLs. There is no URL discovery, no mirror fallback, no dynamic URL construction. Every URL is a literal string pointing at a known upstream mirror: GNU FTP for binutils, GCC, GMP, MPFR, and MPC; kernel.org for the Linux kernel; musl.libc.org for musl; and GitHub (besser82) for libxcrypt. The f_download function iterates over this array, checks each file for existence and integrity, and fetches anything missing.

The integrity check is performed by _archive_sane, which runs tar -tf on the cached file and returns failure if the archive cannot be listed. This is not a cryptographic hash verification; it is a structural integrity check that catches truncated downloads, gzip unexpected-end-of-file errors, and corrupted xz streams. The rationale for not using hash verification at download time is that the upstream tarballs are already trusted by virtue of being fetched from their official sources over HTTPS, and a structural integrity check provides a stronger guarantee against corruption than a hash comparison against a hardcoded value (which could itself be wrong if the upstream release is re-uploaded). After download, f_download generates both MD5 and SHA-512 checksums of each file, written to per-file manifests under ${LOGS}/checksums/. These checksums are for auditing and comparison, not for gate-keeping.

Build Environment Isolation: Ramfs and Path Control

The ramfs cleanroom provides two security properties. First, it ensures that no build artifact persists across build invocations. Each run starts with a clean slate: the ramfs is freshly mounted, sources are freshly extracted, and the sysroot is freshly created. There is no possibility of a stale object file, a cached configure result, or a leftover header contaminating the build. Second, because ramfs lives entirely in memory, it provides a degree of isolation from the host filesystem. A compromised build process cannot write to the host's persistent storage through the normal build paths. The build's output is confined to the ramfs mount point and the persistent directories that BTC.sh explicitly creates (/opt/BTC for state and artifacts, ${LOGS} for build logs).

Path control is the third isolation mechanism. f_setup prepends ${NEWROOT}/bin to the system PATH, which means that the freshly built cross-compiler takes priority over any system-installed compiler. This prevents accidental use of the host GCC or binutils during the build. The f_exec_log wrapper, which is used for every build step, relies on the PATH being correctly set and the set -euo pipefail at the top of the script to catch any command failures.

Hardware Sentinel: Thermal and Memory Guards

The f_guard function implements a simple but effective hardware monitoring system that runs before every build step via f_exec_log. It reads the CPU temperature from /sys/class/thermal/thermal_zone*/temp (with a graceful fallback for containers where thermal zones may not exist) and the available memory from /proc/meminfo via free -m. If the temperature exceeds 85 degrees Celsius, the build pauses for 15 seconds to allow cooling. If available memory drops below 800 MB, the build pauses for 20 seconds. These thresholds are hardcoded constants chosen to prevent thermal throttling and OOM kills during the most memory-intensive build phases (GCC Stage 2 with LTO, which can consume over 2 GB per thread).

Entropy Shield

The f_entropy_shield function monitors the kernel's entropy pool at /proc/sys/kernel/random/entropy_avail. If the pool drops below 1000 bits, it injects hardware jitter by running find /bin /sbin -type f -exec ls -l {} + in the background for two seconds. This is a deliberately simple entropy injection mechanism. It is not intended to be a substitute for a hardware RNG or haveged; it is a safety net for builds running on minimal VMs or containers where the entropy pool can become depleted during parallel compilation. The function is called before every build step, interleaved with the thermal and memory guards.

Thread Allocation and LTO Safety

BTC.sh calculates the number of parallel build threads based on available memory rather than CPU count. The formula in f_set_paths divides free memory (in GB) by 2 and clamps the result between 1 and the number of physical CPUs. This means that on a machine with 8 cores but only 4 GB of free memory, the build will use 2 threads rather than 8. This is a direct response to LTO (Link-Time Optimization), which BTC.sh enables by default via -flto -ffat-lto-objects in GLOBAL_CFLAGS. LTO is extremely memory-hungry during the link phase; each parallel link process can consume 2 GB or more. Without this throttling, a parallel build on memory-constrained hardware would trigger the OOM killer and potentially corrupt the build output.

Kernel Hardening

The kernel built by f_kernel_binary has several hardening measures applied through .config manipulation. CONFIG_MODULES=n disables loadable kernel modules entirely, forcing a monolithic kernel. This eliminates the entire class of kernel module loading attacks (insmod-based rootkits, malicious .ko files) at the cost of larger kernel size and no runtime driver addition. CONFIG_KALLSYMS=n removes the kernel's symbol table from /proc/kallsyms, which prevents attackers from resolving kernel function addresses for exploit development. CONFIG_DEBUG_FS=n removes the debugfs filesystem, which exposes numerous kernel internals that are useful for debugging but also useful for information disclosure in an attack scenario. The LOCALVERSION is set to the system label (-dcosnet-${SYS_LABEL}), which makes the kernel's version string identifiable in uname -a output and correlates it with the BTC toolchain that built it.

The Firewall-First Model and eBPF Warding

One of the more striking architectural decisions across the clean build stack is the firewall-first security model. There is no application-layer TLS or mTLS anywhere in the stack. No certificate management, no key rotation, no CRL propagation. Transport security is delegated entirely to the network boundary — OPNsense or IPFire firewalls isolate the cluster, and all inter-node traffic (sorcery-go to Fester, Fester to workers) is plain HTTP and raw WebSocket.

The defense-in-depth model instead relies on eBPF. Sorcery-go's Warding subsystem includes an eBPF LSM (Linux Security Module) called the "Tomb Guard" that operates at the kernel level to block unauthorized writes to the Tomb (the binary artifact store). eBPF cgroup filters provide device and network control per-process. Content-addressing via Merkle trees ensures that any modification to a stored artifact is detectable by recomputing the root hash. A quarantine system using cgroup freezers can contain a compromised node. And for per-process network filtering, Cilium (eBPF-native, recommended), OpenSnitch, or Portmaster can be layered on top. All crypto/rand, crypto/tls, and crypto/x509 code has been deliberately removed from the sorcery-go codebase. Task IDs are generated deterministically using atomic counters and nanosecond timestamps — no UUIDs, no randomness, no entropy drain.

This is not a security model that works for everyone. If your threat model includes a compromised internal node or a malicious insider, it falls short. But for a self-managed build cluster on trusted hardware behind a dedicated firewall, it eliminates an entire class of operational complexity. There are no certificates to expire, no TLS versions to negotiate, no SNI mismatches to debug, no certificate pinning to maintain across rolling restarts. The security boundary is the network itself, enforced by hardware-level eBPF programs that cannot be bypassed from userspace.

The Integration Manifest and Shared CAS

The f_package function generates a JSON manifest file alongside the golden image tarball. This manifest is the contract between BTC.sh and its consumers (sorcery-go, Fester, and any custom tooling). It contains the complete build provenance: the BTC version, build mode (native or cross), target ID, architecture, CPU, microarchitecture, target triple, host architecture, ISA tier, optimization tag, ABI, C library, endianness, family, description, minimum kernel version, and the versions of every component (kernel, binutils, GCC, glibc/musl, libxcrypt). It also records the exact CFLAGS and LDFLAGS used, the ELF NOTE section name, the xattr field names, the active signature tier and token preview, and two integration blocks.

The sorcery-go integration block specifies the configuration key (toolchain: btc) and the three environment variables that sorcery-go uses to locate and configure a BTC toolchain: SORCERY_GO_BTC_PATH (the path to the golden image directory), SORCERY_GO_BTC_ROOT (the sysroot path), and SORCERY_GO_BTC_SYS_LABEL (the system label for stamp validation). The Fester integration block specifies the YAML configuration keys (btc.enabled, btc.root, btc.target) that Fester's node configuration uses to enable BTC toolchain support on a per-node basis.

The golden image tarball itself is compressed with xz -9 (or the user's choice of compressor) and accompanied by a SHA-256 checksum file. This SHA-256 checksum is the key used by the shared Content-Addressable Store to deduplicate build artifacts across nodes and runtimes. The CAS is the most tangible shared surface between sorcery-go and Fester. The pkg/cas/cas.go package implements a client for Fester's /api/cas/ endpoints. After sorcery-go's Cauldron produces an Essence (a .svb bundle), it can push it to the shared CAS via PushFile. Fester's DAG executor checks the CAS before dispatching builds — if the artifact already exists, the build is skipped entirely.

The CAS key is the SHA-256 of the artifact content, which means the same library compiled with the same toolchain on different runtimes will deduplicate if the output matches. The CAS client supports the full CRUD surface: CheckArtifact (HEAD, returns nil if not cached — this is not an error), PushArtifact and PushFile (PUT, with automatic SHA-256 computation for files), RetrieveArtifact and RetrieveArtifactToFile (GET, streamed), DeleteArtifact (DELETE), and Stats (returns hit rate, utilization, and total bytes). Each artifact carries metadata — source filename, build ID, target triple, runtime type, and producing node — attached as URL query parameters on push and returned in the JSON response on retrieval. There is a 2 GiB size limit per artifact, enforced client-side before reading to prevent unbounded memory allocation.

This design means that sorcery-go and Fester do not need to share a database, a filesystem, or even a runtime. They communicate through a simple HTTP API behind the network firewall. Sorcery-go handles security (eBPF warding, tomb protection, essence verification), spell management, and package lifecycle. Fester handles distributed scheduling, node telemetry, build dispatch, and observability. The CAS is the contract between them. And BTC.sh provides the forensically-stamped compiler that produces every binary in the pipeline.

Putting It All Together

The standard workflow ties the stack into a single pipeline. A spell in the SMGL-compatible grimoire describes a package — its source, dependencies, and build steps. BTC.sh has built a microarchitecture-optimized GCC toolchain for the target architecture, stamped with forensic provenance. Sorcery-go reads the spell, resolves its dependency graph (DAG), and checks the shared CAS for cached artifacts. If the artifact is not cached, sorcery-go delegates the build to Fester via its HTTP API. Fester's scheduler picks the best node based on CPU, thermal, cache, and policy. Fester configures the build environment using the BTC.sh toolchain on that node — setting CC, CXX, CFLAGS, and LDFLAGS automatically via the chroot provider layer. The build executes in an isolated runtime (LXC, Podman, Firecracker, or bare-metal) with eBPF warding active. The resulting binary is verified by sorcery-go's Warding (Merkle root recomputation, BTC stamp validation), sealed as a content-addressable Essence in the Tomb, and pushed to the shared CAS for future deduplication. Target nodes are atomically updated via reflink or hardlink swaps. On first execution, the Warding verifies the Essence again — any mismatch triggers quarantine.

Each project in this stack is independently useful. You can run sorcery-go standalone without Fester or BTC.sh. You can run Fester as a general-purpose distributed build system without sorcery-go. You can run BTC.sh to build toolchains without either of the other two. But the integrations are real, load-bearing code paths — not afterthoughts or loose couplings — and they transform the projects from a collection of tools into a coherent infrastructure management system.

BTC.sh, Sorcery-Go, and Fester are developed by dcos.net and released under AGPL-3.0-or-later. Source Mage GNU/Linux is developed at sourcemage.org under its own project governance. Sorcery-Go implements a compatible spell format for convenience but is an entirely separate and independent project.

38 KiB Executable File Raw Permalink Blame History