dcosnet
/
BuildToolChain
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
BuildToolChain/btc.sh.html

389 lines
39 KiB
HTML
Executable File
Raw Permalink Blame History

<html lang="en"><head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>BTC.sh</title>
<style>
/* ── Reset & Base ── */
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
html { font-size: 16px; -webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; }
body {
font-family: 'Georgia', 'Noto Serif', 'Times New Roman', serif;
line-height: 1.75;
color: #1a1a1a;
background: #fafaf8;
max-width: 42rem;
margin: 0 auto;
padding: 2rem 1.5rem 4rem;
}
/* ── Typography ── */
h1, h2, h3, h4 { font-family: 'Helvetica Neue', 'Arial', 'Noto Sans', sans-serif; font-weight: 700; line-height: 1.3; }
h1 { font-size: 2rem; margin-bottom: 1.5rem; color: #111; letter-spacing: -0.02em; }
h2 { font-size: 1.5rem; margin-top: 3rem; margin-bottom: 1rem; color: #222; border-bottom: 2px solid #e0ddd5; padding-bottom: 0.35rem; }
h3 { font-size: 1.2rem; margin-top: 2rem; margin-bottom: 0.75rem; color: #333; }
p { margin-bottom: 1.25rem; }
a { color: #8b2500; text-decoration: none; border-bottom: 1px solid rgba(139,37,0,0.3); transition: border-color 0.15s; }
a:hover { border-bottom-color: #8b2500; }
/* ── Block Elements ── */
blockquote {
margin: 1.5rem 0;
padding: 1rem 1.25rem;
border-left: 3px solid #c4a96a;
background: #f5f3ed;
font-style: italic;
color: #444;
border-radius: 0 4px 4px 0;
}
blockquote p:last-child { margin-bottom: 0; }
hr { border: none; border-top: 1px solid #ddd; margin: 3rem 0; }
/* ── Code ── */
code {
font-family: 'SFMono-Regular', 'Menlo', 'Consolas', 'DejaVu Sans Mono', monospace;
font-size: 0.88em;
background: #f0eee6;
padding: 0.15em 0.4em;
border-radius: 3px;
color: #5a3e1b;
}
pre {
background: #1e1e1e;
color: #d4d4d4;
padding: 1.25rem 1.5rem;
border-radius: 6px;
overflow-x: auto;
margin: 1.5rem 0;
font-size: 0.85rem;
line-height: 1.6;
}
pre code {
background: none;
padding: 0;
color: inherit;
font-size: inherit;
}
/* ── Tables ── */
table {
width: 100%;
border-collapse: collapse;
margin: 1.5rem 0;
font-size: 0.92rem;
}
thead th {
text-align: left;
font-family: 'Helvetica Neue', 'Arial', sans-serif;
font-weight: 600;
font-size: 0.82rem;
text-transform: uppercase;
letter-spacing: 0.06em;
color: #666;
border-bottom: 2px solid #d0cdc5;
padding: 0.6rem 0.75rem;
}
tbody td {
padding: 0.6rem 0.75rem;
border-bottom: 1px solid #e8e5dd;
vertical-align: top;
}
tbody tr:last-child td { border-bottom: none; }
tbody tr:hover { background: #f5f3ed; }
/* ── Lists ── */
ol, ul { margin: 1rem 0 1.25rem 1.75rem; }
li { margin-bottom: 0.4rem; }
/* ── Strong / Em ── */
strong { font-weight: 700; color: #111; }
em { font-style: italic; }
/* ── Utility ── */
.meta { color: #888; font-size: 0.88rem; margin-bottom: 2rem; font-style: italic; }
.footnote { font-size: 0.85rem; color: #777; margin-top: 3rem; border-top: 1px solid #ddd; padding-top: 1rem; }
/* ── Print ── */
@media print {
body { max-width: none; padding: 0; font-size: 11pt; background: white; }
pre { background: #f5f5f5; color: #333; border: 1px solid #ccc; }
a { color: inherit; text-decoration: underline; border-bottom: none; }
}
/* ── Responsive ── */
@media (max-width: 48rem) {
body { padding: 1.25rem 1rem 3rem; }
h1 { font-size: 1.65rem; }
h2 { font-size: 1.3rem; }
pre { font-size: 0.8rem; padding: 1rem; }
table { font-size: 0.85rem; }
thead th, tbody td { padding: 0.45rem 0.5rem; }
}
</style>
</head>
<body>
<h1>BTC.sh</h1>
<p class="meta">A technical deep-dive into BTC.sh 0.4.1 — the bare-metal toolchain build that produces forensically-stamped GCC cross-compilers for 19 target architectures, integrates with Gentoo, Source Mage, Lunar Linux, LEDE/OpenWrt, and generic Makefile workflows, and binds every binary to its origin through three tiers of cryptographic identity.</p>
<p>There is a specific kind of supply-chain paranoia that is not satisfied by verifying checksums against an upstream release artifact. It wants to know which physical machine compiled the binary, what microarchitecture it was optimized for, which version of the kernel headers were in play, and whether the build environment was the same one that produced every other binary in the deployment. It wants a forensic trail baked into the ELF binary itself — not in a log file that can be deleted, not in a database that can be altered, but in a <code>.note</code> section that survives strip operations, filesystem copies, and package manager reinstallations.</p>
<p>BTC.sh (Build Tool Chain) is a single 1300-line Bash script that does this. It builds a complete cross-compilation toolchain — binutils, two-stage GCC, C library (glibc or musl), kernel headers, libxcrypt, and an optional kernel binary — inside a ramfs mount, stamps every produced binary with forensic provenance, and packages the result as a golden image tarball with a machine-readable JSON manifest. It is the compiler build that feeds sorcery-go's Cauldron and Fester's distributed build nodes. This post explains how it works, how to use its output with five different build systems, and what the fingerprinting strategy actually buys you.</p>
<hr>
<h2>The Cleanroom and the Version Matrix</h2>
<p>BTC.sh builds inside a <strong>ramfs cleanroom</strong> — a volatile filesystem mounted at <code>/usr/src/DCOSNET-{LABEL}-cleanroom/</code> that is created fresh on every invocation and unmounted after packaging. This is not an aesthetic preference. Building on ramfs eliminates I/O wear on SSDs during multi-hour LTO builds (GCC Stage 2 with Link-Time Optimization on 28 threads can run for 45 minutes), guarantees a pristine environment with no stale files from previous runs, and ensures that the build cannot be contaminated by artifacts from the host filesystem. The cleanroom path encodes the full system identity: for a native haswell build, it becomes <code>/usr/src/DCOSNET-HASWELL-AVX2-LTO-cleanroom/</code> — the microarchitecture, ISA tier, and optimization mode are all in the directory name.</p>
<p>The upstream version matrix is defined as a flat set of variables near the top of the script, one per package. As of BTC 0.4.1, the current matrix targets these upstream releases:</p>
<table>
<thead>
<tr><th>Package</th><th>Version</th><th>Upstream</th></tr>
</thead>
<tbody>
<tr><td>Linux Kernel</td><td>7.1</td><td>cdn.kernel.org</td></tr>
<tr><td>Binutils</td><td>2.46.1</td><td>ftp.gnu.org</td></tr>
<tr><td>GCC</td><td>15.3.0</td><td>ftp.gnu.org</td></tr>
<tr><td>Glibc</td><td>2.43</td><td>ftp.gnu.org</td></tr>
<tr><td>Musl</td><td>1.2.6</td><td>musl.libc.org</td></tr>
<tr><td>GMP</td><td>6.3.0</td><td>ftp.gnu.org</td></tr>
<tr><td>MPFR</td><td>4.2.2</td><td>ftp.gnu.org</td></tr>
<tr><td>MPC</td><td>1.4.0</td><td>ftp.gnu.org</td></tr>
<tr><td>libxcrypt</td><td>4.5.2</td><td>github.com/besser82</td></tr>
</tbody>
</table>
<p>All source URLs are centralized in a single Bash associative array called <code>A_SRC_URL</code>, keyed by package name. This is the authoritative source of truth — change a URL here and every download, checksum, and decompress operation follows. The array maps each key to its upstream upstream mirror, constructing the filename from the version variable: <code>${v_binutils}.tar.xz</code> resolves to <code>binutils-2.46.1.tar.xz</code> and fetches from <code>ftp.gnu.org/gnu/binutils/</code>. The download function, <code>f_download()</code>, iterates this array and fetches each tarball with <code>wget -nc</code> (no-clobber, idempotent — safe to re-run). Every file is then validated with <code>_archive_sane()</code>, which runs <code>tar -tf</code> on the downloaded archive to confirm it is not truncated or corrupt. Files that fail validation are deleted and re-fetched automatically. Per-file md5 and sha512 checksums are written to <code>${LOGS}/checksums/</code> for audit trails.</p>
<hr>
<h2>Nineteen Targets, Five Families, Six ISA Tiers</h2>
<p>The target registry is a Bash associative array called <code>BTC_TARGETS</code> where each key is a target identifier and each value is a pipe-delimited specification string containing eleven fields: architecture, CPU, microarchitecture march, ISA tier, ABI, C library, endianness, family, human-readable description, minimum kernel version, and GCC-extra configure flags. The table-driven design follows PEP 868's pattern for dispatch tables and MISRA C's preference for data over control flow — the build logic never contains hardcoded architecture strings, only lookups into this registry.</p>
<p>The nineteen targets span five families:</p>
<table>
<thead>
<tr><th>Family</th><th>Targets</th><th>ISA Tier</th><th>C Library</th></tr>
</thead>
<tbody>
<tr><td>Intel HEDT/Server</td><td>haswell, haswell-ep, skylake, skylake-x, skylake-server</td><td>AVX2 / AVX512</td><td>glibc</td></tr>
<tr><td>AMD Ryzen/EPYC</td><td>znver1, znver2, znver3, znver4</td><td>AVX2 / AVX512</td><td>glibc</td></tr>
<tr><td>AMD APU (mobile)</td><td>apu-zn1, apu-zn2, apu-zn3, apu-zn4</td><td>AVX2</td><td>glibc</td></tr>
<tr><td>Intel Atom (embedded)</td><td>silvermont, goldmont, tremont, sierraforest</td><td>SSE4.2</td><td>glibc</td></tr>
<tr><td>Embedded</td><td>mipselr2, armv7, tilegx</td><td>MIPS32 / NEON / TILE</td><td>musl</td></tr>
</tbody>
</table>
<p>Target selection uses a step-down dispatch in <code>f_silicon_probe()</code>. If you invoke <code>BTC.sh skylake-x</code>, it resolves the target directly from the registry. If you invoke <code>BTC.sh</code> with no arguments or <code>--native</code>, it runs <code>gcc -march=native -Q --help=target</code> to probe the host CPU's microarchitecture, matches the result against the registry, and falls back to <code>haswell</code> if the probed architecture is not registered (Broadwell, for example, maps to Haswell). This probing is deterministic — the same hardware always produces the same target selection, which matters for reproducible builds.</p>
<p>Each target maps to a <strong>custom GCC triple</strong>: not the stock <code>x86_64-pc-linux-gnu</code>, but <code>x86_64-dcosnet-linux-gnu</code>. The <code>dcosnet</code> vendor string is deliberate — it prevents the BTC-built toolchain from colliding with any system-installed compiler, makes the triple identifiable in <code>readelf</code> and <code>file</code> output, and follows the GNU convention that the vendor field is a namespace for distribution-specific toolchains. For musl targets, the triple becomes <code>mipsel-dcosnet-linux-musl</code> or <code>arm-dcosnet-linux-musleabihf</code>, encoding the C library directly into the triple.</p>
<hr>
<h2>The Build Sequence</h2>
<p>The build executes a linear, ordered sequence — there are no concurrent build phases and no dependency resolution at runtime, because the dependency order of a bootstrap toolchain is deterministic and fixed. The sequence, implemented in <code>f_main()</code>, is:</p>
<ol>
<li><strong><code>f_setup</code></strong> — Mount the ramfs cleanroom, create the sysroot directory hierarchy (<code>bin/</code>, <code>usr/</code>, <code>lib/</code>, <code>include/</code>), and symlink <code>lib64/lib</code> to <code>lib</code> for multilib compatibility.</li>
<li><strong><code>f_download</code></strong> — Fetch all upstream source tarballs via <code>A_SRC_URL</code>, validate each with <code>_archive_sane()</code>, and write checksum manifests.</li>
<li><strong><code>f_sig_init</code></strong> — Initialize the forensic signature tier (cluster, TPM, or poly). This runs before any compilation so that the signature token is available to every subsequent <code>f_stamp_binary()</code> call.</li>
<li><strong><code>f_binutils</code></strong> — Build binutils (assembler, linker, object tools) for the target triple. This is the first package built because GCC Stage 1 needs a cross-assembler and cross-linker.</li>
<li><strong><code>f_kernel_headers</code></strong> — Install kernel headers into the sysroot's <code>usr/include/</code>. These headers define the kernel ABI that the C library and all user-space code will be compiled against.</li>
<li><strong><code>f_gcc_p1</code></strong> — Build GCC Stage 1 (C compiler only, no C++ or libstdc++). GMP, MPFR, and MPC are decompressed and nested inside the GCC source tree as <code>gmp/</code>, <code>mpfr/</code>, <code>mpc/</code> for build isolation. Architecture-specific patches are applied here — on x86_64, the dynamic linker path in <code>gcc/config/i386/t-linux64</code> is rewritten from <code>lib64</code> to <code>lib</code> to match the sysroot layout.</li>
<li><strong><code>f_clib</code></strong> — Build the C library. For glibc targets, this runs the full glibc configure/make/install into the sysroot. For musl targets, it builds musl as a static+shared combo. The dispatch is driven by the <code>BTC_T_CLIB</code> field from the target registry — no conditional logic in the build function itself.</li>
<li><strong><code>f_libxcrypt</code></strong> — Build libxcrypt (extended crypt library for bcrypt, scrypt, yescrypt). Skipped entirely for musl targets since musl has built-in crypt support. This is gated by a simple check: if <code>BTC_T_CLIB == "musl"</code>, print a skip message and return.</li>
<li><strong><code>f_gcc_p2</code></strong> — Build GCC Stage 2 (full compiler: C, C++, Fortran, with LTO, PIE, and Stack Smashing Protection enabled). This stage links against the just-built C library, so every binary it produces is natively bound to the target's runtime. GCC is configured with <code>--with-arch={march}</code>, <code>--with-cpu={march}</code>, <code>--enable-lto</code>, <code>--enable-default-pie</code>, <code>--enable-default-ssp</code>, and the architecture-specific extra flags from the target registry.</li>
<li><strong><code>f_kernel_binary</code></strong> — Cross-compile a kernel binary for the target architecture. The appropriate <code>defconfig</code> is selected per-arch (<code>multi_v7_defconfig</code> for ARM, <code>malta_defconfig</code> for MIPS, <code>tilegx_defconfig</code> for Tile-Gx, plain <code>defconfig</code> for x86_64). Enterprise hardening flags are injected (<code>CONFIG_MODULES=n</code>, <code>CONFIG_KALLSYMS=n</code>, <code>CONFIG_DEBUG_FS=n</code>), the kernel is stamped with <code>CONFIG_LOCALVERSION=-dcosnet-{LABEL}</code>, and cross-compiled with <code>ARCH={arch} CROSS_COMPILE={triple}-</code>. After the build, every binary in the sysroot is stamped with <code>f_stamp_binary()</code>.</li>
<li><strong><code>f_package</code></strong> — Package the entire sysroot as a golden image tarball (<code>{SYS_LABEL}-toolchain-golden.tar.xz</code>) with a JSON manifest sidecar containing the full build metadata.</li>
</ol>
<p>Every build function logs its configure and make output to timestamped files under <code>${LOGS}/</code>, making it possible to reconstruct the exact build sequence after the fact. A tmux dashboard provides live build progress if the script is run inside a tmux session.</p>
<hr>
<h2>Forensic Stamping: The Silicon Birth Certificate</h2>
<p>Every ELF binary produced by the build — compilers, assemblers, linkers, kernel image, and every binary in the sysroot — receives two layers of forensic identification through <code>f_stamp_binary()</code>. This function is called in a <code>find</code> loop that walks <code>${NEWROOT}/bin/</code> and <code>${NEWROOT}/usr/bin/</code>, so nothing escapes stamping.</p>
<h3>Layer 1: ELF NOTE Section (<code>.note.BTC</code>)</h3>
<p>The first layer injects an ELF NOTE section named <code>.note.BTC</code> into each binary. This is done by assembling a small object file with a <code>.note</code> section containing structured key-value pairs, then using <code>objcopy --add-section</code> to merge it into the target binary. The note payload contains: the organization identifier (<code>DCOSNET</code>), the kernel version used for headers (<code>K:${v_linux}</code>), the target microarchitecture ID (<code>Arch:${BTC_T_ID}</code>), the full system label (<code>Label:${SYS_LABEL}</code>), the build log filename (<code>Stage:${log_base}</code>), the active signature tier (<code>SigTier:${BTC_SIG_TIER}</code>), and the cryptographic signature token (<code>Sig:${BTC_SIG_TOKEN}</code>).</p>
<p>This data is readable at any time with <code>readelf -n /path/to/binary</code>. It survives <code>strip</code> operations because <code>objcopy --strip-unneeded</code> is applied <em>after</em> the note section is injected, and note sections are not stripped by default. It survives package manager reinstallations because the package was built with the note already embedded. It survives filesystem copies because it is in the ELF binary, not in a sidecar file. The assembly source is straightforward:</p>
<pre><code>.section .note.BTC,"a",@note
.long 2f - 1f /* namesz */
.long 4f - 3f /* descsz */
.long 1 /* type */
1: .asciz "DCOSNET" /* owner */
2: .align 4
3: .ascii "Org: dcos.net|K:linux-7.1|Arch:haswell|Label:DCOSNET-HASWELL-AVX2-LTO|..."
4: .align 4</code></pre>
<h3>Layer 2: Extended Filesystem Attributes</h3>
<p>The second layer writes four extended attributes on every stamped binary using <code>setfattr</code>: <code>user.btc.identity</code> (the full system label and tier), <code>user.btc.hash</code> (the SHA-256 hash of the binary contents), <code>user.btc.sig.tier</code> (the active signature tier name), and <code>user.btc.sig.token</code> (the cryptographic token). These attributes are queryable with <code>getfattr -d /path/to/binary</code> and persist independently of the ELF file — they survive renames, hardlinks (on the same filesystem), and are preserved by <code>cp --preserve=xattr</code>. They are lost by <code>cp</code> without attribute preservation, by tar archives that do not store xattrs, and by filesystems that do not support extended attributes (notably, some tmpfs configurations and older FAT32 mounts).</p>
<p>The dual-layer approach provides defense in depth. The ELF NOTE is tamper-evident within the binary itself — any modification to the binary changes its SHA-256 hash, which a verification step can detect. The xattrs provide a second, independent verification channel at the filesystem level. If the ELF NOTE is present but the xattr hash does not match the file's current SHA-256, something modified the binary after stamping. If the xattrs are missing entirely, the binary was copied without attribute preservation. This is not a cryptographic signature scheme — there is no asymmetric key involved — but it is a practical, zero-dependency provenance system that works on any Linux system with <code>objcopy</code> and <code>setfattr</code>.</p>
<h3>Debug Symbols</h3>
<p>After stamping, <code>f_stamp_binary()</code> extracts debug symbols into a separate <code>.debug</code> file stored under <code>${BTC_ARCHIVE}/symbols/${SYS_LABEL}/</code>, then strips the binary with <code>--strip-unneeded</code>. The debug symbols link back to the original binary path via the debug link section. This means you get full debug info for GDB without bloating the production binary, and the debug symbol files are also stamped and stored centrally.</p>
<hr>
<h2>Three Tiers of Cryptographic Identity</h2>
<p>The <code>SigTier</code> and <code>Sig</code> fields in the forensic stamp are populated by <code>f_sig_init()</code>, which implements a three-tier identity system. The tier selection follows a strict priority: cluster-join overrides TPM, TPM overrides poly, and poly is the default. This is not a negotiation — it is a deterministic cascade that runs once at the start of the build and produces a single signature token that is folded into every binary.</p>
<h3>Tier 1: Cluster</h3>
<p>A cluster deployment shares a single identity token across multiple build machines. When <code>BTC.sh --join-cluster=TOKEN skylake-x</code> is invoked, the provided token is written to <code>${BTC_ARCHIVE}/.btc-cluster-token</code> and adopted as <code>BTC_SIG_TOKEN</code>. On subsequent builds, the file is detected automatically — <code>f_sig_init()</code> checks for the cluster token file before falling through to TPM or poly — and the same token is reused. This means that every binary produced by any machine in the cluster carries the same <code>Sig:</code> field, making it possible to identify cluster membership by inspecting a single binary with <code>readelf -n</code>.</p>
<p>The token itself is an arbitrary string provided by the operator. In a Fester-managed cluster, this token is typically distributed via the cluster configuration and stored in each node's <code>/opt/BTC/</code> directory. sorcery-go's <code>pkg/cluster/cluster.go</code> reads the BTC manifest to extract the tier and token for cluster-wide provenance queries.</p>
<h3>Tier 2: TPM</h3>
<p>TPM-bound signatures tie the build identity to the physical hardware. When <code>BTC.sh --tpm-seal</code> is invoked, <code>f_sig_init()</code> calls <code>f_tpm_pcr_digest()</code>, which attempts to read Platform Configuration Registers (PCRs) 0 through 7 — the registers that cover BIOS, firmware, bootloader, and early boot configuration — from the system's TPM. For TPM 2.0, it uses <code>tpm2_pcrread sha256:0,1,2,3,4,5,6,7</code> and hashes the output to produce a 256-bit digest. For TPM 1.2, it reads <code>/sys/class/tpm/tpm0/pcrs</code> and hashes the file contents. The resulting digest is prefixed with <code>tpm:</code> and stored as the signature token.</p>
<p>This creates a hardware-bound identity: the same BTC.sh build run on different physical machines (even identical models) will produce different signature tokens because the PCR state includes platform-specific firmware measurements. If the BIOS is flashed, the secure boot state changes, or the TPM is cleared, the PCR digest will change and subsequent builds will produce a different token — making it possible to detect hardware state changes by comparing the <code>Sig:</code> field across binaries. TPM sealing is restricted to singular deployments (one target per machine) because the PCR state is a single global value, not per-target.</p>
<h3>Tier 3: Poly (Default)</h3>
<p>The poly tier is the default when no cluster token is provided and <code>--tpm-seal</code> is not specified. It generates a random 128-bit hex string via <code>openssl rand -hex 16</code> and writes it to <code>${BTC_ARCHIVE}/.btc-salt</code>. On subsequent builds, the existing salt file is read and reused — the token is write-once, never re-rolled. This means that two independent machines running the same <code>BTC.sh skylake-x</code> build will produce forensically distinct toolchains: the binaries will be bit-identical (same GCC, same flags, same source), but the <code>Sig:</code> field in the ELF NOTE will differ, making it possible to determine which physical machine produced any given binary.</p>
<p>The poly tier is the simplest identity layer, but it is sufficient for most deployments. It answers the question "which machine built this?" without requiring any hardware support or cluster coordination. The salt file persists under <code>/opt/BTC/</code>, so it survives reboots and rebuilds.</p>
<hr>
<h2>Integration with External Build Systems</h2>
<p>The golden image produced by BTC.sh is a self-contained sysroot — it includes the cross-compiler, linker, assembler, C library headers, kernel headers, and (optionally) a kernel binary. The JSON manifest sidecar describes every property of the toolchain: target triple, microarchitecture, ISA tier, optimization flags, C library version, and signature metadata. This manifest is how external build systems discover and configure the toolchain. The following sections describe five integration patterns, each grounded in how real distributions consume cross-compilers.</p>
<h3>Gentoo</h3>
<p>Gentoo's cross-compilation support revolves around the <code>CROSS_COMPILE</code> environment variable and the <code>crossdev</code> utility. A BTC-built toolchain integrates by pointing Gentoo's <code>CBUILD</code>, <code>CHOST</code>, <code>CTARGET</code>, <code>CC</code>, <code>CXX</code>, <code>AR</code>, <code>NM</code>, <code>RANLIB</code>, and <code>STRIP</code> variables at the BTC sysroot. The golden image tarball is extracted to a stable path (e.g., <code>/opt/btc/skylake-x/</code>), and the target triple from the manifest (<code>x86_64-dcosnet-linux-gnu</code>) is used as <code>CTARGET</code>. Gentoo's <code>make.conf</code> entries for the cross-build would look like this:</p>
<pre><code>ROOT="/opt/btc/skylake-x/"
CBUILD="x86_64-pc-linux-gnu"
CHOST="x86_64-dcosnet-linux-gnu"
CC="${ROOT}/bin/x86_64-dcosnet-linux-gnu-gcc"
CXX="${ROOT}/bin/x86_64-dcosnet-linux-gnu-g++"
AR="${ROOT}/bin/x86_64-dcosnet-linux-gnu-ar"
NM="${ROOT}/bin/x86_64-dcosnet-linux-gnu-nm"
RANLIB="${ROOT}/bin/x86_64-dcosnet-linux-gnu-ranlib"
STRIP="${ROOT}/bin/x86_64-dcosnet-linux-gnu-strip"
CFLAGS="-O2 -march=skylake-avx512 -mtune=skylake-avx512 -pipe"
CXXFLAGS="${CFLAGS}"
LDFLAGS="-Wl,-O2 -Wl,--as-needed"
PKG_CONFIG_SYSROOT_DIR="${ROOT}"
PKG_CONFIG_LIBDIR="${ROOT}/usr/lib/pkgconfig:${ROOT}/usr/share/pkgconfig"</code></pre>
<p>The BTC toolchain's advantage over <code>crossdev</code> is microarchitecture specificity. <code>crossdev</code> builds a generic <code>x86_64-pc-linux-gnu</code> toolchain — it does not distinguish between Skylake and Haswell, between AVX512 and AVX2. A BTC-built Skylake-X toolchain defaults to <code>-march=skylake-avx512</code> at the compiler level, so every package built with it (without explicit <code>CFLAGS</code> overrides) is optimized for that specific microarchitecture. For Gentoo users building for specific hardware — a fleet of Skylake-SP servers, or a cluster of Zen4 EPYC nodes — this eliminates the need to maintain per-architecture <code>make.conf</code> fragments. The compiler does the right thing by default.</p>
<h3>Source Mage GNU/Linux</h3>
<p>Source Mage is the direct ancestor of sorcery-go's spell format. Integration with BTC.sh operates through sorcery-go's <code>pkg/toolchain/btc.go</code> module, which probes for the golden image tarball, validates its <code>.note.BTC</code> section with <code>readelf -n</code>, and parses the forensic stamp into a <code>BTCStamp</code> struct. When sorcery-go's Cauldron builds a spell for a specific architecture, it uses the BTC stamp's target triple and ISA tier to configure the build environment automatically — <code>CC</code>, <code>CXX</code>, <code>CFLAGS</code>, and <code>LDFLAGS</code> are all derived from the stamp.</p>
<p>For standalone Source Mage installations (without sorcery-go), the integration is manual but straightforward. The golden image is extracted to a path like <code>/opt/btc/</code>, and Source Mage's <code>CAST_ARGS</code> or spell-level <code>CONFIGURE</code> scripts set the cross-compiler variables. The BTC manifest JSON can be parsed with <code>jshn</code> or <code>jq</code> to extract the target triple and optimization flags. Because Source Mage spells are just Bash scripts, they can source the manifest directly:</p>
<pre><code>BTC_MANIFEST="/opt/btc/DCOSNET-SKYLAKE-X-AVX512-LTO-manifest.json"
TARGET_TRIPLE=$(jq -r .target_triple "$BTC_MANIFEST")
MARCH=$(jq -r .target_march "$BTC_MANIFEST")
export CC="/opt/btc/DCOSNET-SKYLAKE-X-AVX512-LTO/bin/${TARGET_TRIPLE}-gcc"
export CFLAGS="-O2 -march=${MARCH} -mtune=${MARCH} -pipe"</code></pre>
<h3>Lunar Linux</h3>
<p>Lunar Linux uses a Bash-based module system where each package has a <code>BUILD</code> script that runs in a chrooted environment. The cross-compilation entry point is the <code>CROSS_COMPILE</code> variable, which Lunar's build engine prefixes onto tool names. A BTC-built toolchain integrates by setting the compiler path and flags in Lunar's <code>lunar.conf</code> or per-module overrides. The key difference from Gentoo is that Lunar's build isolation is stronger — each module builds in its own chroot — so the sysroot must be fully self-contained, which the BTC golden image is by design.</p>
<p>The integration pattern is to extract the golden image to a known location, set <code>HOST</code> and <code>HOST_PREFIX</code> in the lunar configuration, and let the module build system discover the cross-compiler. The BTC manifest provides the triple prefix, and the ELF NOTE stamping means that every binary Lunar produces can be traced back to the BTC build that built the compiler — even if the compiler was installed months ago and the build logs have been rotated.</p>
<h3>LEDE / OpenWrt</h3>
<p>LEDE and OpenWrt are the primary consumers of embedded cross-compilers. They expect a standard <code>{triple}-</code> prefixed toolchain in <code>staging_dir/toolchain-{arch}/</code> and use <code>ARCH={arch} CROSS_COMPILE={triple}-</code> as Make variables passed to every <code>make</code> invocation. BTC.sh's target registry includes LEDE-relevant targets directly: <code>mipselr2</code> (MIPS32R2 little-endian, matching the MALTA reference platform used by most LEDE router profiles), <code>armv7</code> (ARMv7-A hard-float NEON, covering RPi 2/3 32-bit, BeagleBone, Odroid, and virtually every Cortex-A SoC), and the Intel Atom family (silvermont, goldmont, tremont, sierraforest) for x86_64 embedded gateways and routers.</p>
<p>Integration with OpenWrt's build system (<code>openwrt/Makefile</code> and <code>rules.mk</code>) is done by overriding <code>CONFIG_TARGET_OPTERON</code> paths or by symlinking the BTC sysroot into <code>staging_dir/</code>. The <code>.config</code> fragment for a MIPS LEDE build using a BTC-built toolchain would set:</p>
<pre><code>CONFIG_TARGET_ARCH="mipsel"
CONFIG_TARGET_OPTERON=""
CONFIG_TOOLCHAIN_ROOT="/opt/btc/DCOSNET-MIPSELR2-MIPS32-LTO"
CONFIG_TOOLCHAIN_PREFIX="mipsel-dcosnet-linux-musl-"
CROSS_COMPILE="${CONFIG_TOOLCHAIN_ROOT}/bin/${CONFIG_TOOLCHAIN_PREFIX}"</code></pre>
<p>BTC.sh's musl targets are particularly relevant here — OpenWrt moved to musl as its default C library years ago, and the <code>mipselr2</code> and <code>armv7</code> targets produce musl-based toolchains that match OpenWrt's expectations. The forensic stamping provides an additional benefit in the embedded context: when a firmware image is deployed to a router, the <code>.note.BTC</code> section in every binary on the filesystem makes it possible to audit which toolchain produced the firmware, which is valuable for supply-chain verification and compliance audits.</p>
<h3>Generic Makefile Workflows</h3>
<p>For projects that are not distribution-specific — standalone C/C++ projects, embedded firmware builds, or CI/CD pipelines — the BTC toolchain integrates through standard environment variables. The golden image is extracted, and the <code>PATH</code>, <code>CC</code>, <code>CXX</code>, <code>CFLAGS</code>, and <code>LDFLAGS</code> are set to point at the BTC sysroot. The manifest JSON provides all the metadata needed for automation:</p>
<pre><code>#!/bin/bash
SYSROOT="/opt/btc/DCOSNET-ZNVER4-AVX512-LTO"
MANIFEST="${SYSROOT%/*}/${SYSROOT##*/}-manifest.json"
TRIPLE=$(jq -r .target_triple "$MANIFEST")
MARCH=$(jq -r .target_march "$MANIFEST")
CLIB=$(jq -r .clib "$MANIFEST")
export PATH="${SYSROOT}/bin:${PATH}"
export CC="${TRIPLE}-gcc"
export CXX="${TRIPLE}-g++"
export CFLAGS="-O2 -march=${MARCH} -mtune=${MARCH} -pipe"
export CXXFLAGS="${CFLAGS}"
export PKG_CONFIG_SYSROOT_DIR="${SYSROOT}"
export PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig"
make CROSS_COMPILE="${TRIPLE}-" ARCH=$(jq -r .target_arch "$MANIFEST")</code></pre>
<p>The BTC manifest also includes the integration keys for sorcery-go and Fester, making it possible for automated systems to discover and configure the toolchain without human intervention. The <code>integrations.sorcery-go</code> block specifies the environment variables that sorcery-go expects (<code>SORCERY_GO_BTC_PATH</code>, <code>SORCERY_GO_BTC_ROOT</code>, <code>SORCERY_GO_BTC_SYS_LABEL</code>), and the <code>integrations.fester</code> block specifies the YAML configuration keys that Fester uses to enable BTC toolchain support on a per-node basis.</p>
<hr>
<h2>Security Properties Derived from the Fingerprinting Strategy</h2>
<p>The three-tier identity system and dual-layer forensic stamping are not security features in the cryptographic sense — they do not prevent an attacker from modifying a binary, and they do not provide integrity verification through digital signatures. What they provide is <strong>supply-chain provenance</strong>: the ability to answer, with high confidence, questions about the origin and consistency of a binary artifact. The specific security properties are:</p>
<h3>Origin Attribution</h3>
<p>Given any binary produced by a BTC-built toolchain, <code>readelf -n</code> reveals the exact build machine (via the signature tier and token), the target microarchitecture, the kernel version used for headers, and the system label. This makes it possible to trace a binary back to the specific BTC invocation that produced it. In a cluster deployment, the shared cluster token identifies which cluster produced the binary. In a TPM-sealed deployment, the PCR digest identifies the specific physical machine and its firmware state at build time. In a poly deployment, the per-machine salt identifies the specific host.</p>
<h3>Build Consistency Verification</h3>
<p>The <code>user.btc.hash</code> xattr records the SHA-256 hash of the binary at stamp time. By recomputing the hash and comparing it to the stored value, a verification script can detect any post-stamping modification to the binary. This catches both accidental corruption (bit rot, truncated copies) and deliberate tampering (modified binaries, injected code). The verification is trivial: <code>sha256sum /path/to/binary | awk '{print $1}'</code> compared against <code>getfattr -n user.btc.hash --only-values /path/to/binary</code>. If they differ, the binary has been modified since it left the build.</p>
<h3>Cluster Identity Assurance</h3>
<p>In a multi-node deployment where all machines use the same cluster token, the <code>Sig:</code> field in the ELF NOTE provides a quick cluster-membership test. Any binary from the cluster will carry the same token. A binary with a different (or missing) token either came from outside the cluster, was built before the machine joined the cluster, or was built with a different toolchain entirely. This is particularly useful in Fester-managed clusters, where nodes may join and leave over time — the signature token provides a persistent identity that survives node churn.</p>
<h3>Hardware State Change Detection (TPM)</h3>
<p>The TPM tier provides a hardware-bound identity that changes if the platform configuration changes. PCR registers 0-7 cover the BIOS, firmware, bootloader configuration, and secure boot state. If any of these change — a BIOS update, a firmware flash, a secure boot key rotation — the PCR digest will differ, and the next BTC build will produce a different signature token. This provides a passive detection mechanism: by comparing the <code>Sig:</code> fields across binaries built at different times, an operator can determine whether the underlying hardware state changed between builds, without needing to separately audit the TPM state.</p>
<h3>Supply Chain Audit Trail</h3>
<p>The JSON manifest sidecar provides a machine-readable record of every build parameter: package versions, target triple, optimization flags, C library choice, signature tier, and token. When combined with the per-binary ELF NOTE data and the checksum manifests from <code>f_download()</code>, this creates a complete audit trail from upstream source tarball to final binary. An auditor can verify that the binutils version in the manifest matches the checksum on disk, that the GCC flags match the expected microarchitecture, and that every binary in the sysroot carries the expected <code>.note.BTC</code> section. This is not a formal SBOM (Software Bill of Materials) in the SPDX or CycloneDX sense, but it contains the same information in a format that is directly queryable from the binaries themselves.</p>
<hr>
<h2>Decompression and Packaging</h2>
<p>BTC.sh provides three utility functions that form the acquire-and-package lifecycle. <code>f_download()</code> fetches and validates upstream source tarballs. <code>f_decompress()</code> auto-detects the compression format from the file extension and dispatches to the appropriate tool: <code>tar -axf</code> for xz, <code>tar -xzf</code> for gzip, <code>tar -xjf</code> for bzip2, <code>tar -axf</code> for lzip, <code>tar -x -I lrzip</code> for lrzip, and <code>unzip</code> for zip archives. It accepts either a bare filename (looked up in <code>SOURCE_CACHE</code>) or an absolute path, making it usable both during the build sequence and for ad-hoc operations. <code>f_compress()</code> provides the inverse operation, stepping down through a <code>USE_COMPRESSOR</code> variable to select the final packaging algorithm — xz (maximum, default), gzip, bzip2, lz, or lrzip.</p>
<p>The decompression function is called at every point in the build sequence where source extraction is needed — binutils, kernel headers, GCC (twice, for each stage), glibc, musl, libxcrypt, and the support libraries (GMP, MPFR, MPC). Each call is a single line: <code>f_decompress "${v_binutils}.tar.xz"</code>. The function resolves the file from the source cache, changes to the ramfs cleanroom, and extracts. There are no hardcoded <code>tar</code> commands anywhere in the build functions — every extraction goes through <code>f_decompress()</code>, which means adding support for a new compression format requires changing exactly one case branch.</p>
<hr>
<h2>The Manifest and Downstream Consumption</h2>
<p>The JSON manifest is the integration contract. It is written at the end of <code>f_package()</code> and contains every piece of metadata that an external system needs to configure the toolchain: the target triple, the microarchitecture march string, the ISA tier, the C library and its version, the full CFLAGS and LDFLAGS used during the build, the signature tier and token, the paths to the forensic stamp locations (both ELF NOTE and xattr names), and explicit integration keys for sorcery-go and Fester. The manifest filename encodes the system label: <code>DCOSNET-SKYLAKE-X-AVX512-LTO-manifest.json</code>.</p>
<p>Sorcery-go's <code>pkg/toolchain/btc.go</code> reads this manifest to populate its <code>BTCStamp</code> struct. Fester's <code>backend/toolchain/btc.py</code> reads it to configure per-node build environments. A standalone user can <code>jq</code> it to extract whatever they need. The manifest is the reason the golden image is not just a tarball — it is a self-describing artifact that carries its own configuration documentation.</p>
<p class="footnote"><em>BTC.sh is developed by dcos.net and released under AGPL-3.0-or-later. The reference implementation discussed here is version 0.4.1. Source Mage GNU/Linux is developed at sourcemage.org under its own project governance. Gentoo, Lunar Linux, LEDE, and OpenWrt are the respective trademarks of their communities.</em></p>
</body></html>
Reference in New Issue View Git Blame Copy Permalink