BTC.sh 0.4.1

This commit is contained in:
Jeremy Anderson 2026-07-24 17:22:38 -04:00
parent 712241660c
commit ea83eac47e
4 changed files with 6 additions and 6 deletions

2
BTC.sh
View File

@ -628,7 +628,7 @@ EOF
# Associative array: package stem -> upstream URL
# Only packages consumed by the build sequence are listed.
# URLs are canonical upstream mirrors — replace to point at a local mirror.
# URLs are official upstream mirrors — replace to point at a local mirror.
declare -A A_SRC_URL=(
[binutils]="https://ftp.gnu.org/gnu/binutils/${v_binutils}.tar.xz"
[linux]="https://cdn.kernel.org/pub/linux/kernel/v7.x/${v_linux}.tar.xz"

View File

@ -135,9 +135,9 @@ BTC.sh's security model operates at three levels: supply chain control, build en
### Supply Chain Control: Single Source of Truth
All upstream source URLs are defined in a single associative array, `A_SRC_URL`, at the top of the script. This array maps package stems (binutils, linux, gcc, glibc, musl, gmp, mpfr, mpc, libxcrypt) to their canonical upstream URLs. There is no URL discovery, no mirror fallback, no dynamic URL construction. Every URL is a literal string pointing at a known upstream mirror: GNU FTP for binutils, GCC, GMP, MPFR, and MPC; kernel.org for the Linux kernel; musl.libc.org for musl; and GitHub (besser82) for libxcrypt. The `f_download` function iterates over this array, checks each file for existence and integrity, and fetches anything missing.
All upstream source URLs are defined in a single associative array, `A_SRC_URL`, at the top of the script. This array maps package stems (binutils, linux, gcc, glibc, musl, gmp, mpfr, mpc, libxcrypt) to their official upstream URLs. There is no URL discovery, no mirror fallback, no dynamic URL construction. Every URL is a literal string pointing at a known upstream mirror: GNU FTP for binutils, GCC, GMP, MPFR, and MPC; kernel.org for the Linux kernel; musl.libc.org for musl; and GitHub (besser82) for libxcrypt. The `f_download` function iterates over this array, checks each file for existence and integrity, and fetches anything missing.
The integrity check is performed by `_archive_sane`, which runs `tar -tf` on the cached file and returns failure if the archive cannot be listed. This is not a cryptographic hash verification; it is a structural integrity check that catches truncated downloads, gzip unexpected-end-of-file errors, and corrupted xz streams. The rationale for not using hash verification at download time is that the upstream tarballs are already trusted by virtue of being fetched from their canonical sources over HTTPS, and a structural integrity check provides a stronger guarantee against corruption than a hash comparison against a hardcoded value (which could itself be wrong if the upstream release is re-uploaded). After download, `f_download` generates both MD5 and SHA-512 checksums of each file, written to per-file manifests under `${LOGS}/checksums/`. These checksums are for auditing and comparison, not for gate-keeping.
The integrity check is performed by `_archive_sane`, which runs `tar -tf` on the cached file and returns failure if the archive cannot be listed. This is not a cryptographic hash verification; it is a structural integrity check that catches truncated downloads, gzip unexpected-end-of-file errors, and corrupted xz streams. The rationale for not using hash verification at download time is that the upstream tarballs are already trusted by virtue of being fetched from their official sources over HTTPS, and a structural integrity check provides a stronger guarantee against corruption than a hash comparison against a hardcoded value (which could itself be wrong if the upstream release is re-uploaded). After download, `f_download` generates both MD5 and SHA-512 checksums of each file, written to per-file manifests under `${LOGS}/checksums/`. These checksums are for auditing and comparison, not for gate-keeping.
### Build Environment Isolation: Ramfs and Path Control
@ -187,7 +187,7 @@ This design means that sorcery-go and Fester do not need to share a database, a
## Putting It All Together
The canonical workflow ties the stack into a single pipeline. A spell in the SMGL-compatible grimoire describes a package — its source, dependencies, and build steps. BTC.sh has built a microarchitecture-optimized GCC toolchain for the target architecture, stamped with forensic provenance. Sorcery-go reads the spell, resolves its dependency graph (DAG), and checks the shared CAS for cached artifacts. If the artifact is not cached, sorcery-go delegates the build to Fester via its HTTP API. Fester's scheduler picks the best node based on CPU, thermal, cache, and policy. Fester configures the build environment using the BTC.sh toolchain on that node — setting CC, CXX, CFLAGS, and LDFLAGS automatically via the chroot provider layer. The build executes in an isolated runtime (LXC, Podman, Firecracker, or bare-metal) with eBPF warding active. The resulting binary is verified by sorcery-go's Warding (Merkle root recomputation, BTC stamp validation), sealed as a content-addressable Essence in the Tomb, and pushed to the shared CAS for future deduplication. Target nodes are atomically updated via reflink or hardlink swaps. On first execution, the Warding verifies the Essence again — any mismatch triggers quarantine.
The standard workflow ties the stack into a single pipeline. A spell in the SMGL-compatible grimoire describes a package — its source, dependencies, and build steps. BTC.sh has built a microarchitecture-optimized GCC toolchain for the target architecture, stamped with forensic provenance. Sorcery-go reads the spell, resolves its dependency graph (DAG), and checks the shared CAS for cached artifacts. If the artifact is not cached, sorcery-go delegates the build to Fester via its HTTP API. Fester's scheduler picks the best node based on CPU, thermal, cache, and policy. Fester configures the build environment using the BTC.sh toolchain on that node — setting CC, CXX, CFLAGS, and LDFLAGS automatically via the chroot provider layer. The build executes in an isolated runtime (LXC, Podman, Firecracker, or bare-metal) with eBPF warding active. The resulting binary is verified by sorcery-go's Warding (Merkle root recomputation, BTC stamp validation), sealed as a content-addressable Essence in the Tomb, and pushed to the shared CAS for future deduplication. Target nodes are atomically updated via reflink or hardlink swaps. On first execution, the Warding verifies the Essence again — any mismatch triggers quarantine.
Each project in this stack is independently useful. You can run sorcery-go standalone without Fester or BTC.sh. You can run Fester as a general-purpose distributed build system without sorcery-go. You can run BTC.sh to build toolchains without either of the other two. But the integrations are real, load-bearing code paths — not afterthoughts or loose couplings — and they transform the projects from a collection of tools into a coherent infrastructure management system.

View File

@ -19,7 +19,7 @@ executes in four phases:
available RAM to prevent LTO thrashing. The target registry is loaded.
2. **Setup** — A volatile cleanroom (ramfs) is provisioned at the configured
mount point. Source tarballs are verified against their SHA-256 checksums.
3. **Build** — Core components are built sequentially:
3. **STOP USING THIS WORD >>>Build** — Core components are built sequentially:
Binutils → Kernel Headers → GCC Stage 1 → C Library (glibc or musl) → GCC Stage 2 → Kernel.
Both GCC stages are configured with `--with-arch=<march>` and `--with-cpu=<march>`
to default to the target microarchitecture.

View File

@ -155,7 +155,7 @@
</tbody>
</table>
<p>All source URLs are centralized in a single Bash associative array called <code>A_SRC_URL</code>, keyed by package name. This is the authoritative source of truth — change a URL here and every download, checksum, and decompress operation follows. The array maps each key to its canonical upstream mirror, constructing the filename from the version variable: <code>${v_binutils}.tar.xz</code> resolves to <code>binutils-2.46.1.tar.xz</code> and fetches from <code>ftp.gnu.org/gnu/binutils/</code>. The download function, <code>f_download()</code>, iterates this array and fetches each tarball with <code>wget -nc</code> (no-clobber, idempotent — safe to re-run). Every file is then validated with <code>_archive_sane()</code>, which runs <code>tar -tf</code> on the downloaded archive to confirm it is not truncated or corrupt. Files that fail validation are deleted and re-fetched automatically. Per-file md5 and sha512 checksums are written to <code>${LOGS}/checksums/</code> for audit trails.</p>
<p>All source URLs are centralized in a single Bash associative array called <code>A_SRC_URL</code>, keyed by package name. This is the authoritative source of truth — change a URL here and every download, checksum, and decompress operation follows. The array maps each key to its upstream upstream mirror, constructing the filename from the version variable: <code>${v_binutils}.tar.xz</code> resolves to <code>binutils-2.46.1.tar.xz</code> and fetches from <code>ftp.gnu.org/gnu/binutils/</code>. The download function, <code>f_download()</code>, iterates this array and fetches each tarball with <code>wget -nc</code> (no-clobber, idempotent — safe to re-run). Every file is then validated with <code>_archive_sane()</code>, which runs <code>tar -tf</code> on the downloaded archive to confirm it is not truncated or corrupt. Files that fail validation are deleted and re-fetched automatically. Per-file md5 and sha512 checksums are written to <code>${LOGS}/checksums/</code> for audit trails.</p>
<hr>