scuttle/blog.md

144 lines
6.8 KiB
Markdown
Executable File

# scuttle v1.0 — A Modern Data Sanitization Framework in Rust
**August 2026**
DBAN and nwipe served the ITAD community for two decades and inspired this project. They work, they
are trusted, and they produce results that pass audit. But the C codebase
has accumulated decades of patches, the option parsing predates modern CLI
conventions, and adding a new PRNG or wipe method requires touching five
files across the wipe engine, the method table, the options parser, the
GUI, and the help text.
Scuttle is a from-scratch Rust rewrite that preserves the operator
contract — boot, detect drives, securely erase everything — while adding
a policy engine, modern cryptographic primitives, firmware-level erase,
TPM-bound crypto erase, free-space-only mode, SMART monitoring, and six
audit certificate formats with Ed25519 signing.
## What shipped
Scuttle v1.0 contains 24 Rust crates, 150 passing tests, and 12,000 lines
of code. Every layer of the architectural manifest has an implementation:
- **12 PRNG providers** with KAT self-tests: ChaCha20, AES-256-CTR,
ISAAC-64 (bit-exact port of Bob Jenkins' C reference), BLAKE3-XOF,
XChaCha20, SHAKE128, SHAKE256, Salsa20, MT19937, XOROSHIRO-256,
SplitMix64, Lagged Fibonacci.
- **4 hash providers**: SHA-256, SHA-512, BLAKE2b-512, BLAKE3-256.
- **10 wipe methods**: Zero, One, PRNG Stream, DoD 5220.22-M, DoD Short,
Gutmann 35-pass, RCMP TSSIT OPS-II, HMG IS5 Enhanced, Schneier 7-Pass,
BMB21-2019.
- **20 profiles** (9 legacy + 11 modern): Quick Clear, Modern Random,
NIST Clear, NIST Purge, Enterprise, Paranoid, Research, Forensic,
Government, Air Gap, Custom.
- **Policy engine**: maps (device media class, operator intent) to a wipe
plan. The policy decides the method, the firmware erase step, the
verification level, and the certificate format.
- **Firmware erase**: ATA Secure Erase (standard + Enhanced), NVMe Sanitize
(Block/Crypto/Overwrite) with status polling, NVMe Format NVM, SCSI
Sanitize, SCSI Format Unit, TRIM, FITRIM, HPA/DCO detect and disable.
- **Free-space-only mode**: fills filesystem free space with temp files
containing the wipe pattern, then deletes them. User data is untouched.
- **SMART data**: health, temperature, wear-level, error count, NVMe health
log — all via `smartctl --json`.
- **TPM erasing**: seal AES keys to TPM PCRs, then erase the key to make
encrypted data permanently unrecoverable.
- **Verification**: static-pattern, PRNG-stream, whole-device hash, spot
(N% of blocks), block, and statistical (Shannon entropy, chi-square,
byte frequency) with LBA-range failure mapping.
- **Audit certificates**: JSON (canonical, deterministic), PDF (A4
single-page), XML, CSV, HTML (self-contained), YAML. Merkle tree
construction. Ed25519 signing with key fingerprint binding.
- **Job scheduler**: sequential, parallel (thread pool with semaphore),
priority, and groups modes.
- **Batch mode**: YAML or JSON spec file for automated multi-device wipes.
- **JSON API**: Unix-domain-socket server with line-delimited JSON protocol.
- **TUI**: interactive terminal UI with device list, detail pane, command
palette, and mouse support.
- **Security hardening**: secure memory (zeroize on drop), constant-time
comparison, startup KAT self-tests, continuous RNG health checks (NIST
SP 800-90B), FIPS mode flag, binary self-hash, reproducible build
verification.
- **Legacy compatibility**: all legacy CLI flags accepted with deprecation
warnings. Invoking via a `nwipe` symlink enables legacy compatibility
mode.
- **SBOM**: CycloneDX 1.4 Software Bill of Materials generation.
## Design decisions
**Why Rust?** Memory safety without garbage collection. The wipe engine
handles untrusted block device I/O, cryptographic seed material, and
PRNG state. Rust's ownership model eliminates use-after-free, buffer
overflow, and data race classes of bugs that C would require manual
discipline to avoid.
**Why a policy engine?** The operator should not need to know whether a
Samsung 980 Pro supports NVMe Sanitize Crypto Erase. The operator selects
a profile (e.g. "Paranoid"), and the policy engine inspects the device's
media class and capabilities to produce the right wipe plan. Adding a new
storage technology means adding a policy function — the wipe engine,
methods, and PRNGs are untouched.
**Why shell out to hdparm / nvme-cli?** These tools are maintained by the
kernel and vendor communities, handle device-specific quirks, and are
already installed on most systems. Reimplementing ATA passthrough in Rust
would duplicate their effort and introduce bugs. Scuttle detects each tool
at runtime and produces a clear error if it is missing.
**Why file-fill for free-space-only?** Block-device access on a mounted
filesystem corrupts the filesystem. File-fill is the only safe approach:
create temp files, write the pattern, delete them. The filesystem
allocator places the files in free blocks, achieving the same coverage as
block-device overwrite without touching user data.
## The ISAAC-64 story
The v0.1 release shipped a deterministic stub for ISAAC-64 — the interface
was present but the round function was a placeholder. The v0.3 release
replaced it with a bit-exact port of Bob Jenkins' 1996 C reference. The
port required understanding three subtle details:
1. The `ind()` macro uses byte addressing (`x & 2040`), not array
indexing (`x & 255`). The original C casts to `ub1*` and adds a byte
offset.
2. The two-loop structure in `isaac64()` uses pointer pairs `(m, m2)`
where `m2` starts at `RANDSIZ/2` in the first loop and wraps to 0 in
the second.
3. The expression `y >> RANDSIZL` where `RANDSIZL = 2048` is undefined
behavior on a 64-bit type. GCC compiles this to 0, which we replicate.
The KAT verifies the first 8 u64 outputs against the C reference compiled
with GCC on x86-64.
## Audit certificates
Every wipe produces an audit certificate binding the job ID (UUID v4),
timestamp (RFC 3339 UTC), operator identity, machine hostname, device
identity (path, model, serial, WWN, firmware), method, PRNG(s) and seed
digest, hash algorithm, per-pass results, final verification result,
performance metrics, and firmware erase notes.
The JSON format uses canonical (sorted-key) serialization for
determinism. The Ed25519 signature covers the canonical JSON bytes. The
key fingerprint (SHA-256 of the Ed25519 public key) is embedded in the
certificate, enabling third-party verification.
## What is next
Scuttle v1.0 is a stable release. The v1.x ABI is frozen. Future work
includes OpenPGP and X.509 signing backends, O_DIRECT with alignment
for maximum throughput, and a plugin sandbox using seccomp.
## Try it
```bash
cargo build --workspace --release
./target/release/scuttle selftest
./target/release/scuttle list
./target/release/scuttle wipe /tmp/test.bin --method dod --certificate all
```
See [quickstart.md](quickstart.md) for the full guide.
Scuttle is GPL-2.0-or-later. The full source is available.