dcosnet
/
scuttle
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
scuttle/docs/MANIFEST.md

2271 lines
145 KiB
Markdown
Executable File
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Scuttle Master Manifest
### A Next Generation Open Source Data Sanitization Framework
| | |
|---|---|
| **Document** | Master Manifest & Architectural Roadmap |
| **Working Title** | Scuttle (interim; successor name TBD by community) |
| **Lineage** | Inspired by [nwipe](https://github.com/martijnvanbrummelen/nwipe) and DBAN; independent reimplementation |
| **License** | GPL-2.0-or-later; all new code GPL-2.0+ compatible |
| **Status** | Draft v0.1 — pre-implementation architectural manifest |
| **Steward** | Open community; governance model defined in §13 |
| **Scope** | Architectural, cryptographic, operational, and conformance specification |
---
> **Reading guide.** This document is the canonical architectural reference for the Scuttle fork. It is written for contributors, security reviewers, compliance auditors, and integrators. It is not a user manual — operator documentation will be generated from this manifest and the inline docstrings of the implementation. Where the user-facing surface and the internal architecture diverge, the operator surface is the source of truth for *behavior* and this manifest is the source of truth for *structure*.
---
## 1. Mission and Philosophy
### 1.1 The Sanitization Ethos, Restated
The classic tools (DBAN and its userspace successor nwipe) were built around a singular, almost austere operator contract:
> **Boot. Detect drives. Securely erase everything.**
That contract made those tools ubiquitous in ITAD (IT Asset Disposition) workflows, hobbyist recycling, and incident response. It worked because it refused to be clever: no cloud, no accounts, no telemetry, no decisions to make beyond "which drive" and "which method". The operator could hand the disc to a stranger with a printed certificate and walk away.
Scuttle inherits that contract as a non-negotiable baseline. Any feature that breaks the "boot, detect, erase" loop for a walk-up operator is, by default, out of scope for the default profile. Complexity is permitted only when it is *invisible* to the operator — surfaced on demand, never imposed.
### 1.2 The Modern Operator Contract
The threat landscape, storage technology stack, and assurance expectations of 2026 are not those of 2003. Rotational media coexist with NVMe, eMMC, UFS, persistent memory, and SMR drives whose overwrite semantics are non-trivial. Firmware-level sanitization (ATA Secure Erase, NVMe Sanitize, SCSI Sanitize) is in many cases *more* reliable than overwrite and is the only meaningful option for SSDs. Regulators and customers expect cryptographic evidence — not a printed screenshot.
The Scuttle operator contract therefore expands to four stages:
> **Boot. Identify media. Choose the most appropriate sanitization method. Produce cryptographic evidence.**
"Most appropriate" is doing a great deal of work in that sentence. It means the operator should not have to know whether a particular NVMe drive supports Crypto Erase vs. Block Erase vs. Overwrite — the framework should recommend, the operator should confirm, and the recommendation should be defensible to an auditor. "Cryptographic evidence" means a tamper-evident, signed record that survives the media being shredded, decommissioned, or shipped to a recycler.
### 1.3 The Architectural Stance
These two contracts appear to be in tension: the first demands simplicity, the second demands sophistication. The resolution — and the central architectural bet of this project — is the strict separation of **algorithms**, **policies**, and **profiles**, formalized in §4. The operator-facing surface stays DBAN-simple. The internal machinery becomes DBAN-sophisticated. The two meet at a thin, well-defined boundary.
This is the same trick that let Linux remain usable as a personal OS while becoming the default substrate for hyperscalers: a small, stable core with composable, replaceable subsystems. DBAN's small core was *boot + ncurses + dd-style passes*. Scuttle's small core is *boot + media model + policy engine + verification engine + certificate emitter*. Everything else — including the wipe methods themselves — is a plugin.
### 1.4 What This Fork Is Not
To prevent scope creep, the following are explicitly out of scope:
- **A general-purpose disk benchmark.** Layer 16 (Performance Laboratory) benchmarks providers *for the purpose of selecting the fastest secure one*, not for marketing.
- **A forensic acquisition tool.** Scuttle destroys data. It does not image it. The Forensic profile (§7) is about *chain-of-custody for destruction*, not evidence preservation.
- **A cloud data sanitization framework.** Cloud volumes are out of scope. The framework erases physical and virtual block devices that the kernel can address.
- **A key-management system.** Layer 18 reserves hooks for HSM/TPM, but Scuttle is a consumer of key material, not a custodian.
- **A replacement for physical destruction.** NIST SP 800-88 defines a Destroy class for a reason. Scuttle supports Clear and Purge; Destroy remains a physical process for which Scuttle produces the pre-destruction audit trail.
---
## 2. Goals
The goals below are listed in priority order. Where two goals conflict, the earlier one wins.
1. **100% Open Source.** Every line shipped in a release tarball is open. No binary blobs, no shim loaders for proprietary firmware erase utilities. Where a vendor SDK is required to invoke a sanitize command, the SDK is documented as a build-time dependency and the *invocation path* is open.
2. **GPL compatible.** All code is GPL-2.0+ compatible. Cryptographic primitives that are patented or restricted in any jurisdiction are isolated behind a provider interface and may be built as out-of-tree modules, but are not shipped in the default release artifact.
3. **No proprietary dependencies.** The default build pulls from system packages only. Optional providers (e.g. PKCS#11, OpenSSL engine) are dlopen'd at runtime and the binary degrades gracefully in their absence.
4. **Modular architecture.** Every layer in §5 is independently testable, independently replaceable, and communicates with adjacent layers through a versioned C ABI. A replacement PRNG provider must compile without touching the wipe engine.
5. **Reproducible builds.** Release artifacts are bit-for-bit reproducible from a tagged commit + a published build environment. CI verifies reproducibility on every release candidate. (See §11.)
6. **Long-term maintainability.** The codebase favors explicit, boring C with a small public surface per module. New cryptography is added as providers, not as patches to `pass.c`. New storage tech is added as drivers, not as special cases in `device.c`.
7. **Embedded friendly.** The default binary runs on a 64 MB initramfs with no Python, no JS, no runtime loader beyond libc and libcrypto. The ncurses UI is the lowest-common-denominator interface and is the only UI guaranteed to work on every supported target.
8. **Enterprise capable.** The same binary, in a different launch mode, exposes a JSON/REST API, integrates with LDAP/AD for operator auth, ships audit logs to a remote collector, and can be driven by a PXE-booted fleet agent. The enterprise features are *additive*: an embedded deployment does not pay for them.
---
## 3. Core Design Principles
The architecture is organized as a six-stage pipeline. Each stage is a hard boundary: contracts above it cannot be violated by behavior below it, and contracts below it cannot be observed by code above it except through the published interface.
```
Small Core ← one binary, one entrypoint, one operator surface
↓
Modular Components ← every subsystem is a separately testable unit
↓
Plugin Architecture ← crypto, profiles, reports, drivers load at runtime
↓
Media Awareness ← the framework understands what it is wiping
↓
Cryptographic Verification ← every claim is backed by hashable evidence
↓
Automation Friendly ← every operator action has a scriptable equivalent
```
### 3.1 Small Core
The core is the *minimum* code required to boot, enumerate block devices, dispatch a wipe job, and emit a result. It contains:
- Process lifecycle and signal handling
- The device enumeration shim (sysfs + ioctl)
- The plugin loader
- The job scheduler (sequential mode only; parallel mode is a plugin)
- The certificate emitter (one format: JSON)
- The ncurses UI (the only UI compiled in by default)
Everything else — additional PRNGs, additional hashes, additional wipe methods, additional report formats, additional UIs, the policy engine itself — is a plugin. A core-only build produces a binary that can run `Zero` and `One` passes against `/dev/sdX` and emit a JSON certificate. That is the floor.
### 3.2 Modular Components
Modules are statically or dynamically linked units that implement one layer of §5. Each module exposes a `<module>_vN.h` header declaring its public ABI. Module versioning follows the GTK rule: even minor = stable, odd minor = development. Modules may only depend on layers *below* themselves in the §5 stack; cross-layer calls are forbidden except through the policy engine.
### 3.3 Plugin Architecture
Plugins are dynamically loaded modules installed under `$prefix/lib/scuttle/<kind>/`. The kinds are: `algorithms/`, `verification/`, `reports/`, `exporters/`, `profiles/`, `drivers/`. The core enumerates these directories at startup, dlopens each `.so`, calls `<kind>_probe()` to populate the registry, and exposes them through the UI and API. A plugin that fails to load is logged but does not abort the process — the operator sees "BLAKE3 provider unavailable" and can proceed with the bundled providers.
### 3.4 Media Awareness
This is the layer DBAN never had. Before any wipe, the framework queries the device for: rotational/flash/NVMe/eMMC/PMEM classification, SMART health, wear level (SSD), sanitize command support (ATA Secure Erase / NVMe Sanitize / SCSI Sanitize), HPA/DCO status, sector size, and total addressable bytes. From this it derives a *media descriptor* consumed by the policy engine. The operator never has to specify "this is an SSD"; the framework knows.
### 3.5 Cryptographic Verification
Every wipe produces at minimum:
- A **seed hash** — the hash of the PRNG seed material used (for reproducible random passes)
- A **verification hash** — the hash of the final state of the wiped region (for `Zero` and `One` passes, the hash of all-zero / all-one; for random passes, the hash of the PRNG output stream up to the device size, which is reproducible from the seed)
- A **job descriptor** — operator, machine, device, serial, firmware, method, PRNG, verification result, duration, bandwidth
These are bound together in a certificate (§7) and, in advanced mode, into a Merkle tree whose root is signed.
### 3.6 Automation Friendly
Every operator action in the ncurses UI has a CLI equivalent. Every CLI action has a JSON API equivalent. Every JSON API action has a documented request/response schema and an OpenAPI spec. The PXE fleet mode is the same binary running headless with a config file. There is no "scripting layer" bolted on; the binary *is* the scripting layer.
---
## 4. The Defining Feature: Algorithms, Policies, and Profiles
This section is the architectural keystone of the fork. If only one chapter of this manifest is read by a contributor, it should be this one. The separation of algorithms, policies, and profiles is what allows Scuttle to absorb new cryptographic primitives, new storage technologies, and new operator workflows *without redesigning the wipe engine*. It is the architectural feature that distinguishes Scuttle from legacy tools and from every other open-source sanitization tool we are aware of.
### 4.1 The Three Tiers
**Algorithms** are the building blocks. They are pure computational primitives with no knowledge of storage, no knowledge of policies, no knowledge of operator intent. Examples: ChaCha20 as a stream cipher, BLAKE3 as an XOF (extendable output function), SHAKE256, Ascon-128a, Serpent in CTR mode, ISAAC64. An algorithm exposes a uniform provider interface (§5 Layer 4) and is otherwise a black box. A new algorithm can be added by dropping a `.so` into `providers/` and shipping test vectors — no other change is required.
**Policies** decide *how* to sanitize a particular class of storage. A policy is a function from (media descriptor, operator intent) to (sequence of operations). Examples: "for an HDD with no sanitize command, run DoD 5220.22-M 3-pass followed by verification"; "for an NVMe SSD that supports Crypto Erase, invoke NVMe Sanitize with crypto-erase action and verify the resulting namespace is empty"; "for an SMR drive, run a single random pass with host-managed zone awareness to avoid write amplification". Policies are the only place where storage-class knowledge lives. They are versioned and individually testable.
**Profiles** package policies into operator-facing choices. A profile is a named, stable, documented bundle of: which policy to use for which media class, which PRNG provider to prefer, which verification level to apply, which certificate format to emit, which report format to generate, and which UI flow to present. Examples: `Quick Clear`, `NIST Purge`, `Enterprise`, `Paranoid`, `Research`, `Forensic`, `Government`, `Air Gap`, `Custom`. Profiles are the *only* thing the operator sees in the default UI; policies and algorithms are invisible unless the operator opts into an "advanced" view.
### 4.2 Why This Separation Matters
Consider three concrete evolution scenarios:
**Scenario A: A new cryptographic primitive becomes fashionable.** Suppose NIST standardizes a new XOF in 2028 and the community wants to use it as a PRNG. In upstream nwipe, this requires editing `prng.c` to add a new branch, editing `method.c` to expose it in the menu, editing `options.c` to add a CLI flag, editing `pass.c` to dispatch to it, and editing the GUI to display it. Every change touches the wipe engine. In Scuttle, the same change is: drop a `.so` implementing the `prng_provider_v1` interface into `providers/`, ship KAT vectors in `tests/kat/`, add a row to the provider catalog table in the docs. The wipe engine, the policy engine, and the profile catalog are untouched.
**Scenario B: A new storage technology appears.** Suppose a new non-volatile memory class — call it XPMEM — ships in 2029 with a vendor-defined sanitize command. In upstream nwipe, this requires editing `device.c` to detect it, editing `pass.c` to handle its quirks, and editing `se_*.c` to invoke its sanitize command. In Scuttle, this is: drop a `drivers/xpmem/` module implementing the `device_driver_v1` interface, drop a `policies/xpmem-purge.policy` file mapping the media class to the sanitize command, and (optionally) update the default profile catalog to reference the new policy. Existing profiles that say "use the default policy for media class X" pick it up automatically.
**Scenario C: A new compliance regime emerges.** Suppose a regulator publishes a new sanitization standard in 2030 that requires three passes with three distinct PRNGs, an entropy check on each pass, and a signed certificate. In upstream nwipe, this is a deep rewrite. In Scuttle, this is: write a new profile file `profiles/regulator-2030.profile` that bundles three existing policies with three existing PRNG providers, sets the verification level to "every pass + entropy", and points the certificate emitter at the signing key. The profile is shipped as a single declarative file. No C code is touched.
These three scenarios are the stress test for the architecture. If any of them requires touching the wipe engine, the separation has failed.
### 4.3 The Boundary Contracts
```
┌──────────────────────────────────────────────────────────────┐
│ PROFILE LAYER (operator-facing, declarative, versioned) │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Quick │ │ NIST │ │ Paranoid │ │ Custom │ ... │
│ │ Clear │ │ Purge │ │ │ │ │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │ │
│ └────────────┴────────────┴────────────┘ │
│ │ selects │
│ ▼ │
│ POLICY LAYER (storage-class knowledge, testable) │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ HDD │ │ NVMe │ │ SMR │ │ PMEM │ ... │
│ │ overwrite│ │ sanitize │ │ zone-aware│ │ crypto │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │ │
│ └────────────┴────────────┴────────────┘ │
│ │ invokes │
│ ▼ │
│ ALGORITHM LAYER (pure computational primitives, plugins) │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ ChaCha20 │ │ BLAKE3 │ │ SHAKE256 │ │ Ascon │ ... │
│ │ (PRNG) │ │ XOF │ │ (XOF) │ │ (PRNG) │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
└──────────────────────────────────────────────────────────────┘
```
The contracts at each boundary are:
- **Profile → Policy**: a profile selects a policy *by media class*. It does not select a policy by device path, by operator, or by time of day. Selection by media class is the only allowed coupling, because media class is what the policy was written to handle. Operator-specific overrides happen *above* the profile layer (in the UI/API), not within it.
- **Policy → Algorithm**: a policy invokes algorithms *by capability*, not by name. A policy that needs "a CSPRNG with at least 256 bits of state and ≥ 1 GB/s throughput on the current CPU" asks the provider registry for one, rather than naming ChaCha20 directly. This lets the hardware optimization layer (§5 Layer 8) substitute AES-NI ChaCha20 for portable ChaCha20 without the policy's knowledge. A policy *may* name a specific algorithm when the choice is semantically meaningful (e.g. a regulatory profile that mandates a particular PRNG), but this is the exception.
- **Algorithm → (nothing)**: algorithms know nothing about policies or profiles. They expose `init/seed/generate/benchmark/self_test/cleanup` and that is all. An algorithm that tries to introspect its caller is a bug.
### 4.4 Profile Definition Language
Profiles are declarative files. The working format is TOML with a JSON Schema for validation. A profile file specifies:
```toml
# profiles/paranoid.profile.toml
[meta]
name = "Paranoid"
version = 1
description = "Multi-pass, multi-algorithm, full verification. For media that must not be recoverable under any plausible adversary."
nist_class = "Purge"
target = "classified, high-value, adversary-rich environments"
[defaults]
prng_pool = ["BLAKE3_XOF", "XChaCha20", "Serpent-CTR"] # round-robin across passes
hash = "BLAKE3"
verify = "every_pass"
certificate = "signed_pdf" # requires operator key
report = ["detailed", "compliance"]
[policy_map]
hdd_smr = "hdd_overwrite_7pass"
hdd_cmr = "hdd_overwrite_7pass"
ssd_sata = "ssd_purge_then_overwrite_3pass" # firmware first, overwrite belt-and-braces
ssd_nvme = "nvme_sanitize_crypto_then_overwrite_3pass"
pmem = "pmem_crypto_erase_then_overwrite"
emmc = "emmc_trim_then_overwrite_3pass"
[constraints]
require_secure_erase_capable = false # proceed with overwrite if no SE
abort_on_verify_failure = true
require_signed_certificate = true
minimum_passes = 3
```
The policy engine loads this file at startup, registers the profile under its `name`, and offers it in the UI. The profile's `policy_map` is consulted at job-dispatch time against the media descriptor of the selected device. If the device's media class is not in the map, the profile refuses to run and the operator is told why.
### 4.5 Algorithm Capability Tags
Algorithms are tagged with capabilities so the policy layer can request by capability rather than by name:
| Tag | Meaning |
|---|---|
| `csprng` | Cryptographically strong pseudo-random number generator |
| `xof` | Extendable output function (variable-length output) |
| `stream_cipher` | Stream cipher usable as a PRNG keystream |
| `block_cipher_ctr` | Block cipher in CTR mode, usable as a PRNG |
| `hash_256` | Cryptographic hash with ≥ 256-bit output |
| `hash_512` | Cryptographic hash with ≥ 512-bit output |
| `aead` | Authenticated encryption with associated data |
| `post_quantum` | Believed secure against quantum adversaries (best-effort) |
| `hardware_accelerated` | Has a SIMD/AES-NI/NEON fast path on this CPU |
| `fips_eligible` | Eligible for FIPS 140-3 validation (no experimental algorithms) |
A policy that needs "a CSPRNG with `hardware_accelerated` and `fips_eligible`" will get AES-NI AES-CTR on x86 and ChaCha20 with NEON on ARM, transparently. This is the mechanism by which §5 Layer 8 (Hardware Optimization) becomes useful without scattering CPU-detection logic through every policy.
### 4.6 The Default Profile Catalog
The default catalog shipped with the framework (§7 contains the full per-profile specification) is:
| Profile | Audience | NIST class | Default PRNG | Passes | Verify | Certificate |
|---|---|---|---|---|---|---|
| Quick Clear | ITAD high-throughput | Clear | AES-CTR (AES-NI) | 1 | Final pass | JSON |
| Modern Random | General-purpose | Clear | ChaCha20 | 1 | Final pass | JSON |
| NIST Clear | NIST 800-88 Clear | Clear | ChaCha20 | 1 | Final pass + spot 5% | JSON+PDF |
| NIST Purge | NIST 800-88 Purge | Purge | (firmware) | firmware + 1 overwrite | Final pass | JSON+PDF, signed |
| Enterprise | Corporate fleets | Purge | BLAKE3 XOF | firmware + 3 overwrite | Every pass | JSON+PDF, signed, Merkle root |
| Paranoid | Adversary-rich | Purge | Multi (BLAKE3+XChaCha+Serpent) | 7 | Every pass + entropy | Signed PDF + Merkle root |
| Research | Academic | n/a | Configurable | Configurable | Full + statistical | JSON + CSV + raw hashes |
| Forensic | Chain-of-custody destruction | Purge | ISAAC64 | 3 + final zero | Entire device | Signed PDF + Merkle root + manifest |
| Government | Federal / regulated | Purge | AES-CTR (FIPS path) | per policy | Every pass + KAT | Signed PDF, FIPS-mode |
| Air Gap | Classified, offline | Purge | SHAKE256 | 7 | Every pass | Offline-signed PDF + Merkle root |
| Custom | Operator-defined | — | — | — | — | — |
The catalog is intentionally short. New profiles are added by dropping a TOML file into `profiles/`; the catalog is regenerated at startup. There is no compile-time profile registry.
---
## 5. Layered Architecture
The framework is organized into eighteen layers. Layers are ordered from the bottom up: a higher layer may depend on a lower layer, never the reverse. The layering is a *compile-time* constraint enforced by the build system (each layer may only `#include` headers from layers at or below its own index) and a *runtime* constraint enforced by the plugin loader (a plugin may only resolve symbols from layers at or below its own index from the core's exported symbol table).
For each layer below, the structure is:
- **Purpose** — what problem this layer solves
- **Scope** — what is in and what is out
- **Public interface** — the C ABI that other layers and plugins see (sketched in pseudocode)
- **Dependencies** — what this layer requires from lower layers and external libraries
- **Integration points** — where this layer touches other layers (the "mixed combinations")
- **Conformance** — how a build proves this layer works
### Layer 1 — Device Discovery Engine
**Purpose.** Enumerate every block device the kernel can address, classify it, and surface a uniform `nwipe_device_t` descriptor to the rest of the framework. This layer replaces the ad-hoc `/proc/partitions` + `lsblk` + `smartctl` shelling-out that upstream nwipe does, with a direct syscall + sysfs + ioctl implementation.
**Scope — in:**
- Detection of: SATA HDD, SAS HDD, USB HDD, SATA SSD, NVMe (PCIe and M.2), eMMC, SD, CF, UFS, SMR HDD (host-managed and host-aware), persistent memory (PMEM, NVDIMM-N/P/B), loop devices, md RAID members, LVM physical volumes, LUKS containers, dm-crypt containers, ZFS vdev members, BTRFS members, VirtIO-blk, VMware virtual disks, Hyper-V virtual disks, QEMU virtual disks.
**Scope — out:**
- Cloud block storage (EBS, Azure Managed Disks, GCE PD).
- Network-attached storage (the framework erases local block devices only).
- Tape (out of scope for v1.0; reserved for v2.0).
**Public interface (sketch):**
```c
typedef enum {
NWIPE_BUS_SATA, NWIPE_BUS_SAS, NWIPE_BUS_USB, NWIPE_BUS_NVME,
NWIPE_BUS_MMC, NWIPE_BUS_SD, NWIPE_BUS_UFS, NWIPE_BUS_PMEM,
NWIPE_BUS_LOOP, NWIPE_BUS_MD, NWIPE_BUS_DM, NWIPE_BUS_VIRTIO,
NWIPE_BUS_VIRTUAL,
} nwipe_bus_t;
typedef struct nwipe_device {
char path[256]; // /dev/sda, /dev/nvme0n1, /dev/mmcblk0
char model[64];
char serial[64];
char wwn[32]; // World Wide Name
char firmware_rev[16];
nwipe_bus_t bus;
uint64_t size_bytes;
uint32_t logical_block_size;
uint32_t physical_block_size;
int rotational; // 0 = flash/non-rotational, 1 = HDD
int removable;
int smart_health_ok; // -1 = unknown, 0 = failing, 1 = ok
int wear_level_pct; // SSD remaining life; -1 if N/A
int supports_ata_se; // ATA Secure Erase
int supports_ata_se_enhanced;
int supports_nvme_sanitize;
int supports_nvme_format;
int supports_scsi_sanitize;
int hpa_present; // Host Protected Area
int dco_present; // Device Configuration Overlay
char media_class[32]; // "hdd_cmr", "hdd_smr", "ssd_sata",
// "ssd_nvme", "emmc", "pmem", ...
char sysfs_path[512];
char driver[64]; // "ahci", "nvme", "usb-storage", ...
} nwipe_device_t;
int nwipe_device_enumerate(nwipe_device_t **out, size_t *count);
void nwipe_device_free(nwipe_device_t *list, size_t count);
int nwipe_device_refresh(nwipe_device_t *dev); // re-query SMART/wear
```
**Dependencies:** libc, libudev (optional, falls back to sysfs walking), libnvme (for NVMe-specific queries), libatasmart (optional, for SMART), `ioctl(SG_IO)` for SCSI/SAS.
**Integration points:**
- Feeds the **media descriptor** consumed by Layer 2 (Media Intelligence) and the **policy engine** in Layer 3.
- The `supports_*` and `media_class` fields drive Layer 9 (Secure Erase Integration) recommendation logic.
- The `serial` and `wwn` fields are bound into Layer 7 (Cryptographic Audit) certificates.
- SMART health and wear level feed Layer 11 (Reporting) graphs and Layer 17 (Research Mode) wear statistics.
**Conformance:**
- Unit test: a fixture of `/sys/block/` snapshots (committed under `tests/fixtures/sysblock/`) must produce a stable `nwipe_device_t` list under `nwipe_device_enumerate`.
- Integration test: against a loopback file + `nvme-cli` mock NVMe target, the enumeration must succeed without crashing on missing optional libs.
- Negative test: devices flagged `hpa_present` or `dco_present` must be reported, not silently erased past.
### Layer 2 — Media Intelligence
**Purpose.** Take a `nwipe_device_t` and produce a `nwipe_media_descriptor_t` that classifies the device along the axes that matter for sanitization. This is the layer that translates "this is a Samsung 980 Pro" into "this is an NVMe SSD that supports Crypto Erase and should be purged, not overwritten".
**Scope — in:**
- Classification into: rotational HDD (CMR), rotational HDD (SMR, host-managed), rotational HDD (SMR, host-aware), flash SSD (SATA), flash SSD (NVMe), hybrid (SSHD), persistent memory, eMMC, SD, CF, UFS, virtual.
- Recommendation, per NIST SP 800-88 Rev. 1, of `Clear`, `Purge`, or `Destroy` based on the device class and the operator's stated assurance target.
**Scope — out:**
- The actual execution of the recommended action — that is Layer 3's job.
- The decision of *which* PRNG or *how many* passes — that is the policy engine's job (§4).
**Public interface (sketch):**
```c
typedef enum {
NWIPE_NIST_CLEAR, // logical sanitization (overwrite, trim)
NWIPE_NIST_PURGE, // firmware-level (SE, Sanitize, Crypto Erase)
NWIPE_NIST_DESTROY, // physical destruction (out of scope for execution;
// framework only emits pre-destruction certificate)
} nwipe_nist_class_t;
typedef struct nwipe_media_descriptor {
nwipe_device_t *dev;
char media_class[32];
char media_subclass[32]; // "cmr", "smr_host_managed", ...
int recommends_clear;
int recommends_purge;
int recommends_destroy;
int purge_method; // SE_ATA, SE_ATA_ENH, NVME_SANITIZE_CRYPTO,
// NVME_SANITIZE_BLOCK, NVME_SANITIZE_OVERWRITE,
// SCSI_SANITIZE, PMEM_CRYPTO_ERASE
int overwrite_recommended_after_purge; // belt-and-braces
char rationale[512]; // human-readable, auditable
} nwipe_media_descriptor_t;
int nwipe_media_classify(nwipe_device_t *dev, nwipe_media_descriptor_t *out);
```
**Dependencies:** Layer 1.
**Integration points:**
- The `purge_method` field is consumed by Layer 9 (Secure Erase Integration) to dispatch the right firmware command.
- The `recommends_*` flags and `rationale` are surfaced to the operator in the UI (Layer 13) and bound into the certificate (Layer 7).
- The `media_class` field is the join key against the profile's `policy_map` (§4.4).
**Conformance:**
- For every device in the `tests/fixtures/sysblock/` corpus, the classification must be deterministic and the rationale string must contain the keywords that justify it (e.g. an SSD with `supports_nvme_sanitize` must have "sanitize" in its rationale).
### Layer 3 — Wipe Engine
**Purpose.** Execute a sanitization job against a device, given a policy decision. This is the spiritual descendant of upstream nwipe's `pass.c` and `method.c`, refactored so that the *dispatch* of methods is driven by the policy engine (§4), not by a hard-coded menu.
**Scope — in:**
- All existing nwipe methods preserved for legacy compatibility: `Zero`, `One`, `PRNG (Stream)`, `DoD 5220.22-M`, `Gutmann`, `RCMP TSSIT OPS-II`, `HMG IS5 (Baseline/Enhanced)`, `Schneier`, `BMB (German)`, `Random Pass`.
- The new profile-driven dispatch (§4.4).
- Pass interleaving: a single job may run multiple passes with different PRNGs (round-robin), per profile.
- Zone-aware writing for SMR drives (host-managed): write sequentially within zones, do not issue random writes that would trigger GC amplification.
**Scope — out:**
- The choice of *which* method to run — that is the policy engine's job.
- Firmware-level sanitize — that is Layer 9's job; the wipe engine invokes Layer 9 as a sub-step when the policy says so.
**Public interface (sketch):**
```c
typedef struct nwipe_job {
uuid_t job_id;
nwipe_device_t *dev;
nwipe_media_descriptor_t *media;
const nwipe_profile_t *profile;
const nwipe_policy_t *policy;
// resolved by policy engine at dispatch time:
nwipe_pass_plan_t *passes; // array, see below
size_t n_passes;
nwipe_prng_t *prng_pool[8]; // resolved providers
size_t n_prngs;
nwipe_hash_t *hash; // for verification
nwipe_verify_level_t verify;
// callbacks for progress, abort, log
nwipe_progress_cb *on_progress;
nwipe_abort_cb *on_abort;
nwipe_log_cb *on_log;
} nwipe_job_t;
typedef struct nwipe_pass_plan {
nwipe_pass_kind_t kind; // ZERO, ONE, PRNG_STREAM, FIRMWARE_SE,
// FIRMWARE_SANITIZE, TRIM, VERIFY
nwipe_prng_t *prng; // NULL for ZERO/ONE/FIRMWARE
const uint8_t *static_pattern; // for static-pattern passes
size_t static_pattern_len;
int round_robin_index; // -1 if not in a round-robin pool
} nwipe_pass_plan_t;
int nwipe_job_run(nwipe_job_t *job, nwipe_result_t *out);
```
**Dependencies:** Layers 1, 2, 4 (PRNG providers), 6 (verification), 9 (firmware erase), 7 (audit, invoked as a sink for events).
**Integration points:**
- The wipe engine emits progress events consumed by Layer 13 (UI) and Layer 11 (Reporting).
- The wipe engine calls Layer 6 (Verification) after each pass whose policy says "verify", and after the final pass unconditionally.
- The wipe engine calls Layer 7 (Audit) to append each pass result to the job's audit record.
- The wipe engine queries Layer 8 (Hardware Optimization) at job start to confirm the fastest available provider for the requested capability is the one actually selected.
**Conformance:**
- All upstream nwipe method test vectors (the `tests/unit/test_round_size.c` family and the `tests/ci/loopback_e2e.sh` e2e) must continue to pass, byte-for-byte, against the legacy method dispatch path.
- A "round-robin" job with 3 PRNGs and 3 passes must produce a device state whose per-pass hash matches the hash of each PRNG's output stream up to the device size.
### Layer 4 — PRNG Provider Framework
**Purpose.** Provide a uniform interface to every pseudo-random number generator the framework can use for wipe passes. Replace the `prng.c` switch statement in upstream nwipe with a registry of dynamically-loaded providers.
**Scope — in:** every PRNG listed in the user manifest, organized into three tiers:
*Current (preserved from upstream nwipe):*
- AES-CTR (already in `src/aes/`)
- ChaCha20 (already in `src/chacha20/`)
- ISAAC, ISAAC64 (already in `src/isaac_rand/`)
- MT19937 (already in `src/mt19937ar-cok/`)
- SplitMix64 (already in `src/splitmix64/`)
- xoroshiro256 (already in `src/xor/`)
- Lagged Fibonacci (already in `src/alfg/`)
*Modern (new):*
- BLAKE3 XOF — extendable output, very fast with SIMD
- XChaCha20 — extended-nonce variant of ChaCha20
- Ascon-128a — NIST lightweight cryptography standard, good for embedded
- SHAKE128, SHAKE256 — SHA-3 XOFs
- KangarooTwelve — Keccak-based, parallelizable
- HC-256 — eSTREAM portfolio
- Rabbit — eSTREAM portfolio
- Salsa20 — predecessor of ChaCha20
- AES-CTR (re-exposed as a provider, with AES-NI fast path)
- Serpent-CTR, Twofish-CTR, Camellia-CTR — alternate block ciphers in CTR mode
*Experimental (build-time flag, not in default release):*
- Threefish, Skein, Whirlpool, BLAKE2X
**Public interface (sketch):**
```c
typedef struct nwipe_prng_v1 {
const char *name; // "BLAKE3-XOF", "ChaCha20", ...
const char **capabilities; // {"csprng","xof","hardware_accelerated",NULL}
uint32_t min_seed_bytes;
uint32_t state_size;
int (*init)(void **state);
int (*seed)(void *state, const uint8_t *seed, size_t len);
int (*generate)(void *state, uint8_t *out, size_t len);
int (*benchmark)(void *state, double *out_mbps, double *out_cpu_pct);
int (*self_test)(void); // KAT vectors
void (*cleanup)(void *state);
} nwipe_prng_v1;
int nwipe_prng_register(const nwipe_prng_v1 *provider);
const nwipe_prng_v1 *nwipe_prng_by_name(const char *name);
const nwipe_prng_v1 *nwipe_prng_by_capability(const char **required,
const char **preferred,
nwipe_hw_caps_t hw);
```
**Dependencies:** Layer 8 (for `nwipe_hw_caps_t` capability discovery, used to choose SIMD vs. portable implementations).
**Integration points:**
- Consumed by Layer 3 (Wipe Engine) when a pass plan says `PRNG_STREAM`.
- The `seed` callback is invoked with material from `/dev/urandom` (or a TPM/HSM, via Layer 18) at job start. The seed material is hashed by Layer 5 and bound into the certificate (Layer 7) so the random stream is reproducible from the certificate.
- The `benchmark` callback is invoked by Layer 16 (Performance Laboratory) to populate the historical performance table.
- The `self_test` callback is invoked at startup (Layer 15, Security) and on demand; a failing self-test removes the provider from the registry for the rest of the process.
**Conformance:**
- Every provider ships with KAT vectors in `tests/kat/<provider>/`. The vectors are checked at `make test` and at startup.
- Every provider ships with a benchmark entry in `tests/bench/<provider>.bench`. The bench is run in CI and the result is recorded in the release notes.
- A provider that fails `self_test` at runtime must log a CRITICAL event and be unavailable for selection, but must not crash the process.
### Layer 5 — Hash Framework
**Purpose.** Provide a uniform interface to every hash function / XOF used for verification, certificate binding, and audit integrity.
**Scope — in:** SHA-256, SHA-512, SHA-3 (256/384/512), BLAKE2 (b/s, 256/512), BLAKE3, SHAKE128, SHAKE256, KangarooTwelve.
**Public interface (sketch):**
```c
typedef struct nwipe_hash_v1 {
const char *name;
uint32_t output_bytes; // 32, 64, ...; for XOF, the chosen output length
int is_xof; // 1 if output is variable-length
int (*init)(void **state);
int (*update)(void *state, const uint8_t *in, size_t len);
int (*final)(void *state, uint8_t *out, size_t out_len);
int (*self_test)(void);
void (*cleanup)(void *state);
} nwipe_hash_v1;
int nwipe_hash_register(const nwipe_hash_v1 *provider);
const nwipe_hash_v1 *nwipe_hash_by_name(const char *name);
```
**Dependencies:** None below the libc/libcrypto boundary. May use OpenSSL, libsodium, or a bundled implementation per provider.
**Integration points:**
- Consumed by Layer 6 (Verification) to compute per-block and whole-device hashes.
- Consumed by Layer 7 (Audit) to compute the Merkle tree root and to bind the seed material.
- Consumed by Layer 4 (PRNG) — every PRNG's `seed()` material is hashed by a Layer 5 hash to produce a stable seed digest for the certificate.
- The Merkle tree construction (Layer 7) uses one hash from this layer; the choice is profile-driven.
**Conformance:**
- NIST CAVP test vectors for SHA-2, SHA-3, SHAKE.
- Official BLAKE2/BLAKE3 test vectors.
- KangarooTwelve reference test vectors.
### Layer 6 — Verification Engine
**Purpose.** After a pass (or after the whole job), prove that the device's content matches what the wipe engine claims it wrote. This is the difference between "I overwrote it" and "I can prove I overwrote it".
**Scope — in:**
- *Sector verification*: re-read every logical sector and compare against the expected pattern.
- *Random spot verification*: re-read N random sectors (default N=5%, cryptographically-chosen offsets).
- *Block verification*: re-read fixed-size blocks (e.g. 1 MiB) — useful for very large devices where sector-by-sector is too slow.
- *Entire device verification*: hash the entire device post-wipe and compare against the expected hash.
- *Entropy analysis*: for random passes, compute Shannon entropy, chi-square, and byte-frequency distribution of the wiped region. Expected: entropy ≈ 8.0 bits/byte, chi-square within tolerance.
- *Failure mapping*: when a verification fails, record the LBA range, the expected vs. actual content, and surface it in the report and certificate.
**Scope — out:**
- The choice of verification level — that is profile-driven.
- The actual hash computation — that is delegated to Layer 5.
**Public interface (sketch):**
```c
typedef enum {
NWIPE_VERIFY_NONE,
NWIPE_VERIFY_FINAL_PASS, // verify after last pass
NWIPE_VERIFY_EVERY_PASS, // verify after each pass
NWIPE_VERIFY_SPOT_5PCT, // random 5% of sectors
NWIPE_VERIFY_ENTIRE_DEVICE, // hash whole device
NWIPE_VERIFY_FULL_STATISTICAL, // + entropy, chi-square, byte freq
} nwipe_verify_level_t;
typedef struct nwipe_verify_result {
nwipe_verify_level_t level;
int pass; // -1 for "final"
int ok; // 1 = verified, 0 = mismatch
size_t failed_ranges_count;
nwipe_lba_range_t *failed_ranges;
double shannon_entropy; // 0.0–8.0
double chi_square; // for uniform distribution
double byte_freq_max_dev; // max |freq - 1/256|
uint8_t hash[64]; // final hash of device
} nwipe_verify_result_t;
int nwipe_verify(nwipe_job_t *job, nwipe_pass_plan_t *pass,
nwipe_verify_result_t *out);
```
**Dependencies:** Layer 1 (to issue reads), Layer 5 (for hashing).
**Integration points:**
- Called by Layer 3 (Wipe Engine) per the policy.
- Results bound into Layer 7 (Audit) certificate.
- Failures flagged in Layer 11 (Reporting) and surfaced in Layer 13 (UI).
- Layer 17 (Research Mode) consumes the statistical results for academic study.
**Conformance:**
- A controlled test: wipe a loopback device, then `dd` a single byte at a known LBA to flip it. The verification engine must report the exact LBA in `failed_ranges`.
- For random passes, the Shannon entropy of a 1 GiB wiped loopback must be ≥ 7.999 with high probability (the test asserts ≥ 7.998 to allow for variance).
### Layer 7 — Cryptographic Audit Engine
**Purpose.** Bind together everything the framework did into a single, tamper-evident, optionally-signed record. This is the layer that turns "I ran nwipe" into "here is cryptographically verifiable evidence that this media was sanitized, by this operator, on this machine, using this method, on this date, with these verification results".
**Scope — in:**
- A **job descriptor** capturing: job ID (UUID v4), timestamp (UTC, RFC 3339), operator identity, machine identity (hostname + chassis serial), device path/model/serial/WWN/firmware rev, method, PRNG(s) and their seed digests, hash algorithm, verification results, duration, bandwidth, average speed, retry counts, SMART delta (health before/after).
- An **advanced mode** that builds a Merkle tree over per-block hashes of the wiped device, exposes the root, and signs the root (and the job descriptor) with an operator key.
- **Output formats:** JSON (canonical, deterministic), XML, CSV, PDF (typeset), HTML (self-contained), YAML. JSON is mandatory; others are exporter plugins.
**Scope — out:**
- Key management — keys are supplied by Layer 18 (HSM/TPM/OpenSSL engine) or loaded from a configured path. The audit engine does not generate or store private keys.
- Long-term archival — the certificate is the artifact; archival is the operator's responsibility.
**Public interface (sketch):**
```c
typedef struct nwipe_audit_record {
uuid_t job_id;
char timestamp[32]; // RFC 3339 UTC
char operator_id[64];
char operator_key_fingerprint[64]; // OpenPGP or X.509 fingerprint
char machine_hostname[256];
char machine_chassis_serial[64];
nwipe_device_t *dev; // borrowed, do not free
nwipe_media_descriptor_t *media;
const nwipe_profile_t *profile;
const nwipe_policy_t *policy;
// per-pass results
nwipe_pass_result_t *passes;
size_t n_passes;
// verification
nwipe_verify_result_t final_verify;
// performance
double duration_sec;
double avg_bandwidth_mbps;
uint64_t bytes_written;
uint64_t bytes_verified;
uint32_t retry_count;
// crypto
char hash_algorithm[32];
uint8_t seed_digest[64]; // hash of PRNG seed material
char prng_names[256]; // "BLAKE3-XOF,XChaCha20"
// Merkle (advanced mode only)
int has_merkle_root;
uint8_t merkle_root[64];
char merkle_hash_algorithm[32];
// signature (advanced mode only)
int has_signature;
char signature_algorithm[32]; // "ed25519", "rsa-pss-4096"
uint8_t *signature;
size_t signature_len;
} nwipe_audit_record_t;
int nwipe_audit_emit(nwipe_audit_record_t *rec, const char *format,
int fd_out);
// format ∈ {"json","xml","csv","pdf","html","yaml"}
int nwipe_audit_sign(nwipe_audit_record_t *rec, nwipe_signer_t *signer);
int nwipe_audit_build_merkle(nwipe_audit_record_t *rec,
nwipe_block_hash_iter_t *iter);
```
**Merkle tree construction.** For the *Entire Device Verification* mode, the framework computes a hash for every fixed-size block (default 1 MiB) of the wiped device. These leaf hashes are assembled into a binary Merkle tree; the root is included in the certificate. An independent reviewer, given the certificate and read access to the post-wipe device, can verify any individual block hash by recomputing it and checking its path to the published root. This converts "trust the certificate" into "trust the root and verify the leaves you care about".
**Dependencies:** Layers 1, 3, 5, 6, 18 (signer, optional).
**Integration points:**
- Consumed by Layer 11 (Reporting) for the human-readable report.
- Consumed by Layer 13 (UI) for the on-screen "success" view.
- Consumed by Layer 14 (Enterprise Features) for remote logging and webhook emission.
- The JSON form is the canonical on-disk artifact; all other formats are derived from it.
**Conformance:**
- The JSON form must be canonical (sorted keys, no extra whitespace) so that the same record produces the same byte sequence on any build. This is required for signature reproducibility.
- A signed certificate must verify against the operator's published public key using a standard tool (`gpg --verify` for OpenPGP, `openssl cms -verify` for X.509).
- The Merkle root construction must match RFC 6962's structure (so it can be verified by third-party tooling).
### Layer 8 — Hardware Optimization
**Purpose.** Detect the host CPU's cryptographic capabilities and ensure the fastest *secure* provider is selected for any given capability request. Without this layer, ChaCha20 falls back to a portable C implementation on a CPU that has AVX-512, leaving 5× performance on the table.
**Scope — in:**
- CPU feature detection: x86 (AES-NI, AVX, AVX2, AVX-512, SSE2, SSE4.2, SHA-NI), ARM (NEON, ARMv8 Crypto Extensions, SVE, SVE2), RISC-V (Vector extension).
- Provider micro-benchmarking at startup: AES, ChaCha20, BLAKE3, Ascon, Twofish, Serpent — measured in MB/s, cycles/byte, and CPU %.
- Provider selection: when a policy asks for a CSPRNG by capability, the registry returns the fastest provider on *this CPU* that satisfies the capability tags.
- Persistent cache: the benchmark results are cached to `/var/lib/scuttle/hwbench.json` keyed by CPU model string, so the startup cost is paid once per host.
**Scope — out:**
- Power measurement (handled in Layer 16, Performance Laboratory).
- GPU offload — explicitly out of scope; the framework runs on hosts without GPUs.
**Public interface (sketch):**
```c
typedef struct nwipe_hw_caps {
int has_aes_ni;
int has_avx;
int has_avx2;
int has_avx512;
int has_sse2;
int has_sse4_2;
int has_sha_ni;
int has_arm_neon;
int has_arm_crypto;
int has_arm_sve;
int has_arm_sve2;
int has_riscv_vector;
char cpu_model[128];
char cpu_vendor[32];
} nwipe_hw_caps_t;
int nwipe_hw_detect(nwipe_hw_caps_t *out);
int nwipe_hw_benchmark_provider(const nwipe_prng_v1 *p,
const nwipe_hw_caps_t *hw,
double *out_mbps, double *out_cpu_pct);
const nwipe_prng_v1 *nwipe_prng_select_fastest(const char **required_caps,
const char **preferred_caps,
const nwipe_hw_caps_t *hw);
```
**Dependencies:** libc, `<cpuid.h>` (x86), `/proc/cpuinfo` (ARM fallback).
**Integration points:**
- The `nwipe_hw_caps_t` is consumed by Layer 4's `nwipe_prng_by_capability`.
- Benchmark results are emitted to Layer 16 for the historical record.
- When a profile's `prng_pool` names a specific provider (e.g. `Paranoid` mandates `BLAKE3_XOF`), the hardware layer is consulted only to choose *which implementation* of that provider (portable vs. AVX-512); the algorithm name is not overridden.
**Conformance:**
- On an x86-64-v3 host (Haswell+), AES-CTR with AES-NI must benchmark at ≥ 3 GB/s single-threaded; portable AES-CTR at ≥ 200 MB/s. The framework must select the AES-NI variant when capability `hardware_accelerated` is requested.
- The benchmark cache must invalidate when the CPU model string changes (e.g. container migration between hosts).
### Layer 9 — Secure Erase Integration
**Purpose.** Provide a unified interface to firmware-level sanitization commands across ATA, NVMe, and SCSI. This is the layer that lets Scuttle recommend "purge, don't overwrite" for SSDs — the right answer for flash media.
**Scope — in:**
- **ATA Secure Erase** (already partially in `src/se_ata.c`): set password, issue SECURITY ERASE UNIT, clear password. Both standard and Enhanced variants.
- **ATA Enhanced Erase**: vendor-specific random data write, mandated by the standard to be cryptographically robust.
- **NVMe Format**: format the namespace with a user-selected pattern (0, 1, 0xFF) or no pattern.
- **NVMe Sanitize** (already partially in `src/se_nvme.c`): Block Erase, Crypto Erase, Overwrite — three actions, each with status polling.
- **NVMe Crypto Erase**: invoke Sanitize with Crypto Erase action; on self-encrypting drives this rotates the data encryption key, cryptographically destroying all data in milliseconds.
- **SCSI SANITIZE** (Block Erase, Crypto Erase, Overwrite) and **SCSI FORMAT UNIT**.
- **TRIM / Discard**: issue `BLKDISCARD` / `FITRIM` to mark all blocks as unused. Useful as a pre-step before overwrite on SSDs, and as a stand-alone "clear" action for non-sensitive data.
**Scope — out:**
- The *decision* to use firmware erase vs. overwrite — that is the policy engine's job (§4).
- Vendor-specific proprietary utilities (e.g. Samsung Magician, Intel MAS) — out of scope; only standards-based commands.
**Public interface (sketch):**
```c
typedef enum {
NWIPE_SE_ATA_STANDARD,
NWIPE_SE_ATA_ENHANCED,
NWIPE_NVME_FORMAT_ZERO,
NWIPE_NVME_FORMAT_ONE,
NWIPE_NVME_FORMAT_FF,
NWIPE_NVME_SANITIZE_BLOCK_ERASE,
NWIPE_NVME_SANITIZE_CRYPTO_ERASE,
NWIPE_NVME_SANITIZE_OVERWRITE,
NWIPE_SCSI_SANITIZE_BLOCK_ERASE,
NWIPE_SCSI_SANITIZE_CRYPTO_ERASE,
NWIPE_SCSI_FORMAT,
NWIPE_TRIM,
} nwipe_firmware_op_t;
typedef struct nwipe_firmware_result {
int ok;
int duration_sec;
char command[64];
char status_string[256]; // from the device
int post_op_verify_ok; // re-read after op
} nwipe_firmware_result_t;
int nwipe_firmware_invoke(nwipe_device_t *dev, nwipe_firmware_op_t op,
nwipe_firmware_result_t *out);
int nwipe_firmware_poll(nwipe_device_t *dev, nwipe_firmware_op_t op,
int (*progress)(int pct, void *u), void *user);
```
**Dependencies:** Layer 1 (device), libata (kernel, via ioctl), libnvme, Linux SG_IO for SCSI.
**Integration points:**
- Invoked by Layer 3 (Wipe Engine) when the policy says so.
- The result is bound into Layer 7 (Audit) as a "pass" of kind `FIRMWARE_SE` / `FIRMWARE_SANITIZE`.
- For NVMe Crypto Erase, the duration is typically milliseconds; the framework must not artificially delay the polling loop, but must verify the namespace is empty afterward via Layer 6.
- Layer 2 (Media Intelligence) sets `purge_method` on the descriptor, which the policy uses to choose the firmware op.
**Conformance:**
- A test against a Samsung 980 Pro (or any drive implementing NVMe Sanitize) must succeed in Crypto Erase, must report the device empty afterward, and must complete in under 5 seconds for a 1 TB drive.
- The framework must refuse to issue a firmware erase command to a device that doesn't claim support for it (per Layer 1's `supports_*` flags), regardless of operator pressure.
### Layer 10 — Job Scheduler
**Purpose.** Manage the lifecycle of one or more wipe jobs on one or more devices. Replace upstream nwipe's one-job-per-thread model with an explicit scheduler that supports parallelism, priorities, and grouping.
**Scope — in:**
- **Sequential mode**: one device at a time. Default. Lowest CPU and power draw.
- **Parallel mode**: up to N jobs concurrently, where N is the count of CPU cores minus headroom (default `min(cpus-1, 8)`). Each job gets its own PRNG state and verification stream.
- **Priority queue**: jobs tagged `low`/`normal`/`high`. A `high` job preempts a `normal` job's slot when one frees up.
- **Device groups**: jobs grouped by rack, by host, by operator. A group can be paused/resumed/cancelled atomically.
- **Rack wipe**: an operator-scoped collection of jobs across multiple hosts (driven by Layer 14).
- **Cluster wipe**: a fleet-scoped collection driven by Layer 14, with a single coordinator and many workers.
**Scope — out:**
- Distributed coordination (Raft, etcd, etc.) — out of scope for v1.0. Cluster wipe (Layer 14) is driven by an external coordinator (a simple JSON-over-HTTP orchestrator) and the framework is a worker.
**Public interface (sketch):**
```c
typedef struct nwipe_scheduler nwipe_scheduler_t;
typedef struct nwipe_job_handle { uuid_t id; int slot; int priority; } nwipe_job_handle_t;
nwipe_scheduler_t *nwipe_scheduler_new(nwipe_sched_mode_t mode, int max_slots);
int nwipe_scheduler_submit(nwipe_scheduler_t *s, nwipe_job_t *job, nwipe_job_handle_t *h);
int nwipe_scheduler_wait(nwipe_scheduler_t *s, nwipe_job_handle_t *h,
nwipe_result_t *out);
int nwipe_scheduler_cancel(nwipe_scheduler_t *s, nwipe_job_handle_t *h);
int nwipe_scheduler_pause_group(nwipe_scheduler_t *s, const char *group_id);
int nwipe_scheduler_resume_group(nwipe_scheduler_t *s, const char *group_id);
```
**Dependencies:** pthreads, Layer 3.
**Integration points:**
- The scheduler is the single owner of all worker threads; Layer 3's `nwipe_job_run` is invoked from a worker thread.
- Progress callbacks from Layer 3 are routed through the scheduler so the UI sees a consistent view of all running jobs.
- Layer 14 (Enterprise) talks to the scheduler over a Unix-domain socket for remote control.
**Conformance:**
- 8 parallel jobs on 8 SATA SSDs must complete in the wall-clock time of the slowest single job, plus ≤ 10% overhead.
- Cancelling a group mid-wipe must leave all member devices in a known state (verified-empty or original — the certificate records which).
### Layer 11 — Reporting
**Purpose.** Produce human-readable reports for operators, auditors, executives, and researchers. Reports are derived from the canonical audit record (Layer 7); they are presentation, not source of truth.
**Scope — in:**
- **Summary**: one page per job, one row per device, key facts only.
- **Detailed**: per-pass breakdown, verification results, SMART delta, throughput graph.
- **Executive**: aggregate across many jobs (e.g. a rack wipe), charts of throughput, completion %, exceptions.
- **Compliance**: maps results to a named compliance regime (NIST 800-88, ISO 27040, GDPR Art. 32, HIPAA Security Rule). Output is a checklist with evidence references.
- **Research**: full statistical data (entropy, chi-square, byte distribution, recovery-attempt logs).
- **Debug**: full internal log with timestamps, thread IDs, syscall latencies.
**Scope — out:**
- The certificate (Layer 7) is the source of truth; reports are derived and may be regenerated from a certificate at any time.
**Public interface (sketch):**
```c
typedef enum {
NWIPE_REPORT_SUMMARY, NWIPE_REPORT_DETAILED, NWIPE_REPORT_EXECUTIVE,
NWIPE_REPORT_COMPLIANCE, NWIPE_REPORT_RESEARCH, NWIPE_REPORT_DEBUG,
} nwipe_report_kind_t;
int nwipe_report_generate(nwipe_audit_record_t *rec, nwipe_report_kind_t kind,
const char *format, int fd_out);
// format ∈ {"pdf","html","csv","md","txt"}
```
**Report contents always include:**
- Speed-over-time graph (per-pass throughput)
- Temperature-over-time graph (where SMART temperature is available)
- SMART health before/after (any attribute deltas flagged)
- Wear level before/after (SSD remaining life)
- Bandwidth distribution (histogram of per-second MB/s)
- Errors and retry count
- Verification result summary (pass/fail per pass, with LBA ranges on failure)
**Dependencies:** Layer 7 (audit), Layer 5 (hashes for integrity stamping), a charting library (matplotlib via a Python exporter plugin, or a bundled C charting lib for embedded builds).
**Integration points:**
- Reports are generated at job completion by Layer 3 invoking Layer 11 with the audit record.
- Reports may be regenerated later from a stored certificate (the JSON form) — this is how an auditor pulls a report months after the wipe.
- Layer 13 (UI) shows a simplified live "report" during the wipe; the full report is post-completion.
**Conformance:**
- A compliance report generated from a certificate must, when regenerated from the same certificate months later, produce a byte-identical PDF (deterministic layout, no embedded timestamps).
- The compliance report must cite specific fields of the audit record by JSON path (e.g. "NIST 800-88 Purge: see `audit_record.final_verify.ok == true`").
### Layer 12 — Plugin System
**Purpose.** Provide the runtime infrastructure for loading, registering, and isolating the six plugin kinds: `algorithms/`, `verification/`, `reports/`, `exporters/`, `profiles/`, `drivers/`. This is the layer that makes §4's evolution scenarios mechanical rather than surgical.
**Scope — in:**
- Plugin discovery: scan `$prefix/lib/scuttle/<kind>/*.so` at startup.
- Plugin loading: `dlopen` with `RTLD_LOCAL | RTLD_NOW` (no lazy binding; failures surface at startup, not mid-wipe).
- Plugin registration: each plugin exports a single symbol `<kind>_probe_v1(nwipe_registry_t *)` that registers its providers.
- Plugin isolation: a plugin's symbols are not exported into the global namespace; it can only call into the core through the published ABI table.
- Plugin versioning: each plugin declares the ABI version it was built against; the core refuses to load a plugin built against a future ABI.
- Plugin sandbox (future, v2.0): seccomp filter restricting plugins to a syscall whitelist (no network, no fork, no exec). v1.0 ships without the sandbox; plugins are trusted.
**Scope — out:**
- Cross-plugin communication. Plugins do not see each other; they communicate only through the registry.
**Public interface (sketch):**
```c
typedef struct nwipe_registry {
// hashmaps by name and by capability
nwipe_hashmap_t *prngs;
nwipe_hashmap_t *hashes;
nwipe_hashmap_t *verify_levels;
nwipe_hashmap_t *reports;
nwipe_hashmap_t *exporters;
nwipe_hashmap_t *profiles;
nwipe_hashmap_t *drivers;
} nwipe_registry_t;
typedef int (*nwipe_plugin_probe_v1)(nwipe_registry_t *reg);
int nwipe_plugin_load_directory(nwipe_registry_t *reg, const char *dir);
int nwipe_plugin_load_file(nwipe_registry_t *reg, const char *path);
```
**Dependencies:** libdl, libc.
**Integration points:**
- Consumed by Layer 3 (Wipe Engine), Layer 4 (PRNG), Layer 5 (Hash), Layer 6 (Verification), Layer 7 (Audit), Layer 11 (Reporting), and Layer 1 (Device Discovery, for `drivers/`).
- The `profiles/` plugin kind is special: it loads TOML files (not `.so`), validating them against the JSON Schema (§4.4) and registering the resulting `nwipe_profile_t` structs.
**Conformance:**
- A build with the `providers/` directory emptied (core-only build) must still produce a working binary that can do `Zero` and `One` passes.
- A plugin built against ABI v1 must load into a v1 core and refuse to load into a v2 core with a clear error message.
- Loading a plugin whose `self_test` fails must not abort startup; the plugin's providers must be unavailable but the rest of the registry must be intact.
### Layer 13 — User Interfaces
**Purpose.** Expose the framework to operators, scripts, and remote systems through a family of interchangeable frontends. All frontends talk to the same core; none of them embed business logic.
**Scope — in:**
*Classic:*
- **ncurses** — the upstream nwipe UI, preserved and lightly modernized. This is the only UI guaranteed to work on a serial console, a PXE-booted initramfs, and a 64 MB embedded target.
*Modern:*
- **TUI** — a rethought text UI (think `lazygit`, `gh dash`, `k9s`) with split panes for device list / job detail / log, mouse support where available, and a command palette. Built on the same ncurses backend but with a richer widget set.
- **CLI** — a non-interactive command-line tool with subcommands (`scuttle list`, `scuttle wipe`, `scuttle verify`, `scuttle audit`). Every operator action is scriptable here.
- **Batch** — read a YAML/JSON job specification and execute it headless. Used by PXE mode and by cron.
- **JSON API** — a Unix-domain-socket server exposing the same operations as the CLI, with JSON in/out. Used by automation.
- **REST API** — same surface as JSON API, over HTTPS, with operator authentication (Layer 14). Used by enterprise integrations.
- **PXE Mode** — a boot-time entry point that fetches a job spec from a configured URL, executes it, posts the certificate to a configured URL, and shuts down. No operator present.
*Optional (build-time flag, not in default binary):*
- **Cockpit module** — for RHEL/Fedora hosts with Cockpit installed.
- **WebUI** — a single-page app served by the REST API backend, for browser-based operation. Not intended for production wipes (a browser tab is a fragile substrate for a 12-hour job); intended for *review* of certificates and *configuration* of profiles.
- **Remote Console** — a thicker client (Electron or Tauri) for fleet operators.
**Scope — out:**
- Mobile app — explicitly out of scope. Sanitization is not a phone-friendly activity.
**Public interface (sketch):** the CLI surface, which is the canonical contract:
```
scuttle list [--bus=TYPE] [--json]
scuttle inspect <device> [--json]
scuttle wipe <device> --profile=NAME [--prng=NAME] [--verify=LEVEL]
[--no-sign] [--report=FORMAT...] [--dry-run]
scuttle batch <spec.yaml|json>
scuttle verify <device> --certificate=FILE
scuttle audit search --operator=ID --from=DATE --to=DATE
scuttle benchmark [--provider=NAME] [--json]
scuttle selftest
scuttle serve --mode={json|rest} --listen=ADDR [--auth=MECH]
```
**Dependencies:** ncurses (classic UI), libreadline (CLI), libmicrohttpd or equivalent (REST), libcurl (PXE fetch).
**Integration points:**
- All UIs call the same core API; the CLI is the reference implementation of the contract.
- The CLI's `--json` flag must produce output that round-trips through the JSON API without loss.
- Layer 14 (Enterprise) consumes the REST API for its own integrations.
**Conformance:**
- Every operator action available in the ncurses UI must have a CLI equivalent. (Reverse is not required: some CLI actions are automation-only.)
- A `--dry-run` of a wipe must produce a complete plan (passes, PRNGs, verification level, estimated duration) without touching any device.
- The CLI must be scriptable in `set -e` / `set -o pipefail` shells: predictable exit codes (0 success, 1 verification failure, 2 device error, 3 operator abort, 4 policy refusal), no ANSI escapes when stdout is not a TTY.
### Layer 14 — Enterprise Features
**Purpose.** Provide the additive features needed to deploy Scuttle across a fleet of hosts in an ITAD facility, a data center, or an enterprise decommissioning workflow. None of these features are required for the default walk-up operator; all of them are required for fleet operation.
**Scope — in:**
- **Remote Agent** — a daemon mode that exposes the JSON/REST API on a stable port, accepts jobs from a coordinator, and reports results. This is the *worker* half of cluster wipe.
- **PXE Boot Integration** — a documented PXE config + initramfs hook that boots the worker, fetches its job spec, executes it, and posts the certificate.
- **Wake-on-LAN** — the coordinator can wake hosts that have been powered down for storage.
- **Inventory** — the worker reports its device list to the coordinator at boot, before any wipe. This is the "what could be wiped" view.
- **Asset Tracking** — each device's serial/WWN is matched against an asset database (configurable connector: CSV import, REST API, or SQL).
- **LDAP / Active Directory** — operator authentication for the REST API. Operators are identified by distinguished name; the DN is bound into the audit record.
- **Certificate Authority Integration** — the worker enrolls with an internal CA to obtain a short-lived code-signing certificate for signing audit records. This avoids per-operator key management.
- **Remote Logging** — audit records are streamed to a remote syslog collector (RFC 5424) or to a SIEM (Splunk HEC, Elastic _bulk, or generic HTTP webhook) in real time, not just at job completion.
- **Webhook Notifications** — at configurable lifecycle events (job started, pass complete, job completed, job failed), the worker POSTs a JSON event to a configured URL.
**Scope — out:**
- The coordinator itself. Scuttle ships the worker; the coordinator is a separate, simpler project (a Flask/FastAPI app with a database) that consumes the worker's API. Keeping it separate prevents the framework from growing a database dependency.
**Public interface (sketch):** the worker's REST surface (subset):
```
GET /v1/inventory → list of nwipe_device_t
POST /v1/jobs → submit a wipe job spec, returns job_id
GET /v1/jobs/{id} → status + progress + audit (if done)
POST /v1/jobs/{id}/cancel
GET /v1/jobs/{id}/certificate → audit record (JSON / PDF / ...)
POST /v1/jobs/{id}/report → regenerate a report from the cert
GET /v1/profiles → list registered profiles
GET /v1/providers → list registered PRNG/hash/etc.
GET /v1/healthz
```
**Dependencies:** Layer 13 (UI server), Layer 7 (audit), Layer 10 (scheduler), libcurl (webhooks), libldap (AD), GSSAPI/Kerberos (optional).
**Integration points:**
- The remote agent is the REST API in `serve --mode=rest` mode (Layer 13).
- The audit records produced are the same canonical JSON as in single-host mode; the only addition is the operator DN and the CA-signed certificate chain.
- The webhook payload is a subset of the audit record, suitable for Slack/Teams/email gateways.
**Conformance:**
- A worker, after a power loss mid-wipe, must on restart detect the in-progress job (from a local state file), report it as `interrupted` to the coordinator, and either resume (if the policy says resumable) or mark the device as `verification_required` for operator review.
- A worker whose CA-signed certificate has expired must refuse to start new jobs but must continue to serve already-completed certificates.
### Layer 15 — Security
**Purpose.** Ensure the framework itself is not a vector for compromise. A tool whose job is to destroy data must not, in doing so, leak data, leak keys, or be subverted by a malicious input.
**Scope — in:**
- **Secure Memory** — all key material (PRNG seeds, signing keys, ATA Secure Erase passwords) is allocated through `libsodium`'s `sodium_malloc` (or an equivalent), which guards the region with `mlock` and zeroizes it on free.
- **Constant-time routines** — all cryptographic comparisons (e.g. signature verification, hash equality checks used in access-control decisions) use constant-time implementations.
- **Memory zeroization** — every buffer that held key material, PRNG state, or hashed seed is explicitly zeroized before free. `memset` is not sufficient (the compiler may elide it); `sodium_memzero` or `explicit_bzero` is used.
- **Self-tests at startup** — every provider's `self_test()` is run at process startup. A failure removes the provider from the registry.
- **Known Answer Tests (KAT)** — every cryptographic primitive ships with KAT vectors; the startup self-test runs them.
- **Continuous RNG Tests** — the entropy source (`/dev/urandom` or `/dev/hwrng`) is health-checked at startup (NIST SP 800-90B-style repetition and adaptive proportion tests). A failure aborts startup; the framework refuses to wipe with a broken entropy source.
- **Startup Validation** — the binary self-hashes at startup (optional, build-time flag) and refuses to run if the on-disk hash does not match the embedded hash. Defends against binary patching on a compromised host.
- **FIPS-style Health Checks** — when built in `--enable-fips-mode` (links against OpenSSL FIPS module), the framework defers all crypto decisions to the FIPS module and inherits its self-tests.
**Scope — out:**
- Side-channel resistance beyond constant-time comparisons (e.g. power analysis, cache timing) — out of scope for v1.0. Documented as a known limitation.
- Full sandboxing of plugins — v2.0 (§5 Layer 12).
**Public interface (sketch):**
```c
void *nwipe_secure_alloc(size_t len); // mlock + canary
void nwipe_secure_free(void *p, size_t len); // zeroize + munlock
int nwipe_secure_memcmp(const void *a, const void *b, size_t len); // constant-time
int nwipe_selftest_run_all(void); // returns 0 on success
int nwipe_rng_health_check(void); // returns 0 on healthy entropy source
```
**Dependencies:** libsodium (or bundled equivalent), `/dev/urandom`, optional OpenSSL FIPS module.
**Integration points:**
- Layer 4 (PRNG) uses `nwipe_secure_alloc` for state buffers.
- Layer 7 (Audit) uses `nwipe_secure_memcmp` for signature verification.
- Layer 14 (Enterprise) invokes `nwipe_rng_health_check` before each job start as a defense-in-depth measure.
**Conformance:**
- A `valgrind --tool=memcheck` run over a complete wipe job must report zero leaks of memory allocated through `nwipe_secure_alloc` (it may report the canary overhead as "still reachable"; that's expected).
- A failing KAT at startup must cause the provider to be unavailable and a CRITICAL log entry to be emitted, but must not crash the process.
- The startup entropy health check must abort the process if `/dev/urandom` returns 1024 consecutive identical bytes (a deliberately broken RNG mock is used to test this).
### Layer 16 — Performance Laboratory
**Purpose.** Measure every provider on every supported host, record the results, and use them to drive hardware-aware provider selection (Layer 8). A side benefit: the laboratory produces the data for the public performance page in the project documentation.
**Scope — in:**
- Per-provider micro-benchmarks: MB/s, CPU %, cycles/byte, peak memory, output entropy (for PRNGs).
- Per-hash benchmarks: MB/s, cycles/byte.
- Per-combination benchmarks: end-to-end wipe speed for representative profiles on representative hardware.
- Power measurement (where `powertop` or RAPL is available): watts drawn during the wipe.
- Historical storage: results are appended to `/var/lib/scuttle/bench-history.jsonl` keyed by CPU model + provider name + ABI version. This is the dataset used by Layer 8's selection logic when the local cache is cold.
**Scope — out:**
- Comparative marketing benchmarks against other tools. The lab measures Scuttle providers; it does not measure `shred`, `scrub`, or `dd`.
**Public interface (sketch):**
```c
typedef struct nwipe_bench_result {
char provider[64];
char hash[32];
char cpu_model[128];
char abi_version[16];
double mbps;
double cpu_pct;
double cycles_per_byte;
double peak_memory_mb;
double output_entropy; // for PRNGs
double power_watts; // -1 if unavailable
char timestamp[32];
} nwipe_bench_result_t;
int nwipe_bench_run_provider(const nwipe_prng_v1 *p, nwipe_bench_result_t *out);
int nwipe_bench_run_hash(const nwipe_hash_v1 *h, nwipe_bench_result_t *out);
int nwipe_bench_run_profile(const nwipe_profile_t *prof, nwipe_device_t *dev,
nwipe_bench_result_t *out);
int nwipe_bench_history_append(const nwipe_bench_result_t *r);
int nwipe_bench_history_query(const char *cpu_model, const char *provider,
nwipe_bench_result_t *out);
```
**Dependencies:** Layer 8, libc, optional RAPL access (`/sys/class/powercap/`).
**Integration points:**
- The selection logic in Layer 8 calls `nwipe_bench_history_query` first; only on a miss does it run a live benchmark.
- Layer 11 (Reporting) can include benchmark data in the Debug report.
- Layer 17 (Research Mode) consumes the raw cycles/byte and power data for academic study.
**Conformance:**
- The benchmark must produce stable results: three consecutive runs of the same provider on the same idle host must report MB/s within 5% of each other.
- The benchmark must explicitly *not* use the host's primary disk as a target — it benchmarks the PRNG/hash in memory, not the I/O path.
### Layer 17 — Research Mode
**Purpose.** Make Scuttle a useful instrument for academic study of storage sanitization, data recovery resistance, and media reliability. This is not a profile (though a `Research` profile exists); it is a *mode* that any profile can opt into, which records extra data and exposes extra hooks.
**Scope — in:**
- **Compression ratio** of the wiped device — compress the wiped region with zstd and record the ratio. A truly random wipe should not compress; a `Zero` wipe compresses maximally. Useful for detecting PRNG flaws.
- **Entropy** at multiple granularities (per-MiB, per-GiB, whole-device).
- **Pattern analysis** — run the wiped region through `ent`-style tests: serial correlation coefficient, arithmetic mean, Monte Carlo π estimation.
- **Recovery attempts** — optionally, after a wipe, attempt to read back previously-known patterns (for a test device pre-loaded with a known pattern) and report how much was recoverable. This requires a controlled test device and is opt-in.
- **ECC behavior** — on devices that expose ECC information (some NVMe drives via vendor-specific logs), record ECC error rates before and after wipe.
- **Wear statistics** — for SSDs, record SMART wear-level attributes before and after, and the change in available spare blocks.
**Scope — out:**
- Physical recovery attempts (MFM, STM) — out of scope; the framework records *logical* recoverability, not physical.
**Public interface (sketch):**
```c
typedef struct nwipe_research_data {
double compression_ratio; // wiped_bytes / compressed_bytes
double entropy_per_mib_mean;
double entropy_per_mib_stdev;
double entropy_per_gib_mean;
double serial_correlation;
double arithmetic_mean;
double monte_carlo_pi_estimate;
double chi_square;
size_t recovery_known_pattern_bytes;
size_t recovery_attempted_bytes;
size_t recovery_recovered_bytes;
int ecc_pre_wipe_errors;
int ecc_post_wipe_errors;
int wear_pre_pct;
int wear_post_pct;
} nwipe_research_data_t;
int nwipe_research_collect(nwipe_job_t *job, nwipe_research_data_t *out);
```
**Dependencies:** Layer 6 (verification reads), libzstd (for compression), Layer 5 (hashes for the patterns).
**Integration points:**
- The research data block is appended to the audit record (Layer 7) under a `research` key.
- Research data is included in the Research report (Layer 11).
- The Research profile (§7) sets `research_mode = true`, which enables collection regardless of which wipe method runs.
**Conformance:**
- For a 1 GiB wiped region of pure `Zero`, `compression_ratio` must be ≥ 1000 (zstd level 19).
- For a 1 GiB wiped region of ChaCha20 output, `compression_ratio` must be ≤ 1.01 (within zstd's overhead).
- The arithmetic mean of a 1 GiB random wipe must be in [127.0, 128.5] (expected 127.5 for uniform bytes).
### Layer 18 — Future Expansion
**Purpose.** Reserve the architectural hooks for capabilities that are not yet implemented but that the framework must not preclude. Each hook is an interface with no implementation in v1.0; the interface is published so that downstream forks and enterprise integrators can plug in without modifying the core.
**Reserved interfaces:**
- **Post-Quantum PRNG Providers** — `nwipe_prng_v1` already has a `post_quantum` capability tag. Providers like Kyber/ML-KEM-derived XOFs, or NIST PQC finalists in CTR mode, can be added as `providers/` plugins. No core change required.
- **Hardware Security Modules (HSM)** — the `nwipe_signer_t` interface (Layer 7) is opaque; it can be backed by an in-process key, an OpenSSL engine, an HSM via PKCS#11, or a cloud KMS. The HSM path is a `exporters/hsm/` plugin that implements `nwipe_signer_t` against the PKCS#11 API.
- **Trusted Platform Module (TPM)** — TPM 2.0 can be used (a) as an entropy source via `nwipe_entropy_source_t` (a Layer 4 extension), (b) as a sealed-key store for the operator signing key, and (c) as a remote-attestation anchor for Layer 14 fleet deployments. Each is a separate plugin.
- **OpenSSL 3.0 Providers** — OpenSSL 3.0's provider model is a natural fit for the `nwipe_prng_v1` / `nwipe_hash_v1` interfaces. A `providers/openssl/` plugin bridges OpenSSL providers into the Scuttle registry, gaining access to FIPS-validated implementations without bundling them.
- **PKCS#11** — for HSM and smartcard signing. Same `nwipe_signer_t` interface as above.
- **Remote Attestation** — the worker (Layer 14) emits a TPM quote at job start, binding the binary hash, the device list, and the job spec into a single attested record. The coordinator (a separate project) verifies the quote against expected measurements. This closes the loop: the coordinator can prove *which binary* ran *on which host* against *which device*. Reserved as a `exporters/attest/` plugin.
**Scope — out:**
- Cloud KMS support (AWS KMS, Azure Key Vault, GCP KMS) — same `nwipe_signer_t` interface, but the implementation lives in a separate repo to avoid pulling cloud SDKs into the core.
**Public interface (sketch):**
```c
typedef struct nwipe_signer {
const char *backend; // "openpgp", "x509", "pkcs11", "tpm", "kms"
const char *key_id;
int (*sign)(void *ctx, const uint8_t *msg, size_t msg_len,
uint8_t **sig, size_t *sig_len);
int (*fingerprint)(void *ctx, char *out, size_t out_len);
void (*free)(void *ctx);
} nwipe_signer_t;
typedef struct nwipe_entropy_source {
const char *name;
int (*read)(void *ctx, uint8_t *out, size_t len);
int (*health_check)(void *ctx);
} nwipe_entropy_source_t;
```
**Dependencies:** None in v1.0 (no implementation); plugins bring their own.
**Integration points:**
- A signing plugin is selected by profile (e.g. `Air Gap` profile mandates an offline OpenPGP signer; `Enterprise` profile mandates a CA-backed X.509 signer).
- An entropy-source plugin is selected by configuration; the default is `/dev/urandom`.
- An attestation plugin is invoked by Layer 14 at job start and at job completion.
**Conformance:**
- The interfaces above must remain stable across v1.x releases. A v1.0 signer plugin must load and operate in a v1.5 binary without recompilation.
- Adding a new signer backend must not require any change to Layer 7 (Audit) code; Layer 7 calls `nwipe_signer_t->sign()` and does not know which backend is in use.
---
## 6. Cryptographic Provider Catalog
This section enumerates every algorithm the framework ships, organized by tier. The catalog is the source of truth for the `providers/` directory layout. Each entry specifies: name, class, capability tags (§4.5), minimum seed/key bytes, hardware acceleration, KAT vector source, and license.
### 6.1 PRNG Providers
| Name | Class | Capabilities | Seed (bytes) | HW accel | KAT source | License |
|---|---|---|---|---|---|---|
| AES-CTR (portable) | Block cipher CTR | `csprng`,`block_cipher_ctr`,`fips_eligible` | 32 | — | NIST CAVP | GPL-2.0+ |
| AES-CTR (AES-NI) | Block cipher CTR | `csprng`,`block_cipher_ctr`,`hardware_accelerated`,`fips_eligible` | 32 | x86 AES-NI | NIST CAVP | GPL-2.0+ |
| AES-CTR (ARMv8 CE) | Block cipher CTR | `csprng`,`block_cipher_ctr`,`hardware_accelerated`,`fips_eligible` | 32 | ARMv8 CE | NIST CAVP | GPL-2.0+ |
| ChaCha20 (portable) | Stream cipher | `csprng`,`stream_cipher` | 32 | — | RFC 8439 | GPL-2.0+ |
| ChaCha20 (AVX2) | Stream cipher | `csprng`,`stream_cipher`,`hardware_accelerated` | 32 | x86 AVX2 | RFC 8439 | GPL-2.0+ |
| ChaCha20 (AVX-512) | Stream cipher | `csprng`,`stream_cipher`,`hardware_accelerated` | 32 | x86 AVX-512 | RFC 8439 | GPL-2.0+ |
| ChaCha20 (NEON) | Stream cipher | `csprng`,`stream_cipher`,`hardware_accelerated` | 32 | ARM NEON | RFC 8439 | GPL-2.0+ |
| XChaCha20 | Stream cipher, ext-nonce | `csprng`,`stream_cipher` | 32 | (inherits ChaCha20) | RFC 8439 (draft) | GPL-2.0+ |
| Salsa20 | Stream cipher | `csprng`,`stream_cipher` | 32 | — | eSTREAM | GPL-2.0+ |
| HC-256 | Stream cipher | `csprng`,`stream_cipher` | 32 | — | eSTREAM | GPL-2.0+ |
| Rabbit | Stream cipher | `csprng`,`stream_cipher` | 16 | — | eSTREAM | GPL-2.0+ |
| ISAAC | PRNG | `csprng` | 256 | — | Bob Jenkins' ref | Public domain |
| ISAAC64 | PRNG (64-bit) | `csprng` | 256 | — | Bob Jenkins' ref | Public domain |
| MT19937 | PRNG (not crypto) | (none — non-crypto) | 8 | — | Matsumoto ref | BSD |
| SplitMix64 | PRNG (not crypto) | (none — non-crypto) | 8 | — | Stafford ref | Public domain |
| xoroshiro256** | PRNG (not crypto) | (none — non-crypto) | 32 | — | Vigna ref | Public domain |
| Lagged Fibonacci | PRNG (not crypto) | (none — non-crypto) | 55 × 8 | — | Knuth ref | GPL-2.0+ |
| BLAKE3 XOF | XOF | `csprng`,`xof`,`hardware_accelerated` | 32 | AVX2/AVX-512/NEON | BLAKE3 spec | Apache-2.0 |
| SHAKE128 | SHA-3 XOF | `csprng`,`xof`,`fips_eligible` | 32 | (SHA-NI on some x86) | NIST CAVP | GPL-2.0+ |
| SHAKE256 | SHA-3 XOF | `csprng`,`xof`,`fips_eligible` | 64 | (SHA-NI on some x86) | NIST CAVP | GPL-2.0+ |
| KangarooTwelve | Keccak XOF | `csprng`,`xof` | 32 | (Keccak SIMD) | K12 spec | CC0 |
| Ascon-128a | Lightweight AEAD/XOF | `csprng`,`xof`,`aead` | 16 | (SIMD on x86) | NIST LWC | GPL-2.0+ |
| Serpent-CTR | Block cipher CTR | `csprng`,`block_cipher_ctr` | 32 | — | AES finals | Public domain |
| Twofish-CTR | Block cipher CTR | `csprng`,`block_cipher_ctr` | 32 | — | AES finals | BSD |
| Camellia-CTR | Block cipher CTR | `csprng`,`block_cipher_ctr`,`fips_eligible` | 32 | (some x86) | RFC 3713 | GPL-2.0+ |
*Experimental (build-time flag `--enable-experimental-providers`, not in default release):*
| Name | Class | Notes |
|---|---|---|
| Threefish-CTR | Block cipher CTR | Skein's tweakable block cipher |
| Skein-512 XOF | XOF | Skein hash in XOF mode |
| Whirlpool | Hash | Legacy 512-bit hash |
| BLAKE2X | XOF | BLAKE2 extended-output variant |
### 6.2 Hash Providers
| Name | Output (bytes) | XOF | HW accel | KAT source | License |
|---|---|---|---|---|---|
| SHA-256 | 32 | no | SHA-NI (x86), ARMv8 CE | NIST CAVP | GPL-2.0+ |
| SHA-512 | 64 | no | — | NIST CAVP | GPL-2.0+ |
| SHA-3-256 | 32 | no | — | NIST CAVP | GPL-2.0+ |
| SHA-3-512 | 64 | no | — | NIST CAVP | GPL-2.0+ |
| SHAKE128 | variable | yes | — | NIST CAVP | GPL-2.0+ |
| SHAKE256 | variable | yes | — | NIST CAVP | GPL-2.0+ |
| BLAKE2b | 64 | no | AVX2/NEON | RFC 7693 | GPL-2.0+ |
| BLAKE2s | 32 | no | AVX/NEON | RFC 7693 | GPL-2.0+ |
| BLAKE3 | variable | yes | AVX2/AVX-512/NEON | BLAKE3 spec | Apache-2.0 |
| KangarooTwelve | variable | yes | Keccak SIMD | K12 spec | CC0 |
### 6.3 Cipher Providers (for Crypto Erase and signing)
| Name | Class | Key (bytes) | Notes |
|---|---|---|---|
| AES-256-GCM | AEAD | 32 | For internal authenticated channels (e.g. webhook delivery) |
| ChaCha20-Poly1305 | AEAD | 32 | Same |
| Ed25519 | Signature | 32 | Default for signing audit records |
| RSA-PSS-4096 | Signature | — | For X.509 / CA-backed signing |
### 6.4 Signer Backends
| Backend | Use case | Plugin path |
|---|---|---|
| OpenPGP (gpgme) | Air-gapped / individual operator signing | `exporters/signer_openpgp/` |
| X.509 (OpenSSL) | Enterprise / CA-issued certificates | `exporters/signer_x509/` |
| PKCS#11 | HSM, smartcard | `exporters/signer_pkcs11/` |
| TPM 2.0 | Sealed keys, remote attestation | `exporters/signer_tpm/` |
| Ed25519 (raw key) | Default, low-friction | bundled in core |
---
## 7. Profile Catalog
The default profile catalog. Each profile is a TOML file in `profiles/` (§4.4). The table below summarizes the operator-facing surface; the per-profile TOML is the source of truth.
### 7.1 Legacy Profiles (compatibility shims)
These profiles exist to give upstream nwipe users a 1:1 migration path (§12). They are not recommended for new deployments.
| Profile | Upstream method | PRNG | Passes | Verify | Notes |
|---|---|---|---|---|---|
| Legacy Zero | Zero | n/a | 1 | none | Compatibility |
| Legacy DoD | DoD 5220.22-M | Mersenne Twister (legacy) | 3 | none | Compatibility |
| Legacy Gutmann | Gutmann | Mersenne Twister (legacy) | 35 | none | Compatibility |
| Legacy RCMP | RCMP TSSIT OPS-II | Mersenne Twister (legacy) | 7 | none | Compatibility |
| Legacy HMG IS5 | HMG IS5 Enhanced | Mersenne Twister (legacy) | 3 | none | Compatibility |
| Legacy Schneier | Schneier 7-pass | Mersenne Twister (legacy) | 7 | none | Compatibility |
| Legacy BMB | BMB (German) | Mersenne Twister (legacy) | per spec | none | Compatibility |
| Legacy PRNG | Random (1 pass) | user-selectable | 1 | none | Compatibility |
### 7.2 Modern Profiles (recommended)
#### Quick Clear
- **Audience**: ITAD high-throughput, internal IT, decommissioning of low-sensitivity media
- **NIST class**: Clear
- **PRNG**: AES-CTR (AES-NI auto-selected)
- **Passes**: 1 random pass
- **Verify**: final pass only, spot 5%
- **Certificate**: JSON, unsigned
- **Report**: summary + detailed
- **Typical duration**: 60–90 min for a 1 TB HDD
- **Rationale**: For non-sensitive data, one pass with a CSPRNG is sufficient to defeat all software recovery. Cryptographic verification confirms the overwrite happened.
#### Modern Random
- **Audience**: General-purpose default
- **NIST class**: Clear
- **PRNG**: ChaCha20 (AVX2 if available)
- **Passes**: 1 random pass
- **Verify**: final pass, spot 5%
- **Certificate**: JSON, unsigned
- **Report**: detailed
- **Rationale**: Modern default for rotational media. ChaCha20 is faster than AES on hosts without AES-NI (e.g. older ARM, low-power x86).
#### NIST Clear
- **Audience**: NIST SP 800-88 Clear compliance
- **NIST class**: Clear
- **PRNG**: ChaCha20
- **Passes**: 1 random pass (for HDD) / firmware trim+overwrite (for SSD)
- **Verify**: final pass, spot 5%
- **Certificate**: JSON + PDF, unsigned
- **Report**: compliance (NIST 800-88 Clear checklist)
- **Rationale**: Maps directly to NIST SP 800-88 Rev. 1 Clear definition: "logical sanitization that prevents data retrieval with laboratory techniques."
#### NIST Purge
- **Audience**: NIST SP 800-88 Purge compliance
- **NIST class**: Purge
- **PRNG**: (firmware-driven) + ChaCha20 for post-purge overwrite
- **Passes**: 1 firmware purge + 1 random overwrite
- **Verify**: final pass, entire device
- **Certificate**: JSON + PDF, signed (Ed25519 by default)
- **Report**: compliance (NIST 800-88 Purge checklist) + detailed
- **Rationale**: Purge is "physical or cryptographic sanitization that prevents data retrieval with laboratory techniques." For SSDs/NVMe, this means firmware Crypto Erase. The post-purge overwrite is belt-and-braces: cheap insurance against firmware bugs.
#### Enterprise
- **Audience**: Corporate fleets, ITAD facilities
- **NIST class**: Purge
- **PRNG**: BLAKE3 XOF (round-robin with XChaCha20)
- **Passes**: 1 firmware purge + 3 random overwrites (round-robin)
- **Verify**: every pass, entire device + entropy
- **Certificate**: JSON + PDF, signed (X.509 via internal CA), Merkle root
- **Report**: executive + detailed + compliance
- **Integration**: LDAP operator auth, remote logging, webhook notifications
- **Rationale**: Fleet operation. The Merkle root lets an auditor spot-verify any region of any device without re-reading every device end-to-end. Round-robin PRNGs mean a single PRNG flaw cannot compromise the wipe.
#### Paranoid
- **Audience**: Adversary-rich environments, classified-adjacent, high-value IP
- **NIST class**: Purge
- **PRNG**: BLAKE3 XOF, XChaCha20, Serpent-CTR (round-robin, 7 passes)
- **Passes**: 1 firmware purge (where supported) + 7 random overwrites (3 distinct PRNGs)
- **Verify**: every pass, entire device + entropy + statistical
- **Certificate**: signed PDF + Merkle root + manifest
- **Report**: detailed + research + compliance
- **Constraints**: requires signed certificate; aborts on any verify failure
- **Rationale**: Defense in depth. Three cryptographically distinct PRNGs means a flaw in any one does not weaken the wipe. 7 passes is excessive by modern standards (NIST says 1 is enough for Purge), but this profile is for operators whose threat model includes future advances in recovery and whose regulators insist on multi-pass.
#### Research
- **Audience**: Universities, storage researchers, forensics labs
- **NIST class**: n/a (research mode)
- **PRNG**: configurable per run
- **Passes**: configurable
- **Verify**: full statistical
- **Certificate**: JSON + CSV + raw per-block hashes
- **Report**: research
- **Research mode**: enabled (compression ratio, entropy per MiB, pattern analysis, ECC behavior, wear statistics)
- **Rationale**: Not for production sanitization. For producing data about sanitization. The framework records everything it can measure so the researcher can analyze post-hoc.
#### Forensic
- **Audience**: Chain-of-custody destruction of evidence
- **NIST class**: Purge
- **PRNG**: ISAAC64 (deterministic, well-studied)
- **Passes**: 3 random + 1 final zero
- **Verify**: entire device, every pass
- **Certificate**: signed PDF + Merkle root + manifest
- **Report**: detailed + compliance
- **Chain of custody**: operator identity, timestamp, and signature bound into the certificate; the certificate hash is logged to an external system (syslog/SIEM) at the moment of completion
- **Rationale**: For situations where the destruction itself must be provable. ISAAC64 is chosen for its long history of cryptanalysis and its deterministic behavior given a known seed, which makes the wipe reproducible from the certificate.
#### Government
- **Audience**: Federal, regulated industries
- **NIST class**: Purge
- **PRNG**: AES-CTR (FIPS path; AES-NI if available)
- **Passes**: per applicable policy (default 3)
- **Verify**: every pass, entire device
- **Certificate**: signed PDF, FIPS-mode
- **Report**: compliance + executive
- **Build flag**: `--enable-fips-mode` (links against OpenSSL FIPS module; all crypto deferred to it)
- **Rationale**: For deployments where the cryptography itself must be FIPS-validated. AES-CTR is the only FIPS-eligible PRNG in the catalog; it is selected automatically by the FIPS build flag.
#### Air Gap
- **Audience**: Classified, offline
- **NIST class**: Purge
- **PRNG**: SHAKE256 (deterministic, XOF, no hardware dependency)
- **Passes**: 7 random
- **Verify**: every pass, entire device + statistical
- **Certificate**: offline-signed PDF + Merkle root
- **Signer**: OpenPGP, key on removable media; signing happens on a separate, air-gapped signing host
- **Constraints**: refuses to start if any network interface is up; refuses to use a PRNG tagged `hardware_accelerated` (to avoid microarchitectural side channels); refuses to use any plugin under `exporters/` that talks to a network
- **Rationale**: For environments where the wipe host must not be networked and the signing key must not touch the wipe host. The two-phase sign (wipe host emits unsigned certificate; signing host signs) preserves the air gap.
#### Custom
- **Audience**: Power users
- **Configuration**: TOML file, validated against the profile JSON Schema
- **Rationale**: For cases the catalog does not anticipate. The schema validation ensures even custom profiles respect the layering (§4); a custom profile that tries to invoke a non-existent policy or PRNG will fail validation at load time.
---
## 8. Mixed Combinations and Reference Deployments
This section catalogs the *mixed combinations* the architecture enables — deployments where multiple layers and features compose into a coherent operational pattern. Each pattern is described as a worked scenario with the layer stack identified. These are not exhaustive; they are the patterns we expect to see in practice and have designed for explicitly.
### 8.1 ITAD High-Throughput Line
**Pattern**: Quick Clear + AES-NI auto-select + final-pass verification + JSON certificate + remote logging
**Stack**: Layer 1 → 2 → 3 → 4(AES-CTR AES-NI) → 6(final) → 7(JSON) → 14(remote syslog)
A row of 16 wipe stations, each with a 4-bay hot-swap backplane. Operator slides drives in, presses "go" on the ncurses UI, the framework auto-detects (Layer 1), classifies (Layer 2 — typically `hdd_cmr` or `ssd_sata`), picks the `Quick Clear` profile, selects AES-CTR with AES-NI (Layer 8 — 4+ GB/s on a modern host), runs one pass, verifies the final state, and emits a JSON certificate. The certificate is streamed to a central syslog collector (Layer 14). Total wall-clock per 1 TB HDD: ~5 minutes. Per drive cost: power + operator time. Auditable: the central log has every certificate.
### 8.2 Enterprise Fleet Decommission
**Pattern**: Enterprise profile + PXE boot + LDAP auth + CA-signed certificates + Merkle root + webhook
**Stack**: Layer 1 → 2 → 3 → 4(BLAKE3 XOF round-robin with XChaCha20) → 6(every pass + entire device) → 7(JSON+PDF, X.509 signed, Merkle root) → 10(parallel) → 13(PXE + REST) → 14(LDAP + CA + webhook + remote logging)
A data center is being decommissioned. 500 hosts, each with 4 NVMe drives. A central coordinator issues Wake-on-LAN to all hosts; each boots via PXE into the Scuttle initramfs. The worker enrolls with the internal CA, gets a short-lived signing certificate, authenticates the operator via LDAP (the operator's DN is recorded per job), runs the `Enterprise` profile against all 4 drives in parallel, posts certificates to the coordinator, and shuts down. The coordinator's database now has 2000 signed certificates with Merkle roots. An auditor samples 50 drives: for each, they re-read the (now wiped) device, recompute the per-block hashes, and verify they chain to the published Merkle root. Total wall-clock: ~2 hours including reboot time.
### 8.3 Air-Gapped Classified Destruction
**Pattern**: Air Gap profile + SHAKE256 + offline OpenPGP signing + Merkle root + network refusal
**Stack**: Layer 1 → 2 → 3 → 4(SHAKE256) → 6(every pass + statistical) → 7(PDF, OpenPGP signed on separate host, Merkle root) → 15(strict memory zeroization, startup self-test mandatory)
A classified facility. The wipe host has no network interfaces configured (Layer 13 verifies this on startup and refuses to proceed otherwise). The operator boots from read-only media, selects the `Air Gap` profile, and wipes the target. The unsigned certificate is written to a transfer medium (e.g. a write-once optical disc). On a separate, air-gapped signing host, the operator loads the certificate, signs it with their OpenPGP key, and writes the signed certificate back to the transfer medium. The signed certificate is the artifact presented to the regulator. The Merkle root lets the regulator verify the certificate against the (still-wiped) device if needed.
### 8.4 Academic Research Study
**Pattern**: Research profile + Ascon + KangarooTwelve + statistical verification + CSV report + raw per-block hashes
**Stack**: Layer 1 → 2 → 3 → 4(Ascon-128a) → 5(KangarooTwelve) → 6(full statistical) → 7(JSON + CSV + raw hashes) → 11(research report) → 16(perf lab) → 17(research mode enabled)
A university storage lab is studying the reliability of post-wipe recovery on SSDs from different vendors. They use the `Research` profile with Ascon-128a as the PRNG and KangarooTwelve as the hash. For each drive, the framework records: compression ratio, per-MiB entropy distribution, chi-square, byte frequency, ECC behavior, wear statistics, and per-block hashes. The CSV report is loaded into R/Python for analysis. The raw per-block hashes let the researchers verify that two drives wiped with the same seed produce identical block-hash sequences (a control for the experiment).
### 8.5 Federal FIPS Deployment
**Pattern**: Government profile + AES-CTR (FIPS) + KAT self-tests + FIPS-mode build + signed PDF
**Stack**: Layer 1 → 2 → 3 → 4(AES-CTR FIPS path via OpenSSL FIPS module) → 6(every pass) → 7(PDF, FIPS-signed) → 15(startup validation, continuous RNG tests, KAT)
A federal agency requires FIPS 140-3 validated cryptography. The binary is built with `--enable-fips-mode`, linking against OpenSSL's FIPS module. The framework's startup runs the FIPS module's self-tests (Layer 15), runs KAT vectors on every provider (Layer 4), runs the continuous RNG health check (Layer 15), and only then offers the UI. The `Government` profile is selected, which mandates AES-CTR as the only FIPS-eligible PRNG. The signed certificate is emitted with a FIPS-validated signing algorithm. The compliance report maps directly to FIPS 140-3 sections.
### 8.6 Forensic Chain-of-Custody
**Pattern**: Forensic profile + ISAAC64 + entire-device verification + Merkle root + manifest + syslog binding
**Stack**: Layer 1 → 2 → 3 → 4(ISAAC64) → 6(entire device, every pass) → 7(PDF, signed, Merkle root, manifest) → 11(detailed + compliance) → 14(remote syslog at completion)
A law-environment forensic lab must destroy seized media after the chain of custody closes. The `Forensic` profile is selected. ISAAC64 is the PRNG (chosen for its long cryptanalytic history and deterministic reproducibility from a known seed). 3 random passes + 1 final zero pass. Every pass is verified against the entire device. The certificate includes a manifest (list of all per-block hashes) and a Merkle root. The certificate's hash is logged to an external syslog server the instant the wipe completes — creating a tamper-evident timestamp of the destruction. The signed PDF is filed in the case record.
### 8.7 NVMe Data Center Refresh
**Pattern**: NIST Purge + NVMe Sanitize Crypto Erase + post-purge overwrite + JSON+PDF signed certificate
**Stack**: Layer 1(NVMe) → 2(NVMe, recommends purge) → 3 → 9(NVME_SANITIZE_CRYPTO_ERASE) → 4(ChaCha20) → 6(entire device) → 7(JSON+PDF signed)
A cloud provider is refreshing its NVMe fleet. Each drive is self-encrypting; the `NIST Purge` profile's policy for `ssd_nvme` invokes NVMe Sanitize with the Crypto Erase action, which rotates the data encryption key — destroying all user data in milliseconds. The framework then runs one ChaCha20 overwrite pass (belt-and-braces), verifies the entire device, and emits a signed JSON+PDF certificate. Total time per 4 TB drive: ~90 seconds (vs. 8+ hours for an overwrite-only wipe). The signed certificate satisfies the customer's data-destruction attestation requirement.
### 8.8 Embedded SMR Drive Recommissioning
**Pattern**: Custom profile (SMR-aware) + zone-sequential writes + Ascon + minimal PRNG + CSV report
**Stack**: Layer 1(SMR detection) → 2(`hdd_smr_host_managed`) → 3(zone-aware pass) → 4(Ascon-128a — small code size) → 6(spot 5%) → 7(JSON+CSV) → 11(summary)
An embedded appliance (router with an SMR drive) needs periodic sanitization on decommission. The host has 64 MB of RAM. The framework's embedded build excludes everything except: core, Ascon-128a PRNG (smallest code size in the catalog), SHA-256 hash (hardware-accelerated on the appliance's ARM SoC), and the ncurses UI. The wipe is zone-sequential (Layer 3's SMR-aware pass) to avoid triggering the drive's background GC and destroying throughput. The CSV report is small enough to fit on the appliance's flash. Total binary size: under 800 KB stripped.
### 8.9 Paranoid Multi-Algorithm Sweep
**Pattern**: Paranoid profile + BLAKE3 XOF + XChaCha20 + Serpent-CTR round-robin + 7 passes + per-pass verification + entropy + statistical + signed PDF + Merkle root
**Stack**: Layer 1 → 2 → 3(round-robin dispatch) → 4(BLAKE3/XChaCha20/Serpent-CTR) → 5(BLAKE3) → 6(every pass + entropy + statistical) → 7(signed PDF + Merkle root + manifest) → 11(detailed + research + compliance)
A high-value IP holder (defense contractor, financial exchange) is destroying drives that contained classified design material. Threat model includes future advances in recovery and an adversary with nation-state resources. The `Paranoid` profile runs 7 passes round-robin across three cryptographically distinct PRNGs (so a flaw in any one algorithm is not catastrophic). Per-pass verification with full statistical analysis. The signed PDF + Merkle root + manifest lets an independent reviewer verify any block of any pass against the certificate. The research data block (compression ratio, entropy distribution) provides extra assurance that the random data was, in fact, random.
### 8.10 PXE Boot Kiosk
**Pattern**: Batch mode + PXE + Quick Clear + webhook + auto-shutdown
**Stack**: Layer 1 → 2 → 3 → 4(AES-CTR AES-NI) → 6(final) → 7(JSON) → 13(PXE + batch) → 14(webhook)
An ITAD kiosk: operator plugs in a drive, kiosk detects it, fetches a job spec from a central URL ("wipe with Quick Clear, post certificate to /api/jobs"), executes, posts the certificate via webhook, and shuts down. No keyboard, no monitor, no operator authentication (the kiosk is in a locked room). The webhook payload includes the device serial, which the central system matches to the asset database and marks the asset as "sanitized". Total operator interaction: insert drive, close door, press button.
---
## 9. NIST SP 800-88 Conformance Map
NIST SP 800-88 Rev. 1 defines three sanitization classes. Scuttle maps each profile to one of these classes and provides the conformance evidence required by each.
### 9.1 Class Definitions
| Class | Definition | Implementation in Scuttle |
|---|---|---|
| **Clear** | Logical sanitization that protects against data retrieval by ordinary keyboard recovery techniques; sufficient for media that will remain in a controlled environment. | Overwrite with a CSPRNG; for SSDs, TRIM + overwrite. |
| **Purge** | Physical or logical sanitization that protects against laboratory retrieval techniques; required before media leaves a controlled environment. | Firmware-level: ATA Secure Erase (Enhanced), NVMe Sanitize (Crypto or Block Erase), SCSI Sanitize. For SEDs, Crypto Erase (key rotation). Optionally followed by overwrite. |
| **Destroy** | Physical destruction that renders the media unusable. | Out of scope for execution. Scuttle produces a pre-destruction certificate that attests the media was sanitized before destruction; physical destruction is the operator's responsibility. |
### 9.2 Profile → Class Map
| Profile | NIST class | Mechanism | Verification | Certificate |
|---|---|---|---|---|
| Quick Clear | Clear | CSPRNG overwrite (1 pass) | Final, spot 5% | JSON |
| Modern Random | Clear | CSPRNG overwrite (1 pass) | Final, spot 5% | JSON |
| NIST Clear | Clear | CSPRNG overwrite or TRIM+overwrite | Final, spot 5% | JSON+PDF |
| NIST Purge | Purge | Firmware purge + 1 overwrite | Final, entire device | JSON+PDF signed |
| Enterprise | Purge | Firmware purge + 3 overwrites | Every pass, entire device | JSON+PDF signed, Merkle |
| Paranoid | Purge | Firmware purge + 7 multi-algo overwrites | Every pass, entire device + statistical | Signed PDF + Merkle + manifest |
| Research | n/a | Configurable | Full statistical | JSON + CSV + raw hashes |
| Forensic | Purge | 3 overwrites + final zero | Every pass, entire device | Signed PDF + Merkle + manifest |
| Government | Purge | Firmware purge + 3 AES-CTR overwrites (FIPS) | Every pass, entire device | Signed PDF, FIPS-mode |
| Air Gap | Purge | 7 overwrites (no firmware path assumed) | Every pass, entire device + statistical | Offline-signed PDF + Merkle |
### 9.3 Conformance Evidence Per Class
For each class, the framework emits a structured conformance block in the certificate:
```json
{
"nist_800_88": {
"class": "Purge",
"mechanism": "nvme_sanitize_crypto_erase + chacha20_overwrite_1pass",
"evidence": {
"firmware_command_issued": "NVME_SANITIZE crypto-erase",
"firmware_command_exit_status": "success",
"firmware_command_duration_sec": 0.87,
"post_purge_verify_hash": "blake3:7f3a...",
"post_purge_verify_pass": true,
"overwrite_passes": [
{ "prng": "ChaCha20", "seed_digest": "sha256:9c1d...", "verify_hash": "blake3:7f3a...", "verify_pass": true }
]
},
"rationale": "Per NIST SP 800-88 Rev.1 §4.4, Purge on NVMe SSDs is achieved via Sanitize Crypto Erase, which cryptographically destroys data by erasing the data encryption key. The post-purge overwrite and full-device verification are belt-and-braces controls."
}
}
```
### 9.4 Other Compliance Regimes
| Regime | Profile | Mapping notes |
|---|---|---|
| ISO/IEC 27040:2024 (Storage Security) | Enterprise | §6.4.2 sanitization requirements; certificate provides evidence of method, verification, and chain-of-custody. |
| GDPR Article 32 | Quick Clear / NIST Purge | "Appropriate technical measures" — the certificate is the artifact. Profile selection is the organization's risk decision. |
| HIPAA Security Rule §164.310(d)(2)(i) | NIST Purge | "Device and media controls" — requires documented disposal; the signed certificate is the documentation. |
| PCI DSS v4 §9.2 | NIST Purge | "Media destruction" — signed certificate with operator identity satisfies the requirement. |
| DoD 5220.22-M | Paranoid (or Legacy DoD for compatibility) | The original DoD method (3-pass) is preserved as `Legacy DoD`; modern DoD practice accepts NIST Purge. |
| BSI BSI-TR-03112 | NIST Purge | German federal guidance aligns with NIST Purge for modern media. |
| HMG IS5 (Baseline & Enhanced) | Legacy HMG IS5 (compat) or NIST Clear/Purge | UK CESG guidance; preserved as a legacy profile, modern practice prefers NIST classes. |
---
## 10. Threat Model
### 10.1 Adversary Classes
| Class | Capabilities | What defeats them |
|---|---|---|
| **Curious operator** | Boot the host, run `dd` or file-recovery tools | Any overwrite (even a single zero pass) |
| **ITAD recipient** | Receive the drive second-hand, run commercial recovery tools (e.g. Recuva, PhotoRec) | Any single-pass CSPRNG overwrite (Clear) |
| **Forensic recovery lab** | Specialized hardware: MFM, STM, spin-stand, advanced pattern recovery | Firmware purge (Purge) per NIST 800-88; or multi-pass overwrite for HDDs |
| **Nation-state adversary** | Decap, electron microscopy, advanced signal processing on residual magnetic domains | Physical destruction (Destroy) — out of scope for the framework, but the pre-destruction certificate is provided |
| **Insider with framework access** | Can modify the binary, alter the audit record, or skip steps | Signed certificates with offline keys (Air Gap profile); Merkle root verification; remote logging that the operator cannot suppress |
| **Firmware adversary** | Compromised drive firmware that lies about Secure Erase success | Post-purge overwrite + full-device verification; statistical verification (a fake "all zeros" post-purge state from compromised firmware would have anomalous entropy if the drive actually retained data) |
### 10.2 What Each Profile Defends Against
| Profile | Defends against | Does NOT defend against |
|---|---|---|
| Quick Clear | Curious operator, ITAD recipient | Forensic lab, nation-state, firmware adversary |
| Modern Random | Curious operator, ITAD recipient | Forensic lab, nation-state, firmware adversary |
| NIST Clear | ITAD recipient, commercial recovery | Forensic lab on rotational media; nation-state |
| NIST Purge | ITAD recipient, commercial recovery, forensic lab | Nation-state with physical access to platters; compromised firmware (without post-purge overwrite) |
| Enterprise | All of the above + insider with framework access (via signed certs, Merkle, remote logging) | Nation-state with physical access |
| Paranoid | All of the above + future advances in recovery + single-PRNG flaws (via round-robin) | Nation-state with physical access |
| Forensic | All of NIST Purge + chain-of-custody attack (via signing + syslog binding) | Nation-state with physical access |
| Government | All of NIST Purge, with FIPS-validated cryptography | Nation-state with physical access; non-FIPS PRNGs (intentionally excluded) |
| Air Gap | All of Paranoid + insider with network access (via network refusal + offline signing) | Nation-state with physical access |
### 10.3 Known Limitations
- **Firmware-level Purge relies on the drive.** If the drive's firmware is buggy or compromised, Sanitize may report success without actually erasing. The framework mitigates this with post-purge overwrite and full-device verification, but cannot fully eliminate it. Operators with the highest assurance requirements should pair Purge with Destroy.
- **Wear-leveling on SSDs defeats overwrite.** A single overwrite pass may not touch every physical block, because the SSD's controller remaps logical blocks to physical pages transparently. The framework addresses this by recommending Purge (firmware-level) for SSDs rather than overwrite. Profiles that mandate overwrite on SSDs (e.g. `Paranoid`) do so with the understanding that firmware Purge has already been attempted.
- **SMR drives have non-deterministic write behavior.** Random writes can trigger background GC that exposes stale data. The framework's zone-aware write path (Layer 3) addresses this for host-managed SMR; host-aware SMR remains a known weak point.
- **PMEM (persistent memory) may have unique sanitization requirements.** The framework supports Crypto Erase on PMEM that exposes it, and falls back to overwrite. Operators should consult the vendor's sanitization guidance for their specific PMEM product.
- **The framework cannot prevent host compromise.** If the host running Scuttle is itself compromised, no amount of cryptography in the framework can produce a trustworthy certificate. The Air Gap profile exists for this scenario; FIPS mode exists for federal hosts with rigorous hardening.
---
## 11. Build and Reproducibility
### 11.1 Reproducible Build Policy
Every tagged release of Scuttle must be bit-for-bit reproducible from:
1. The tagged commit on the canonical git repository.
2. A published build environment specification (a Dockerfile or a Nix flake).
The CI pipeline verifies reproducibility on every release candidate by building twice on different hosts and comparing SHA-256 of the resulting tarball. A non-reproducible release candidate blocks the release.
The build environment specification includes:
- Exact OS and version (e.g. Debian 12.5)
- Exact versions of all build dependencies (gcc, glibc, libcrypto, libsodium, ncurses, etc.)
- Exact versions of all bundled dependencies (BLAKE3, Ascon reference code, etc.)
- The `SOURCE_DATE_EPOCH` environment variable, set to the release tag's commit timestamp
- The `TZ=UTC` and `LC_ALL=C` environment variables
### 11.2 Signed Releases
Every release artifact (tarball, detached signature, checksum file) is signed with the project's OpenPGP release key. The public key is published on the project website and on a well-known keyserver. The signing happens on an air-gapped host; the private key never touches a networked machine.
### 11.3 Supply Chain
- **SBOM (Software Bill of Materials)**: every release ships a CycloneDX-format SBOM enumerating every direct and transitive dependency (including vendored code in `third_party/`).
- **Dependency auditing**: all dependencies are audited for known vulnerabilities (via `osv-scanner` or equivalent) on every release. New dependencies require maintainer review of license, security history, and maintenance status.
- **Vendored code**: third-party cryptographic implementations that are vendored (rather than linked from system packages) are isolated under `third_party/<name>/` and have their origin (upstream URL, commit hash, license) documented in `third_party/<name>/ORIGIN.md`.
- **Pinned dependencies**: system-package dependencies are pinned to a minimum version; the build refuses to proceed if a pinned dependency is older.
### 11.4 Build Configuration
The build system is meson (replacing autotools from upstream nwipe — meson's `--reproducible` flag and cleaner cross-compilation support make the reproducible-build policy easier to enforce). The configure flags of interest:
| Flag | Effect |
|---|---|
| `--enable-fips-mode` | Link against OpenSSL FIPS module; restrict to FIPS-eligible providers |
| `--enable-embedded` | Strip optional UIs, REST, LDAP, PKCS#11 — minimal binary |
| `--enable-experimental-providers` | Build Threefish, Skein, Whirlpool, BLAKE2X |
| `--disable-plugins` | Statically link all providers; no `dlopen` at runtime |
| `--enable-sandbox` | Enable seccomp filter on plugins (v2.0; experimental in v1.5) |
| `--with-signer=openpgp\|x509\|ed25519-default` | Default signer backend |
---
## 12. Migration from Nwipe
### 12.1 What Stays the Same
- The `nwipe` binary name is preserved as a symlink to `scuttle` for compatibility with existing PXE configs and operator muscle memory. (`scuttle` is the canonical name; `nwipe` is the alias.)
- The ncurses UI is recognizable to any upstream nwipe user. The device list, method selection, and progress display are in the same places. New features (profiles, verification level, certificate preview) are accessible but not imposed.
- Every upstream wipe method is preserved, byte-for-byte, under the `Legacy` profile family (§7.1). A user running `nwipe --method=dod` will get the same bytes on disk as upstream nwipe.
### 12.2 What Changes
- The default method changes from "DoD 5220.22-M" (upstream) to "Modern Random" (NG). DoD is still available as `Legacy DoD`. Modern Random is faster, cryptographically stronger (CSPRNG vs. MT19937), and produces a verifiable certificate.
- The default PRNG for random passes changes from "Mersenne Twister" (upstream) to "ChaCha20" (NG). MT19937 is not cryptographically secure and should not be used for sanitization; it is preserved only for legacy reproducibility.
- The PDF certificate is enriched: signed, includes verification results, includes the seed digest, includes the Merkle root (when advanced mode is on). Upstream's PDF is preserved as a "legacy certificate" exporter.
- The CLI gains subcommands. `nwipe --help` continues to work (showing the upstream options); `scuttle --help` shows the new structure. Operators can use either.
### 12.3 Migration Path
1. **Read-only evaluation**: install Scuttle alongside nwipe. Run `scuttle list` and `scuttle inspect <device>` to see the richer device information. No writes occur.
2. **Test wipe**: run `scuttle wipe <test-device> --profile=Quick Clear --dry-run` to see the plan. Then run it for real on a non-production device. Compare the certificate to an upstream nwipe PDF.
3. **Profile selection**: choose a profile that matches the operator's threat model (§7, §10). For most ITAD use cases, `Quick Clear` or `Modern Random` is sufficient. For regulated industries, `NIST Purge` or `Enterprise`.
4. **Fleet rollout**: deploy via PXE. The PXE config is a drop-in replacement for upstream nwipe's PXE config; only the initramfs URL changes.
### 12.4 Backward Compatibility Promises
- The `nwipe` command name and the upstream method names will continue to work for the entire v1.x release series.
- Upstream nwipe's PDF certificate format will be producible (via `--report=legacy-pdf`) for the entire v1.x release series.
- The upstream option flags (`--method=`, `--prng=`, etc.) will be accepted and mapped to the equivalent profile for the entire v1.x release series. A deprecation warning is emitted when they are used.
- v2.0 may remove the legacy compatibility shims. The decision will be made based on adoption metrics and community feedback, not unilaterally.
### 12.5 What Does NOT Migrate
- Upstream nwipe's `pass.c` / `method.c` dispatch logic is replaced. The new dispatch goes through the policy engine. The bytes-on-disk for each method are preserved, but the code path is different.
- Upstream nwipe's `customers.c` (a hard-coded list of customer names) is removed. Operator identity is now an authenticated, auditable field (Layer 14), not a hard-coded string.
- Upstream nwipe's bundled PDF generator (`src/PDFGen/`) is replaced by a proper PDF library (libharu or wkhtmltopdf-derived). The new generator produces accessible, tagged PDFs.
- Upstream nwipe's `temperature.c` (re-implemented from `hddtemp`) is replaced by direct SMART temperature reads via Layer 1's `nwipe_device_refresh()`.
---
## 13. Project Governance
### 13.1 Maintainership
The project is governed by a small core team (3–5 maintainers) with commit access, supported by a wider community of contributors. Maintainers are added by consensus of the existing core team; a maintainer may be removed by a 2/3 majority vote of the core team (this has never happened and is reserved for severe conduct or security violations).
The core team's responsibilities:
- Review and merge pull requests. Maintainers review within their areas of expertise; no maintainer is expected to review every PR.
- Tag releases. The release manager role rotates among maintainers.
- Triage security reports (§13.3).
- Maintain the architectural integrity of the layering (§4, §5). PRs that violate the layering are rejected with explanation.
### 13.2 Contribution Model
Contributions follow the standard fork-and-PR model. All contributors must sign off on their commits (`git commit -s`), attesting to the Developer Certificate of Origin. The project does not require a CLA.
Contribution areas particularly welcome:
- New PRNG providers (drop a `.so` + KAT vectors in `tests/kat/`)
- New hash providers (same)
- New profile TOMLs (validated against the schema, with rationale)
- New device drivers (for storage tech the core doesn't yet detect)
- New report formats
- New language bindings for the JSON/REST API (Python, Go, Rust)
- Conformance test fixtures (sysfs snapshots for unusual devices)
### 13.3 Security Disclosure
Security vulnerabilities are reported privately to `security@scuttle.example.org` (PGP-encrypted). The core team acknowledges receipt within 48 hours, provides an initial assessment within 7 days, and coordinates a fix and disclosure timeline with the reporter.
Disclosure timeline:
- **Day 0**: private report received and acknowledged.
- **Day 7**: initial assessment; severity assigned (Critical / High / Medium / Low).
- **Day 30** (Critical) / **Day 90** (others): fix released, public disclosure.
- The team reserves the right to extend the timeline if a fix requires coordinated disclosure with downstream packagers (Debian, Fedora, etc.).
Public disclosures are published as GitHub Security Advisories with associated CVEs (requested from MITRE or via the GitHub CNA).
### 13.4 CVE Handling
CVEs are requested for any vulnerability that meets at least one of:
- Allows an attacker to make the framework report a successful wipe when the wipe did not occur
- Allows an attacker to recover key material from process memory
- Allows an attacker to forge an audit signature
- Allows a malicious plugin to escape its expected scope
- Crashes the framework (denial of service during a wipe)
CVEs are not requested for:
- Issues that require the operator to already have root on the host
- Issues that require physical access to the device being wiped
- Theoretical weaknesses with no plausible attack path
### 13.5 Code of Conduct
The project follows the Contributor Covenant 2.1 code of conduct. Enforcement is by the core team; reports go to `conduct@scuttle.example.org` and are handled by a designated maintainer who is not the subject of the report.
---
## 14. Source Tree (Expanded)
The expanded source tree below reflects the layered architecture. Each subdirectory maps to a layer or a plugin kind. The tree is the canonical reference for contributor orientation.
```
scuttle/
├── core/ # Layer 3 (Wipe Engine) + process lifecycle
│ ├── lifecycle.c # main, signal handling, plugin loader init
│ ├── job.c # nwipe_job_run
│ ├── pass.c # pass dispatch (round-robin, zone-aware)
│ ├── options.c # CLI parsing
│ ├── scheduler.c # Layer 10
│ └── registry.c # Layer 12 (plugin registry)
│
├── devices/ # Layer 1 (Device Discovery)
│ ├── enumerate.c # nwipe_device_enumerate
│ ├── sysfs.c # /sys/block walker
│ ├── ata.c # ATA/SATA-specific queries
│ ├── nvme.c # NVMe-specific queries (libnvme)
│ ├── scsi.c # SCSI/SAS queries (SG_IO)
│ ├── mmc.c # eMMC, SD
│ ├── pmem.c # /dev/pmem*
│ ├── virtual.c # VirtIO, VMware, Hyper-V, QEMU
│ └── drivers/ # Layer 12 plugin kind: drivers/
│ └── README.md
│
├── media/ # Layer 2 (Media Intelligence)
│ ├── classify.c # nwipe_media_classify
│ ├── nist_map.c # Clear/Purge/Destroy recommendation
│ └── rationale.c # human-readable rationale strings
│
├── wipe/ # Layer 3 (Wipe Engine) - method implementations
│ ├── zero.c # Zero pass
│ ├── one.c # One pass
│ ├── prng_stream.c # PRNG stream pass
│ ├── static_pattern.c # Static-pattern passes (e.g. DoD patterns)
│ ├── firmware.c # Wrapper that invokes Layer 9
│ ├── verify.c # Wrapper that invokes Layer 6
│ ├── smr_zone_aware.c # SMR host-managed zone-sequential pass
│ └── round_robin.c # Multi-PRNG round-robin dispatch
│
├── providers/ # Layer 4 (PRNG) + Layer 5 (Hash) - plugin kind: algorithms/
│ ├── aes/
│ │ ├── aes_ctr_prng.c # portable
│ │ ├── aes_ctr_prng_aesni.c # x86 AES-NI
│ │ ├── aes_ctr_prng_armce.c # ARMv8 CE
│ │ └── aes_ctr_prng.h
│ ├── chacha/
│ │ ├── chacha20.c
│ │ ├── chacha20_avx2.c
│ │ ├── chacha20_avx512.c
│ │ ├── chacha20_neon.c
│ │ ├── xchacha20.c
│ │ └── chacha20.h
│ ├── blake3/
│ │ ├── blake3_xof.c
│ │ ├── blake3_avx2.c
│ │ ├── blake3_avx512.c
│ │ ├── blake3_neon.c
│ │ └── blake3.h
│ ├── ascon/
│ │ ├── ascon_xof.c
│ │ └── ascon.h
│ ├── serpent/
│ │ ├── serpent_ctr.c
│ │ └── serpent.h
│ ├── twofish/
│ │ ├── twofish_ctr.c
│ │ └── twofish.h
│ ├── camellia/
│ │ ├── camellia_ctr.c
│ │ └── camellia.h
│ ├── shake/
│ │ ├── shake128.c
│ │ ├── shake256.c
│ │ ├── kangaroo12.c
│ │ └── shake.h
│ ├── hc256/
│ │ ├── hc256.c
│ │ └── hc256.h
│ ├── isaac/ # preserved from upstream
│ │ ├── isaac.c
│ │ ├── isaac64.c
│ │ └── isaac.h
│ ├── mt19937/ # preserved (legacy/non-crypto)
│ │ ├── mt19937.c
│ │ └── mt19937.h
│ ├── splitmix64/ # preserved
│ ├── xor/ # preserved (xoroshiro)
│ ├── alfg/ # preserved (lagged fibonacci)
│ ├── salsa20/
│ ├── rabbit/
│ └── experimental/ # build flag --enable-experimental-providers
│ ├── threefish/
│ ├── skein/
│ ├── whirlpool/
│ └── blake2x/
│
├── verify/ # Layer 6 (Verification Engine)
│ ├── sector.c # per-sector verify
│ ├── spot.c # random spot verify
│ ├── block.c # block verify
│ ├── entire_device.c # whole-device hash
│ ├── entropy.c # Shannon entropy
│ ├── chi_square.c
│ ├── byte_freq.c
│ ├── failure_map.c # LBA range tracking
│ └── merkle.c # Merkle tree construction (used by Layer 7)
│
├── certificates/ # Layer 7 (Cryptographic Audit)
│ ├── audit_record.c # nwipe_audit_record build / serialize
│ ├── canonical_json.c # canonical (sorted-key) JSON
│ ├── sign.c # nwipe_audit_sign dispatcher
│ ├── merkle.c # Merkle root computation
│ └── exporters/ # Layer 12 plugin kind: exporters/
│ ├── json/
│ ├── xml/
│ ├── csv/
│ ├── pdf/ # typeset PDF via libharu
│ ├── html/
│ ├── yaml/
│ ├── signer_openpgp/ # uses gpgme
│ ├── signer_x509/ # uses OpenSSL
│ ├── signer_pkcs11/ # uses PKCS#11
│ ├── signer_tpm/ # uses tpm2-tss
│ └── attest/ # remote attestation (v2.0)
│
├── benchmark/ # Layer 16 (Performance Laboratory)
│ ├── bench_provider.c
│ ├── bench_hash.c
│ ├── bench_profile.c
│ ├── history.c # /var/lib/scuttle/bench-history.jsonl
│ └── power.c # RAPL / powertop integration
│
├── security/ # Layer 15 (Security)
│ ├── secure_alloc.c # sodium_malloc wrapper
│ ├── secure_memcmp.c # constant-time compare
│ ├── selftest.c # startup KAT runner
│ ├── rng_health.c # NIST SP 800-90B-style health checks
│ ├── startup_validate.c # binary self-hash (optional)
│ └── fips_mode.c # FIPS module integration
│
├── hardware/ # Layer 8 (Hardware Optimization)
│ ├── cpuid.c # x86 cpuid
│ ├── arm_features.c # ARM HWCAP
│ ├── riscv_features.c # RISC-V V extension detection
│ ├── bench_cache.c # /var/lib/scuttle/hwbench.json
│ └── select.c # nwipe_prng_select_fastest
│
├── firmware/ # Layer 9 (Secure Erase Integration)
│ ├── ata_secure_erase.c # ATA SE / Enhanced
│ ├── nvme_sanitize.c # NVMe Sanitize (Block/Crypto/Overwrite)
│ ├── nvme_format.c # NVMe Format NVM
│ ├── scsi_sanitize.c # SCSI SANITIZE
│ ├── scsi_format.c # SCSI FORMAT UNIT
│ ├── trim.c # BLKDISCARD / FITRIM
│ └── hpa_dco.c # HPA/DCO detect & disable (preserved from upstream)
│
├── reports/ # Layer 11 (Reporting) - plugin kind: reports/
│ ├── summary.c
│ ├── detailed.c
│ ├── executive.c
│ ├── compliance.c # NIST/ISO/GDPR/HIPAA/PCI/DoD/BSI mappings
│ ├── research.c
│ ├── debug.c
│ ├── charts.c # speed, temp, bandwidth graphs
│ └── formatters/
│ ├── pdf.c
│ ├── html.c
│ ├── csv.c
│ ├── md.c
│ └── txt.c
│
├── ui/ # Layer 13 (User Interfaces)
│ ├── ncurses/ # Classic UI (preserved, lightly modernized)
│ │ ├── gui.c
│ │ ├── widgets.c
│ │ └── theme.c
│ ├── tui/ # Modern TUI
│ │ ├── panes.c
│ │ ├── command_palette.c
│ │ └── mouse.c
│ ├── cli/ # CLI subcommands
│ │ ├── list.c
│ │ ├── inspect.c
│ │ ├── wipe.c
│ │ ├── batch.c
│ │ ├── verify.c
│ │ ├── audit.c
│ │ ├── benchmark.c
│ │ ├── selftest.c
│ │ └── serve.c
│ ├── batch/ # YAML/JSON spec parser
│ ├── json_api/ # Unix-domain-socket JSON API
│ ├── rest_api/ # HTTPS REST API
│ ├── pxe/ # PXE entry point
│ └── optional/
│ ├── cockpit/ # Cockpit module
│ ├── webui/ # SPA (separate repo, built into release)
│ └── remote_console/ # Electron/Tauri client (separate repo)
│
├── enterprise/ # Layer 14 (Enterprise Features)
│ ├── agent.c # remote agent daemon
│ ├── pxe_boot.c # PXE initramfs hook
│ ├── wol.c # Wake-on-LAN
│ ├── inventory.c
│ ├── asset_tracking.c # CSV/REST/SQL connectors
│ ├── ldap.c # LDAP/AD auth
│ ├── ca_enroll.c # internal CA enrollment
│ ├── remote_logging.c # RFC 5424 syslog, Splunk HEC, Elastic
│ └── webhook.c # lifecycle event POSTs
│
├── research/ # Layer 17 (Research Mode)
│ ├── compression.c # zstd ratio
│ ├── entropy_dist.c # per-MiB / per-GiB entropy
│ ├── pattern_analysis.c # serial correlation, Monte Carlo π
│ ├── recovery_attempt.c # controlled known-pattern recovery
│ ├── ecc_behavior.c # NVMe vendor log reads
│ └── wear_stats.c # SMART wear-level deltas
│
├── profiles/ # Layer 12 plugin kind: profiles/ (TOML files)
│ ├── quick_clear.profile.toml
│ ├── modern_random.profile.toml
│ ├── nist_clear.profile.toml
│ ├── nist_purge.profile.toml
│ ├── enterprise.profile.toml
│ ├── paranoid.profile.toml
│ ├── research.profile.toml
│ ├── forensic.profile.toml
│ ├── government.profile.toml
│ ├── air_gap.profile.toml
│ ├── legacy_zero.profile.toml
│ ├── legacy_dod.profile.toml
│ ├── legacy_gutmann.profile.toml
│ ├── legacy_rcmp.profile.toml
│ ├── legacy_hmg.profile.toml
│ ├── legacy_schneier.profile.toml
│ ├── legacy_bmb.profile.toml
│ └── schema.json # JSON Schema for profile validation
│
├── policies/ # Policy definitions (referenced by profiles)
│ ├── hdd_overwrite_1pass.policy
│ ├── hdd_overwrite_3pass.policy
│ ├── hdd_overwrite_7pass.policy
│ ├── ssd_purge_then_overwrite_1pass.policy
│ ├── ssd_purge_then_overwrite_3pass.policy
│ ├── nvme_sanitize_crypto_then_overwrite.policy
│ ├── nvme_sanitize_block_then_overwrite.policy
│ ├── emmc_trim_then_overwrite.policy
│ ├── smr_zone_aware_overwrite.policy
│ ├── pmem_crypto_erase_then_overwrite.policy
│ └── scsi_sanitize_then_overwrite.policy
│
├── future/ # Layer 18 (Future Expansion) - reserved interfaces
│ ├── signer.h # nwipe_signer_t
│ ├── entropy_source.h # nwipe_entropy_source_t
│ └── attest.h # remote attestation (v2.0)
│
├── tests/
│ ├── unit/
│ │ ├── test_device_enumerate.c
│ │ ├── test_media_classify.c
│ │ ├── test_pass_dispatch.c
│ │ ├── test_round_robin.c
│ │ ├── test_zone_aware.c
│ │ ├── test_verify_*.c
│ │ ├── test_audit_canonical.c
│ │ ├── test_merkle.c
│ │ └── test_round_size.c # preserved from upstream
│ ├── kat/ # Known Answer Test vectors
│ │ ├── aes_ctr/
│ │ ├── chacha20/
│ │ ├── blake3/
│ │ ├── ascon/
│ │ ├── shake128/
│ │ ├── shake256/
│ │ ├── kangaroo12/
│ │ ├── serpent_ctr/
│ │ ├── twofish_ctr/
│ │ ├── camellia_ctr/
│ │ └── ...
│ ├── integration/
│ │ ├── loopback_e2e.sh # preserved from upstream tests/ci/
│ │ ├── nvme_mock_target.sh # nvme-cli mock target
│ │ ├── parallel_8disks.sh
│ │ └── fault_injection.sh # preserved from upstream
│ ├── fixtures/
│ │ └── sysblock/ # /sys/block snapshots for enumeration tests
│ ├── conformance/
│ │ ├── nist_800_88_clear.test
│ │ ├── nist_800_88_purge.test
│ │ ├── fips_mode.test
│ │ └── reproducible_build.test
│ └── ci/
│ ├── build_matrix.yml # OS × arch × build-flag matrix
│ └── reproducibility_check.sh
│
├── docs/
│ ├── MANIFEST.md # this file
│ ├── architecture/ # per-layer deep dives
│ ├── api/ # CLI, JSON API, REST API reference
│ ├── profiles/ # per-profile documentation
│ ├── providers/ # per-provider documentation
│ ├── compliance/ # NIST, ISO, GDPR, HIPAA, PCI, DoD, BSI mappings
│ ├── deployment/ # PXE, enterprise, embedded, air-gap guides
│ └── migration/ # upstream nwipe → NG migration guide
│
├── tools/
│ ├── cert_verify.py # verify a signed certificate
│ ├── merkle_check.py # verify a device against a published Merkle root
│ ├── profile_validate.py # validate a profile TOML against the schema
│ ├── bench_compare.py # compare two bench-history files
│ └── sbom_generate.sh # CycloneDX SBOM generator
│
├── third_party/ # vendored crypto (with ORIGIN.md each)
│ ├── blake3/
│ ├── ascon/
│ ├── kangaroo12/
│ └── ...
│
├── meson.build # top-level build
├── meson.options # build flags
├── README.md
├── CONTRIBUTING.md
├── SECURITY.md # disclosure policy
├── CODE_OF_CONDUCT.md
├── COPYING # GPL-2.0+
├── CHANGELOG.md
└── AUTHORS
```
---
## 15. Roadmap
The roadmap is phased to deliver usable, auditable increments. Each phase ends with a tagged release and a published conformance report. Dates are indicative, not committed.
### v0.1 — Architectural Bootstrap (Q1)
- Meson build system, core source tree scaffolding
- Layer 1 (Device Discovery) — ported from upstream `device.c`, expanded
- Layer 2 (Media Intelligence) — new
- Layer 3 (Wipe Engine) — ported from upstream `pass.c`, refactored
- Layer 4 (PRNG) — ported upstream PRNGs as plugins (AES-CTR, ChaCha20, ISAAC, MT19937, SplitMix64, xoroshiro, ALFG)
- Layer 5 (Hash) — SHA-256, SHA-512, BLAKE2, BLAKE3
- Layer 6 (Verification) — sector + final pass only
- Layer 7 (Audit) — JSON output only, unsigned
- Layer 13 (UI) — ncurses (ported from upstream) + CLI (`list`, `inspect`, `wipe`)
- Conformance: upstream nwipe e2e tests pass; legacy profiles produce byte-identical output
### v0.2 — Legacy Compatibility (Q2)
- All upstream methods preserved as Legacy profiles
- Upstream option flags accepted (with deprecation warnings)
- Legacy PDF certificate exporter
- `nwipe` symlink for binary compatibility
- Existing PXE configs work unchanged
### v0.3 — Modern Providers (Q3)
- BLAKE3 XOF, XChaCha20, SHAKE128, SHAKE256, KangarooTwelve, Ascon, HC-256, Rabbit, Salsa20, Serpent-CTR, Twofish-CTR, Camellia-CTR providers
- KAT vectors for all new providers
- Layer 8 (Hardware Optimization) — AES-NI, AVX2, NEON fast paths
- Layer 16 (Performance Laboratory) — benchmarks + history
### v0.4 — Profiles and Policies (Q4)
- Layer 12 (Plugin System) — full plugin loading
- Profile TOML schema and validator
- All Modern profiles from §7.2
- Policy engine — policy_map dispatch from profile → policy → wipe plan
### v0.5 — Verification and Audit (Q1, year 2)
- Layer 6 — spot, block, entire-device, entropy, chi-square, byte frequency, failure mapping
- Layer 7 — XML, CSV, PDF, HTML, YAML exporters; Merkle tree; signing (Ed25519, OpenPGP, X.509)
- Compliance reports (NIST 800-88 Clear/Purge)
### v0.6 — Firmware Erase (Q2)
- Layer 9 — ATA SE, ATA Enhanced, NVMe Format, NVMe Sanitize (all actions), SCSI Sanitize, SCSI Format, TRIM
- HPA/DCO detect + disable (ported from upstream)
- NVMe Sanitize status polling
### v0.7 — Scheduler and UI (Q3)
- Layer 10 — sequential, parallel, priority, groups
- Layer 13 — Modern TUI, batch mode, JSON API (Unix socket)
- Per-job progress callbacks
### v0.8 — Enterprise (Q4)
- Layer 14 — remote agent, PXE, WoL, inventory, asset tracking, LDAP, CA enrollment, remote logging, webhook
- REST API (HTTPS, with auth)
- Cluster wipe worker
### v0.9 — Security Hardening (Q1, year 3)
- Layer 15 — secure memory, constant-time routines, memory zeroization, KAT self-tests, continuous RNG health checks, startup validation
- FIPS mode build flag
- Reproducible build verification in CI
### v1.0 — First Stable Release (Q2)
- All layers feature-complete
- SBOM generation, signed releases
- Full documentation (architecture, API, profiles, providers, compliance, deployment, migration)
- Conformance report: all NIST 800-88 Clear/Purge tests pass; all KAT vectors pass; reproducibility verified on 3+ OS/arch combinations
- API stability commitment: the v1.x ABI is frozen; v2.0 may break it with a deprecation cycle
### v1.x — Maintenance and Provider Expansion
- New PRNG/hash providers as community contributions
- New device drivers (new storage tech)
- New profile TOMLs (new compliance regimes)
- Bug fixes, performance improvements
- No core architectural changes
### v2.0 — Sandbox and Future Expansion
- Plugin sandbox (seccomp filter)
- Layer 18 — TPM 2.0 entropy source, sealed keys, remote attestation
- PKCS#11 signer backend
- Post-quantum PRNG providers (when standardized)
- Possible: removal of legacy compatibility shims (with community consultation)
- API break: clean up any v1.x warts; provide a migration guide
---
## 16. Conformance and Testing
### 16.1 Test Pyramid
- **Unit tests** — per-module, in `tests/unit/`. Fast (seconds). Run on every PR.
- **KAT vectors** — per-provider, in `tests/kat/`. Run at `make test` and at process startup. A failure removes the provider from the registry.
- **Integration tests** — in `tests/integration/`. Use loopback devices and mock NVMe targets. Run on every PR (in containers) and nightly (on bare metal).
- **Conformance tests** — in `tests/conformance/`. Map to specific compliance regimes (NIST 800-88 Clear/Purge, FIPS mode, reproducible build). Run on every release candidate.
- **Hardware-in-loop tests** — manual, in `tests/hil/`. A documented procedure for testing against a curated set of real drives (a Samsung 980 Pro, a WD Red CMR, a Seagate SMR, an Intel Optane PMEM). Run before each release.
### 16.2 CI Matrix
CI runs on every push and every PR, on:
- **OS**: Debian 12, Ubuntu 22.04/24.04, Fedora 40, Alpine 3.20, Arch rolling
- **Arch**: x86-64, aarch64 (via QEMU on x86-64 hosts), armv7 (cross-compiled, run via QEMU)
- **Build flags**: default, `--enable-fips-mode`, `--enable-embedded`, `--disable-plugins`
The full matrix runs nightly; a representative subset (Debian 12 x86-64 default, Ubuntu 24.04 x86-64 default, Debian 12 aarch64 default) runs on every PR.
### 16.3 Reproducibility Verification
Every release candidate is built twice on two different hosts (one x86-64, one aarch64). The SHA-256 of the resulting tarball must match. If it does not, the release is blocked and the cause is investigated.
### 16.4 Performance Regression Detection
The Performance Laboratory (Layer 16) runs benchmarks on every CI build. The results are compared to the previous build; a regression of more than 10% on any provider triggers a warning, and a regression of more than 25% blocks the PR (pending maintainer override).
### 16.5 Fuzzing
The CLI, the JSON/REST API, and the profile TOML parser are fuzz-tested continuously via OSS-Fuzz. The PRNG providers' `seed()` and `generate()` are fuzzed with malformed input to ensure they fail safe (return an error, never crash).
---
## 17. Glossary
| Term | Definition |
|---|---|
| **ATA Secure Erase** | ATA command that instructs the drive to erase all user data internally. Standard and Enhanced variants. |
| **Crypto Erase** | Sanitization technique that destroys the data encryption key, rendering the encrypted data unreadable. Applicable to SEDs and NVMe drives with crypto-sanitize support. |
| **CSPRNG** | Cryptographically Secure Pseudo-Random Number Generator. A PRNG whose output is computationally indistinguishable from true random. |
| **DBAN** | Darik's Boot and Nuke. The original boot-and-wipe live CD; one of the projects that inspired Scuttle. |
| **HPA / DCO** | Host Protected Area / Device Configuration Overlay. Vendor areas of the drive that may not be visible to normal read/write commands; must be disabled before wiping to ensure full coverage. |
| **ITAD** | IT Asset Disposition. The industry of disposing of end-of-life IT equipment. |
| **KAT** | Known Answer Test. A test vector: given input X, the algorithm must produce output Y. |
| **Merkle Tree** | A tree of hashes where each leaf is the hash of a data block and each internal node is the hash of its children. The root authenticates all leaves. |
| **NIST SP 800-88** | NIST Special Publication 800-88, "Guidelines for Media Sanitization." Defines Clear, Purge, Destroy. |
| **NVMe Sanitize** | NVMe command that instructs the drive to perform a sanitization operation: Block Erase, Crypto Erase, or Overwrite. |
| **PMEM** | Persistent Memory. Non-volatile memory accessed via the memory bus (e.g. Intel Optane DC PMM). |
| **PRNG** | Pseudo-Random Number Generator. |
| **PXE** | Preboot eXecution Environment. Network boot protocol. |
| **SED** | Self-Encrypting Drive. A drive with hardware encryption; Crypto Erase rotates the key. |
| **SMR** | Shingled Magnetic Recording. HDD technology with overlapping tracks; random writes are expensive. |
| **TPM** | Trusted Platform Module. Hardware security chip on the motherboard. |
| **XOF** | eXtendable Output Function. A hash whose output can be any length (e.g. SHAKE, BLAKE3 in XOF mode). |
---
## 18. References
- NIST SP 800-88 Rev. 1 — *Guidelines for Media Sanitization*
- NIST SP 800-90A/B/C — *Recommendation for Random Number Generation*
- NIST FIPS 140-3 — *Cryptographic Module Security Requirements*
- NIST Lightweight Cryptography — Ascon-128a standardization
- RFC 8439 — *ChaCha20 and Poly1305 for IETF Protocols*
- RFC 7693 — *BLAKE2*
- BLAKE3 Specification — *BLAKE3: an extensible, high-speed, parallel cryptographic hash*
- KangarooTwelve Specification — Bertoni et al.
- ISO/IEC 27040:2024 — *Storage Security*
- DBAN homepage and historical documentation
- Nwipe upstream — `https://github.com/martijnvanbrummelen/nwipe`
- OpenSSL 3.0 Provider documentation
- libsodium documentation — *Securing memory and constant-time operations*
- RFC 6962 — *Certificate Transparency* (Merkle tree structure used as reference)
- Developer Certificate of Origin — `https://developercertificate.org/`
- Contributor Covenant 2.1 — *Code of Conduct*
- CycloneDX — *Software Bill of Materials format*
---
*End of manifest. This document is the architectural source of truth for Scuttle. Implementation follows this manifest; deviations require a manifest revision and a maintainer vote. The manifest is versioned with the same scheme as the code: even minor versions are stable, odd minor versions are development drafts.*
Reference in New Issue View Git Blame Copy Permalink