corbel/README.md

# CorbelPurge

> Strict Rust document sanitizer & threat neutralizer for PDF, EPUB, Markdown, and DOCX.

**Author:** Jeremy Anderson — [https://git.dcos.net/dcosnet/corbel](https://git.dcos.net/dcosnet/corbel)
**License:** GPL-3.0-or-later

CorbelPurge parses documents into a unified intermediate representation (`Document` struct), runs a layered contextual scanner that distinguishes educational security literature from active malicious injections, and produces cleansed derivatives with all executable content stripped. Malicious payloads are carved into quarantine tarballs with full forensic reports.

## What it does

Given a file path, `Pipeline::run()` in `src/core/pipeline.rs`:

1. **Detects format** via `DocumentFormat::from_path()` (extension-based: `.pdf`, `.epub`, `.md`/`.markdown`, `.docx`).
2. **Parses** via `parsers::Dispatcher` into a `Document` containing `TextNode` (static text with semantic context) and `ExecutableVector` (active content like JS streams, embedded files, script tags, VBA macros) items.
3. **Scans** with a two-pass engine:
   - `heuristics::inspect_vector()` classifies every executable vector against file-signature tables, shellcode patterns, phishing heuristics, and URI allowlists.
   - `context_filter::evaluate()` checks text nodes for suspicious signatures and determines whether the surrounding context is educational (code blocks, CVE writeups, academic language) or weaponized.
   - `cve_tags::match_cve()` annotates findings with known exploit IDs.
4. **Quarantines** (when malicious findings exist) — `quarantine::handle()` carves payloads into `quarantine_<ts>_<sha>.tar.gz` with `original.<ext>`, `report.json`, `report.md`, and one `.bin` per payload.
5. **Cleanses** (when recommended) — produces a sanitized derivative:
   - **Markdown mode** (default): `cleanse::sanitizer::sanitize()` emits a safe Markdown file. Text nodes at malicious locations are stripped; hyperlinks lose their destinations.
   - **PreserveFormat mode** (`--preserve-format`): `cleanse::repackage::repackage()` rebuilds the original format with malicious entries removed. EPUB entries are stripped from the ZIP, DOCX macros/embeddings/external-links are removed, PDF objects are deleted via lopdf.

Supported formats: **PDF**, **EPUB**, **Markdown**, **DOCX**.

## Build

```bash
# Headless CLI (default — no GUI deps)
cargo build --release

# With iced GUI
cargo build --release --features gui --bin corbel-purge-gui
```

Requires Rust 1.70+ (stable).

Binaries:
- `target/release/corbel-purge` — headless CLI
- `target/release/corbel-purge-gui` — iced dashboard GUI (`gui` feature)

## CLI usage

The CLI is in `src/main.rs` and parses its own arguments (no `clap` dependency).

```bash
# Scan a single file
corbel-purge scan path/to/suspicious.pdf

# Scan with format-preserving output (keeps original format)
corbel-purge scan path/to/book.epub --preserve-format

# Scan with a custom workspace
corbel-purge scan path/to/file.pdf --workspace /tmp/corbel

# Scan a directory recursively
corbel-purge scan-dir path/to/inbox --recursive --workspace /tmp/corbel

# CI gate: exit non-zero on threat
corbel-purge scan path/to/file.pdf --abort-on-threat

# Quiet mode: print only the JSON report path
corbel-purge scan path/to/file.pdf --quiet

# Version
corbel-purge --version
```

### Exit codes

| Code | Meaning |
|------|----------|
| 0 | No threats found |
| 1 | Hard error (parse failure, IO, etc.) |
| 2 | One or more malicious findings |

### Environment variables

| Variable | Default | Purpose |
|----------|---------|---------|
| `CORBEL_QUARANTINE_DIR` | `./corbel_quarantine` | Quarantine tarball output dir |
| `CORBEL_CLEANSE_DIR` | `./corbel_clean` | Cleansed document output dir |
| `CORBEL_ABORT_ON_THREAT` | `false` | Abort on first malicious finding |
| `CORBEL_EMIT_SUSPICIOUS` | `true` | Include Suspicious findings in report |
| `CORBEL_EMIT_MARKDOWN_REPORT` | `true` | Generate Markdown report alongside JSON |

## Library usage

```rust
use corbel_purge::{Pipeline, Config, CleanseMode};

// Scan a file on disk
let pipeline = Pipeline::with_config(Config::with_workspace("/tmp/output"));
let result = pipeline.run("suspicious.pdf")?;
println!("malicious: {}", result.scan_report.malicious_count());

// Scan in-memory bytes (e.g. from an email gateway)
let bytes = std::fs::read("upload.docx")?;
let result = pipeline.run_on_bytes(
    bytes,
    corbel_purge::DocumentFormat::Docx,
    Some("upload.docx".into()),
)?;

// PreserveFormat mode
let mut config = Config::default();
config.cleanse_mode = CleanseMode::PreserveFormat;
let pipeline = Pipeline::with_config(config);
let result = pipeline.run("book.epub")?;
```

## Source layout

```text
src/
├── main.rs                  # CLI: scan, scan-dir, --version, arg parsing
├── lib.rs                   # Crate root, CorbelError enum, public re-exports
├── util.rs                  # sha256_hex(), truncate_with_ellipsis(), read_with_cap()
├── bin/
│   └── gui.rs               # iced 0.13 dashboard GUI (gui feature)
├── core/
│   ├── types.rs             # Document, TextNode, ExecutableVector, Finding, ScanReport,
│   │                         # Location, TextContext, VectorType, ThreatClassification,
│   │                         # MaliciousType, Recommendation, DocumentMetadata
│   ├── config.rs            # Config, CleanseMode, env overrides, workspace dirs
│   └── pipeline.rs          # Pipeline, PipelineResult, DocumentFormat::from_path()
├── parsers/
│   ├── mod.rs                # DocumentParser trait, Dispatcher
│   ├── pdf_parser.rs         # lopdf-based PDF structure walker
│   ├── epub_parser.rs        # ZIP-based EPUB container inspector
│   ├── md_parser.rs          # pulldown-cmark text extractor
│   └── docx_parser.rs        # OOXML ZIP inspector (regex XML extraction)
├── scanner/
│   ├── mod.rs                # scan() entrypoint, CVE tag injection
│   ├── heuristics.rs         # classify_vector() for executable vectors
│   ├── context_filter.rs     # evaluate() for text nodes, educational vs weaponized
│   ├── signatures.rs         # PHISHING_TLDS, KNOWN_FILE_SIGNATURES,
│   │                         # SHELLCODE_PATTERNS, COMMON_PHISHING_BRANDS,
│   │                         # SUSPICIOUS_URL_KEYWORDS, URL_SHORTENER_DOMAINS
│   └── cve_tags.rs           # CVE_TABLE (7 entries), match_cve(), cve_tag()
├── quarantine/
│   ├── mod.rs                # handle() — tarball writer, QuarantineOutcome
│   ├── extractor.rs          # extract_payloads(), ExtractedPayload, filename sanitization
│   └── reporter.rs           # build_json_report(), build_markdown_report()
├── cleanse/
│   ├── mod.rs                # cleanse() dispatcher (Markdown vs PreserveFormat)
│   ├── sanitizer.rs          # sanitize() — Markdown re-serializer, build_clean_markdown()
│   └── repackage.rs          # repackage() — format-preserving (EPUB/DOCX/PDF)
└── ui/
    ├── mod.rs                # GUI module gate (behind `gui` feature)
    ├── viewer.rs              # Stub
    └── alert_modal.rs        # Stub
```

## Testing

```bash
# Regenerate test fixtures (requires pypdf, reportlab, python-docx)
pip install pypdf reportlab python-docx
python3 scripts/gen_fixtures.py
python3 scripts/gen_md_epub_fixtures.py
python3 scripts/gen_docx_fixtures.py
python3 scripts/gen_zip_bomb_fixtures.py

# Run all tests
cargo test
```

## Adding a new format

1. Create `src/parsers/<format>_parser.rs` implementing `DocumentParser`.
2. Add a variant to `DocumentFormat` in `src/core/types.rs` (and its `Display` impl).
3. Add a match arm in `Dispatcher::parse()` in `src/parsers/mod.rs`.
4. Add an extension check in `DocumentFormat::from_path()` in `src/core/pipeline.rs`.
5. Add repackage support in `src/cleanse/repackage.rs` (optional).

The scanner, quarantine, and cleanse modules consume only the `Document` UIR and require no changes for basic format support.

## Non-goals

- No document editing or annotation.
- No execution of embedded scripting (PDF JS, EPUB scripts, VBA macros).
- No network access during scanning — all analysis is local.
- No sandbox hardening of the tool itself.

## License

GPL-3.0-or-later. Copyright (c) 2026 Jeremy Anderson. https://git.dcos.net/dcosnet/corbel