12 KiB

Raw Blame History

Build & Operation Guide

This document is the authoritative build guide and operator reference for CorbelPurge. For the project overview and detection model, see README.md.

Prerequisites

Tool	Version	Purpose
Rust	1.70+ stable	Compiles the scanner, CLI, and GUI
Python 3	any	Only needed to regenerate test fixtures

Install Rust via rustup:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"

The headless CLI has zero GUI dependencies and no native libraries. The GUI adds iced, rfd, and tokio (all pure-Rust).

Step 1 — Get the source

From a release tarball:

tar xzf corbel-purge-0.4.2.tar.gz
cd corbel-purge-0.4.2

From the repository:

git clone https://git.dcos.net/dcosnet/corbel.git
cd corbel

Step 2 — Build the CLI

cargo build --release

The binary lands at target/release/corbel-purge. Verify it works:

./target/release/corbel-purge --help

You should see the usage banner listing scan, scan-dir, study, and the supported flags (--workspace, --abort-on-threat, --quiet, --recursive, --preserve-format, --rules, --cve-db, --no-clean-output, --move-clean).

Building the GUI (optional)

The iced 0.13 dashboard GUI is behind the gui feature flag:

cargo build --release --features gui --bin corbel-purge-gui
./target/release/corbel-purge-gui

Build profiles

The release profile (in Cargo.toml) is tuned for production:

[profile.release]
opt-level = 3
lto = "thin"
codegen-units = 1
strip = "symbols"

For faster debug builds (no optimizations, faster compile):

cargo build            # debug profile

Step 3 — Run the test suite

cargo test

Expected output: 154 lib tests + 24 pipeline integration tests + 4 zip-bomb defense tests, all passing. The pipeline integration tests include the corpus regression test (benign_realworld_pdf_produces_zero_findings) which asserts that a 3-page PDF with 13 hyperlinks (kernel.org, linuxfromscratch.org, github.com/microsoft/vscode, .ru URLs, mailto:, tel:, and URLs containing "support", "account", "verify" in their paths) produces zero findings.

Regenerating test fixtures

The fixtures in tests/fixtures/ are checked in. Regenerate them only when changing the parser or scanner behavior:

pip install pypdf reportlab python-docx
python3 scripts/gen_fixtures.py
python3 scripts/gen_md_epub_fixtures.py
python3 scripts/gen_docx_fixtures.py
python3 scripts/gen_zip_bomb_fixtures.py
python3 scripts/gen_benign_realworld_pdf.py

Step 4 — Scan your first file

./target/release/corbel-purge scan suspicious_document.pdf

If the file contains threats, the summary block prints to stdout and the pipeline writes:

A quarantine tarball in ./corbel_quarantine/ (quarantine_<ts>_<sha>.tar.gz) containing original.<ext>, report.json, report.md, and one .bin per carved payload (plus paired .hex and .info files for each payload).
A cleansed Markdown derivative in ./corbel_clean/ (cleansed_<ts>_<sha>.md).
Standalone report_<ts>_<sha>.json and .md for programmatic access in ./corbel_quarantine/.

If the file is clean (zero malicious findings), the pipeline writes:

A clean-output copy in ./corbel_clean/ (clean_<ts>_<sha>.<ext>) — the source file byte-for-byte.
Standalone report_<ts>_<sha>.json and .md confirming the scan ran and the document was clean.

Step 5 — Try PreserveFormat mode

For a cleaned version that keeps the original format (e.g. a cleaned .epub you can read in an e-reader):

./target/release/corbel-purge scan research_paper.epub --preserve-format

Produces cleansed_<ts>_<sha>.epub with malicious entries stripped from the ZIP container but chapter text preserved. Works for PDF and DOCX too.

Step 6 — Pipeline-stage mode

The scanner acts as a pipeline stage: input files flow through and clean ones end up in the output folder alongside the cleansed derivatives of malicious ones.

# Default: copy clean files to ./corbel_clean/
./target/release/corbel-purge scan inbox/file.pdf

# Queue-draining: move (not copy) clean files to output, remove source
./target/release/corbel-purge scan inbox/file.pdf --move-clean

# Reports only, no clean-output copy
./target/release/corbel-purge scan inbox/file.pdf --no-clean-output

Downstream processing can then operate on the contents of corbel_clean/ without inspecting each file's report — every file there has been verified clean.

Step 7 — Scan a directory

./target/release/corbel-purge scan-dir /path/to/inbox --recursive --workspace /tmp/corbel

The directory walker picks up .pdf, .epub, .md, .markdown, and .docx files. Each file is logged with a [OK], [MALICIOUS], or [ERROR] tag.

Step 8 — CI gate

Use --abort-on-threat to make CorbelPurge a CI gate. Exit code 2 means threats were found:

./target/release/corbel-purge scan incoming_document.pdf --abort-on-threat --quiet
# exit 0: clean
# exit 2: has threats -> fail the build
# exit 1: hard error (parse failure, IO, etc.)

--quiet suppresses the summary block and prints only the JSON report path, which is handy for piping into downstream tooling.

Step 9 — External threat-intel feeds (optional)

The built-in signature tables and CVE database are static. Layer your own on top at runtime:

# Load additional signature rules
./target/release/corbel-purge scan suspicious.pdf --rules my_rules.json

# Load additional CVE signature entries
./target/release/corbel-purge scan suspicious.docx --cve-db my_cve_db.json

# Or set them via environment variables
export CORBEL_EXTERNAL_RULES=/etc/corbel/rules.json
export CORBEL_EXTERNAL_CVE_DB=/etc/corbel/cve_db.json
./target/release/corbel-purge scan suspicious.pdf

External rules are matched alongside the built-in tables; nothing is overridden. See MANIFEST.md for the JSON schema. Supported external-rule types:

homograph-host-list — additional exact-match homograph host strings
signature-list — additional (offset, magic_bytes, name) entries
shellcode-list — additional shellcode prologue byte patterns

(Legacy tld-list, keyword-list, and brand-list rule types are silently skipped — the scanner no longer consults those tables.)

Step 10 — Study a document in place

The study subcommand renders the source document to a single annotated HTML file with malicious regions wrapped in inline <span> tags, color-coded by classification. Use it when you want to see exactly where the exploit sits in context, without leaving the source format:

./target/release/corbel-purge study suspicious.epub

Output lands at study_<ts>_<sha>.html in the workspace. Quiet mode (-q) prints just the path.

CLI flags reference

Flag	Default	Purpose
`--workspace <dir>`	CWD	Root for `corbel_quarantine/` and `corbel_clean/` output dirs
`--abort-on-threat`	off	Exit with code 2 if any malicious finding fires
`--quiet`, `-q`	off	Print only the JSON report path on success
`--recursive`, `-r`	off	Recurse into subdirectories (`scan-dir`)
`--preserve-format`	off	Repackage cleansed document in original format
`--rules <path>`	unset	Path to external signature-rules JSON
`--cve-db <path>`	unset	Path to external CVE database JSON
`--no-clean-output`	off	Do NOT copy clean documents to output folder
`--move-clean`	off	Move (not copy) clean source files to output folder

Exit codes

Code	Meaning
`0`	No threats found
`1`	Hard error (parse failure, IO failure, etc.)
`2`	One or more malicious findings (file was processed)

Environment variables

Variable	Default	Purpose
`CORBEL_QUARANTINE_DIR`	`./corbel_quarantine`	Quarantine tarball + reports output dir
`CORBEL_CLEANSE_DIR`	`./corbel_clean`	Cleansed document + clean-output copy dir
`CORBEL_ABORT_ON_THREAT`	`false`	Abort on first malicious finding
`CORBEL_EMIT_SUSPICIOUS`	`false`	Include Suspicious findings (no default detector produces any)
`CORBEL_EMIT_MARKDOWN_REPORT`	`true`	Generate Markdown report alongside JSON
`CORBEL_EMIT_CLEAN_OUTPUT`	`true`	Copy clean documents to output folder
`CORBEL_MOVE_CLEAN_TO_OUTPUT`	`false`	Move (not copy) clean source files to output
`CORBEL_TOTAL_ARCHIVE_SCAN_CAP`	`268435456` (256 MiB)	Cumulative cap across all entries in a multi-entry archive
`CORBEL_EXTERNAL_RULES`	(unset)	Path to external signature-rules JSON
`CORBEL_EXTERNAL_CVE_DB`	(unset)	Path to external CVE database JSON

Expected output examples

Clean file

────────────────────────────────────────────────────────────
scan complete: pdf
  source:      benign.pdf
  sha256:      9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
  text nodes:  12
  vectors:     0
  findings:    0 malicious, 0 educational, 0 total
  cleansed:    ./corbel_clean/clean_20260801T120000_9f86d081.pdf
  json report: ./corbel_quarantine/report_20260801T120000_9f86d081.json
  md report:   ./corbel_quarantine/report_20260801T120000_9f86d081.md
────────────────────────────────────────────────────────────

Threat found

────────────────────────────────────────────────────────────
scan complete: pdf
  source:      suspicious.pdf
  sha256:      a1b2c3...
  text nodes:  8
  vectors:     1
  findings:    1 malicious, 0 educational, 1 total
  quarantine:  ./corbel_quarantine/quarantine_20260801T120000_abc12345.tar.gz
  cleansed:    ./corbel_clean/cleansed_20260801T120000_abc12345.pdf
  json report: ./corbel_quarantine/report_20260801T120000_abc12345.json
  md report:   ./corbel_quarantine/report_20260801T120000_abc12345.md
────────────────────────────────────────────────────────────

Exit code is 2 when any malicious finding is produced.

Troubleshooting

Build fails on `lopdf` or `zip`

These crates occasionally need a newer Rust than the MSRV declared in their Cargo.toml. Update Rust:

rustup update stable

GUI binary not found

The GUI is behind the gui feature flag. Build with:

cargo build --release --features gui --bin corbel-purge-gui

Scanner reports zero findings on a document I expected to be flagged

Confirm the document actually contains a Category 1 or Category 2 detector trigger (see README.md for the table). The scanner does not flag based on reputation, file source, or filename. Run with cargo run -- scan file.pdf for verbose output.

Quarantine tarball missing

The tarball is written only when malicious_count() > 0. Clean documents produce only report_<ts>_<sha>.{json,md} and a clean_<ts>_<sha>.<ext> copy in the output folder.

Test failure on `benign_realworld_pdf_produces_zero_findings`

This is the corpus regression test. If it fails, a detector is wrong by construction — the detector fired on a clean document. Inspect the test's failure output for the specific detector that fired and tighten that detector's rule.

Installation

After building, install the binary to a system path:

cargo install --path .
# or
sudo cp target/release/corbel-purge /usr/local/bin/

For system-wide configuration, set environment variables in /etc/corbel/env or your shell profile:

export CORBEL_QUARANTINE_DIR=/var/lib/corbel/quarantine
export CORBEL_CLEANSE_DIR=/var/lib/corbel/clean
export CORBEL_EXTERNAL_RULES=/etc/corbel/rules.json
export CORBEL_EXTERNAL_CVE_DB=/etc/corbel/cve_db.json

Next steps

Read README.md for the project overview and detection model
Read MANIFEST.md for the detailed technical specification
Read TODO.md for the development roadmap
Read FIX-NOTES-false-positive-redesign.md for the design notes on the two-category detector model

Contact

Jeremy Anderson — dcos.net — info@dcos.net

12 KiB Raw Blame History