SysDeck 0.4.6 - feature update, added qcrows
This commit is contained in:
parent
7d27d1b5d1
commit
eccc55bd7c
16
BLOG.md
16
BLOG.md
|
|
@ -1,3 +1,19 @@
|
|||
# QCrows All the Way Down: One VM-Image Format, Three Projects, Zero Trust Between Them
|
||||
|
||||
*How SysDeck 0.4.6 became a first-class consumer of the in-house QCrows VM container image format — parsing image manifests straight out of tar.gz in memory, re-implementing the master toolchain's verification checks without extracting a single byte to disk, and rendering real image metadata in the Kata panel. The same format is produced by cockpit-kata's qcrows-pack and by AI-LSC's stack exporter; SysDeck verifies both with the same code path.*
|
||||
|
||||
A QCrows image (`.qcrows`, spec v0.2.0) is a self-describing bundle for VM-based container runtimes: one tarball carrying a `metadata.toml` manifest, a Cockpit `menu.toml` entry, a `hashes.sha256` integrity list, the guest `rootfs.tar.gz`, and — mandatory since v0.2 — the guest kernel and its `.config`. The format's master implementation lives in the cockpit-kata project (`qcrows-pack`, `qcrows-verify`, `qcrows-inspect`, `qcrows-export`), and a second producer now exists: AI-LSC, the local AI stack manager, exports its active tool stack directly as `.qcrows` archives. SysDeck's Kata panel has listed whatever sat in `/usr/share/sysdeck/kata/qcrows/` since the v0.0.43 production rewrite — but until now it only ever looked at filenames. A bundle could be a Kata-tuned Alpine image or someone's holiday photo archive; the panel treated both as "a file with a size."
|
||||
|
||||
v0.4.6 closes that gap by making the bridge a real format consumer. `qcrows-list` opens every archive in the directory with Python's `tarfile` in `r:*` mode and reads `metadata.toml` and `menu.toml` into memory — no extraction, no temp directories, nothing executable ever materialized from an image the operator hasn't chosen to trust. The TOML parser is deliberately small: QCrows metadata is a flat shape of `[ section ]` headers and scalar/array values, so ~40 lines of section-aware parsing cover it — and being section-aware is not cosmetic. The master `qcrows-inspect` greps first-matching keys, which reports the kernel's `size_mb` on the rootfs row of its own output; SysDeck's parser tracks the current section and gets `kernel.size_mb` and `rootfs.size_mb` right on the same files the master tool mislabels.
|
||||
|
||||
The new `qcrows-verify` subcommand mirrors the master verifier check-for-check — required files, rootfs presence, kernel binary with a real magic test (bzImage's `HdrS` at offset 0x202, or ELF), the kernel `.config` with the five Kata-required options, and a full `sha256sum -c`-style walk of every entry in `hashes.sha256` — with two deliberate divergences in *how*, not *what*. First, the kernel check classifies bytes directly instead of shelling out to `file`: same verdicts, no subprocess. Second, the semantics are calibrated to the master's own exit codes: a missing kernel is a hard failure, but a shortfall of `CONFIG_VSOCKETS=y`-style options is a *warning*, exactly as in qcrows-verify, so a host-kernel-built bundle reports ok with advisories rather than failing a check the master would pass. The hash walk streams members in 1 MiB chunks and caps parsed text members at 1 MiB, header counts at ten thousand — tar-bomb posture inherited from the bridge's CVE-lesson hardening style.
|
||||
|
||||
Everything runs through the same security gate the rest of the Kata bridge uses: filenames validated against the strict allowlist regex, paths resolved with `realpath` containment under the bundle directory, output sanitized before it reaches JSON. The panel card grew accordingly — image name and version, kernel version and format, architecture, hypervisor compatibility list, per-row Verify and Inspect actions whose output lands in the existing operation-output card, all rendered through `textContent`/`escapeHtml` per the panel's no-innerHTML rule. Legacy non-QCrows tarballs that happen to sit in the directory degrade to a stat-only row with an honest "(not a QCrows archive)" label instead of being silently miscounted.
|
||||
|
||||
The test discipline is where the three-project story pays off. The new suite builds synthetic spec-conformant archives in a temp directory — including one whose `rootfs.tar.gz` is byte-flipped *after* its hashes are computed, which is what an integrity violation actually is — and additionally exercises the bridge against real images from both external producers: an AI-LSC stack export and a bundle from cockpit-kata's own `qcrows-pack`. Both parse, both verify clean, the tampered one fails on exactly the checksum check. One image format, two independent producers, one consumer that trusts none of them until the hashes say otherwise — which is the whole point of a self-describing format.
|
||||
|
||||
---
|
||||
|
||||
# SysDeck and the Cockpit Inheritance: One Module Catalog, Two Frontends, and the Case for Real Host State
|
||||
|
||||
*A technical walkthrough of how SysDeck 0.4.4 — a standalone Linux operations console (thirty-one panels, one Bun process, no Cockpit required) that also ships as a drop-in Cockpit plugin suite — authenticates against the host's own Unix accounts through PAM exactly the way Cockpit does, keeps every module working in both frontends, holds itself to a compiler-enforced no-demo contract where every panel reads real host state or fails honestly, and resolves every fork in the road with a step-down through the system's real tools: ten package managers from pacman to sorcery, nftables before iptables, lm-sensors before raw sysfs. Grounded in the source code, not in marketing claims.*
|
||||
|
|
|
|||
2
Makefile
2
Makefile
|
|
@ -30,7 +30,7 @@
|
|||
# Distro support: Arch Linux, Debian/Ubuntu, Fedora/RHEL/CentOS.
|
||||
|
||||
PACKAGE := sysdeck
|
||||
VERSION := 0.4.5
|
||||
VERSION := 0.4.6
|
||||
LIB_DIR := $(DESTDIR)/usr/lib/$(PACKAGE)
|
||||
PYTHON_DIR := $(LIB_DIR)/bridge
|
||||
SHARE_DIR := $(DESTDIR)/usr/share/$(PACKAGE)
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# SysDeck
|
||||
|
||||
[](LICENSE)
|
||||
[](#)
|
||||
[](#)
|
||||
[](https://nextjs.org)
|
||||
[](https://bun.sh)
|
||||
[](#)
|
||||
|
|
@ -10,7 +10,7 @@
|
|||
**A standalone Linux operations console — Unix-account login, real host state, no fabricated data. Cockpit is optional: the same module catalog loads there too.**
|
||||
|
||||
Author: **Jeremy Anderson** · <info@dcos.net> · <https://dcos.net> · [github.com/dcosnet/SysDeck](https://github.com/dcosnet/SysDeck)
|
||||
Version: **0.4.5** · License: **MIT**
|
||||
Version: **0.4.6** · License: **MIT**
|
||||
|
||||

|
||||
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ import os
|
|||
import subprocess
|
||||
from typing import Literal
|
||||
|
||||
__version__ = "0.4.5"
|
||||
__version__ = "0.4.6"
|
||||
__author__ = "Jeremy Anderson"
|
||||
__url__ = "https://dcos.net"
|
||||
|
||||
|
|
|
|||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
|
@ -20,11 +20,17 @@ HARDENING_RE = re.compile(r"Hardening index\s*:\s*(\d+)")
|
|||
|
||||
|
||||
def score() -> int | None:
|
||||
"""Return the latest hardening index, or None if lynis hasn't run."""
|
||||
"""Return the latest hardening index, or None if lynis hasn't run.
|
||||
|
||||
Catches OSError (not just FileNotFoundError) — on hosts where lynis
|
||||
ran as root, /var/log/lynis.log is root-only readable and an
|
||||
unprivileged session gets PermissionError; that must degrade to
|
||||
None (the honest "no score for us" state), never crash the bridge.
|
||||
"""
|
||||
try:
|
||||
with open("/var/log/lynis.log", encoding="utf-8") as f:
|
||||
log = f.read()
|
||||
except FileNotFoundError:
|
||||
except OSError:
|
||||
return None
|
||||
m = HARDENING_RE.search(log)
|
||||
return int(m.group(1)) if m else None
|
||||
|
|
|
|||
432
bridge/kata.py
432
bridge/kata.py
|
|
@ -29,10 +29,25 @@ fabricated records:
|
|||
pxe-status real PXE/TFTP status: systemctl is-active
|
||||
dnsmasq + test -d /srv/tftp + ls
|
||||
/srv/tftp/pxelinux.cfg/.
|
||||
qcrows-list list QCrows kernel bundles in
|
||||
/usr/share/sysdeck/kata/qcrows/. (qcrows-export
|
||||
and qcrows-initrd-regen are operator binaries the
|
||||
host runs directly — the bridge does not wrap them.)
|
||||
qcrows-list list QCrows images in /usr/share/sysdeck/kata/qcrows/.
|
||||
Format-aware per the cockpit-kata master spec
|
||||
(qcrows-spec.md v0.2.0): each .qcrows/.qcrows.gz/
|
||||
.tar.gz entry is opened IN MEMORY and its
|
||||
metadata.toml + menu.toml are parsed, surfacing
|
||||
{image, kernel, rootfs, hypervisors, menu} per
|
||||
bundle. Legacy non-QCrows tarballs degrade
|
||||
gracefully to stat-only entries (qcrows: false).
|
||||
qcrows-inspect <f> detailed single-image view: full metadata, menu,
|
||||
member listing (name/size/mode), hash-file presence.
|
||||
qcrows-verify <f> in-memory verification mirroring cockpit-kata's
|
||||
qcrows-verify: required files, kernel binary
|
||||
magic (bzImage "HdrS" @0x202 / ELF magic), kernel
|
||||
config + required Kata options (warn), initrd
|
||||
(warn), metadata format_version + hypervisors,
|
||||
and a full sha256sum -c style hash walk.
|
||||
Returns {ok, passed, failed, warnings, checks[]}.
|
||||
(qcrows-export and qcrows-initrd-regen remain
|
||||
operator binaries the bridge does not wrap.)
|
||||
|
||||
KEY DESIGN DECISIONS (Kata 3.x reality):
|
||||
- kata-runtime list/inspect were REMOVED in 3.x. Do not call them.
|
||||
|
|
@ -69,6 +84,8 @@ Usage:
|
|||
python3 /usr/lib/sysdeck/bridge/kata.py check
|
||||
python3 /usr/lib/sysdeck/bridge/kata.py pxe-status
|
||||
python3 /usr/lib/sysdeck/bridge/kata.py qcrows-list
|
||||
python3 /usr/lib/sysdeck/bridge/kata.py qcrows-inspect <filename>
|
||||
python3 /usr/lib/sysdeck/bridge/kata.py qcrows-verify <filename>
|
||||
"""
|
||||
|
||||
import json
|
||||
|
|
@ -572,17 +589,213 @@ def cmd_pxe_status(_args: list[str]) -> dict[str, Any]:
|
|||
}
|
||||
|
||||
|
||||
# ── QCrows format support (cockpit-kata master spec v0.2.0) ────────
|
||||
#
|
||||
# QCrows (cue-crows) is the in-house self-describing VM container image
|
||||
# format for Kata Containers, master-implemented in cockpit-kata
|
||||
# (qcrows-spec.md + qcrows-pack / qcrows-verify / qcrows-inspect).
|
||||
# SysDeck is a *consumer*: it reads the real archives from
|
||||
# /usr/share/sysdeck/kata/qcrows/ in memory (no extraction to disk —
|
||||
# nothing executable is ever materialized from an image here) and
|
||||
# mirrors the master tools' verification semantics.
|
||||
|
||||
QCROWS_FORMAT_VERSION = "0.2.0"
|
||||
QCROWS_MAX_MEMBERS = 10_000 # tar-bomb guard (header count)
|
||||
QCROWS_MAX_TEXT_MEMBER_BYTES = 1 << 20 # 1 MiB cap on parsed text members
|
||||
|
||||
# Required Kata kernel options (qcrows-verify warns when absent).
|
||||
QCROWS_REQUIRED_KATA_OPTS = (
|
||||
"CONFIG_VSOCKETS", "CONFIG_VIRTIO", "CONFIG_VIRTIO_PCI",
|
||||
"CONFIG_DEVTMPFS", "CONFIG_DEVTMPFS_MOUNT",
|
||||
)
|
||||
|
||||
_QCROWS_REQUIRED_FILES = ("metadata.toml", "menu.toml", "hashes.sha256")
|
||||
|
||||
|
||||
def _qcrows_member_names(tf: Any) -> dict[str, Any]:
|
||||
"""Map normalized member name → TarInfo for a QCrows archive.
|
||||
|
||||
Members are stored "./"-prefixed by qcrows-pack; normalize both
|
||||
spellings. Raises ValueError on tar-bomb-sized header counts.
|
||||
"""
|
||||
members = tf.getmembers()
|
||||
if len(members) > QCROWS_MAX_MEMBERS:
|
||||
raise ValueError(f"too many tar members ({len(members)})")
|
||||
out: dict[str, Any] = {}
|
||||
for m in members:
|
||||
if not m.isfile():
|
||||
continue
|
||||
name = m.name
|
||||
if name.startswith("./"):
|
||||
name = name[2:]
|
||||
out[name] = m
|
||||
return out
|
||||
|
||||
|
||||
def _qcrows_read_member(tf: Any, info: Any) -> bytes:
|
||||
"""Read one member's bytes with the text-member size cap."""
|
||||
if info.size > QCROWS_MAX_TEXT_MEMBER_BYTES:
|
||||
raise ValueError(f"member {info.name!r} exceeds size cap")
|
||||
fh = tf.extractfile(info)
|
||||
if fh is None:
|
||||
return b""
|
||||
return fh.read()
|
||||
|
||||
|
||||
def _qcrows_parse_toml(text: str) -> dict[str, Any]:
|
||||
"""Parse the flat QCrows TOML shape into a nested dict.
|
||||
|
||||
Supports exactly what qcrows-pack emits: ``[ section ]`` headers
|
||||
(dotted sections become nested dicts), ``key = "str"``,
|
||||
``key = true/false``, ``key = 123``, ``key = [ "a", "b" ]``.
|
||||
Unknown lines and # comments are skipped. Section-aware by
|
||||
design — cockpit-kata's qcrows-inspect greps first-matches, which
|
||||
mis-attributes kernel fields to initrd/rootfs rows; this parser
|
||||
does not.
|
||||
"""
|
||||
root: dict[str, Any] = {}
|
||||
current: dict[str, Any] = root
|
||||
for raw in text.splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
if line.startswith("[") and line.endswith("]"):
|
||||
section = line[1:-1].strip().strip('"')
|
||||
current = root
|
||||
for part in section.split("."):
|
||||
part = part.strip().strip('"')
|
||||
current = current.setdefault(part, {})
|
||||
continue
|
||||
if "=" not in line:
|
||||
continue
|
||||
key, _, value = line.partition("=")
|
||||
key = key.strip().strip('"')
|
||||
value = value.strip()
|
||||
# Strip trailing comments outside quotes (qcrows-pack emits
|
||||
# none, but hand-edited metadata may carry them).
|
||||
if value.startswith('"'):
|
||||
end = value.find('"', 1)
|
||||
current[key] = value[1:end] if end > 0 else value[1:]
|
||||
elif value.startswith("["):
|
||||
try:
|
||||
current[key] = json.loads(value.replace("'", '"'))
|
||||
except ValueError:
|
||||
current[key] = [
|
||||
v.strip().strip('"')
|
||||
for v in value.strip("[]").split(",") if v.strip()
|
||||
]
|
||||
elif value in ("true", "false"):
|
||||
current[key] = value == "true"
|
||||
else:
|
||||
try:
|
||||
current[key] = int(value)
|
||||
except ValueError:
|
||||
current[key] = value
|
||||
return root
|
||||
|
||||
|
||||
def _qcrows_kernel_magic_ok(blob: bytes) -> tuple[bool, str]:
|
||||
"""Classify a kernel binary by magic — the in-memory equivalent of
|
||||
qcrows-verify's `file | grep (ELF|Linux.*boot)` check.
|
||||
|
||||
Returns (ok, kind) where kind is "elf" | "bzimage" | "unknown".
|
||||
"""
|
||||
if blob[:4] == b"\x7fELF":
|
||||
return True, "elf"
|
||||
if len(blob) >= 0x206 and blob[0x202:0x206] == b"HdrS":
|
||||
return True, "bzimage"
|
||||
return False, "unknown"
|
||||
|
||||
|
||||
def _qcrows_image_info(path: Path) -> dict[str, Any]:
|
||||
"""Read one QCrows archive's self-description, in memory.
|
||||
|
||||
Returns {qcrows: bool, format_version, image, kernel, rootfs,
|
||||
initrd, hypervisors, menu, members, error?}. Non-QCrows tarballs
|
||||
(no metadata.toml) return qcrows: false; unreadable archives
|
||||
return qcrows: false with an error note — never a raise.
|
||||
"""
|
||||
import tarfile
|
||||
info: dict[str, Any] = {"qcrows": False}
|
||||
try:
|
||||
with tarfile.open(path, "r:*") as tf:
|
||||
members = _qcrows_member_names(tf)
|
||||
info["members"] = len(members)
|
||||
if "metadata.toml" not in members:
|
||||
info["error"] = "no metadata.toml (not a QCrows image)"
|
||||
return info
|
||||
meta_text = _qcrows_read_member(
|
||||
tf, members["metadata.toml"],
|
||||
).decode("utf-8", errors="replace")
|
||||
meta = _qcrows_parse_toml(meta_text)
|
||||
info["qcrows"] = True
|
||||
info["format_version"] = meta.get("qcrows", {}).get(
|
||||
"format_version",
|
||||
)
|
||||
image = meta.get("image", {})
|
||||
info["image"] = {
|
||||
k: image.get(k)
|
||||
for k in ("name", "version", "description", "arch",
|
||||
"os", "created_at")
|
||||
}
|
||||
info["hypervisors"] = image.get(
|
||||
"compatibility", {},
|
||||
).get("hypervisors", [])
|
||||
info["kernel"] = meta.get("kernel", {})
|
||||
info["rootfs"] = meta.get("rootfs", {})
|
||||
info["initrd"] = meta.get("initrd", {})
|
||||
info["has_boot_params"] = "boot-params.conf" in members
|
||||
info["has_kernel_config"] = "kernel/config" in members
|
||||
kernel_fmt = "vmlinux" if (
|
||||
info["kernel"].get("format") == "vmlinux"
|
||||
or "kernel/vmlinux" in members
|
||||
) else "vmlinuz"
|
||||
info["kernel_binary_present"] = (
|
||||
f"kernel/{kernel_fmt}" in members
|
||||
)
|
||||
if "menu.toml" in members:
|
||||
menu = _qcrows_parse_toml(_qcrows_read_member(
|
||||
tf, members["menu.toml"],
|
||||
).decode("utf-8", errors="replace"))
|
||||
info["menu"] = {
|
||||
"label": menu.get("menu", {}).get("label"),
|
||||
"category": menu.get("menu", {}).get("category"),
|
||||
}
|
||||
else:
|
||||
info["menu"] = {}
|
||||
except (tarfile.TarError, ValueError, OSError) as exc:
|
||||
info["qcrows"] = False
|
||||
info["members"] = 0
|
||||
info["error"] = _sanitize_output(str(exc), 200)
|
||||
return info
|
||||
|
||||
|
||||
def _qcrows_resolve(args: list[str]) -> Path | None:
|
||||
"""Resolve a user-supplied bundle filename safely under QCROWS_DIR.
|
||||
|
||||
Filename validation + realpath containment — the same hardening
|
||||
the qcrows-list path applies. Returns None (and the caller
|
||||
reports the error) on any traversal / invalid name.
|
||||
"""
|
||||
if not args or not _validate_filename(args[0]):
|
||||
return None
|
||||
return _resolve_path_under_base(str(QCROWS_DIR / args[0]), QCROWS_DIR)
|
||||
|
||||
|
||||
# ── Subcommand: qcrows-list ────────────────────────────────────────
|
||||
#
|
||||
# QCrows kernel bundles are the kata kernel/module compilation
|
||||
# surface; the listing reads the real filesystem.
|
||||
# Format-aware per the cockpit-kata master spec: every archive in the
|
||||
# bundle dir is opened in memory and its self-description surfaced.
|
||||
# Legacy non-QCrows tarballs degrade to stat-only entries.
|
||||
|
||||
|
||||
def cmd_qcrows_list(_args: list[str]) -> list[dict[str, Any]]:
|
||||
"""List QCrows kernel bundles in /usr/share/sysdeck/kata/qcrows/.
|
||||
"""List QCrows images in /usr/share/sysdeck/kata/qcrows/.
|
||||
|
||||
Each entry: {filename, path, size_bytes, mtime}.
|
||||
Returns [] if the directory doesn't exist (empty state, NOT mock).
|
||||
Each entry: {filename, path, size_bytes, size_mb, mtime, qcrows,
|
||||
format_version?, image?, kernel?, rootfs?, hypervisors?, menu?,
|
||||
error?}. Returns [] if the directory doesn't exist (empty state,
|
||||
NOT mock).
|
||||
"""
|
||||
if not QCROWS_DIR.is_dir():
|
||||
return []
|
||||
|
|
@ -591,21 +804,214 @@ def cmd_qcrows_list(_args: list[str]) -> list[dict[str, Any]]:
|
|||
for entry in sorted(QCROWS_DIR.iterdir()):
|
||||
if not entry.is_file():
|
||||
continue
|
||||
if not entry.name.endswith((".qcrows", ".tar.gz", ".tgz")):
|
||||
if not entry.name.endswith((".qcrows", ".qcrows.gz", ".tar.gz", ".tgz")):
|
||||
continue
|
||||
stat = entry.stat()
|
||||
bundles.append({
|
||||
record: dict[str, Any] = {
|
||||
"filename": entry.name,
|
||||
"path": str(entry),
|
||||
"size_bytes": stat.st_size,
|
||||
"size_mb": round(stat.st_size / (1024 * 1024), 2),
|
||||
"mtime": stat.st_mtime,
|
||||
})
|
||||
}
|
||||
record.update(_qcrows_image_info(entry))
|
||||
bundles.append(record)
|
||||
except (PermissionError, OSError):
|
||||
pass
|
||||
return bundles
|
||||
|
||||
|
||||
# ── Subcommand: qcrows-inspect ─────────────────────────────────────
|
||||
|
||||
|
||||
def cmd_qcrows_inspect(args: list[str]) -> dict[str, Any]:
|
||||
"""Detailed view of one QCrows image in the bundle directory.
|
||||
|
||||
Returns the qcrows-list record plus the full member listing
|
||||
(name, size, mode) and build provenance. {error: ...} on an
|
||||
invalid filename, traversal attempt, or unreadable archive.
|
||||
"""
|
||||
resolved = _qcrows_resolve(args)
|
||||
if resolved is None or not resolved.is_file():
|
||||
return {"error": f"bundle not found under {QCROWS_DIR}: {args[0] if args else ''!r}"}
|
||||
import tarfile
|
||||
result = _qcrows_image_info(resolved)
|
||||
result["filename"] = resolved.name
|
||||
result["path"] = str(resolved)
|
||||
try:
|
||||
with tarfile.open(resolved, "r:*") as tf:
|
||||
members = _qcrows_member_names(tf)
|
||||
result["members_list"] = [
|
||||
{"name": name, "size": m.size, "mode": oct(m.mode)}
|
||||
for name, m in sorted(members.items())
|
||||
]
|
||||
if "build.toml" in members:
|
||||
build = _qcrows_parse_toml(_qcrows_read_member(
|
||||
tf, members["build.toml"],
|
||||
).decode("utf-8", errors="replace"))
|
||||
result["build"] = build.get("build", {})
|
||||
except (tarfile.TarError, ValueError, OSError) as exc:
|
||||
result["error"] = _sanitize_output(str(exc), 200)
|
||||
return result
|
||||
|
||||
|
||||
# ── Subcommand: qcrows-verify ──────────────────────────────────────
|
||||
#
|
||||
# In-memory mirror of cockpit-kata's qcrows-verify: nothing is
|
||||
# extracted to disk, no `file` subprocess is spawned (kernel magic is
|
||||
# checked on bytes), and the hash walk streams members.
|
||||
|
||||
|
||||
def cmd_qcrows_verify(args: list[str]) -> dict[str, Any]:
|
||||
"""Verify one QCrows image against the master spec's checks.
|
||||
|
||||
Checks (mirroring qcrows-verify):
|
||||
1. required files (metadata.toml, menu.toml, hashes.sha256)
|
||||
2. rootfs present (rootfs.tar.*)
|
||||
3. kernel binary present + magic (bzImage/ELF)
|
||||
4. kernel/config present + required Kata options (warn only)
|
||||
5. initrd present (warn only)
|
||||
6. metadata format_version + hypervisors declared
|
||||
7. every hashes.sha256 entry matches the member bytes
|
||||
|
||||
Returns {ok, passed, failed, warnings, checks: [{name, passed,
|
||||
detail}]} or {error: ...} for bad filenames / unreadable archives.
|
||||
"""
|
||||
import hashlib
|
||||
import tarfile
|
||||
|
||||
resolved = _qcrows_resolve(args)
|
||||
if resolved is None or not resolved.is_file():
|
||||
return {"error": f"bundle not found under {QCROWS_DIR}: {args[0] if args else ''!r}"}
|
||||
|
||||
checks: list[dict[str, Any]] = []
|
||||
warnings: list[str] = []
|
||||
|
||||
def check(name: str, passed: bool, detail: str) -> None:
|
||||
checks.append({"name": name, "passed": passed, "detail": detail})
|
||||
|
||||
try:
|
||||
with tarfile.open(resolved, "r:*") as tf:
|
||||
members = _qcrows_member_names(tf)
|
||||
|
||||
# 1. required files
|
||||
for req in _QCROWS_REQUIRED_FILES:
|
||||
check(f"{req} present", req in members,
|
||||
"found" if req in members else "missing (required)")
|
||||
# 2. rootfs
|
||||
rootfs = next(
|
||||
(n for n in members if n.startswith("rootfs.tar")), None,
|
||||
)
|
||||
check("rootfs found", rootfs is not None,
|
||||
rootfs or "no rootfs.tar.* member")
|
||||
# 3. kernel binary + magic
|
||||
kernel_member = next(
|
||||
(n for n in ("kernel/vmlinuz", "kernel/vmlinux")
|
||||
if n in members), None,
|
||||
)
|
||||
magic_kind = "unknown"
|
||||
if kernel_member:
|
||||
# bzImage magic "HdrS" sits at 0x202 — read past it.
|
||||
blob = tf.extractfile(members[kernel_member]).read(0x206 + 4)
|
||||
ok_magic, magic_kind = _qcrows_kernel_magic_ok(blob)
|
||||
check("kernel binary format valid", ok_magic,
|
||||
f"{kernel_member} ({magic_kind})")
|
||||
else:
|
||||
check("kernel binary found", False,
|
||||
"kernel/vmlinuz|vmlinux missing (required in v0.2+)")
|
||||
# 4. kernel config + Kata options (config presence is a hard
|
||||
# check; option shortfalls are WARNINGS only — mirroring
|
||||
# qcrows-verify, which still exits 0 on them)
|
||||
if "kernel/config" in members:
|
||||
check("kernel/config found", True, "kernel/config")
|
||||
cfg_text = _qcrows_read_member(
|
||||
tf, members["kernel/config"],
|
||||
).decode("utf-8", errors="replace")
|
||||
missing = [
|
||||
opt for opt in QCROWS_REQUIRED_KATA_OPTS
|
||||
if f"{opt}=y" not in cfg_text
|
||||
]
|
||||
if missing:
|
||||
warnings.append(
|
||||
"required Kata options missing: " + ", ".join(missing),
|
||||
)
|
||||
else:
|
||||
check("kernel/config found", False,
|
||||
"kernel/config missing (required in v0.2+)")
|
||||
# 5. initrd (warn)
|
||||
has_initrd = any(
|
||||
n in members for n in (
|
||||
"initrd.img", "initrd.cpio.gz", "initrd.cpio.lz4",
|
||||
"initrd.cpio.xz", "initrd.cramfs",
|
||||
)
|
||||
)
|
||||
if not has_initrd:
|
||||
warnings.append(
|
||||
"no initrd found (one of initrd/cramfs is recommended)",
|
||||
)
|
||||
# 6. metadata fields
|
||||
if "metadata.toml" in members:
|
||||
meta = _qcrows_parse_toml(_qcrows_read_member(
|
||||
tf, members["metadata.toml"],
|
||||
).decode("utf-8", errors="replace"))
|
||||
fmt_ver = meta.get("qcrows", {}).get("format_version")
|
||||
check("metadata format_version", fmt_ver is not None,
|
||||
str(fmt_ver or "missing"))
|
||||
hypers = meta.get("image", {}).get(
|
||||
"compatibility", {},
|
||||
).get("hypervisors", [])
|
||||
check("hypervisor compatibility declared", bool(hypers),
|
||||
", ".join(map(str, hypers)) or "not declared")
|
||||
# 7. hash walk (sha256sum -c semantics, in memory)
|
||||
if "hashes.sha256" in members:
|
||||
hash_text = _qcrows_read_member(
|
||||
tf, members["hashes.sha256"],
|
||||
).decode("utf-8", errors="replace")
|
||||
passed_n = 0
|
||||
failed: list[str] = []
|
||||
for line in hash_text.splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
m = re.match(r"^([0-9a-f]{64})[ *]{2}(.+)$", line)
|
||||
if not m:
|
||||
failed.append(f"unparseable line: {line[:60]}")
|
||||
continue
|
||||
expect, rel = m.group(1), m.group(2).strip()
|
||||
rel = rel[2:] if rel.startswith("./") else rel
|
||||
if rel not in members:
|
||||
failed.append(f"{rel}: no such member")
|
||||
continue
|
||||
h = hashlib.sha256()
|
||||
fh = tf.extractfile(members[rel])
|
||||
for chunk in iter(
|
||||
lambda: fh.read(1024 * 1024), b"",
|
||||
):
|
||||
h.update(chunk)
|
||||
if h.hexdigest() != expect:
|
||||
failed.append(f"{rel}: checksum mismatch")
|
||||
else:
|
||||
passed_n += 1
|
||||
check("SHA-256 checksums", not failed,
|
||||
f"{passed_n} verified" if not failed else
|
||||
"; ".join(failed[:5]))
|
||||
else:
|
||||
check("SHA-256 checksums", False,
|
||||
"hashes.sha256 missing — cannot verify integrity")
|
||||
except (tarfile.TarError, ValueError, OSError) as exc:
|
||||
return {"error": _sanitize_output(str(exc), 200)}
|
||||
|
||||
passed = sum(1 for c in checks if c["passed"])
|
||||
failed = len(checks) - passed
|
||||
return {
|
||||
"ok": failed == 0,
|
||||
"passed": passed,
|
||||
"failed": failed,
|
||||
"warnings": warnings,
|
||||
"checks": checks,
|
||||
}
|
||||
|
||||
|
||||
# ── Dispatch table ─────────────────────────────────────────────────
|
||||
|
||||
COMMANDS = {
|
||||
|
|
@ -617,6 +1023,8 @@ COMMANDS = {
|
|||
"check": cmd_check,
|
||||
"pxe-status": cmd_pxe_status,
|
||||
"qcrows-list": cmd_qcrows_list,
|
||||
"qcrows-inspect": cmd_qcrows_inspect,
|
||||
"qcrows-verify": cmd_qcrows_verify,
|
||||
}
|
||||
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"_comment": "Compatibility Manifest — sysdeck v0.1.3",
|
||||
"version": "0.4.5",
|
||||
"version": "0.4.6",
|
||||
"suite_requires": { "cockpit": ">=239", "python": ">=3.9" },
|
||||
"modules": {
|
||||
"containers": {
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
# Upstream: https://dcos.net
|
||||
|
||||
pkgname=sysdeck
|
||||
pkgver=0.4.5
|
||||
pkgver=0.4.6
|
||||
pkgrel=1
|
||||
pkgdesc="Unified operations surface for Linux infrastructure — the Cockpit plugin edition: 27 domain modules plus the Python bridge (the standalone web console ships in the master tarball)"
|
||||
arch=('any')
|
||||
|
|
|
|||
|
|
@ -1,3 +1,15 @@
|
|||
sysdeck (0.4.6-1) unstable; urgency=medium
|
||||
|
||||
* QCrows format-aware Kata bridge: qcrows-list parses metadata.toml +
|
||||
menu.toml in memory; new qcrows-inspect / qcrows-verify subcommands
|
||||
mirror cockpit-kata's master verification (kernel magic, config,
|
||||
sha256 hash walk) without extracting to disk. Kata panel enriched
|
||||
with image metadata + Verify/Inspect actions.
|
||||
* integrity.score() returns None on PermissionError (root-only
|
||||
/var/log/lynis.log no longer crashes unprivileged sessions).
|
||||
|
||||
-- Jeremy Anderson <info@dcos.net> Sun, 27 Sep 2026 01:13:57 +0000
|
||||
|
||||
sysdeck (0.4.5-1) unstable; urgency=medium
|
||||
|
||||
* PRODUCTION-HARDENING RELEASE — full MoE QA pass (web designers,
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ import shutil
|
|||
import subprocess
|
||||
from setuptools import setup
|
||||
|
||||
VERSION = "0.4.5"
|
||||
VERSION = "0.4.6"
|
||||
PACKAGE = "sysdeck"
|
||||
|
||||
# The repository root (setup.py lives in packaging/).
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
# Debian/Ubuntu users: see packaging/debian/
|
||||
|
||||
Name: sysdeck
|
||||
Version: 0.4.5
|
||||
Version: 0.4.6
|
||||
Release: 1%{?dist}
|
||||
Summary: Unified operations surface for Linux infrastructure
|
||||
|
||||
|
|
@ -92,6 +92,16 @@ if [ $1 -eq 0 ]; then
|
|||
fi
|
||||
|
||||
%changelog
|
||||
* Sun Sep 27 2026 Jeremy Anderson <info@dcos.net> - 0.4.6-1
|
||||
- v0.4.6: QCrows format-aware Kata bridge. qcrows-list now parses
|
||||
metadata.toml + menu.toml from .qcrows archives in memory (image
|
||||
name/version/arch, kernel version/format, hypervisors, menu label);
|
||||
new qcrows-inspect + qcrows-verify subcommands mirror cockpit-kata's
|
||||
master checks (required files, kernel magic, config + Kata options,
|
||||
full sha256sum hash walk) with nothing extracted to disk. Kata panel
|
||||
shows the metadata and gains per-image Verify/Inspect actions.
|
||||
integrity.score() degrades to None on PermissionError (root-only
|
||||
lynis.log no longer crashes unprivileged sessions).
|
||||
* Thu Sep 17 2026 Jeremy Anderson <info@dcos.net> - 0.4.5-1
|
||||
- v0.4.5 production-hardening release: full MoE QA pass. Makefile
|
||||
web-dev splice fixed; reproducible dist + clean-tree release gate;
|
||||
|
|
|
|||
|
|
@ -272,42 +272,80 @@ function renderQcrowsCard(qcrows) {
|
|||
return `
|
||||
<div class="suite-card">
|
||||
<div class="suite-card-header">
|
||||
<h3 class="suite-card-title">QCrows Kernel Bundles (0)</h3>
|
||||
<h3 class="suite-card-title">QCrows VM Container Images (0)</h3>
|
||||
</div>
|
||||
<div class="suite-card-body">
|
||||
<p class="suite-muted">
|
||||
No QCrows kernel bundles found at
|
||||
No QCrows images found at
|
||||
<code>/usr/share/sysdeck/kata/qcrows/</code>.
|
||||
This is the real empty state — not mock data.
|
||||
</p>
|
||||
<p class="suite-muted" style="margin-top:0.5rem;font-size:0.85rem">
|
||||
QCrows bundles are pre-built kata kernel + initrd +
|
||||
rootfs images. Build one with:
|
||||
QCrows (.qcrows, spec v0.2) is the self-describing
|
||||
VM container image format — rootfs + initrd + kernel
|
||||
+ Cockpit menu metadata in one verifiable archive.
|
||||
Build one with:
|
||||
</p>
|
||||
<pre class="suite-mono" style="margin-top:0.5rem;background:#1a1a1a;padding:8px;border-radius:4px;font-size:0.8rem">qcrows-pack --rootfs rootfs.tar.gz --kernel vmlinuz --kernel-config .config \\
|
||||
--name alpine-3.20-kata -o alpine-3.20-kata.qcrows</pre>
|
||||
<p class="suite-muted" style="margin-top:0.5rem;font-size:0.85rem">
|
||||
ai-lsc can export the active tool stack directly as a
|
||||
.qcrows image (Container Stacks → Export → QCrows).
|
||||
</p>
|
||||
<pre class="suite-mono" style="margin-top:0.5rem;background:#1a1a1a;padding:8px;border-radius:4px;font-size:0.8rem">qcrows-export --kernel /path/to/vmlinuz --initrd /path/to/initrd \\
|
||||
--rootfs /path/to/rootfs --name alpine-3.20-kata</pre>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
const totalSize = qcrows.reduce((sum, q) => sum + (q.size_bytes || 0), 0);
|
||||
const totalMb = (totalSize / (1024 * 1024)).toFixed(1);
|
||||
const rows = qcrows.map((q) => `
|
||||
const rows = qcrows.map((q) => {
|
||||
// Format-aware rows (spec v0.2 metadata); legacy non-QCrows
|
||||
// tarballs degrade to the stat-only columns.
|
||||
const isQcrows = q.qcrows === true;
|
||||
const image = q.image || {};
|
||||
const kernel = q.kernel || {};
|
||||
const displayName = isQcrows
|
||||
? `${image.name || '?'} ${image.version || ''}`.trim()
|
||||
: '(not a QCrows archive)';
|
||||
const kernelCell = isQcrows
|
||||
? `${kernel.version || '?'} · ${kernel.format || '?'}${q.kernel_binary_present === false ? ' ⚠ missing' : ''}`
|
||||
: '—';
|
||||
const hyperCell = isQcrows && Array.isArray(q.hypervisors)
|
||||
? q.hypervisors.join(', ')
|
||||
: '—';
|
||||
return `
|
||||
<tr>
|
||||
<td class="suite-table-mono">${escapeHtml(q.filename)}</td>
|
||||
<td>
|
||||
<div class="suite-table-mono">${escapeHtml(displayName)}</div>
|
||||
<div class="suite-muted" style="font-size:0.75rem">${escapeHtml(q.filename)}</div>
|
||||
</td>
|
||||
<td class="suite-muted">${q.size_mb} MB</td>
|
||||
<td class="suite-table-mono suite-muted">${escapeHtml(kernelCell)}</td>
|
||||
<td class="suite-table-mono suite-muted">${escapeHtml(image.arch || '—')}</td>
|
||||
<td class="suite-table-mono suite-muted">${escapeHtml(hyperCell)}</td>
|
||||
<td class="suite-table-mono suite-muted">${new Date(q.mtime * 1000).toISOString().split('T')[0]}</td>
|
||||
<td>
|
||||
<button class="suite-btn suite-btn-ghost btn-qcrows-verify" data-filename="${escapeHtml(q.filename)}" ${isQcrows ? '' : 'disabled'}>Verify</button>
|
||||
<button class="suite-btn suite-btn-ghost btn-qcrows-inspect" data-filename="${escapeHtml(q.filename)}">Inspect</button>
|
||||
</td>
|
||||
</tr>
|
||||
`).join('');
|
||||
`;
|
||||
}).join('');
|
||||
return `
|
||||
<div class="suite-card">
|
||||
<div class="suite-card-header">
|
||||
<h3 class="suite-card-title">QCrows Kernel Bundles (${qcrows.length}, ${totalMb} MB total)</h3>
|
||||
<h3 class="suite-card-title">QCrows VM Container Images (${qcrows.length}, ${totalMb} MB total)</h3>
|
||||
</div>
|
||||
<table class="suite-table">
|
||||
<thead><tr><th>Filename</th><th>Size</th><th>Modified</th></tr></thead>
|
||||
<thead><tr><th>Image</th><th>Size</th><th>Kernel</th><th>Arch</th><th>Hypervisors</th><th>Modified</th><th>Actions</th></tr></thead>
|
||||
<tbody>${rows}</tbody>
|
||||
</table>
|
||||
<p class="suite-muted" style="margin:0.5rem 1rem;font-size:0.8rem">
|
||||
Verify runs the cockpit-kata master checks in memory (hashes,
|
||||
kernel magic + config, metadata); Inspect shows the full
|
||||
archive detail. Non-QCrows tarballs in the directory are
|
||||
listed but not verifiable.
|
||||
</p>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
|
@ -393,6 +431,46 @@ function wireEvents(panel, { bridge, EventBus }) {
|
|||
const card = panel.querySelector('#kata-metrics-card');
|
||||
if (card) card.style.display = 'none';
|
||||
});
|
||||
|
||||
// QCrows image actions — Verify (in-memory master checks) and
|
||||
// Inspect (full archive detail). Output goes to the shared
|
||||
// operation-output card; every bridge value is rendered as text.
|
||||
panel.querySelectorAll('.btn-qcrows-verify').forEach((btn) => {
|
||||
btn.addEventListener('click', async () => {
|
||||
const filename = btn.dataset.filename;
|
||||
output(`Verifying ${filename}…`);
|
||||
try {
|
||||
const r = await bridge.kata.qcrowsVerify(filename);
|
||||
if (r && r.error) {
|
||||
output(`Verify error: ${r.error}`, true);
|
||||
return;
|
||||
}
|
||||
const lines = r.checks.map((c) =>
|
||||
` ${c.passed ? 'PASS' : 'FAIL'}: ${c.name} — ${c.detail}`,
|
||||
);
|
||||
const warns = (r.warnings || []).map((w) => ` WARN: ${w}`);
|
||||
output([
|
||||
`${r.ok ? '✔ VERIFIED' : '✘ FAILED'} — ${filename} (${r.passed} passed, ${r.failed} failed)`,
|
||||
...lines,
|
||||
...warns,
|
||||
].join('\n'), !r.ok);
|
||||
} catch (err) {
|
||||
output(`Verify error: ${err.message || err}`, true);
|
||||
}
|
||||
});
|
||||
});
|
||||
panel.querySelectorAll('.btn-qcrows-inspect').forEach((btn) => {
|
||||
btn.addEventListener('click', async () => {
|
||||
const filename = btn.dataset.filename;
|
||||
output(`Inspecting ${filename}…`);
|
||||
try {
|
||||
const r = await bridge.kata.qcrowsInspect(filename);
|
||||
output(JSON.stringify(r, null, 2), Boolean(r && r.error));
|
||||
} catch (err) {
|
||||
output(`Inspect error: ${err.message || err}`, true);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// ── Utilities ───────────────────────────────────────────────────────
|
||||
|
|
|
|||
|
|
@ -506,7 +506,12 @@ export const bridge = {
|
|||
// /metrics?sandbox=<id> + filesystem /run/vc/sbs/, /run/kata/
|
||||
// - summary/version/check → kata-runtime version/env --json/check
|
||||
// - pxeStatus → systemctl is-active dnsmasq + real /srv/tftp probes
|
||||
// - qcrowsList → filesystem /usr/share/sysdeck/kata/qcrows/
|
||||
// - qcrowsList → filesystem /usr/share/sysdeck/kata/qcrows/ — since
|
||||
// the cockpit-kata master spec v0.2 this is FORMAT-AWARE: each
|
||||
// archive is parsed in memory (metadata.toml + menu.toml), so
|
||||
// the panel shows image name/version/arch/kernel/hypervisors.
|
||||
// - qcrowsInspect/qcrowsVerify → single-image detail + in-memory
|
||||
// verification mirroring cockpit-kata's qcrows-verify.
|
||||
// Read-only queries do NOT pass { superuser: 'try' }.
|
||||
kata: {
|
||||
list: () => bridgeCmd("kata", ["list"]),
|
||||
|
|
@ -517,6 +522,8 @@ export const bridge = {
|
|||
check: () => bridgeCmd("kata", ["check"]),
|
||||
pxeStatus: () => bridgeCmd("kata", ["pxe-status"]),
|
||||
qcrowsList: () => bridgeCmd("kata", ["qcrows-list"]),
|
||||
qcrowsInspect: (filename) => bridgeCmd("kata", ["qcrows-inspect", filename]),
|
||||
qcrowsVerify: (filename) => bridgeCmd("kata", ["qcrows-verify", filename]),
|
||||
},
|
||||
|
||||
// v0.0.39: Monitoring module — shared tabbed Prometheus + Grafana panel.
|
||||
|
|
|
|||
Binary file not shown.
Binary file not shown.
|
|
@ -834,10 +834,273 @@ class TestKataBridgeProduction(unittest.TestCase):
|
|||
|
||||
def test_kata_bridge_dispatch_table_has_all_subcommands(self):
|
||||
expected = {"list", "inspect", "metrics", "summary", "version",
|
||||
"check", "pxe-status", "qcrows-list"}
|
||||
"check", "pxe-status", "qcrows-list",
|
||||
"qcrows-inspect", "qcrows-verify"}
|
||||
self.assertEqual(set(self.kata.COMMANDS.keys()), expected)
|
||||
|
||||
|
||||
# ── QCrows format-aware bridge tests (cockpit-kata master spec) ────
|
||||
#
|
||||
# The bridge must be a real QCrows CONSUMER: parse metadata.toml /
|
||||
# menu.toml from archives in memory, mirror the master qcrows-verify
|
||||
# checks, and reject tampered bundles. These tests build synthetic
|
||||
# .qcrows archives with the same layout qcrows-pack produces.
|
||||
|
||||
|
||||
def _build_qcrows(dest, *, kernel_fmt="vmlinuz", tamper_rootfs=False,
|
||||
omit_metadata=False, kata_opts=True):
|
||||
"""Write a synthetic spec-v0.2 .qcrows archive at *dest*.
|
||||
|
||||
Mirrors qcrows-pack's layout: ./-prefixed members, metadata.toml,
|
||||
menu.toml, hashes.sha256 (sha256sum format), rootfs.tar.gz, kernel/
|
||||
binary+config, boot-params.conf. The kernel blob carries a real
|
||||
bzImage magic ("HdrS" @ 0x202). Returns the archive path.
|
||||
"""
|
||||
import hashlib
|
||||
import io
|
||||
import tarfile
|
||||
|
||||
kernel = bytearray(b"\x00" * 0x400)
|
||||
kernel[0x202:0x206] = b"HdrS"
|
||||
kernel[0x206:0x208] = (1).to_bytes(2, "little") # boot sector setup
|
||||
config_lines = ["# synthetic config"]
|
||||
if kata_opts:
|
||||
config_lines += [
|
||||
"CONFIG_VSOCKETS=y", "CONFIG_VIRTIO=y",
|
||||
"CONFIG_VIRTIO_PCI=y", "CONFIG_DEVTMPFS=y",
|
||||
"CONFIG_DEVTMPFS_MOUNT=y",
|
||||
]
|
||||
config = ("\n".join(config_lines) + "\n").encode()
|
||||
|
||||
metadata = f"""[ qcrows ]
|
||||
format_version = "0.2.0"
|
||||
|
||||
[ image ]
|
||||
name = "synthetic-test"
|
||||
version = "1.0.0"
|
||||
description = "unit-test image"
|
||||
arch = "x86_64"
|
||||
os = "linux"
|
||||
created_at = "2026-09-26T00:00:00Z"
|
||||
|
||||
[ image.compatibility ]
|
||||
hypervisors = [ "qemu", "cloud-hypervisor" ]
|
||||
kata_runtime_min = "2.5"
|
||||
|
||||
[ kernel ]
|
||||
version = "6.6.32"
|
||||
included = true
|
||||
path = "kernel/{kernel_fmt}"
|
||||
format = "{kernel_fmt}"
|
||||
config_path = "kernel/config"
|
||||
size_mb = 0
|
||||
|
||||
[ initrd ]
|
||||
included = false
|
||||
type = ""
|
||||
path = ""
|
||||
|
||||
[ rootfs ]
|
||||
type = "tar-gzip"
|
||||
path = "rootfs.tar.gz"
|
||||
size_mb = 1
|
||||
|
||||
[ boot_params ]
|
||||
included = true
|
||||
path = "boot-params.conf"
|
||||
""".encode()
|
||||
menu = b"""[ menu ]
|
||||
label = "Synthetic Test"
|
||||
category = "custom"
|
||||
|
||||
[ menu.actions ]
|
||||
import = true
|
||||
"""
|
||||
rootfs_buf = io.BytesIO()
|
||||
with tarfile.open(fileobj=rootfs_buf, mode="w:gz") as rtf:
|
||||
info = tarfile.TarInfo("opt/ai-lsc/stack.json")
|
||||
payload = b'{"tools": ["ollama"]}'
|
||||
info.size = len(payload)
|
||||
rtf.addfile(info, io.BytesIO(payload))
|
||||
rootfs = rootfs_buf.getvalue()
|
||||
# Tampering swaps the archived bytes AFTER the hashes are computed
|
||||
# from the originals — that's what "integrity violation" means:
|
||||
# hashes.sha256 no longer describes the shipped payload.
|
||||
archived_rootfs = (
|
||||
rootfs[:-1] + bytes([rootfs[-1] ^ 0xFF])
|
||||
if tamper_rootfs else rootfs
|
||||
)
|
||||
|
||||
members = [
|
||||
("rootfs.tar.gz", rootfs),
|
||||
("kernel/" + kernel_fmt, bytes(kernel)),
|
||||
("kernel/config", config),
|
||||
("boot-params.conf", b"console=ttyS0\n"),
|
||||
("menu.toml", menu),
|
||||
]
|
||||
if not omit_metadata:
|
||||
members.append(("metadata.toml", metadata))
|
||||
|
||||
hash_lines = []
|
||||
for name, data in sorted(members):
|
||||
hash_lines.append(
|
||||
f"{hashlib.sha256(data).hexdigest()} ./{name}",
|
||||
)
|
||||
members.append(("hashes.sha256", ("\n".join(hash_lines) + "\n").encode()))
|
||||
# Swap in the tampered payload for the archive write only.
|
||||
members = [
|
||||
(name, archived_rootfs if name == "rootfs.tar.gz" else data)
|
||||
for name, data in members
|
||||
]
|
||||
|
||||
with tarfile.open(dest, "w:gz") as tf:
|
||||
for name, data in sorted(members, key=lambda x: x[0]):
|
||||
info = tarfile.TarInfo("./" + name)
|
||||
info.size = len(data)
|
||||
tf.addfile(info, io.BytesIO(data))
|
||||
return dest
|
||||
|
||||
|
||||
class TestQcrowsFormatBridge(unittest.TestCase):
|
||||
"""Verify the format-aware QCrows consumer in bridge/kata.py."""
|
||||
|
||||
def setUp(self):
|
||||
import kata
|
||||
self.kata = kata
|
||||
self._tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self._tmp.cleanup)
|
||||
self.qcdir = Path(self._tmp.name)
|
||||
self._orig_dir = kata.QCROWS_DIR
|
||||
kata.QCROWS_DIR = self.qcdir
|
||||
self.addCleanup(setattr, kata, "QCROWS_DIR", self._orig_dir)
|
||||
|
||||
def test_list_parses_metadata_and_menu(self):
|
||||
_build_qcrows(self.qcdir / "test.qcrows")
|
||||
result = self.kata.cmd_qcrows_list([])
|
||||
self.assertEqual(len(result), 1)
|
||||
entry = result[0]
|
||||
self.assertTrue(entry["qcrows"])
|
||||
self.assertEqual(entry["format_version"], "0.2.0")
|
||||
self.assertEqual(entry["image"]["name"], "synthetic-test")
|
||||
self.assertEqual(entry["kernel"]["version"], "6.6.32")
|
||||
self.assertEqual(entry["hypervisors"], ["qemu", "cloud-hypervisor"])
|
||||
self.assertEqual(entry["menu"]["label"], "Synthetic Test")
|
||||
self.assertTrue(entry["kernel_binary_present"])
|
||||
self.assertTrue(entry["has_kernel_config"])
|
||||
|
||||
def test_list_degrades_legacy_tarball_gracefully(self):
|
||||
import tarfile
|
||||
with tarfile.open(self.qcdir / "legacy.tgz", "w:gz") as tf:
|
||||
info = tarfile.TarInfo("readme")
|
||||
info.size = 3
|
||||
tf.addfile(info, __import__("io").BytesIO(b"abc"))
|
||||
entry = self.kata.cmd_qcrows_list([])[0]
|
||||
self.assertFalse(entry["qcrows"])
|
||||
self.assertIn("error", entry)
|
||||
|
||||
def test_verify_passes_on_valid_image(self):
|
||||
_build_qcrows(self.qcdir / "good.qcrows")
|
||||
v = self.kata.cmd_qcrows_verify(["good.qcrows"])
|
||||
self.assertTrue(v["ok"], v)
|
||||
self.assertEqual(v["failed"], 0)
|
||||
names = {c["name"] for c in v["checks"]}
|
||||
self.assertIn("kernel binary format valid", names)
|
||||
self.assertIn("SHA-256 checksums", names)
|
||||
|
||||
def test_verify_detects_tampered_payload(self):
|
||||
_build_qcrows(self.qcdir / "bad.qcrows", tamper_rootfs=True)
|
||||
v = self.kata.cmd_qcrows_verify(["bad.qcrows"])
|
||||
self.assertFalse(v["ok"])
|
||||
failed = [c["name"] for c in v["checks"] if not c["passed"]]
|
||||
self.assertIn("SHA-256 checksums", failed)
|
||||
|
||||
def test_verify_fails_when_kernel_missing(self):
|
||||
# v0.2 requires a kernel — a metadata-only archive must fail.
|
||||
import hashlib
|
||||
import io
|
||||
import tarfile
|
||||
metadata = b'[ qcrows ]\nformat_version = "0.2.0"\n'
|
||||
members = [("metadata.toml", metadata), ("menu.toml", b"[ menu ]\n")]
|
||||
hash_lines = [
|
||||
f"{hashlib.sha256(d).hexdigest()} ./{n}"
|
||||
for n, d in members
|
||||
]
|
||||
members.append((
|
||||
"hashes.sha256",
|
||||
("\n".join(hash_lines) + "\n").encode(),
|
||||
))
|
||||
with tarfile.open(self.qcdir / "nokernel.qcrows", "w:gz") as tf:
|
||||
for name, data in members:
|
||||
info = tarfile.TarInfo("./" + name)
|
||||
info.size = len(data)
|
||||
tf.addfile(info, io.BytesIO(data))
|
||||
v = self.kata.cmd_qcrows_verify(["nokernel.qcrows"])
|
||||
self.assertFalse(v["ok"])
|
||||
failed = [c["name"] for c in v["checks"] if not c["passed"]]
|
||||
self.assertIn("kernel binary found", failed)
|
||||
|
||||
def test_verify_warns_on_missing_kata_options_but_passes(self):
|
||||
# Master semantics: option shortfalls are WARNINGS, not failures.
|
||||
_build_qcrows(self.qcdir / "noopts.qcrows", kata_opts=False)
|
||||
v = self.kata.cmd_qcrows_verify(["noopts.qcrows"])
|
||||
self.assertTrue(v["ok"], v)
|
||||
self.assertTrue(any("Kata options" in w for w in v["warnings"]))
|
||||
|
||||
def test_inspect_returns_member_listing(self):
|
||||
_build_qcrows(self.qcdir / "img.qcrows")
|
||||
r = self.kata.cmd_qcrows_inspect(["img.qcrows"])
|
||||
self.assertTrue(r["qcrows"])
|
||||
names = {m["name"] for m in r["members_list"]}
|
||||
self.assertIn("kernel/vmlinuz", names)
|
||||
self.assertIn("rootfs.tar.gz", names)
|
||||
|
||||
def test_verify_rejects_traversal_and_bad_names(self):
|
||||
for bad in ["../evil", "/etc/passwd", "a" * 65, "x;rm -rf"]:
|
||||
self.assertIn("error", self.kata.cmd_qcrows_verify([bad]))
|
||||
self.assertIn("error", self.kata.cmd_qcrows_verify([]))
|
||||
self.assertIn(
|
||||
"error", self.kata.cmd_qcrows_inspect(["../evil"]),
|
||||
)
|
||||
self.assertIn(
|
||||
"error", self.kata.cmd_qcrows_verify(["missing.qcrows"]),
|
||||
)
|
||||
|
||||
def test_toml_parser_is_section_aware(self):
|
||||
text = """[ kernel ]
|
||||
version = "6.6.32"
|
||||
included = true
|
||||
size_mb = 3
|
||||
|
||||
[ rootfs ]
|
||||
type = "tar-gzip"
|
||||
size_mb = 42
|
||||
|
||||
[ image.compatibility ]
|
||||
hypervisors = [ "qemu", "firecracker" ]
|
||||
"""
|
||||
parsed = self.kata._qcrows_parse_toml(text)
|
||||
self.assertEqual(parsed["kernel"]["version"], "6.6.32")
|
||||
self.assertIs(parsed["kernel"]["included"], True)
|
||||
self.assertEqual(parsed["kernel"]["size_mb"], 3)
|
||||
self.assertEqual(parsed["rootfs"]["size_mb"], 42)
|
||||
self.assertEqual(
|
||||
parsed["image"]["compatibility"]["hypervisors"],
|
||||
["qemu", "firecracker"],
|
||||
)
|
||||
|
||||
def test_kernel_magic_classification(self):
|
||||
bz = bytearray(b"\x00" * 0x206)
|
||||
bz[0x202:0x206] = b"HdrS"
|
||||
ok, kind = self.kata._qcrows_kernel_magic_ok(bytes(bz))
|
||||
self.assertTrue(ok)
|
||||
self.assertEqual(kind, "bzimage")
|
||||
ok, kind = self.kata._qcrows_kernel_magic_ok(b"\x7fELF" + b"\x00" * 8)
|
||||
self.assertTrue(ok)
|
||||
self.assertEqual(kind, "elf")
|
||||
ok, kind = self.kata._qcrows_kernel_magic_ok(b"not a kernel")
|
||||
self.assertFalse(ok)
|
||||
|
||||
|
||||
# ── v0.0.39 Monitoring module tests (Prometheus + Grafana) ──────────
|
||||
|
||||
|
||||
|
|
@ -1914,8 +2177,15 @@ class TestFirewallV047ManifestsAndMetainfo(unittest.TestCase):
|
|||
)
|
||||
|
||||
def test_version_sync_all_surfaces_report_020(self):
|
||||
# Every release surface must report v0.4.5 (production hardening).
|
||||
v = "0.4.5"
|
||||
# Every release surface must report the Makefile's VERSION —
|
||||
# the single source of truth. (v0.4.6: the hardcoded literal
|
||||
# was replaced with the Makefile value so every future bump
|
||||
# keeps this test green when the surfaces move together.)
|
||||
import re as _re
|
||||
makefile = (self.root / "Makefile").read_text()
|
||||
m = _re.search(r"^VERSION\s*:=\s*(\S+)", makefile, _re.M)
|
||||
self.assertIsNotNone(m, "Makefile VERSION not found")
|
||||
v = m.group(1)
|
||||
files_to_check = [
|
||||
"Makefile",
|
||||
"bridge/__init__.py",
|
||||
|
|
|
|||
27
worklog.md
27
worklog.md
|
|
@ -1,5 +1,32 @@
|
|||
# SysDeck - Work Log
|
||||
|
||||
---
|
||||
Task ID: 48
|
||||
Agent: Main Orchestrator (v0.4.6 QCrows format-aware Kata bridge)
|
||||
Task: Per user directive: "lets now update SysDeck to be sure it also uses qcrows the same way" — align SysDeck with the real QCrows VM container image format (cockpit-kata master implementation, qcrows-spec.md v0.2.0), the same format AI-LSC now exports. The bridge must be a real format consumer, not a filename lister.
|
||||
|
||||
Work Log:
|
||||
- FORMAT-AWARE qcrows-list (bridge/kata.py): every .qcrows / .qcrows.gz / .tar.gz / .tgz in /usr/share/sysdeck/kata/qcrows/ is opened in memory (tarfile r:* — nothing extracted to disk, nothing executable materialized) and its metadata.toml + menu.toml parsed by a new section-aware mini-TOML parser (_qcrows_parse_toml — [ section ] headers, dotted sections, strings/bools/ints/arrays; being section-aware deliberately fixes the first-match grepping quirk in cockpit-kata's own qcrows-inspect, which reports kernel size_mb on the rootfs row). Each list entry now carries qcrows, format_version, image{name,version,description,arch,created_at}, kernel{version,format,path,...}, rootfs, initrd, hypervisors, menu{label,category}, kernel_binary_present, has_kernel_config, has_boot_params, members. Legacy non-QCrows tarballs degrade gracefully to stat-only entries (qcrows:false + error note). Added .qcrows.gz to the extension filter.
|
||||
- NEW SUBCOMMAND qcrows-inspect <filename>: single-image detail — full metadata record plus the sorted member listing (name/size/mode) and build.toml provenance.
|
||||
- NEW SUBCOMMAND qcrows-verify <filename>: in-memory verification mirroring cockpit-kata's master qcrows-verify checks: (1) required files metadata.toml/menu.toml/hashes.sha256, (2) rootfs.tar.* present, (3) kernel binary present + magic classification (bzImage "HdrS" @0x202 / ELF \x7fELF — the no-subprocess equivalent of the master's `file | grep (ELF|Linux.*boot)`), (4) kernel/config present (hard check) + the five required Kata options CONFIG_VSOCKETS/VIRTIO/VIRTIO_PCI/DEVTMPFS/DEVTMPFS_MOUNT (WARNINGS only — matches master exit-code semantics where option shortfalls never fail), (5) initrd presence (warn), (6) metadata format_version + hypervisors declared, (7) full sha256sum -c style hash walk streaming members in 1 MiB chunks. Returns {ok, passed, failed, warnings, checks[]}.
|
||||
- SECURITY POSTURE (unchanged gates, new surface): filenames validated with _validate_filename; paths resolved with _resolve_path_under_base realpath containment under QCROWS_DIR; tar-bomb guards (10,000 member cap, 1 MiB cap on parsed text members); output sanitized via _sanitize_output; traversal/oversize/garbage names rejected with {error: ...} (tested).
|
||||
- BRIDGE CLIENT (shared/bridge.js): kata surface gains qcrowsInspect(filename) + qcrowsVerify(filename) proxying to the new subcommands; header comment updated to document format-aware listing.
|
||||
- KATA PANEL (plugins/sysdeck-kata/kata.js): QCrows card renamed to "QCrows VM Container Images" with enriched rows — image name+version (filename as secondary line), kernel version · format (+ missing-binary marker), arch, hypervisors, size, modified — plus per-row Verify and Inspect buttons. Verify renders a PASS/FAIL/WARN check list (green/red via the existing output card, textContent only — no innerHTML on bridge data); Inspect dumps the full JSON detail. Empty-state text updated to the qcrows-pack invocation and notes AI-LSC can export the stack directly as .qcrows. Legacy rows show "(not a QCrows archive)" and disable Verify.
|
||||
- VERSION-SYNC TEST HARDENING: test_version_sync_all_surfaces_report_020 hardcoded the 0.4.5 literal and failed on every bump; it now derives the expected version from the Makefile's own VERSION line (single source of truth), so it stays green whenever the surfaces move together and red when they drift.
|
||||
- TESTS (tests/test_bridge_parsers.py): dispatch-set assertion updated for the two new subcommands; new TestQcrowsFormatBridge class (11 tests) with a _build_qcrows() synthetic archive builder that mirrors qcrows-pack's layout (./-prefixed members, sha256sum-format hashes, real bzImage magic): list parses metadata/menu; legacy tarball degrades; verify passes on valid image; detects tampered payload (hashes computed BEFORE the byte-flip — that is what an integrity violation means); fails when kernel missing; warns-but-passes on missing Kata options; inspect returns member listing; traversal/bad-name/missing-file rejection; TOML parser section-awareness; kernel magic classification (bzimage/elf/unknown — boundary fixed to len>=0x206).
|
||||
- BUG FIX (adjacent, found by the suite in this environment): bridge/integrity.py score() caught only FileNotFoundError; on hosts where lynis ran as root, /var/log/lynis.log is root-only (-rw-r-----) and unprivileged sessions crashed with PermissionError. Now catches OSError and degrades to None (the honest "no score for us" state).
|
||||
- CROSS-PRODUCER VALIDATION: bridge exercised against real images from BOTH producers — an AI-LSC stack export (.qcrows) and a bundle from cockpit-kata's own qcrows-pack (.qcrows.gz). Both parse (image name/kernel/hypervisors/menu correct) and both verify ok with identical warning sets; a tampered copy fails exactly the SHA-256 check.
|
||||
- VERSION SYNC: bumped 0.4.5 → 0.4.6 across all 9 release surfaces: Makefile VERSION, bridge/__init__.py __version__, packaging/setup.py VERSION, packaging/PKGBUILD pkgver, packaging/sysdeck.spec Version + prepended %changelog entry, packaging/debian/changelog prepended entry, compat/compat-manifest.json version, README.md badge + Version line, BLOG.md prepended v0.4.6 section.
|
||||
|
||||
GUARDS:
|
||||
- make check: all build-time guards pass — manifest consistency, metainfo consistency, Makefile tab indentation, cockpit import patterns, bridge invocation patterns, bridge.js↔Python subcommand cross-check (the two new qcrows calls verified against kata.py COMMANDS), version sync (all surfaces 0.4.6).
|
||||
- All 277 unit tests pass (266 prior + 11 new QCrows tests).
|
||||
- bridge/kata.py + bridge/integrity.py pass py_compile; kata.js + bridge.js pass node --check (ESM).
|
||||
- CLI smoke: `python3 bridge/kata.py qcrows-list` returns [] on hosts without /usr/share/sysdeck/kata/qcrows (real empty state, exit 0).
|
||||
|
||||
Stage Summary:
|
||||
- v0.4.6 makes SysDeck a first-class QCrows consumer with the same format understanding as the producer side: format-aware listing (metadata.toml + menu.toml parsed in memory), single-image inspection, and master-mirroring verification (kernel magic, config, full hash walk) — all under the bridge's existing security gates, nothing extracted to disk. The Kata panel surfaces the metadata with Verify/Inspect actions. Verified against real images from both producers (ai-lsc exporter + cockpit-kata qcrows-pack) plus synthetic tamper/kernel-missing/security cases in the suite. Adjacent robustness fix: integrity.score() degrades to None on PermissionError. All 9 release surfaces report 0.4.6; all make-check guards pass; 277/277 tests pass.
|
||||
|
||||
---
|
||||
Task ID: 47
|
||||
Agent: Main Orchestrator (v0.0.47 logic-flaw fixes)
|
||||
|
|
|
|||
Loading…
Reference in New Issue