AI-LSC: AI - Local Stack Control

This commit is contained in:
Jeremy Anderson 2026-09-27 02:07:26 -04:00
parent 34d429345d
commit d244d69ca6
25 changed files with 2217 additions and 157 deletions

1
.env Normal file
View File

@ -0,0 +1 @@
AI_LSC_BASE_DIR=/mnt/AI

View File

@ -1,5 +1,204 @@
# AI-LSC Changelog # AI-LSC Changelog
## v3.4 — File-system map alignment + QCOW2 exports + real updates
### 1. The base dir now follows the canonical file-system map
`constants.py` / `utils/paths.py` are the single source of truth for
the /mnt/AI layout, and the runtime code now actually honours it:
- **Install-root router** (`utils/paths.py::install_dir_for`): a
tool's declared `filesystem.install` is honoured verbatim; when
absent, web UIs (`flags.has_web`, launcher not pinned to
`{tools_root}`) route to `dashboards/<tool_id>/`, everything else
stays under `tools/<tool_id>/`. `runtime/installer.py` resolves
every per-tool destination through the router.
- **Multi-root detection**: `candidate_install_dirs()` probed by
installer preflight, verification, version detection, and the
main-window drift audit — so tools installed before the routing
change are still detected, verified, and **updated in place**
instead of being orphaned or duplicated.
- **Registry state moved** from `<base>/registry/` (off-map) to
`<base>/configs/registry/` (app state belongs under configs/ per
the map). `main_window._migrate_registry_dir()` migrates existing
installs once on startup; legacy dir removed when emptied.
- **`exports/qcow2/`** added to REQUIRED_DIRS (25 dirs now) alongside
`exports/oci-images/`.
- Ollama server probe order is now map-correct:
`runtime/ollama` → `tools/ollama` → legacy `ollama` / `bin/ollama`.
- Every launcher-template placeholder from the path tree resolves
everywhere (`{dashboards_root}`, `{runtime_root}`, `{configs_root}`,
`{datasets_root}`, …) — in `resolve_launcher_cmd`, the runtime
executor's `format_context`, env overrides, post_install commands,
and the stack-export placeholder chain.
### 2. QCOW2 / QCOW VM-disk exports (the missing export backend)
`stack/export.py::ContainerBackend.write_vm_image()` — a fully rootless
pipeline (no loop mounts, no root):
1. Stage a rootfs tree (`/opt/ai-lsc/stack.json` + compose file,
`/root/start.sh` boot script, `/etc/ai-lsc-release`) under
`exports/qcow2/.staging-*`.
2. Populate an ext4 filesystem straight from the staging dir with
`mkfs.ext4 -d` into a sparse raw image (auto-sized: staging usage
× 4 + headroom, min 2 GiB).
3. `qemu-img convert -c -O qcow2` (also `raw`; legacy `qcow` on
older qemu hosts — modern qemu ≥ ~10 refuses v1 writes and the
exporter surfaces a clean, actionable error suggesting qcow2).
4. Sidecar `<image>.manifest.json` with sha256, virtual size,
`qemu-img info`, tool list, and a boot hint (rootfs disk — pair
with an external kernel such as the Firecracker export's vmlinux,
or attach to any VM).
Tool detection is built in: missing `qemu-img` / `mkfs.ext4` raise
`FileNotFoundError` with the pacman/apt install hint. UI: two new
buttons on the Container Stacks tab ("Export -> QCOW2 Image",
"Export -> QCOW (legacy)"); the file list shows images with sizes.
Formats are registry-driven (`VM_IMAGE_FORMATS`, one output dir per
format under `exports/`).
### 2b. QCrows (.qcrows) VM container image export — the real format
QCrows (cue-crows) is the in-house **self-describing VM container
image format for Kata Containers**, master-implemented in the
**cockpit-kata** project (`qcrows-spec.md` v0.2.0 + the `qcrows-pack`
/ `qcrows-verify` / `qcrows-inspect` / `qcrows-export` /
`qcrows-initrd-regen` tools). (An earlier iteration of this changelog
misidentified qcrows as sorcery-go's Sovereign Bundle — that format is
now supported under its own `svb` key, below.)
`ContainerBackend.write_qcrows()` produces a spec-conformant archive:
```
<tar> (PAX per spec §2.1; .qcrows or .qcrows.gz)
├── metadata.toml generated manifest (qcrows-pack's template
│ incl. section order qcrows-inspect parses)
├── menu.toml Cockpit UI menu entry
├── hashes.sha256 sha256sum -c byte-compatible, ./-prefixed
├── rootfs.tar.gz the staged ai-lsc stack rootfs
├── boot-params.conf console=ttyS0 root=/dev/vda rw
├── build.toml ai-lsc provenance (generator, kernel origin)
├── spec.md per-image build guide
└── kernel/ REQUIRED (v0.2+): vmlinuz|vmlinux + config
```
- **Kernel sourcing** (kernel is REQUIRED by v0.2 — same refusal
behaviour as qcrows-pack): `<base>/runtime/qcrows/` (canonical guest
kernel home: `vmlinuz|vmlinux` + `config`) → the firecracker dir's
`vmlinux` → host kernel fallback (`/boot/vmlinuz-<release>`, any
`/boot/vmlinuz-*` for Arch-style names; config from
`/boot/config-*`, `/usr/lib/modules/<ver>/config`, or
`/proc/config.gz`). Host fallback logs a warning — a Kata-tuned
guest kernel is recommended for real deployments.
- **Pre-flight Kata checks**: kernel-version probe mirrors
qcrows-pack (bzImage strings → `CONFIG_VERSION_SIGNATURE` →
"unknown"); the five required Kata config options
(`CONFIG_VSOCKETS`, `CONFIG_VIRTIO`, `CONFIG_VIRTIO_PCI`,
`CONFIG_DEVTMPFS`, `CONFIG_DEVTMPFS_MOUNT`) are checked and
shortfalls logged + recorded in the sidecar (qcrows-verify warns,
never fails, on these).
- **Master round-trip verified**: images built by ai-lsc pass
cockpit-kata's own `qcrows-verify` (11 PASS / 0 FAIL / exit 0, both
compressed and uncompressed) and render fully in `qcrows-inspect`.
Tar members carry the `./` prefix exactly like qcrows-pack's
`tar czf - ./*` output; hashes match `sha256sum -c` byte-for-byte.
- Output: `exports/qcrows/ai-lsc-stack-<stamp>.qcrows` (+ optional
`.gz`) with an ai-lsc sidecar manifest (sha256, kernel origin,
config warnings, verify hint). UI button
"Export -> QCrows Image (.qcrows)".
**Tool quirks found in cockpit-kata** (their code, reported not
modified — the ai-lsc writer avoids all of them):
1. `qcrows-pack` renames `initrd.cpio.gz` to `initrd.gz` — its
extension grep (`\.[^.]*$`) only captures the last suffix, and the
packer then derives both the on-disk name and `metadata.toml`'s
`initrd.path` from it; `qcrows-verify`'s initrd probe
(`initrd.img|initrd.cpio.gz|…`) then can't find it.
2. `qcrows-pack` writes GNU-format tar although spec §2.1 mandates
PAX (ai-lsc writes PAX).
3. `metadata.toml`'s generated `rootfs.path` assumes the source
tarball is named `rootfs.*` — other names produce a path that
doesn't match the staged file.
4. `qcrows-inspect`'s first-match key parsing reports kernel's
`included`/`size_mb` for the initrd/rootfs rows (visible in its
own output on its own packs).
### 2c. Sovereign Bundle (.svb) export (sorcery-go format)
Kept as its own format key (`svb`) after the qcrows identification
was corrected: `ContainerBackend.write_svb()` implements
`sorcery-sovereign-bundle-v1` (METADATA.json first entry, optional
64-byte ed25519 SIGNATURE.sig over the payload sha256, per-file
MANIFEST.txt, then payload) with deterministic `payload_sha`
(identical content → identical hash) and optional signing via
`AI_LSC_SVB_SIGNING_KEY` + the `cryptography` package. Output:
`exports/svb/*.svb`. Cross-verified with a Go verifier mirroring
sorcery-go's parse path — including the discovery that sorcery-go's
own `VerifySVBSignature` diverges from its builder (it hashes
concatenated entry contents instead of the inner tar.gz), so their
verifier can't validate their own bundles; the ai-lsc writer follows
the builder.
### 3. Updates actually update now
The registry's `installer.update_cmd` existed in the schema since
v3.0 but nothing executed it. New
`InstallerManager.update_tool()`:
- Registry `update_cmd` runs first, rendered with all path-tree
placeholders plus `{install_dir}`, cwd = the *detected* install
location (so pre-routing installs update where they live).
- Per-type defaults otherwise: `uv tool upgrade` (reinstall
step-down preserves `--with` extras), `pipx upgrade`, per-tool
venv `pip install -U`, `git pull --ff-only` (+ yarn rebuild for
git_node), `npm update`.
- `install_with_preflight()` now flows found-tools into
`update_tool()` by default — the UI's single Deploy/Update button
is a real in-place updater instead of returning "already
installed; use force=True".
- New `RuntimeExecutor.update_tool()` facade; service rows pass the
full registry entry through (`registry_entry=`) so routing,
update_cmd, and post_install all flow from one source.
### 4. Open WebUI install/start fixes
- **Step-down data loss fixed**: `install_uv` used to fall back to
`pipx install open-webui` — silently dropping `--with
audioop-lts`, producing an install that later crashed importing
`audioop` on Python 3.13. The chain is now: uv (default Python)
→ uv with a managed CPython 3.12 (`uv` fetches it automatically —
sidesteps wheels not yet rebuilt for newer Pythons) → per-tool pip
venv installing the package **plus every `--with` extra**.
- Launcher pre-creates `workspaces/openwebui` (DATA_DIR) alongside
the existing `configs/auth` bootstrap so first start can't fail on
a missing data dir.
### 5. LibreChat install/start fixes
- **Fresh-clone start bug**: `yarn backend` aborts without `.env`,
and production mode needs `yarn build` assets that plain
`yarn install` never produces. The installer entry now carries
`post_install: "cp -n .env.example .env 2>/dev/null || true;
yarn build"` — runs after clone/update in the project dir.
- **Map-aligned paths**: installs to `dashboards/librechat/`
(was `tools/librechat`), data → `workspaces/librechat` (the
off-map root-level `data/` dir is gone), config stays
`configs/librechat`, launcher uses `{dashboards_root}`. Existing
`tools/librechat` clones are detected and updated in place.
- `agents/librechat_config.py` docstring path corrected to the
dashboards location.
### 6. Housekeeping
- Version 3.3.0 → 3.4.0 (`pyproject.toml`, `constants.APP_VERSION`).
- Container Stacks tab lists qcow2 images (+ manifests, sizes) and
excludes the qcow2 dir from the generic directory listing.
- Verification tab passes the full registry entry so multi-root
detection applies to compliance checks too.
## v3.3 — Registry git-URL sync + bandwidth-aware git-pull + ComfyUI ## v3.3 — Registry git-URL sync + bandwidth-aware git-pull + ComfyUI
Patch release over v3.2. No schema changes, no UI reorganisation, no Patch release over v3.2. No schema changes, no UI reorganisation, no
@ -123,7 +322,7 @@ bandwidth-saving behaviour the user asked for.
- `pyproject.toml`: 3.2.0 → 3.3.0 - `pyproject.toml`: 3.2.0 → 3.3.0
- `src/ai_lsc/constants.py`: `APP_VERSION` 3.2.0 → 3.3.0 - `src/ai_lsc/constants.py`: `APP_VERSION` 3.2.0 → 3.3.0
(`APP_CODENAME` unchanged: `"Decalogue"`) (`APP_CODENAME` unchanged: `"Ankh of Jah"`)
- New file: `docs/REGISTRY-URLS-v3.3.0.md` (per-file old → new URL - New file: `docs/REGISTRY-URLS-v3.3.0.md` (per-file old → new URL
table, also serves as a release audit trail). table, also serves as a release audit trail).
- `gitcommit` rewritten for v3.3.0. - `gitcommit` rewritten for v3.3.0.

View File

@ -187,3 +187,66 @@ WEBUI_DATA_DIR={workspaces_root}/hermes-webui OLLAMA_BASE_URL=http://localhost:1
The ai-lsc runtime resolves env-prefix shells via `shlex.split()` on the The ai-lsc runtime resolves env-prefix shells via `shlex.split()` on the
launcher cmd, so the env-assignment form is parsed correctly. launcher cmd, so the env-assignment form is parsed correctly.
---
## 11. Open WebUI Python-3.13 audioop fix (follow-up #2)
After the launcher fix landed, open-webui started successfully but crashed
at import time inside the uv-managed tool env:
```
ModuleNotFoundError: No module named 'audioop'
…
ModuleNotFoundError: No module named 'pyaudioop'
```
Root cause: Python 3.13 removed the `audioop` stdlib module (PEP 594).
`pydub` (an open-webui dep) imports `audioop` and falls back to
`pyaudioop` on ImportError; neither was present in the uv-managed
open-webui env at `~/.local/share/uv/tools/open-webui/`.
### Fix
`uv tool install` supports `--with <extra>` for additional dependencies.
The `pkg` field in the registry now carries that flag, and `install_uv`
was hardened to shlex-split `pkg` so the flag passes through verbatim:
- `src/ai_lsc/runtime/installer.py` `install_uv` — shlex-splits `pkg`;
validates only the first token as the package name; passes remaining
tokens as additional argv to `uv tool install`. Falls back to pipx
using just the first token.
- `src/ai_lsc/runtime/installer.py` `_detect_installation` /
`_binary_name` / `_detect_version` — all now take only the first
shlex token of `pkg` when looking up the binary on PATH or building
version-detection commands. (Without this, preflight would look for
a binary literally named `open-webui --with audioop-lts`.)
- `src/ai_lsc/registry/layers/user_interfaces.py` — both `openwebui`
and `hermes_webui` installer.pkg changed from `"open-webui"` to
`"open-webui --with audioop-lts"`. Inline comment explains the
Python-3.13 / PEP-594 rationale.
- `src/ai_lsc/registry/defaults.py` — openwebui entry updated to match.
### Verification
- 95 Python files AST-parse cleanly.
- Registry loads 187 tools; `validate_registry()` reports 0 errors.
- `shlex.split("open-webui --with audioop-lts")` yields the expected
argv: `["open-webui", "--with", "audioop-lts"]`.
- `_binary_name("open-webui --with audioop-lts", "uv")` resolves to
`"open_webui"` (the actual on-PATH binary name).
### Operator workaround for existing installs
If open-webui was installed before this fix landed, the audioop-lts
backport can be added in place:
```bash
~/.local/share/uv/tools/open-webui/bin/python -m pip install audioop-lts
```
Or reinstall via the new pkg spec:
```bash
uv tool install --force "open-webui --with audioop-lts"
```

View File

@ -5,12 +5,12 @@
<h1 align="center">AI - Local Stack Control</h1> <h1 align="center">AI - Local Stack Control</h1>
<p align="center"> <p align="center">
<strong>v3.1.1 — Codename: Ankh of Jah (local-coder-mesh build)</strong><br> <strong>v3.4.0 — Codename: Ankh of Jah</strong><br>
Author: <a href="https://dcos.net">Jeremy Anderson</a> &lt;<a href="mailto:info@dcos.net">info@dcos.net</a>&gt;<br> Author: <a href="https://dcos.net">Jeremy Anderson</a> &lt;<a href="mailto:info@dcos.net">info@dcos.net</a>&gt;<br>
Web: <a href="https://dcos.net">https://dcos.net</a> Web: <a href="https://dcos.net">https://dcos.net</a>
</p> </p>
<p align="center"><em>This build includes the local-coder-mesh integration — see <a href="CHANGES.md">CHANGES.md</a> for the full list of changes vs upstream v3.1.</em></p> <p align="center"><em>Includes the local-coder-mesh integration, 10-layer taxonomy migration, MoE QA pass, and openhuman/openwebui hardening — see <a href="CHANGES.md">CHANGES.md</a> for the full version history.</em></p>
<p align="center"> <p align="center">
A PySide6 desktop application for orchestrating local AI/ML tool stacks across a 10-layer architecture. A PySide6 desktop application for orchestrating local AI/ML tool stacks across a 10-layer architecture.

View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""AI Local Stack Control v3.1 — Ankh of Jah """AI Local Stack Control v3.3.0 — Ankh of Jah
Direct launcher. Run from the project root: Direct launcher. Run from the project root:

View File

@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# ────────────────────────────────────────────────────────────── # ──────────────────────────────────────────────────────────────
# AI Local Stack Control v3.1.1 — Ankh of Jah # AI Local Stack Control v3.3.0 — Ankh of Jah
# Bootstrap Script # Bootstrap Script
# #
# Fully portable: works wherever the tarball lands. # Fully portable: works wherever the tarball lands.
@ -46,7 +46,7 @@ export AI_LSC_BASE_DIR="$AI_BASE"
echo "" echo ""
echo -e "${BOLD}╔══════════════════════════════════════════════════════╗${NC}" echo -e "${BOLD}╔══════════════════════════════════════════════════════╗${NC}"
echo -e "${BOLD}║ AI Local Stack Control v3.1.1 — Ankh of Jah ║${NC}" echo -e "${BOLD}║ AI Local Stack Control v3.3.0 — Ankh of Jah ║${NC}"
echo -e "${BOLD}╚══════════════════════════════════════════════════════╝${NC}" echo -e "${BOLD}╚══════════════════════════════════════════════════════╝${NC}"
echo "" echo ""
echo -e "${CYAN} Project root : ${SCRIPT_DIR}${NC}" echo -e "${CYAN} Project root : ${SCRIPT_DIR}${NC}"
@ -316,12 +316,16 @@ else
ERRORS=$((ERRORS + 1)) ERRORS=$((ERRORS + 1))
fi fi
for cmd in ollama podman docker tmux ripgrep fd tree-sitter; do for cmd in ollama podman docker tmux ripgrep fd tree-sitter qemu-img; do
if command -v "$cmd" &>/dev/null; then if command -v "$cmd" &>/dev/null; then
info "${cmd} — found" info "${cmd} — found"
else
if [ "$cmd" = "qemu-img" ]; then
warn "qemu-img — not found (needed for QCOW2 VM-image exports: pacman -S qemu-img / apt install qemu-utils)"
else else
warn "${cmd} — not found (optional)" warn "${cmd} — not found (optional)"
fi fi
fi
done done
# ── Summary ─────────────────────────────────────────────────── # ── Summary ───────────────────────────────────────────────────

View File

@ -92,7 +92,7 @@ Intentionally NOT touched:
- 3 registry-only git tools with real-looking URLs already in - 3 registry-only git tools with real-looking URLs already in
place (goose -> block/goose, nvidia_agent_skills -> place (goose -> block/goose, nvidia_agent_skills ->
NVIDIA/agent-skills, picode -> jasonjmcghee/picode.git). NVIDIA/agent-skills, picode -> jasonjmcghee/picode.git).
- APP_CODENAME ("Decalogue") and all historical codename - APP_CODENAME ("Ankh of Jah") and all historical codename
references in docstrings / UI labels / README banner / bootstrap references in docstrings / UI labels / README banner / bootstrap
banner — left as-is per user's clarification. banner — left as-is per user's clarification.
@ -102,7 +102,7 @@ removed from the registry.
Versioning: Versioning:
- pyproject.toml: 3.2.0 -> 3.3.0 - pyproject.toml: 3.2.0 -> 3.3.0
- src/ai_lsc/constants.py: APP_VERSION 3.2.0 -> 3.3.0 - src/ai_lsc/constants.py: APP_VERSION 3.2.0 -> 3.3.0
(APP_CODENAME unchanged: "Decalogue") (APP_CODENAME unchanged: "Ankh of Jah")
- docs/REGISTRY-URLS-v3.3.0.md: per-file old -> new URL table. - docs/REGISTRY-URLS-v3.3.0.md: per-file old -> new URL table.
Verification: Verification:

View File

@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "ai-lsc" name = "ai-lsc"
version = "3.3.0" version = "3.4.0"
description = "AI Local Stack Control — PySide6 desktop app for orchestrating local AI/ML tool stacks" description = "AI Local Stack Control — PySide6 desktop app for orchestrating local AI/ML tool stacks"
readme = "README.md" readme = "README.md"
license = {text = "AGPL-3.0-or-later"} license = {text = "AGPL-3.0-or-later"}

View File

@ -73,7 +73,7 @@ print(f'Registry loaded: {len(DEFAULT_REGISTRY)} tools')
print(f'Validation errors: {len(errors)}') print(f'Validation errors: {len(errors)}')
" "
# Expected output (v3.1): # Expected output (v3.4.0):
# Registry loaded: 125 tools # Registry loaded: 125 tools
# Validation errors: 0 # Validation errors: 0
``` ```
@ -253,6 +253,6 @@ for f in os.listdir('ai_lsc/registry/layers'):
- Try different **Stack Templates** to find the right combination for your workflow - Try different **Stack Templates** to find the right combination for your workflow
- Set up the **Skills Console** to extend your tool capabilities - Set up the **Skills Console** to extend your tool capabilities
- Use **Code Analysis** to inspect and understand your project dependencies - Use **Code Analysis** to inspect and understand your project dependencies
- Read [CHANGES.md](CHANGES.md) for the full v3.1 changelog - Read [CHANGES.md](CHANGES.md) for the full version history (v3.0 → v3.4.0)
- Read [whatremains.txt](whatremains.txt) for known deferred items (curl|sh installers, etc.) - Read [whatremains.txt](whatremains.txt) for known deferred items (curl|sh installers, etc.)
- Read [docs/ADR-002-pipeline-ticker.md](docs/ADR-002-pipeline-ticker.md) and [docs/ADR-003-workspace-tab.md](docs/ADR-003-workspace-tab.md) for the design rationale behind the two new widgets - Read [docs/ADR-002-pipeline-ticker.md](docs/ADR-002-pipeline-ticker.md) and [docs/ADR-003-workspace-tab.md](docs/ADR-003-workspace-tab.md) for the design rationale behind the two new widgets

View File

@ -1,5 +1,5 @@
""" """
AI Local Stack Control v3.1 — Release codename: Ankh of Jah. AI Local Stack Control v3.3.0 — Release codename: Ankh of Jah.
Extracted from the monolithic ``ai_lsc_v11.py`` in incremental phases. Extracted from the monolithic ``ai_lsc_v11.py`` in incremental phases.
Currently contains: Currently contains:

View File

@ -15,7 +15,7 @@ Usage
config.set_ollama_endpoint(ollama_port=11434) config.set_ollama_endpoint(ollama_port=11434)
config.set_litellm_endpoint(litellm_port=4000) config.set_litellm_endpoint(litellm_port=4000)
config.set_tool_schemas(tool_schemas) config.set_tool_schemas(tool_schemas)
config.save("/mnt/AI/runtime/librechat/config.yaml") config.save("/mnt/AI/dashboards/librechat")
""" """
from __future__ import annotations from __future__ import annotations

View File

@ -1,9 +1,9 @@
""" """
AI-LSC v3.1 — Application-wide constants. AI-LSC v3.3.0 — Application-wide constants.
Release codename: Ankh of Jah Release codename: Ankh of Jah
Pure data: file names, schema version, required directories, default ports, Pure data: file names, schema version, required directories, default ports,
status styles, log colours, service licences, tree-skip patterns, status styles, log colours, service licences, tree-skip patterns,
and the reorganized 10-layer navigation layer order. and the 10-layer navigation layer order.
""" """
import os import os
@ -15,8 +15,8 @@ BASE_DIR: str = os.environ.get("AI_LSC_BASE_DIR", "/mnt/AI")
CANONICAL_BASE_DIR: str = BASE_DIR CANONICAL_BASE_DIR: str = BASE_DIR
# ── Filenames ──────────────────────────────────────────────────────────── # ── Filenames ────────────────────────────────────────────────────────────
APP_VERSION: str = "3.3.0" APP_VERSION: str = "3.4.0"
APP_CODENAME: str = "Decalogue" APP_CODENAME: str = "Ankh of Jah"
APP_DISPLAY_NAME: str = f"AI - Local Stack Control v{APP_VERSION} - http://dcos.net" APP_DISPLAY_NAME: str = f"AI - Local Stack Control v{APP_VERSION} - http://dcos.net"
CONFIG_FILE: str = "controller_config.json" CONFIG_FILE: str = "controller_config.json"
APP_ICON_FILE: str = "ai-lsc-logo.png" APP_ICON_FILE: str = "ai-lsc-logo.png"
@ -51,6 +51,8 @@ REQUIRED_DIRS: list[str] = [
"dashboards", # web UIs and landing pages (Dashy, Open-WebUI, …) "dashboards", # web UIs and landing pages (Dashy, Open-WebUI, …)
"tools", # standalone compiles (built from distfiles) "tools", # standalone compiles (built from distfiles)
"exports/oci-images", # finalized Podman .tar snapshots (-> MinIO registry) "exports/oci-images", # finalized Podman .tar snapshots (-> MinIO registry)
"exports/qcow2", # VM disk-image exports (qemu qcow2 / legacy qcow)
"exports/qcrows", # QCrows VM container images (.qcrows, spec v0.2)
"scripts", # system admin / maintenance automation for the stack "scripts", # system admin / maintenance automation for the stack
"logs", # system, runtime, and pipeline service logs "logs", # system, runtime, and pipeline service logs
] ]
@ -158,11 +160,14 @@ NAV_LAYER_ORDER: list[str] = [
] ]
# ── Ollama server candidate paths (probed in order) ──────────────── # ── Ollama server candidate paths (probed in order) ────────────────
# Map-aligned order: runtime/ (native binaries) is probed first;
# the legacy root-level and bin/ locations remain as fallbacks for
# pre-v3.4 installs.
OLLAMA_SERVER_CANDIDATES: list[str] = [ OLLAMA_SERVER_CANDIDATES: list[str] = [
"ollama", # /mnt/AI/ollama "runtime/ollama", # /mnt/AI/runtime/ollama (canonical)
"tools/ollama", # /mnt/AI/tools/ollama "tools/ollama", # /mnt/AI/tools/ollama
"runtime/ollama", # /mnt/AI/runtime/ollama "ollama", # /mnt/AI/ollama (legacy)
"bin/ollama" # /mnt/AI/bin/ollama "bin/ollama" # /mnt/AI/bin/ollama (legacy)
] ]
# ── Model tier routing (reserved for v4.0 agentic layer) ────────── # ── Model tier routing (reserved for v4.0 agentic layer) ──────────

View File

@ -796,7 +796,10 @@ DEFAULT_REGISTRY: dict = {
}, },
"launcher": { "launcher": {
"type": 'tmux', "type": 'tmux',
"cmd": 'OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve', # Force ollama to use /mnt/AI/models/ollama (not ~/.ollama/models)
# and put its identity-key + runtime state under /mnt/AI/.ollama.
# See inference.py ollama entry for full rationale.
"cmd": 'mkdir -p {base_dir}/.ollama && HOME={base_dir} OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve',
"default_port": 11434, "default_port": 11434,
}, },
"deps": [ "deps": [
@ -2905,19 +2908,25 @@ DEFAULT_REGISTRY: dict = {
"type": 'git_node', "type": 'git_node',
"pkg": 'https://github.com/danny-avila/LibreChat.git', "pkg": 'https://github.com/danny-avila/LibreChat.git',
"update_cmd": 'git pull --ff-only && yarn install && yarn build', "update_cmd": 'git pull --ff-only && yarn install && yarn build',
# Two known fresh-clone start bugs, fixed at install time:
# 1. `yarn backend` aborts without an .env file — seed one
# from the shipped example (never clobber an existing one).
# 2. production mode requires the client assets from
# `yarn build` — plain `yarn install` doesn't produce them.
"post_install": 'cp -n .env.example .env 2>/dev/null || true; yarn build',
}, },
"launcher": { "launcher": {
"type": 'tmux', "type": 'tmux',
"cmd": 'cd {tools_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend', "cmd": 'cd {dashboards_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend',
"default_port": 3080, "default_port": 3080,
}, },
"deps": [ "deps": [
'ollama', 'ollama',
], ],
"filesystem": { "filesystem": {
"install": 'tools/librechat', "install": 'dashboards/librechat',
"config": 'configs/librechat', "config": 'configs/librechat',
"data": 'data/librechat', "data": 'workspaces/librechat',
"logs": 'logs/librechat', "logs": 'logs/librechat',
}, },
"description": 'Default agent interface with native tool execution, system files parsing, and multi-user configurations.', "description": 'Default agent interface with native tool execution, system files parsing, and multi-user configurations.',
@ -3125,21 +3134,30 @@ DEFAULT_REGISTRY: dict = {
"category": 'Extensible Interface', "category": 'Extensible Interface',
"installer": { "installer": {
"type": 'uv', "type": 'uv',
"pkg": 'open-webui', # open-webui's pydub dep imports `audioop`, which Python 3.13
# removed from the stdlib (PEP 594). The `audioop-lts` backport
# restores it; --with pulls it into the tool's isolated env.
"pkg": 'open-webui --with audioop-lts',
"env_overrides": { "env_overrides": {
'OPEN_WEBUI_CONFIG_DIR': '{base_dir}/configs/openwebui', 'OPEN_WEBUI_CONFIG_DIR': '{base_dir}/configs/openwebui',
}, },
}, },
"launcher": { "launcher": {
"type": 'tmux', "type": 'tmux',
"cmd": 'WEBUI_DATA_DIR={workspaces_root}/openwebui open-webui serve --port {port}', # Minimal launcher — see user_interfaces.py openwebui entry for
# the full rationale. Summary: open-webui auto-discovers ollama
# at http://localhost:11434 (do NOT set OLLAMA_BASE_URL here).
# DATA_DIR replaces the removed --data-dir CLI flag. cd to
# configs/auth/ so the auto-generated .webui_secret_key lands
# in the canonical private-data dir, not the ai-lsc run dir.
"cmd": 'mkdir -p {base_dir}/configs/auth {workspaces_root}/openwebui && chmod 700 {base_dir}/configs/auth && cd {base_dir}/configs/auth && DATA_DIR={workspaces_root}/openwebui open-webui serve --port {port}',
"default_port": 8080, "default_port": 8080,
}, },
"deps": [ "deps": [
'ollama', 'ollama',
], ],
"filesystem": { "filesystem": {
"install": 'tools/openwebui', "install": 'dashboards/openwebui',
"config": 'configs/openwebui', "config": 'configs/openwebui',
"data": 'workspaces/openwebui', "data": 'workspaces/openwebui',
"logs": 'logs/openwebui', "logs": 'logs/openwebui',

View File

@ -35,7 +35,24 @@ TOOLS: dict[str, dict] = {
}, },
"launcher": { "launcher": {
"type": "tmux", "type": "tmux",
"cmd": "OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve", # Force ollama to use /mnt/AI/models/ollama (not ~/.ollama/models)
# and put its identity-key + runtime state under /mnt/AI/.ollama
# (not ~/.ollama). We set HOME={base_dir} *only for ollama* —
# this is safe because ollama doesn't depend on uv-tool/pipx
# (which are $HOME-relative). For uv-tool-installed tools like
# open-webui, we CANNOT override HOME globally or the tool's
# own venv lookup breaks; each tool that needs a HOME redirect
# must opt in via its own launcher cmd.
#
# OLLAMA_HOST=0.0.0.0:{port} — bind to all interfaces so
# open-webui and other tools on the box can reach it.
# OLLAMA_MODELS — pins the models dir to the canonical ai-lsc
# location; this is what makes `ollama list` and open-webui's
# model picker see the same set.
# mkdir -p {base_dir}/.ollama — pre-creates the dotdir ollama
# expects so it doesn't trip on a broken ~/.ollama state on
# the host (e.g. a stale file where a dir should be).
"cmd": "mkdir -p {base_dir}/.ollama && HOME={base_dir} OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve",
"default_port": 11434 "default_port": 11434
}, },
"deps": [], "deps": [],

View File

@ -20,16 +20,46 @@ TOOLS: dict[str, dict] = {
"category": 'Extensible Interface', "category": 'Extensible Interface',
"installer": { "installer": {
"type": "uv", "type": "uv",
"pkg": "open-webui" # open-webui's pydub dep imports `audioop`, which Python 3.13
# removed from the stdlib (PEP 594). The `audioop-lts` backport
# restores it; --with pulls it into the tool's isolated env.
"pkg": "open-webui --with audioop-lts"
}, },
"launcher": { "launcher": {
# Minimal launcher — open-webui auto-discovers ollama at
# http://localhost:11434 (its built-in default). Do NOT add
# OLLAMA_BASE_URL / CORS_ORIGINS / ENABLE_DIRECT_CONNECTIONS /
# WEBUI_SECRET_KEY env vars — past iterations proved they break
# model discovery. open-webui handles all of these itself when
# left alone.
#
# Two things we DO control:
# 1. DATA_DIR — points the sqlite db + uploads at the
# canonical workspaces dir. (The old --data-dir CLI flag
# was removed upstream; the env var is the replacement.)
# 2. cd to {base_dir}/configs/auth/ before launching —
# open-webui writes its auto-generated .webui_secret_key
# to Path.cwd() (see backend/open_webui/__init__.py
# KEY_FILE). Running from configs/auth/ keeps the secret
# out of the ai-lsc run dir and in the canonical
# private-data location, with no env-var games.
"type": "tmux", "type": "tmux",
"cmd": "WEBUI_DATA_DIR={workspaces_root}/openwebui open-webui serve --port {port}", "cmd": "mkdir -p {base_dir}/configs/auth {workspaces_root}/openwebui && chmod 700 {base_dir}/configs/auth && cd {base_dir}/configs/auth && DATA_DIR={workspaces_root}/openwebui open-webui serve --port {port}",
"default_port": 8080 "default_port": 8080
}, },
"deps": [ "deps": [
"ollama" "ollama"
], ],
"filesystem": {
# Map-aligned per the v3.4 file-system map: the web UI app
# marker lives under dashboards/, config under configs/, and
# the sqlite/uploads DATA_DIR under workspaces/. The uv-managed
# binary itself lives in tools/.uv/bin (on the managed PATH).
"install": "dashboards/openwebui",
"config": "configs/openwebui",
"data": "workspaces/openwebui",
"logs": "logs/openwebui"
},
"description": "Extensible frontend for LLMs.", "description": "Extensible frontend for LLMs.",
"license": 'MIT', "license": 'MIT',
"flags": { "flags": {
@ -79,11 +109,17 @@ TOOLS: dict[str, dict] = {
"installer": { "installer": {
"type": "git_node", "type": "git_node",
"pkg": "https://github.com/danny-avila/LibreChat.git", "pkg": "https://github.com/danny-avila/LibreChat.git",
"update_cmd": "git pull --ff-only && yarn install && yarn build" "update_cmd": "git pull --ff-only && yarn install && yarn build",
# Two known fresh-clone start bugs, fixed at install time:
# 1. `yarn backend` aborts without an .env file — seed one
# from the shipped example (never clobber an existing one).
# 2. production mode requires the client assets from
# `yarn build` — plain `yarn install` doesn't produce them.
"post_install": "cp -n .env.example .env 2>/dev/null || true; yarn build"
}, },
"launcher": { "launcher": {
"type": "tmux", "type": "tmux",
"cmd": "cd {tools_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend", "cmd": "cd {dashboards_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend",
"default_port": 3080 "default_port": 3080
}, },
"deps": [ "deps": [
@ -101,9 +137,9 @@ TOOLS: dict[str, dict] = {
"is_skills_collection": False "is_skills_collection": False
}, },
"filesystem": { "filesystem": {
"install": "tools/librechat", "install": "dashboards/librechat",
"config": "configs/librechat", "config": "configs/librechat",
"data": "data/librechat", "data": "workspaces/librechat",
"logs": "logs/librechat" "logs": "logs/librechat"
} }
}, },
@ -536,11 +572,21 @@ TOOLS: dict[str, dict] = {
"category": 'Chat Frontend', "category": 'Chat Frontend',
"installer": { "installer": {
"type": "uv", "type": "uv",
"pkg": "open-webui" # Same pydub/audioop Python-3.13 fix as the openwebui entry.
"pkg": "open-webui --with audioop-lts"
}, },
"launcher": { "launcher": {
"type": "tmux", "type": "tmux",
"cmd": "WEBUI_DATA_DIR={workspaces_root}/hermes-webui OLLAMA_BASE_URL=http://localhost:17051 open-webui serve --port {port}", # Same minimal pattern as the openwebui entry — cd to
# configs/auth/ so .webui_secret_key lands there, DATA_DIR
# points at the hermes-webui data volume.
#
# The ONE env var we set here: OLLAMA_BASE_URL. The default is
# http://localhost:11434 (ollama direct); hermes_webui needs to
# point at hermes_agent (17051) instead so every conversation
# flows through the Hermes runtime's tool-use layer. This is
# the only difference from the openwebui entry.
"cmd": "mkdir -p {base_dir}/configs/auth && chmod 700 {base_dir}/configs/auth && cd {base_dir}/configs/auth && DATA_DIR={workspaces_root}/hermes-webui OLLAMA_BASE_URL=http://127.0.0.1:17051 open-webui serve --port {port}",
"default_port": 8081 "default_port": 8081
}, },
"deps": [ "deps": [
@ -625,4 +671,54 @@ TOOLS: dict[str, dict] = {
"is_skills_collection": False "is_skills_collection": False
} }
}, },
'openhuman': {
"name": "OpenHuman",
"level": 10,
"layer": 'Human Interface & System Operations',
"role": 'Personal AI Harness',
"category": 'Personal AI Harness',
"installer": {
# Same curl|sh policy category as ollama / meilisearch / fabric
# (see whatremains.txt C-05). The openhuman README explicitly
# warns this script path is unverified (no detached signature);
# the native-package paths (deb / rpm / AppImage / brew cask)
# are preferred when the user is ready to move off the script
# installer.
"type": "script",
"cmd": "curl -fsSL https://raw.githubusercontent.com/tinyhumansai/openhuman/main/scripts/install.sh | bash",
"pkg": "https://github.com/tinyhumansai/openhuman"
},
"launcher": {
# Desktop GUI app (Rust core + native shell). One-click start
# → talks to localhost ollama. OLLAMA_BASE_URL is preset so
# openhuman's first-run model-router defaults to the local
# ollama instance instead of the managed subscription endpoint.
# Wrapped in `bash -c` because launch_desktop uses
# subprocess.Popen(argv) directly (no shell), so env-assignment
# prefixes must be expanded by an actual shell.
"type": "desktop",
"cmd": "bash -c 'OLLAMA_BASE_URL=http://127.0.0.1:11434 exec openhuman'",
"default_port": None
},
"deps": [
"ollama"
],
"description": "Personal AI harness: persistent local memory tree, "
"agent orchestration on durable graphs, deep-research "
"pipeline. Local-first with opt-in Privacy Mode that "
"forces all inference to localhost ollama (or another "
"managed backend). Native packages (.deb / .rpm / "
".AppImage / brew cask) preferred over the script "
"installer once the user moves past first boot.",
"license": 'GPL-3.0',
"flags": {
"has_cli": False,
"has_gui": True,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
} }

View File

@ -175,7 +175,11 @@ def _check_entry(tool_id: str, entry: dict[str, Any]) -> list[str]:
# Warn if script cmd doesn't contain {tools_root} # Warn if script cmd doesn't contain {tools_root}
if itype == "script" and inst.get("cmd"): if itype == "script" and inst.get("cmd"):
cmd_str = inst["cmd"] cmd_str = inst["cmd"]
if "{tools_root}" not in cmd_str and tool_id != "ollama": # carve-out: ollama and openhuman ship native installers
# (apt/dpkg/AppImage/brew cask) that target system paths,
# not tools_root. Their script installers are accepted as-is
# per the C-05 curl|sh policy (see whatremains.txt).
if "{tools_root}" not in cmd_str and tool_id not in {"ollama", "openhuman"}:
errors.append( errors.append(
f"{tool_id}: script installer cmd should reference " f"{tool_id}: script installer cmd should reference "
f"{{{{tools_root}}}} to avoid polluting system dirs" f"{{{{tools_root}}}} to avoid polluting system dirs"

View File

@ -104,16 +104,28 @@ class RuntimeExecutor:
port: str = "", port: str = "",
model_arg: str = "", model_arg: str = "",
) -> dict[str, str]: ) -> dict[str, str]:
"""Build the ``{placeholders}`` dict used by launcher commands.""" """Build the ``{placeholders}`` dict used by launcher commands.
Every key of the path tree is exposed (``{tools_root}``,
``{dashboards_root}``, ``{runtime_root}``, ``{configs_root}``,
``{datasets_root}``, …) so registry launcher templates can
reference any canonical /mnt/AI location.
"""
from ai_lsc.constants import BASE_DIR from ai_lsc.constants import BASE_DIR
return { from ai_lsc.utils.paths import build_path_tree
ctx = {
str(k): str(v)
for k, v in build_path_tree(BASE_DIR).items()
}
ctx.update({
"base_dir": BASE_DIR, "base_dir": BASE_DIR,
"tools_root": self.tools_root, "tools_root": self.tools_root,
"models_root": self.models_root, "models_root": self.models_root,
"workspaces_root": self.workspaces_root, "workspaces_root": self.workspaces_root,
"port": port, "port": port,
"model_arg": model_arg, "model_arg": model_arg,
} })
return ctx
# -- service lifecycle ----------------------------------------------- # -- service lifecycle -----------------------------------------------
@ -224,15 +236,26 @@ class RuntimeExecutor:
env_overrides: dict[str, str] | None = None, env_overrides: dict[str, str] | None = None,
filesystem: dict[str, str] | None = None, filesystem: dict[str, str] | None = None,
license_spdx: str | None = None, license_spdx: str | None = None,
registry_entry: dict | None = None,
) -> str: ) -> str:
"""Dispatch tool installation to the correct installer. """Dispatch tool installation to the correct installer.
If *tool_id* is provided, the installer uses preflight detection If *tool_id* is provided, the installer uses preflight detection.
and routes artifacts to ``tools_root/<tool_id>/``. When the tool is already installed, the call becomes an in-place
**update** (honouring the registry's ``installer.update_cmd``)
unless *force* is set or the tool is missing.
If *force* is True, skips preflight and installs unconditionally. If *force* is True, skips preflight and installs unconditionally.
*post_install* runs a shell command inside ``tools_root/<tool_id>`` *registry_entry* (the tool's full registry metadata dict) routes
after clone (e.g. ``pip install -r requirements.txt``, ``make``). artifacts to the canonical install root declared by
``filesystem.install`` / the v3.4 router (web UIs →
``dashboards/<id>/``, standalone compiles → ``tools/<id>/``)
and supplies ``update_cmd`` / ``post_install`` defaults.
*post_install* runs a shell command inside the tool's install
directory after clone (e.g. ``pip install -r requirements.txt``,
``make``).
*env_overrides* remaps upstream environment variables (HF_HOME, *env_overrides* remaps upstream environment variables (HF_HOME,
TRANSFORMERS_CACHE, etc.) into ``/mnt/AI/`` paths. TRANSFORMERS_CACHE, etc.) into ``/mnt/AI/`` paths.
@ -259,6 +282,7 @@ class RuntimeExecutor:
post_install=post_install, post_install=post_install,
env_overrides=env_overrides, env_overrides=env_overrides,
license_spdx=license_spdx, license_spdx=license_spdx,
entry=registry_entry,
) )
return self._installer.run( return self._installer.run(
inst_type=inst_type, inst_type=inst_type,
@ -269,6 +293,40 @@ class RuntimeExecutor:
post_install=post_install, post_install=post_install,
env_overrides=env_overrides, env_overrides=env_overrides,
license_spdx=license_spdx, license_spdx=license_spdx,
entry=registry_entry,
)
def update_tool(
self,
tool_id: str,
inst_type: str,
pkg: str,
cmd: str = "",
update_cmd: str = "",
registry_entry: dict | None = None,
env_overrides: dict[str, str] | None = None,
post_install: str | None = None,
) -> str:
"""Update an already-installed tool in place.
Honours the registry's ``installer.update_cmd`` when provided;
otherwise falls back to per-type defaults (``uv tool upgrade``,
``pipx upgrade``, ``git pull --ff-only`` + node rebuild, …).
Runs in the *detected* install location so tools installed
before the v3.4 layout change update where they live.
Returns a description of the result.
"""
_validate_tool_id(tool_id)
return self._installer.update_tool(
tool_id=tool_id,
inst_type=inst_type,
pkg=pkg,
cmd=cmd,
update_cmd=update_cmd,
entry=registry_entry,
env_overrides=env_overrides,
post_install=post_install,
) )
# -- verification --------------------------------------------------- # -- verification ---------------------------------------------------
@ -280,6 +338,7 @@ class RuntimeExecutor:
pkg: str, pkg: str,
cmd: str = "", cmd: str = "",
filesystem: dict[str, str] | None = None, filesystem: dict[str, str] | None = None,
registry_entry: dict | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Run the installation compliance checklist for a tool. """Run the installation compliance checklist for a tool.
@ -291,6 +350,7 @@ class RuntimeExecutor:
pkg=pkg, pkg=pkg,
cmd=cmd, cmd=cmd,
filesystem=filesystem, filesystem=filesystem,
entry=registry_entry,
) )
# -- model management ------------------------------------------------ # -- model management ------------------------------------------------

View File

@ -53,6 +53,11 @@ from typing import Any
from urllib.parse import urlparse from urllib.parse import urlparse
from ai_lsc.utils.logging import get_logger from ai_lsc.utils.logging import get_logger
from ai_lsc.utils.paths import (
build_path_tree,
candidate_install_dirs,
install_dir_for,
)
from ai_lsc.utils.process import enriched_env from ai_lsc.utils.process import enriched_env
logger = get_logger(__name__) logger = get_logger(__name__)
@ -92,6 +97,21 @@ def _validate_url(url: str, *, allow_schemes: tuple[str, ...] = ("http", "https"
raise ValueError(f"unsafe URL rejected: {url!r}") raise ValueError(f"unsafe URL rejected: {url!r}")
return url return url
def _uv_with_extras(tokens: list[str]) -> list[str]:
"""Extract the package names pulled in by ``--with`` flags.
``["open-webui", "--with", "audioop-lts"]`` → ``["audioop-lts"]``.
Multiple ``--with`` flags each contribute one package; unknown
flags contribute nothing. Used by the uv→pip step-down so the
fallback venv keeps every extra the registry declared.
"""
extras: list[str] = []
for i, tok in enumerate(tokens[:-1]):
if tok == "--with":
extras.append(tokens[i + 1])
return extras
# Step-down containment order (most isolated first) # Step-down containment order (most isolated first)
STEP_DOWN_ORDER: list[str] = [ STEP_DOWN_ORDER: list[str] = [
"ollama", "uv", "pipx", "pip", "ollama", "uv", "pipx", "pip",
@ -152,6 +172,43 @@ class InstallerManager:
# e.g. in the UI layer). # e.g. in the UI layer).
self.license_gate = license_gate self.license_gate = license_gate
# ── Install-root routing ─────────────────────────────────────────
def _dest_for(
self,
tool_id: str,
entry: dict[str, Any] | None = None,
) -> str:
"""Resolve the on-disk destination directory for *tool_id*.
Honours the registry entry's ``filesystem.install`` and the
v3.4 install-root router (web UIs → ``dashboards/``, everything
else → ``tools/``). Falls back to the historical
``tools_root/<tool_id>`` when no entry is supplied, so
pre-v3.4 call sites keep working unchanged.
"""
if entry:
return str(install_dir_for(entry, tool_id, self.base_dir))
return os.path.join(self.tools_root, tool_id)
def _existing_dir_for(
self,
tool_id: str,
entry: dict[str, Any] | None = None,
) -> str:
"""Like :meth:`_dest_for`, but prefer an existing install.
Scans the candidate roots in router order and returns the first
directory that already exists; only when none exists is the
canonical (fresh-install) destination returned. This keeps
updates flowing to wherever a tool actually lives today
instead of forking a second copy under the new canonical root.
"""
for cand in candidate_install_dirs(tool_id, entry, self.base_dir):
if cand.is_dir():
return str(cand)
return self._dest_for(tool_id, entry)
# ── Environment construction ───────────────────────────────────── # ── Environment construction ─────────────────────────────────────
def _env( def _env(
@ -198,14 +255,20 @@ class InstallerManager:
env["PIPX_HOME"] = os.path.join(self.tools_root, ".pipx") env["PIPX_HOME"] = os.path.join(self.tools_root, ".pipx")
# ── Per-tool env overrides from registry ──────────────────────── # ── Per-tool env overrides from registry ────────────────────────
# Keys may contain {tools_root}, {base_dir} placeholders. # Values may contain any {path_tree_key} placeholder
# ({tools_root}, {base_dir}, {dashboards_root}, …).
if env_overrides: if env_overrides:
tree = build_path_tree(self.base_dir)
for key, raw_val in env_overrides.items(): for key, raw_val in env_overrides.items():
expanded = raw_val.replace( expanded = raw_val.replace(
"{tools_root}", self.tools_root, "{tools_root}", self.tools_root,
).replace( ).replace(
"{base_dir}", self.base_dir, "{base_dir}", self.base_dir,
) )
for tree_key, tree_val in tree.items():
expanded = expanded.replace(
"{" + tree_key + "}", str(tree_val),
)
env[key] = expanded env[key] = expanded
logger.debug( logger.debug(
"env override: %s=%s (tool %s)", key, expanded, tool_id, "env override: %s=%s (tool %s)", key, expanded, tool_id,
@ -230,6 +293,7 @@ class InstallerManager:
inst_type: str, inst_type: str,
pkg: str, pkg: str,
cmd: str = "", cmd: str = "",
entry: dict[str, Any] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Check whether a tool is already installed before installing. """Check whether a tool is already installed before installing.
@ -246,7 +310,7 @@ class InstallerManager:
} }
location, version = self._detect_installation( location, version = self._detect_installation(
tool_id, inst_type, pkg, cmd, tool_id, inst_type, pkg, cmd, entry,
) )
if location: if location:
result["found"] = True result["found"] = True
@ -262,39 +326,54 @@ class InstallerManager:
inst_type: str, inst_type: str,
pkg: str, pkg: str,
cmd: str = "", cmd: str = "",
entry: dict[str, Any] | None = None,
) -> tuple[str, str]: ) -> tuple[str, str]:
"""Detect existing installation. Returns (location, version).""" """Detect existing installation. Returns (location, version).
# 1. Check tools_root/<tool_id> directory existence Probes every candidate install root (router dir, dashboards/,
tool_dir = os.path.join(self.tools_root, tool_id) tools/) so tools installed before the v3.4 routing change are
still found and offered an update instead of a reinstall.
"""
# `pkg` may carry extra uv/flags tokens (e.g.
# "open-webui --with audioop-lts"); for preflight we only care
# about the first token — the actual package name.
pkg_token = shlex.split(pkg)[0] if pkg else pkg
# 1. Check candidate install directories (router order)
for tool_dir in candidate_install_dirs(
tool_id, entry, self.base_dir,
):
if os.path.isdir(tool_dir): if os.path.isdir(tool_dir):
ver = self._detect_version(inst_type, pkg, cmd, tool_dir) ver = self._detect_version(
return tool_dir, ver inst_type, pkg_token, cmd, str(tool_dir),
)
return str(tool_dir), ver
# 2. Check tools_root/.pipx, tools_root/.uv, tools_root/.local # 2. Check tools_root/.pipx, tools_root/.uv, tools_root/.local
for subdir in [".pipx", ".uv", ".local"]: for subdir in [".pipx", ".uv", ".local"]:
check = os.path.join(self.tools_root, subdir, "bin", pkg) check = os.path.join(self.tools_root, subdir, "bin", pkg_token)
if os.path.exists(check): if os.path.exists(check):
return os.path.dirname(check), "" return os.path.dirname(check), ""
# 3. Check tools_root/bin # 3. Check tools_root/bin
bin_check = os.path.join(self.tools_root, "bin", pkg) bin_check = os.path.join(self.tools_root, "bin", pkg_token)
if os.path.exists(bin_check): if os.path.exists(bin_check):
return os.path.dirname(bin_check), "" return os.path.dirname(bin_check), ""
# 4. Check system PATH via shutil.which # 4. Check system PATH via shutil.which
binary_name = self._binary_name(pkg, inst_type) binary_name = self._binary_name(pkg_token, inst_type)
system_path = shutil.which(binary_name) system_path = shutil.which(binary_name)
if system_path: if system_path:
ver = self._detect_version(inst_type, pkg, cmd) ver = self._detect_version(inst_type, pkg_token, cmd)
return system_path, ver return system_path, ver
# 5. OS package manager query (pacman / dnf / apt) — list-form # 5. OS package manager query (pacman / dnf / apt) — list-form
# subprocess calls, no shell, no interpolation. # subprocess calls, no shell, no interpolation.
_PKG_MGR_QUERIES: dict[str, list[str]] = { _PKG_MGR_QUERIES: dict[str, list[str]] = {
"pacman": ["pacman", "-Qi", pkg], "pacman": ["pacman", "-Qi", pkg_token],
"dnf": ["dnf", "info", pkg], "dnf": ["dnf", "info", pkg_token],
"apt": ["dpkg", "-s", pkg], "apt": ["dpkg", "-s", pkg_token],
} }
if inst_type in _PKG_MGR_QUERIES: if inst_type in _PKG_MGR_QUERIES:
try: try:
@ -306,19 +385,24 @@ class InstallerManager:
for line in proc.stdout.splitlines(): for line in proc.stdout.splitlines():
if line.strip().startswith("Version"): if line.strip().startswith("Version"):
ver = line.split(":", 1)[-1].strip() ver = line.split(":", 1)[-1].strip()
return f"{inst_type}:{pkg}", ver return f"{inst_type}:{pkg_token}", ver
except (OSError, subprocess.SubprocessError): except (OSError, subprocess.SubprocessError):
pass pass
return "", "" return "", ""
def _binary_name(self, pkg: str, inst_type: str) -> str: def _binary_name(self, pkg: str, inst_type: str) -> str:
"""Map a package name to its likely binary name.""" """Map a package name to its likely binary name.
Takes the first shlex token of *pkg* so a spec like
``"open-webui --with audioop-lts"`` resolves to ``open_webui``.
"""
first = shlex.split(pkg)[0] if pkg else pkg
if inst_type == "npm": if inst_type == "npm":
return pkg if "/" not in pkg else pkg.split("/")[-1] return first if "/" not in first else first.split("/")[-1]
if inst_type in ("uv", "pip"): if inst_type in ("uv", "pip"):
return pkg.replace("-", "_").replace(".", "_") return first.replace("-", "_").replace(".", "_")
return pkg return first
def _detect_version( def _detect_version(
self, self,
@ -329,9 +413,19 @@ class InstallerManager:
) -> str: ) -> str:
"""Try to extract the installed version.""" """Try to extract the installed version."""
if inst_type == "git": if inst_type == "git":
git_dir = os.path.join(self.tools_root, pkg.split("/")[-1] # Use only the first token of `pkg` for the dir-name lookup
.replace(".git", "")) # so a spec like "repo --branch foo" doesn't break the
if os.path.isdir(os.path.join(git_dir, ".git")): # rsplit("/"). (Git pkg specs are normally bare URLs, but
# be defensive.)
pkg_first = shlex.split(pkg)[0] if pkg else pkg
repo_name = pkg_first.split("/")[-1].replace(".git", "")
git_dirs = [
str(d) for d in candidate_install_dirs(
repo_name, None, self.base_dir,
)
if os.path.isdir(os.path.join(d, ".git"))
]
for git_dir in git_dirs:
for argv in ( for argv in (
["git", "describe", "--tags", "--abbrev=0"], ["git", "describe", "--tags", "--abbrev=0"],
["git", "rev-parse", "--short", "HEAD"], ["git", "rev-parse", "--short", "HEAD"],
@ -355,7 +449,11 @@ class InstallerManager:
else: else:
tmpl = _VERSION_CMDS.get(inst_type, "") tmpl = _VERSION_CMDS.get(inst_type, "")
if tmpl: if tmpl:
ver_argv = shlex.split(tmpl.format(pkg=pkg, cmd=pkg)) # Use only the first token of `pkg` so a spec like
# "open-webui --with audioop-lts" renders as just the
# binary name in the version-detection command.
pkg_first = shlex.split(pkg)[0] if pkg else pkg
ver_argv = shlex.split(tmpl.format(pkg=pkg_first, cmd=pkg_first))
if not ver_argv: if not ver_argv:
return "" return ""
@ -378,14 +476,16 @@ class InstallerManager:
self, self,
tool_id: str, tool_id: str,
post_install_cmd: str, post_install_cmd: str,
dest: str = "",
) -> str: ) -> str:
"""Run a post-install hook inside ``tools_root/<tool_id>``.""" """Run a post-install hook inside the tool's install directory."""
if not post_install_cmd: if not post_install_cmd:
return "" return ""
dest = os.path.join(self.tools_root, tool_id) cwd = dest or os.path.join(self.tools_root, tool_id)
env = self._env(tool_id) env = self._env(tool_id)
# Replace {tools_root} in the command # Resolve {tools_root} / {base_dir} / {install_dir} and every
cmd = post_install_cmd.replace("{tools_root}", self.tools_root) # path-tree placeholder in the command.
cmd = self._render_cmd(post_install_cmd, cwd)
logger.info("Running post-install for %s: %s", tool_id, cmd) logger.info("Running post-install for %s: %s", tool_id, cmd)
try: try:
# Post-install commands are arbitrary shell snippets supplied by # Post-install commands are arbitrary shell snippets supplied by
@ -395,7 +495,7 @@ class InstallerManager:
# break out of the subprocess call itself. # break out of the subprocess call itself.
subprocess.run( subprocess.run(
["bash", "-c", cmd], check=True, env=env, ["bash", "-c", cmd], check=True, env=env,
timeout=300, cwd=dest, timeout=600, cwd=cwd,
) )
return f"Post-install completed for {tool_id}." return f"Post-install completed for {tool_id}."
except (subprocess.CalledProcessError, OSError) as exc: except (subprocess.CalledProcessError, OSError) as exc:
@ -404,6 +504,20 @@ class InstallerManager:
) )
return f"Post-install FAILED for {tool_id}: {exc}" return f"Post-install FAILED for {tool_id}: {exc}"
def _render_cmd(self, cmd: str, install_dir: str = "") -> str:
"""Resolve path placeholders in a registry shell snippet.
Supports ``{tools_root}``, ``{base_dir}``, ``{install_dir}``
and every key of the path tree (``{dashboards_root}``, …).
"""
rendered = cmd.replace("{tools_root}", self.tools_root)
rendered = rendered.replace("{base_dir}", self.base_dir)
if install_dir:
rendered = rendered.replace("{install_dir}", install_dir)
for key, value in build_path_tree(self.base_dir).items():
rendered = rendered.replace("{" + key + "}", str(value))
return rendered
# ── Strategy methods ──────────────────────────────────────────── # ── Strategy methods ────────────────────────────────────────────
def install_ollama(self, pkg: str, tool_id: str) -> str: def install_ollama(self, pkg: str, tool_id: str) -> str:
@ -438,26 +552,84 @@ class InstallerManager:
return f"Ollama model '{pkg}' queued for pull." return f"Ollama model '{pkg}' queued for pull."
def install_uv(self, pkg: str, tool_id: str, def install_uv(self, pkg: str, tool_id: str,
env_overrides: dict[str, str] | None = None) -> str: env_overrides: dict[str, str] | None = None,
"""Install a Python tool via ``uv tool install`` pinned to tools_root.""" entry: dict[str, Any] | None = None) -> str:
dest = os.path.join(self.tools_root, tool_id) """Install a Python tool via ``uv tool install`` pinned to tools_root.
The ``pkg`` string may carry extra ``uv`` flags after the package
name — e.g. ``"open-webui --with audioop-lts"``. The first token
is validated as the package name; the remaining tokens are passed
through to ``uv tool install`` verbatim. This lets registry
entries pin Python-3.13-deps backports (audioop-lts for pydub,
etc.) without needing new schema fields or threading new params
through every caller.
Step-down containment (fix for the open-webui install bug):
1. ``uv tool install <pkg> [flags]`` with the default Python.
2. If that fails, retry with ``--python 3.12`` — uv fetches a
managed CPython 3.12 on the fly, which sidesteps wheels that
have not been rebuilt for the host's newer Python (the
pydub/audioop class of failure).
3. If uv fails entirely, fall back to a per-tool pip venv that
installs the package **plus every ``--with`` extra** — the
historical pipx fallback silently dropped the extras, which
produced installs that broke at import time.
"""
dest = self._dest_for(tool_id, entry)
os.makedirs(dest, exist_ok=True) os.makedirs(dest, exist_ok=True)
env = self._env(tool_id, env_overrides) env = self._env(tool_id, env_overrides)
# Shlex-split so registry authors can write
# `pkg: "open-webui --with audioop-lts"` and have it land as
# `["uv", "tool", "install", "open-webui", "--with", "audioop-lts"]`.
tokens = shlex.split(pkg)
if not tokens:
raise ValueError(f"invalid pkg spec for {tool_id}: {pkg!r}")
_validate_pkg(tokens[0])
extras = _uv_with_extras(tokens)
try: try:
_validate_pkg(pkg)
subprocess.run( subprocess.run(
["uv", "tool", "install", pkg], ["uv", "tool", "install", *tokens],
check=True, env=env, timeout=300, check=True, env=env, timeout=300,
) )
return f"UV tool '{pkg}' installed to {env['UV_TOOL_DIR']}." return f"UV tool '{tokens[0]}' installed to {env['UV_TOOL_DIR']}."
except subprocess.CalledProcessError: except subprocess.CalledProcessError:
logger.info("uv install failed for %s, stepping down to pipx", pkg) logger.info(
return self.install_pipx(pkg, tool_id, env_overrides) "uv install failed for %s on the default Python; "
"retrying with a managed CPython 3.12", tokens[0],
)
try:
subprocess.run(
["uv", "tool", "install", "--python", "3.12", *tokens],
check=True, env=env, timeout=600,
)
return (
f"UV tool '{tokens[0]}' installed to "
f"{env['UV_TOOL_DIR']} (managed CPython 3.12)."
)
except subprocess.CalledProcessError:
if extras:
logger.info(
"uv failed for %s even on 3.12; falling back to a "
"pip venv WITH the %d --with extra(s) intact",
tokens[0], len(extras),
)
pip_spec = " ".join([tokens[0], *extras])
return self.install_pip(
pip_spec, tool_id, env_overrides, entry,
)
logger.info(
"uv failed for %s; stepping down to pipx", tokens[0],
)
return self.install_pipx(
tokens[0], tool_id, env_overrides, entry,
)
def install_pipx(self, pkg: str, tool_id: str, def install_pipx(self, pkg: str, tool_id: str,
env_overrides: dict[str, str] | None = None) -> str: env_overrides: dict[str, str] | None = None,
entry: dict[str, Any] | None = None) -> str:
"""Install a Python CLI tool via ``pipx`` pinned to tools_root.""" """Install a Python CLI tool via ``pipx`` pinned to tools_root."""
dest = os.path.join(self.tools_root, tool_id) dest = self._dest_for(tool_id, entry)
os.makedirs(dest, exist_ok=True) os.makedirs(dest, exist_ok=True)
env = self._env(tool_id, env_overrides) env = self._env(tool_id, env_overrides)
try: try:
@ -469,12 +641,18 @@ class InstallerManager:
return f"pipx '{pkg}' installed to {env['PIPX_HOME']}." return f"pipx '{pkg}' installed to {env['PIPX_HOME']}."
except subprocess.CalledProcessError: except subprocess.CalledProcessError:
logger.info("pipx install failed for %s, stepping down to pip", pkg) logger.info("pipx install failed for %s, stepping down to pip", pkg)
return self.install_pip(pkg, tool_id, env_overrides) return self.install_pip(pkg, tool_id, env_overrides, entry)
def install_pip(self, pkg: str, tool_id: str, def install_pip(self, pkg: str, tool_id: str,
env_overrides: dict[str, str] | None = None) -> str: env_overrides: dict[str, str] | None = None,
"""Install a Python tool via ``pip`` into a per-tool venv.""" entry: dict[str, Any] | None = None) -> str:
dest = os.path.join(self.tools_root, tool_id) """Install a Python tool via ``pip`` into a per-tool venv.
*pkg* may be a space-separated spec naming several distributions
(``"open-webui audioop-lts"``) — the uv step-down path uses this
to keep ``--with`` extras alive in the fallback venv.
"""
dest = self._dest_for(tool_id, entry)
venv_dir = os.path.join(dest, ".venv") venv_dir = os.path.join(dest, ".venv")
os.makedirs(dest, exist_ok=True) os.makedirs(dest, exist_ok=True)
env = self._env(tool_id, env_overrides) env = self._env(tool_id, env_overrides)
@ -484,16 +662,20 @@ class InstallerManager:
check=True, env=env, timeout=60, check=True, env=env, timeout=60,
) )
pip_bin = os.path.join(venv_dir, "bin", "pip") pip_bin = os.path.join(venv_dir, "bin", "pip")
pip_tokens = shlex.split(pkg) if pkg else []
if not pip_tokens:
raise ValueError(f"invalid pkg spec for {tool_id}: {pkg!r}")
try: try:
_validate_pkg(pkg) for tok in pip_tokens:
_validate_pkg(tok)
subprocess.run( subprocess.run(
[pip_bin, "install", pkg], [pip_bin, "install", *pip_tokens],
check=True, env=env, timeout=300, check=True, env=env, timeout=300,
) )
except subprocess.CalledProcessError as exc: except subprocess.CalledProcessError as exc:
logger.warning("pip install failed for %s: %s", pkg, exc) logger.warning("pip install failed for %s: %s", pkg, exc)
raise raise
self._symlink_venv_bin(tool_id, venv_dir, pkg) self._symlink_venv_bin(tool_id, venv_dir, pip_tokens[0])
return f"pip '{pkg}' installed to {venv_dir}." return f"pip '{pkg}' installed to {venv_dir}."
def _symlink_venv_bin( def _symlink_venv_bin(
@ -544,9 +726,10 @@ class InstallerManager:
return f"Dispatched apt for {pkg}." return f"Dispatched apt for {pkg}."
def install_npm(self, pkg: str, tool_id: str, def install_npm(self, pkg: str, tool_id: str,
env_overrides: dict[str, str] | None = None) -> str: env_overrides: dict[str, str] | None = None,
entry: dict[str, Any] | None = None) -> str:
"""Install an npm package to an isolated prefix under tools_root.""" """Install an npm package to an isolated prefix under tools_root."""
dest = os.path.join(self.tools_root, tool_id) dest = self._dest_for(tool_id, entry)
os.makedirs(dest, exist_ok=True) os.makedirs(dest, exist_ok=True)
env = self._env(tool_id, env_overrides) env = self._env(tool_id, env_overrides)
_validate_pkg(pkg) _validate_pkg(pkg)
@ -562,6 +745,7 @@ class InstallerManager:
tool_id: str, tool_id: str,
post_install: str | None = None, post_install: str | None = None,
env_overrides: dict[str, str] | None = None, env_overrides: dict[str, str] | None = None,
entry: dict[str, Any] | None = None,
) -> str: ) -> str:
"""Clone or update a git repository at ``tools_root/<tool_id>``. """Clone or update a git repository at ``tools_root/<tool_id>``.
@ -571,8 +755,14 @@ class InstallerManager:
fails for any reason — diverged branches, network errors, fails for any reason — diverged branches, network errors,
corrupted index, etc.), move the old dir aside and re-clone corrupted index, etc.), move the old dir aside and re-clone
fresh, so the install always ends in a usable state. fresh, so the install always ends in a usable state.
The destination follows the v3.4 install-root router when
*entry* is supplied (web UIs → ``dashboards/<tool_id>/``).
A pre-existing clone under ``tools/<tool_id>/`` is still
detected and updated in place — the install never forks a
second copy just because the routing changed.
""" """
dest = os.path.join(self.tools_root, tool_id) dest = self._existing_dir_for(tool_id, entry)
git_dir = os.path.join(dest, ".git") git_dir = os.path.join(dest, ".git")
# Step-down: each branch resolves one situation and assigns msg. # Step-down: each branch resolves one situation and assigns msg.
@ -607,7 +797,7 @@ class InstallerManager:
msg = f"Git source cloned: {dest}" msg = f"Git source cloned: {dest}"
if post_install: if post_install:
self._run_post_install(tool_id, post_install) self._run_post_install(tool_id, post_install, dest)
return msg return msg
def install_git_node( def install_git_node(
@ -615,6 +805,7 @@ class InstallerManager:
pkg: str, pkg: str,
tool_id: str, tool_id: str,
post_install: str | None = None, post_install: str | None = None,
entry: dict[str, Any] | None = None,
) -> str: ) -> str:
"""Clone or update a git repo and run ``yarn install``. """Clone or update a git repo and run ``yarn install``.
@ -623,8 +814,11 @@ class InstallerManager:
then ``yarn install`` to pick up any changed dependencies. If then ``yarn install`` to pick up any changed dependencies. If
the destination exists but is not a git repo, or if the pull the destination exists but is not a git repo, or if the pull
fails, move the old dir aside and re-clone fresh. fails, move the old dir aside and re-clone fresh.
Destination routing matches :meth:`install_git` (existing
clones are updated in place wherever they live).
""" """
dest = os.path.join(self.tools_root, tool_id) dest = self._existing_dir_for(tool_id, entry)
git_dir = os.path.join(dest, ".git") git_dir = os.path.join(dest, ".git")
# Step-down: each branch resolves one situation, runs the # Step-down: each branch resolves one situation, runs the
@ -671,7 +865,7 @@ class InstallerManager:
msg = f"Git+Node source synchronized: {dest}" msg = f"Git+Node source synchronized: {dest}"
if post_install: if post_install:
self._run_post_install(tool_id, post_install) self._run_post_install(tool_id, post_install, dest)
return msg return msg
# ── git install helpers ──────────────────────────────────────── # ── git install helpers ────────────────────────────────────────
@ -767,6 +961,7 @@ class InstallerManager:
post_install: str | None = None, post_install: str | None = None,
env_overrides: dict[str, str] | None = None, env_overrides: dict[str, str] | None = None,
license_spdx: str | None = None, license_spdx: str | None = None,
entry: dict[str, Any] | None = None,
) -> str: ) -> str:
"""Dispatch to the correct installer strategy. """Dispatch to the correct installer strategy.
@ -774,6 +969,11 @@ class InstallerManager:
Parameters Parameters
---------- ----------
entry :
The tool's full registry entry. When supplied, artifacts
are routed to the install-root destination declared by
``filesystem.install`` / the v3.4 router instead of the
flat ``tools_root/<tool_id>``.
license_spdx : license_spdx :
SPDX ID for the tool's license. If provided AND a SPDX ID for the tool's license. If provided AND a
``license_gate`` was passed to the InstallerManager ``license_gate`` was passed to the InstallerManager
@ -806,21 +1006,29 @@ class InstallerManager:
strategies: dict[str, Any] = { strategies: dict[str, Any] = {
"ollama": lambda: self.install_ollama(pkg, tool_id), "ollama": lambda: self.install_ollama(pkg, tool_id),
"uv": lambda: self.install_uv(pkg, tool_id, env_overrides), "uv": lambda: self.install_uv(
"pipx": lambda: self.install_pipx(pkg, tool_id, env_overrides), pkg, tool_id, env_overrides, entry,
"pip": lambda: self.install_pip(pkg, tool_id, env_overrides), ),
"pipx": lambda: self.install_pipx(
pkg, tool_id, env_overrides, entry,
),
"pip": lambda: self.install_pip(
pkg, tool_id, env_overrides, entry,
),
"script": lambda: self.install_script( "script": lambda: self.install_script(
cmd, ctx, tool_id, env_overrides, cmd, ctx, tool_id, env_overrides,
), ),
"pacman": lambda: self.install_pacman(pkg), "pacman": lambda: self.install_pacman(pkg),
"dnf": lambda: self.install_dnf(pkg), "dnf": lambda: self.install_dnf(pkg),
"apt": lambda: self.install_apt(pkg), "apt": lambda: self.install_apt(pkg),
"npm": lambda: self.install_npm(pkg, tool_id, env_overrides), "npm": lambda: self.install_npm(
pkg, tool_id, env_overrides, entry,
),
"git": lambda: self.install_git( "git": lambda: self.install_git(
pkg, tool_id, post_install, env_overrides, pkg, tool_id, post_install, env_overrides, entry,
), ),
"git_node": lambda: self.install_git_node( "git_node": lambda: self.install_git_node(
pkg, tool_id, post_install, pkg, tool_id, post_install, entry,
), ),
"custom": lambda: self.install_custom(pkg, tool_id), "custom": lambda: self.install_custom(pkg, tool_id),
} }
@ -852,6 +1060,7 @@ class InstallerManager:
inst_type=meta.get("installer", {}).get("type", "pacman"), inst_type=meta.get("installer", {}).get("type", "pacman"),
pkg=meta.get("installer", {}).get("pkg", ""), pkg=meta.get("installer", {}).get("pkg", ""),
cmd=meta.get("installer", {}).get("cmd", ""), cmd=meta.get("installer", {}).get("cmd", ""),
entry=meta,
) )
for tid, meta in tools.items() for tid, meta in tools.items()
} }
@ -867,15 +1076,28 @@ class InstallerManager:
post_install: str | None = None, post_install: str | None = None,
env_overrides: dict[str, str] | None = None, env_overrides: dict[str, str] | None = None,
license_spdx: str | None = None, license_spdx: str | None = None,
entry: dict[str, Any] | None = None,
update_if_found: bool = True,
) -> str: ) -> str:
"""Install a tool with preflight detection. """Install a tool with preflight detection.
If the tool is already installed and *force* is False, returns If the tool is already installed and *force* is False:
a message saying the tool exists and suggesting an update.
* ``update_if_found=True`` (default) — run the update path
(:meth:`update_tool`), honouring the registry's
``installer.update_cmd`` when declared. This is what makes
the UI's single Deploy/Update button behave as a smooth
in-place updater.
* ``update_if_found=False`` — return the historical
"already installed" notice without touching anything.
If *force* is True, proceeds with installation regardless. If *force* is True, proceeds with installation regardless.
Parameters Parameters
---------- ----------
entry :
The tool's full registry entry (routes artifacts + enables
update_cmd).
license_spdx : license_spdx :
SPDX ID for the tool's license. Forwarded to ``run()`` SPDX ID for the tool's license. Forwarded to ``run()``
for gate checking. for gate checking.
@ -884,16 +1106,152 @@ class InstallerManager:
# subprocess call on a blocked tool. # subprocess call on a blocked tool.
self._check_license(tool_id, license_spdx) self._check_license(tool_id, license_spdx)
check = self.preflight(tool_id, inst_type, pkg, cmd) check = self.preflight(tool_id, inst_type, pkg, cmd, entry)
if check["found"] and not force: if check["found"] and not force:
if not update_if_found:
return ( return (
f"Tool '{tool_id}' already installed at {check['location']}. " f"Tool '{tool_id}' already installed at "
f"{check['location']}. "
f"Version: {check['version'] or 'unknown'}. " f"Version: {check['version'] or 'unknown'}. "
f"Use force=True to update." f"Use force=True to update."
) )
update_cmd = (entry or {}).get("installer", {}).get(
"update_cmd", "",
) if entry else ""
return self.update_tool(
tool_id=tool_id,
inst_type=inst_type,
pkg=pkg,
cmd=cmd,
update_cmd=update_cmd or "",
entry=entry,
env_overrides=env_overrides,
post_install=post_install,
detected=check,
)
return self.run( return self.run(
inst_type, pkg, cmd, ctx, tool_id, inst_type, pkg, cmd, ctx, tool_id,
post_install, env_overrides, post_install, env_overrides, license_spdx, entry,
)
# ── Updates ─────────────────────────────────────────────────────
def update_tool(
self,
tool_id: str,
inst_type: str,
pkg: str,
cmd: str = "",
update_cmd: str = "",
entry: dict[str, Any] | None = None,
env_overrides: dict[str, str] | None = None,
post_install: str | None = None,
detected: dict[str, Any] | None = None,
) -> str:
"""Update an already-installed tool in place.
Resolution order:
1. **Registry ``update_cmd``** — rendered with the path-tree
placeholders plus ``{install_dir}`` and executed in the
tool's install directory (e.g. LibreChat's
``git pull --ff-only && yarn install && yarn build``).
2. **Per-type defaults** — ``uv tool upgrade``, ``pipx
upgrade``, per-tool venv ``pip install -U``, ``git pull
--ff-only`` (+ node rebuild for ``git_node``), ``npm
update``.
3. **System packages** (pacman / dnf / apt) and ``custom``
installs return guidance instead of shelling out.
The working directory is the *detected* install location when
known, else the router destination — so updates land where the
tool actually lives, never forking a second copy.
"""
_validate_tool_id(tool_id)
dest = (
detected.get("location")
if detected and detected.get("location")
else self._existing_dir_for(tool_id, entry)
)
env = self._env(tool_id, env_overrides)
# 1. Registry-declared update command wins.
if update_cmd:
rendered = self._render_cmd(update_cmd, dest)
logger.info("Updating %s via update_cmd: %s", tool_id, rendered)
subprocess.run(
["bash", "-c", rendered], check=True, env=env,
cwd=dest if os.path.isdir(dest) else None,
timeout=1800,
)
return f"Update completed for {tool_id} (update_cmd) in {dest}."
# 2. Per-type defaults.
pkg_first = shlex.split(pkg)[0] if pkg else pkg
if inst_type == "uv":
try:
subprocess.run(
["uv", "tool", "upgrade", pkg_first],
check=True, env=env, timeout=600,
)
return f"UV tool '{pkg_first}' upgraded."
except subprocess.CalledProcessError:
# upgrade failed (e.g. broken env) — reinstall keeps
# --with extras alive via install_uv's step-down.
logger.info(
"uv upgrade failed for %s; reinstalling", pkg_first,
)
return self.install_uv(
pkg, tool_id, env_overrides, entry,
)
if inst_type == "pipx":
try:
subprocess.run(
["pipx", "upgrade", pkg_first],
check=True, env=env, timeout=600,
)
return f"pipx '{pkg_first}' upgraded."
except subprocess.CalledProcessError:
logger.info(
"pipx upgrade failed for %s; reinstalling", pkg_first,
)
return self.install_pipx(
pkg, tool_id, env_overrides, entry,
)
if inst_type == "pip":
venv_pip = os.path.join(dest, ".venv", "bin", "pip")
if os.path.isfile(venv_pip):
_validate_pkg(pkg_first)
subprocess.run(
[venv_pip, "install", "--upgrade", pkg_first],
check=True, env=env, timeout=600,
)
return f"pip '{pkg_first}' upgraded in {dest}/.venv."
return (
f"No managed venv for {tool_id}; re-run with force to "
f"reinstall."
)
if inst_type in ("git", "git_node"):
# install_git / install_git_node already implement
# pull-ff-only-with-fallback + node rebuild semantics.
handler = (self.install_git_node if inst_type == "git_node"
else self.install_git)
msg = handler(
pkg, tool_id, post_install, env_overrides, entry,
)
return f"Update: {msg}"
if inst_type == "npm":
_validate_pkg(pkg_first)
subprocess.run(
["npm", "update", "--prefix", dest, pkg_first],
check=True, env=env, timeout=600,
)
return f"NPM '{pkg_first}' updated in {dest}."
# 3. Types we deliberately don't auto-update.
return (
f"'{tool_id}' is a {inst_type}-managed install — update it "
f"with the system package manager or its upstream installer."
) )
# ── Installation verification ─────────────────────────────────── # ── Installation verification ───────────────────────────────────
@ -905,6 +1263,7 @@ class InstallerManager:
pkg: str, pkg: str,
cmd: str = "", cmd: str = "",
filesystem: dict[str, str] | None = None, filesystem: dict[str, str] | None = None,
entry: dict[str, Any] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Run a compliance checklist against a single tool installation. """Run a compliance checklist against a single tool installation.
@ -927,9 +1286,10 @@ class InstallerManager:
fs = filesystem or {} fs = filesystem or {}
tool_dir = os.path.join(self.tools_root, tool_id) tool_dir = os.path.join(self.tools_root, tool_id)
# 1. Native install detected # 1. Native install detected (multi-root: router + dashboards
# + tools, so pre-v3.4 installs still verify)
location, version = self._detect_installation( location, version = self._detect_installation(
tool_id, inst_type, pkg, cmd, tool_id, inst_type, pkg, cmd, entry,
) )
checks.append(VerifyCheck( checks.append(VerifyCheck(
name="Native Install", name="Native Install",

File diff suppressed because it is too large Load Diff

View File

@ -219,6 +219,7 @@ if _HAS_QT:
self._setup_environment_hierarchy() self._setup_environment_hierarchy()
self._migrate_legacy_state_files() self._migrate_legacy_state_files()
self._migrate_registry_dir()
self.dtach_bin: str | None = find_binary("dtach-ng", "dtach") self.dtach_bin: str | None = find_binary("dtach-ng", "dtach")
# License gate — checks every tool's license before the # License gate — checks every tool's license before the
# installer dispatches to a subprocess. SaaS-blocked # installer dispatches to a subprocess. SaaS-blocked
@ -245,7 +246,7 @@ if _HAS_QT:
self.txt_base_dir.setReadOnly(True) self.txt_base_dir.setReadOnly(True)
self.registry_mgr = RegistryManager( self.registry_mgr = RegistryManager(
os.path.join(self.base_dir, "registry") os.path.join(self.base_dir, "configs", "registry")
) )
self.skill_resolver = ( self.skill_resolver = (
SkillRuntimeResolver(self.skills_root) SkillRuntimeResolver(self.skills_root)
@ -360,6 +361,40 @@ if _HAS_QT:
filter(None, [uv_bin, npm_bin]) filter(None, [uv_bin, npm_bin])
) )
def _migrate_registry_dir(self) -> None:
"""One-time move of the app registry into configs/registry.
v3.4 aligned the base dir with the canonical file-system
map: the app-internal ecosystem registry (ecosystem.json +
user tweaks) is app state and belongs under ``configs/``.
Pre-v3.4 installs kept it at ``<base_dir>/registry/`` —
move it when the legacy dir exists and the new one does
not; remove the legacy dir when it is left empty. Never
raises.
"""
import shutil
from ai_lsc.utils.paths import legacy_registry_dir
old = legacy_registry_dir(self.base_dir)
new = os.path.join(self.base_dir, "configs", "registry")
try:
if os.path.isdir(old):
os.makedirs(new, exist_ok=True)
for fname in os.listdir(old):
src = os.path.join(old, fname)
dst = os.path.join(new, fname)
if not os.path.exists(dst):
shutil.move(src, dst)
if not os.listdir(old):
os.rmdir(old)
from ai_lsc.utils.logging import get_logger
get_logger(__name__).info(
"Migrated tool registry to %s (legacy %s "
"removed)", new, old,
)
except OSError:
pass
# ─────────────────────────────────────────────────────────────── # ───────────────────────────────────────────────────────────────
# LEFT NAVIGATION RACK LAYOUT # LEFT NAVIGATION RACK LAYOUT
# ─────────────────────────────────────────────────────────────── # ───────────────────────────────────────────────────────────────
@ -964,7 +999,7 @@ if _HAS_QT:
layout.setContentsMargins(30, 20, 30, 20) layout.setContentsMargins(30, 20, 30, 20)
header = QHBoxLayout() header = QHBoxLayout()
lbl_title = QLabel("<b>AI-LSC v3.1 — Ankh of Jah</b>") lbl_title = QLabel("<b>AI-LSC v3.3.0 — Ankh of Jah</b>")
lbl_title.setFont(QFont("Segoe UI", 16)) lbl_title.setFont(QFont("Segoe UI", 16))
header.addWidget(lbl_title) header.addWidget(lbl_title)
header.addStretch() header.addStretch()
@ -1368,11 +1403,14 @@ if _HAS_QT:
if handler: if handler:
handler() handler()
# Drift detection # Drift detection (multi-root: checks the router dir,
# dashboards/, and tools/ so pre-v3.4 installs are not
# false-flagged)
state_path = self._get_active_state_file() state_path = self._get_active_state_file()
drift: list[str] = [] drift: list[str] = []
if state_path: if state_path:
try: try:
from ai_lsc.utils.paths import candidate_install_dirs
with open(state_path) as f: with open(state_path) as f:
state = json.load(f) state = json.load(f)
git_types = {"git", "git_node"} git_types = {"git", "git_node"}
@ -1383,8 +1421,12 @@ if _HAS_QT:
and self.registry_mgr.get_tool(tid) and self.registry_mgr.get_tool(tid)
.get("installer", {}) .get("installer", {})
.get("type") in git_types .get("type") in git_types
and not os.path.exists( and not any(
os.path.join(self.tools_root, tid) d.is_dir() for d in candidate_install_dirs(
tid,
self.registry_mgr.get_tool(tid),
self.base_dir,
)
) )
] ]
except Exception: except Exception:
@ -1449,6 +1491,11 @@ if _HAS_QT:
label = { label = {
"lxc": "LXC configs + launch script", "lxc": "LXC configs + launch script",
"firecracker": "Firecracker microVM configs + launch script", "firecracker": "Firecracker microVM configs + launch script",
"qcow2": "QCOW2 VM disk image",
"qcow": "QCOW (legacy) VM disk image",
"raw": "Raw VM disk image",
"qcrows": "QCrows VM container image (.qcrows)",
"svb": "Sovereign Bundle (.svb)",
}.get(backend_type, f"{backend_type.capitalize()} Compose") }.get(backend_type, f"{backend_type.capitalize()} Compose")
self.log( self.log(
f"{label} generated: {out_file}", f"{label} generated: {out_file}",

View File

@ -1,7 +1,8 @@
"""ContainerStacksTab widget — lists exported stack files. """ContainerStacksTab widget — lists exported stack files.
Displays available stack snapshots and provides export buttons for Displays available stack snapshots and provides export buttons for
Podman Compose and Docker Compose outputs. Podman Compose, Docker Compose, LXC, Firecracker, and QCOW2/QCOW VM
disk-image outputs.
""" """
import os import os
@ -60,6 +61,32 @@ if _HAS_QT:
lambda: self.main.finalize_stack_export("firecracker") lambda: self.main.finalize_stack_export("firecracker")
) )
header.addWidget(btn_firecracker) header.addWidget(btn_firecracker)
btn_qcow2 = QPushButton("Export -> QCOW2 Image")
btn_qcow2.setStyleSheet("background-color: #d35400;")
btn_qcow2.clicked.connect(
lambda: self.main.finalize_stack_export("qcow2")
)
header.addWidget(btn_qcow2)
btn_qcrows = QPushButton("Export -> QCrows Image (.qcrows)")
btn_qcrows.setStyleSheet("background-color: #7f8c8d;")
btn_qcrows.clicked.connect(
lambda: self.main.finalize_stack_export("qcrows")
)
header.addWidget(btn_qcrows)
btn_svb = QPushButton("Export -> Sovereign .svb")
btn_svb.clicked.connect(
lambda: self.main.finalize_stack_export("svb")
)
header.addWidget(btn_svb)
btn_qcow = QPushButton("Export -> QCOW (legacy)")
btn_qcow.clicked.connect(
lambda: self.main.finalize_stack_export("qcow")
)
header.addWidget(btn_qcow)
layout.addLayout(header) layout.addLayout(header)
self.file_list = QListWidget() self.file_list = QListWidget()
@ -75,12 +102,27 @@ if _HAS_QT:
# the LXC and Firecracker backends. # the LXC and Firecracker backends.
if fname.endswith((".yml", ".yaml", ".json", ".sh")): if fname.endswith((".yml", ".yaml", ".json", ".sh")):
self.file_list.addItem(fname) self.file_list.addItem(fname)
# Also surface subdirectories that hold per-container / # Image-format dirs (qcow2/, and any future format like the
# per-VM configs (lxc/, firecracker/). # in-house qcrows/) list their images + sidecar manifests
# with sizes; config dirs (lxc/, firecracker/) get a plain
# directory entry.
for fname in sorted(os.listdir(self.main.exports_root)): for fname in sorted(os.listdir(self.main.exports_root)):
if fname.startswith("."):
# transient .staging-* dirs from in-flight exports
continue
fpath = os.path.join(self.main.exports_root, fname) fpath = os.path.join(self.main.exports_root, fname)
if os.path.isdir(fpath): if not os.path.isdir(fpath):
continue
if fname in ("lxc", "firecracker"):
self.file_list.addItem(f"{fname}/ (directory)") self.file_list.addItem(f"{fname}/ (directory)")
continue
for sub in sorted(os.listdir(fpath)):
sub_path = os.path.join(fpath, sub)
if os.path.isfile(sub_path) and not sub.startswith("."):
size_mb = os.path.getsize(sub_path) // (1024 * 1024)
self.file_list.addItem(
f"{fname}/{sub} ({size_mb} MiB)"
)
else: else:
ContainerStacksTab = None ContainerStacksTab = None

View File

@ -612,13 +612,17 @@ if _HAS_QT:
mgr = InstallerManager( mgr = InstallerManager(
tools_root=self.main.tools_root, tools_root=self.main.tools_root,
base_bin_dir=self.main.base_bin_dir, base_bin_dir=self.main.base_bin_dir,
base_dir=self.main.base_dir,
) )
try: try:
if inst_type == "git_node": if inst_type == "git_node":
msg = mgr.install_git_node(pkg, tool_id, post_install) msg = mgr.install_git_node(
pkg, tool_id, post_install, entry=meta,
)
else: else:
msg = mgr.install_git( msg = mgr.install_git(
pkg, tool_id, post_install, env_overrides or None, pkg, tool_id, post_install, env_overrides or None,
entry=meta,
) )
self.main.log(msg, "GitRepo") self.main.log(msg, "GitRepo")
self.refresh() self.refresh()

View File

@ -4,7 +4,11 @@ Renders one tool (or skill:-prefixed behavior binding) inside the
Tools/Services page. Each row shows the service name, live status, Tools/Services page. Each row shows the service name, live status,
CPU load, port input, model selector (for engine/LLM-runtime services), CPU load, port input, model selector (for engine/LLM-runtime services),
Ollama pull controls, launcher buttons (CLI/GUI/Web), and Ollama pull controls, launcher buttons (CLI/GUI/Web), and
Install/Sync + Start/Stop action buttons. Install/Update + Start/Stop action buttons. The install button runs
preflight first: missing tools install to their map-aligned root
(dashboards/ for web UIs, tools/ otherwise); already-installed tools
update in place (registry ``update_cmd`` when declared, per-type
default otherwise).
All process management is delegated to All process management is delegated to
:class:`~ai_lsc.runtime.executor.RuntimeExecutor` -- this widget :class:`~ai_lsc.runtime.executor.RuntimeExecutor` -- this widget
@ -222,7 +226,7 @@ if _HAS_QT:
btn_api.clicked.connect(self._open_api_dialog) btn_api.clicked.connect(self._open_api_dialog)
layout.addWidget(btn_api) layout.addWidget(btn_api)
self.btn_update = QPushButton("Install / Sync") self.btn_update = QPushButton("Install / Update")
self.btn_update.setStyleSheet("background-color: #8e44ad;") self.btn_update.setStyleSheet("background-color: #8e44ad;")
self.btn_update.clicked.connect(self.smart_install) self.btn_update.clicked.connect(self.smart_install)
self._install_btn_original_text = self.btn_update.text() self._install_btn_original_text = self.btn_update.text()
@ -331,6 +335,7 @@ if _HAS_QT:
tool_id=self.tool_id, tool_id=self.tool_id,
ctx=ctx, ctx=ctx,
license_spdx=license_spdx, license_spdx=license_spdx,
registry_entry=self.meta,
) )
QTimer.singleShot( QTimer.singleShot(
0, lambda d=desc, t=self.tool_id: self._on_install_done(d, t) 0, lambda d=desc, t=self.tool_id: self._on_install_done(d, t)

View File

@ -85,6 +85,7 @@ class VerificationWorker(QThread if _HAS_QT else object): # type: ignore[misc]
pkg=installer.get("pkg", ""), pkg=installer.get("pkg", ""),
cmd=installer.get("cmd", ""), cmd=installer.get("cmd", ""),
filesystem=fs, filesystem=fs,
entry=meta,
) )
self.tool_done.emit(tool_id, result) self.tool_done.emit(tool_id, result)
completed += 1 completed += 1

View File

@ -4,14 +4,26 @@ AI-LSC — Centralised path definitions.
Every path in the application is derived from ``BASE_DIR`` Every path in the application is derived from ``BASE_DIR``
using :mod:`pathlib`. Import these in UI and orchestration code using :mod:`pathlib`. Import these in UI and orchestration code
instead of constructing paths with ``os.path.join`` ad-hoc. instead of constructing paths with ``os.path.join`` ad-hoc.
This module also owns the **install-root router** — the mapping that
decides where a tool's artifacts live under the canonical /mnt/AI
layout (``tools/`` for standalone compiles, ``dashboards/`` for web
UIs, ``runtime/`` for native binaries) — plus the multi-root
candidate list used for detection of pre-existing installs.
""" """
from __future__ import annotations from __future__ import annotations
import re
from pathlib import Path from pathlib import Path
from typing import Any
from ai_lsc.constants import BASE_DIR, REQUIRED_DIRS from ai_lsc.constants import BASE_DIR, REQUIRED_DIRS
# Only path-tree keys are auto-resolvable as launcher placeholders.
# `{port}` / `{model_arg}` are runtime values resolved separately.
_TOOL_ID_RE = re.compile(r"^[A-Za-z0-9_.:\-]+$")
def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]: def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
"""Return a dict of well-known absolute paths under *base_dir*. """Return a dict of well-known absolute paths under *base_dir*.
@ -41,19 +53,21 @@ def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
"datasets_root": Path("/mnt/AI/datasets"), # parent of wordlists/, huggingface/, github/ "datasets_root": Path("/mnt/AI/datasets"), # parent of wordlists/, huggingface/, github/
"pipelines_root": Path("/mnt/AI/pipelines"), # ETL / chunking / routing scripts "pipelines_root": Path("/mnt/AI/pipelines"), # ETL / chunking / routing scripts
"configs_root": Path("/mnt/AI/configs"), # app state + templated configs "configs_root": Path("/mnt/AI/configs"), # app state + templated configs
"registry_root": Path("/mnt/AI/registry"), # app-internal: ecosystem.json + manifests/ "registry_root": Path("/mnt/AI/configs/registry"), # app-internal: ecosystem.json
"agents_root": Path("/mnt/AI/agents"), # configs and chains for autonomous actors "agents_root": Path("/mnt/AI/agents"), # configs and chains for autonomous actors
"skills_root": Path("/mnt/AI/skills"), # 3rd-party integrations and tool wrappers "skills_root": Path("/mnt/AI/skills"), # 3rd-party integrations and tool wrappers
"projects_root": Path("/mnt/AI/projects"), # parent of active/, labs/, vault/ "projects_root": Path("/mnt/AI/projects"), # parent of active/, labs/, vault/
"blueprints_root": Path("/mnt/AI/blueprints"), # Dockerfiles / build contexts for Podman exports "blueprints_root": Path("/mnt/AI/blueprints"), # Dockerfiles / build contexts for Podman exports
"workspaces_root": Path("/mnt/AI/workspaces"), # Jupyter, OpenNotebook, etc. "workspaces_root": Path("/mnt/AI/workspaces"), # Jupyter, OpenNotebook, etc.
"dashboards_root": Path("/mnt/AI/dashboards"), # web UIs (Dashy, Open-WebUI, Hermes WebUI, etc.) "dashboards_root": Path("/mnt/AI/dashboards"), # web UIs (Dashy, Homepage, Open-WebUI, etc.)
"exports_root": Path("/mnt/AI/exports"), # parent of oci-images/ "exports_root": Path("/mnt/AI/exports"), # parent of oci-images/, qcow2/, qcrows/
"exports_qcow2": Path("/mnt/AI/exports/qcow2"), # VM disk images (qcow2 / legacy qcow)
"exports_qcrows": Path("/mnt/AI/exports/qcrows"),# QCrows VM container images (.qcrows)
"scripts_root": Path("/mnt/AI/scripts"), # system admin / maintenance automation "scripts_root": Path("/mnt/AI/scripts"), # system admin / maintenance automation
"logs_root": Path("/mnt/AI/logs"), "logs_root": Path("/mnt/AI/logs"),
"backends_root": Path("/mnt/AI/backends"), # S3/MinIO/Ceph connection profiles "backends_root": Path("/mnt/AI/backends"), # S3/MinIO/Ceph connection profiles
"distfiles_root": Path("/mnt/AI/distfiles"), # permanent local mirror of source tarballs "distfiles_root": Path("/mnt/AI/distfiles"), # permanent local mirror of source tarballs
"config_root": Path("/mnt/AI/configs"), # app state + templated configs "config_root": Path("/mnt/AI/configs"), # alias of configs_root
} }
""" """
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR) root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
@ -67,7 +81,11 @@ def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
"corpus_root": root / "corpus", "corpus_root": root / "corpus",
"datasets_root": root / "datasets", "datasets_root": root / "datasets",
"pipelines_root": root / "pipelines", "pipelines_root": root / "pipelines",
"registry_root": root / "registry", # App-internal registry state (ecosystem.json + user tweaks)
# lives under configs/ per the v3.4 file-system map. Pre-v3.4
# installs used <base>/registry — main_window migrates that
# directory on startup.
"registry_root": root / "configs" / "registry",
"agents_root": root / "agents", "agents_root": root / "agents",
"skills_root": root / "skills", "skills_root": root / "skills",
"projects_root": root / "projects", "projects_root": root / "projects",
@ -75,6 +93,8 @@ def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
"workspaces_root": root / "workspaces", "workspaces_root": root / "workspaces",
"dashboards_root": root / "dashboards", "dashboards_root": root / "dashboards",
"exports_root": root / "exports", "exports_root": root / "exports",
"exports_qcow2": root / "exports" / "qcow2",
"exports_qcrows": root / "exports" / "qcrows",
"scripts_root": root / "scripts", "scripts_root": root / "scripts",
"logs_root": root / "logs", "logs_root": root / "logs",
"backends_root": root / "backends", "backends_root": root / "backends",
@ -89,6 +109,121 @@ def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
} }
# ── Install-root router ─────────────────────────────────────────────
#
# The v3.4 file-system map gives every top-level directory a purpose:
# tools/ — standalone compiles (built from distfiles)
# runtime/ — native compiled binaries (ollama, llama.cpp, MinIO…)
# dashboards/ — web UIs and landing pages (Open-WebUI, LibreChat…)
#
# A tool's declared ``filesystem.install`` (a path relative to the base
# dir) is the single source of truth. When absent, the router picks a
# root by tool kind — but only when the launcher does not already pin
# the tool to ``{tools_root}`` via a ``cd {tools_root}/<id>`` command
# (a pinned launcher would break if the artifacts moved).
INSTALL_ROOT_DEFAULT = "tools"
INSTALL_ROOT_ROUTES: dict[str, str] = {
"web": "dashboards", # flags.has_web → web UI apps
}
def install_dir_for(
entry: dict[str, Any] | None,
tool_id: str = "",
base_dir: str | Path | None = None,
) -> Path:
"""Return the canonical install directory for a registry *entry*.
Resolution order:
1. ``filesystem.install`` declared on the entry (relative to the
base dir) — honoured verbatim.
2. ``flags.has_web`` is set AND the launcher does not reference
``{tools_root}`` → ``<base>/dashboards/<tool_id>/``.
3. Otherwise → ``<base>/tools/<tool_id>/`` (the historical
default, still correct for standalone compiles).
Falls back to ``tools/<tool_id>`` for empty / malformed entries.
"""
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
entry = entry or {}
rel = (entry.get("filesystem") or {}).get("install", "")
if rel:
# Normalise without allowing traversal outside the base dir.
parts = [p for p in Path(rel).parts if p not in (".", "..")]
clean = Path(*parts) if parts else Path("")
if str(clean) not in ("", "."):
return root / clean
if tool_id and not _TOOL_ID_RE.fullmatch(tool_id):
tool_id = ""
launcher_cmd = (entry.get("launcher") or {}).get("cmd", "") or ""
pins_tools_root = "{tools_root}" in launcher_cmd
flags = entry.get("flags") or {}
if (tool_id and flags.get("has_web")
and INSTALL_ROOT_ROUTES.get("web")
and not pins_tools_root):
return root / INSTALL_ROOT_ROUTES["web"] / tool_id
return root / INSTALL_ROOT_DEFAULT / (tool_id or "")
def candidate_install_dirs(
tool_id: str,
entry: dict[str, Any] | None = None,
base_dir: str | Path | None = None,
) -> list[Path]:
"""Return every directory an install of *tool_id* might live in.
Ordered best-candidate-first: the router's canonical dir, then the
other plausible roots (dashboards/, tools/), de-duplicated. Used
by installer preflight / verification / drift detection so tools
installed before the v3.4 routing change are still detected.
"""
if not tool_id or not _TOOL_ID_RE.fullmatch(tool_id):
return []
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
entry = entry or {}
cands: list[Path] = [install_dir_for(entry, tool_id, root)]
for dirname in (INSTALL_ROOT_ROUTES.get("web", "dashboards"),
INSTALL_ROOT_DEFAULT):
cands.append(root / dirname / tool_id)
# Pre-v3.4 git tools could land in a dir named after the repo
# (not the tool_id) — cover the common LibreChat-style casing too.
repo_hint = (entry.get("installer", {}) or {}).get("pkg", "")
if isinstance(repo_hint, str) and repo_hint.rstrip("/").endswith(".git"):
repo_name = repo_hint.rstrip("/").rsplit("/", 1)[-1][:-4]
if repo_name and _TOOL_ID_RE.fullmatch(repo_name):
cands.append(root / INSTALL_ROOT_DEFAULT / repo_name)
seen: set[Path] = set()
ordered: list[Path] = []
for c in cands:
if c not in seen:
seen.add(c)
ordered.append(c)
return ordered
def legacy_registry_dir(base_dir: str | Path | None = None) -> Path:
"""Pre-v3.4 location of the app-internal registry dir."""
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
return root / "registry"
def ensure_required_dirs(base_dir: str | Path | None = None) -> list[Path]:
"""Create every REQUIRED_DIRS entry under *base_dir*; return the list."""
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
created: list[Path] = []
for rel in REQUIRED_DIRS:
p = root / rel
p.mkdir(parents=True, exist_ok=True)
created.append(p)
return created
def resolve_launcher_cmd( def resolve_launcher_cmd(
cmd_template: str, cmd_template: str,
base_dir: str | Path | None = None, base_dir: str | Path | None = None,
@ -97,17 +232,30 @@ def resolve_launcher_cmd(
) -> str: ) -> str:
"""Resolve ``{placeholder}`` tokens in a launcher command template. """Resolve ``{placeholder}`` tokens in a launcher command template.
Recognised placeholders (case-sensitive): Every key of :func:`build_path_tree` is recognised as a
``{port}``, ``{base_dir}``, ``{tools_root}``, placeholder (``{base_dir}``, ``{tools_root}``, ``{dashboards_root}``,
``{models_root}``, ``{workspaces_root}``, ``{model_arg}`` ``{runtime_root}``, ``{configs_root}``, …), plus the runtime values
``{port}`` and ``{model_arg}``. Unknown placeholders are left
untouched so callers can chain their own substitution.
""" """
paths = build_path_tree(base_dir) paths = build_path_tree(base_dir)
resolved = cmd_template
for key, value in paths.items():
resolved = resolved.replace("{" + key + "}", str(value))
return ( return (
cmd_template resolved
.replace("{port}", str(port or "")) .replace("{port}", str(port or ""))
.replace("{base_dir}", str(paths["base_dir"]))
.replace("{tools_root}", str(paths["tools_root"]))
.replace("{models_root}", str(paths["models_root"]))
.replace("{workspaces_root}", str(paths["workspaces_root"]))
.replace("{model_arg}", model_arg) .replace("{model_arg}", model_arg)
) )
__all__ = [
"build_path_tree",
"install_dir_for",
"candidate_install_dirs",
"legacy_registry_dir",
"ensure_required_dirs",
"resolve_launcher_cmd",
"INSTALL_ROOT_DEFAULT",
"INSTALL_ROOT_ROUTES",
]