AI-LSC: AI - Local Stack Control
This commit is contained in:
parent
34d429345d
commit
d244d69ca6
201
CHANGES.md
201
CHANGES.md
|
|
@ -1,5 +1,204 @@
|
||||||
# AI-LSC Changelog
|
# AI-LSC Changelog
|
||||||
|
|
||||||
|
## v3.4 — File-system map alignment + QCOW2 exports + real updates
|
||||||
|
|
||||||
|
### 1. The base dir now follows the canonical file-system map
|
||||||
|
|
||||||
|
`constants.py` / `utils/paths.py` are the single source of truth for
|
||||||
|
the /mnt/AI layout, and the runtime code now actually honours it:
|
||||||
|
|
||||||
|
- **Install-root router** (`utils/paths.py::install_dir_for`): a
|
||||||
|
tool's declared `filesystem.install` is honoured verbatim; when
|
||||||
|
absent, web UIs (`flags.has_web`, launcher not pinned to
|
||||||
|
`{tools_root}`) route to `dashboards/<tool_id>/`, everything else
|
||||||
|
stays under `tools/<tool_id>/`. `runtime/installer.py` resolves
|
||||||
|
every per-tool destination through the router.
|
||||||
|
- **Multi-root detection**: `candidate_install_dirs()` probed by
|
||||||
|
installer preflight, verification, version detection, and the
|
||||||
|
main-window drift audit — so tools installed before the routing
|
||||||
|
change are still detected, verified, and **updated in place**
|
||||||
|
instead of being orphaned or duplicated.
|
||||||
|
- **Registry state moved** from `<base>/registry/` (off-map) to
|
||||||
|
`<base>/configs/registry/` (app state belongs under configs/ per
|
||||||
|
the map). `main_window._migrate_registry_dir()` migrates existing
|
||||||
|
installs once on startup; legacy dir removed when emptied.
|
||||||
|
- **`exports/qcow2/`** added to REQUIRED_DIRS (25 dirs now) alongside
|
||||||
|
`exports/oci-images/`.
|
||||||
|
- Ollama server probe order is now map-correct:
|
||||||
|
`runtime/ollama` → `tools/ollama` → legacy `ollama` / `bin/ollama`.
|
||||||
|
- Every launcher-template placeholder from the path tree resolves
|
||||||
|
everywhere (`{dashboards_root}`, `{runtime_root}`, `{configs_root}`,
|
||||||
|
`{datasets_root}`, …) — in `resolve_launcher_cmd`, the runtime
|
||||||
|
executor's `format_context`, env overrides, post_install commands,
|
||||||
|
and the stack-export placeholder chain.
|
||||||
|
|
||||||
|
### 2. QCOW2 / QCOW VM-disk exports (the missing export backend)
|
||||||
|
|
||||||
|
`stack/export.py::ContainerBackend.write_vm_image()` — a fully rootless
|
||||||
|
pipeline (no loop mounts, no root):
|
||||||
|
|
||||||
|
1. Stage a rootfs tree (`/opt/ai-lsc/stack.json` + compose file,
|
||||||
|
`/root/start.sh` boot script, `/etc/ai-lsc-release`) under
|
||||||
|
`exports/qcow2/.staging-*`.
|
||||||
|
2. Populate an ext4 filesystem straight from the staging dir with
|
||||||
|
`mkfs.ext4 -d` into a sparse raw image (auto-sized: staging usage
|
||||||
|
× 4 + headroom, min 2 GiB).
|
||||||
|
3. `qemu-img convert -c -O qcow2` (also `raw`; legacy `qcow` on
|
||||||
|
older qemu hosts — modern qemu ≥ ~10 refuses v1 writes and the
|
||||||
|
exporter surfaces a clean, actionable error suggesting qcow2).
|
||||||
|
4. Sidecar `<image>.manifest.json` with sha256, virtual size,
|
||||||
|
`qemu-img info`, tool list, and a boot hint (rootfs disk — pair
|
||||||
|
with an external kernel such as the Firecracker export's vmlinux,
|
||||||
|
or attach to any VM).
|
||||||
|
|
||||||
|
Tool detection is built in: missing `qemu-img` / `mkfs.ext4` raise
|
||||||
|
`FileNotFoundError` with the pacman/apt install hint. UI: two new
|
||||||
|
buttons on the Container Stacks tab ("Export -> QCOW2 Image",
|
||||||
|
"Export -> QCOW (legacy)"); the file list shows images with sizes.
|
||||||
|
|
||||||
|
Formats are registry-driven (`VM_IMAGE_FORMATS`, one output dir per
|
||||||
|
format under `exports/`).
|
||||||
|
|
||||||
|
### 2b. QCrows (.qcrows) VM container image export — the real format
|
||||||
|
|
||||||
|
QCrows (cue-crows) is the in-house **self-describing VM container
|
||||||
|
image format for Kata Containers**, master-implemented in the
|
||||||
|
**cockpit-kata** project (`qcrows-spec.md` v0.2.0 + the `qcrows-pack`
|
||||||
|
/ `qcrows-verify` / `qcrows-inspect` / `qcrows-export` /
|
||||||
|
`qcrows-initrd-regen` tools). (An earlier iteration of this changelog
|
||||||
|
misidentified qcrows as sorcery-go's Sovereign Bundle — that format is
|
||||||
|
now supported under its own `svb` key, below.)
|
||||||
|
|
||||||
|
`ContainerBackend.write_qcrows()` produces a spec-conformant archive:
|
||||||
|
|
||||||
|
```
|
||||||
|
<tar> (PAX per spec §2.1; .qcrows or .qcrows.gz)
|
||||||
|
├── metadata.toml generated manifest (qcrows-pack's template
|
||||||
|
│ incl. section order qcrows-inspect parses)
|
||||||
|
├── menu.toml Cockpit UI menu entry
|
||||||
|
├── hashes.sha256 sha256sum -c byte-compatible, ./-prefixed
|
||||||
|
├── rootfs.tar.gz the staged ai-lsc stack rootfs
|
||||||
|
├── boot-params.conf console=ttyS0 root=/dev/vda rw
|
||||||
|
├── build.toml ai-lsc provenance (generator, kernel origin)
|
||||||
|
├── spec.md per-image build guide
|
||||||
|
└── kernel/ REQUIRED (v0.2+): vmlinuz|vmlinux + config
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Kernel sourcing** (kernel is REQUIRED by v0.2 — same refusal
|
||||||
|
behaviour as qcrows-pack): `<base>/runtime/qcrows/` (canonical guest
|
||||||
|
kernel home: `vmlinuz|vmlinux` + `config`) → the firecracker dir's
|
||||||
|
`vmlinux` → host kernel fallback (`/boot/vmlinuz-<release>`, any
|
||||||
|
`/boot/vmlinuz-*` for Arch-style names; config from
|
||||||
|
`/boot/config-*`, `/usr/lib/modules/<ver>/config`, or
|
||||||
|
`/proc/config.gz`). Host fallback logs a warning — a Kata-tuned
|
||||||
|
guest kernel is recommended for real deployments.
|
||||||
|
- **Pre-flight Kata checks**: kernel-version probe mirrors
|
||||||
|
qcrows-pack (bzImage strings → `CONFIG_VERSION_SIGNATURE` →
|
||||||
|
"unknown"); the five required Kata config options
|
||||||
|
(`CONFIG_VSOCKETS`, `CONFIG_VIRTIO`, `CONFIG_VIRTIO_PCI`,
|
||||||
|
`CONFIG_DEVTMPFS`, `CONFIG_DEVTMPFS_MOUNT`) are checked and
|
||||||
|
shortfalls logged + recorded in the sidecar (qcrows-verify warns,
|
||||||
|
never fails, on these).
|
||||||
|
- **Master round-trip verified**: images built by ai-lsc pass
|
||||||
|
cockpit-kata's own `qcrows-verify` (11 PASS / 0 FAIL / exit 0, both
|
||||||
|
compressed and uncompressed) and render fully in `qcrows-inspect`.
|
||||||
|
Tar members carry the `./` prefix exactly like qcrows-pack's
|
||||||
|
`tar czf - ./*` output; hashes match `sha256sum -c` byte-for-byte.
|
||||||
|
- Output: `exports/qcrows/ai-lsc-stack-<stamp>.qcrows` (+ optional
|
||||||
|
`.gz`) with an ai-lsc sidecar manifest (sha256, kernel origin,
|
||||||
|
config warnings, verify hint). UI button
|
||||||
|
"Export -> QCrows Image (.qcrows)".
|
||||||
|
|
||||||
|
**Tool quirks found in cockpit-kata** (their code, reported not
|
||||||
|
modified — the ai-lsc writer avoids all of them):
|
||||||
|
|
||||||
|
1. `qcrows-pack` renames `initrd.cpio.gz` to `initrd.gz` — its
|
||||||
|
extension grep (`\.[^.]*$`) only captures the last suffix, and the
|
||||||
|
packer then derives both the on-disk name and `metadata.toml`'s
|
||||||
|
`initrd.path` from it; `qcrows-verify`'s initrd probe
|
||||||
|
(`initrd.img|initrd.cpio.gz|…`) then can't find it.
|
||||||
|
2. `qcrows-pack` writes GNU-format tar although spec §2.1 mandates
|
||||||
|
PAX (ai-lsc writes PAX).
|
||||||
|
3. `metadata.toml`'s generated `rootfs.path` assumes the source
|
||||||
|
tarball is named `rootfs.*` — other names produce a path that
|
||||||
|
doesn't match the staged file.
|
||||||
|
4. `qcrows-inspect`'s first-match key parsing reports kernel's
|
||||||
|
`included`/`size_mb` for the initrd/rootfs rows (visible in its
|
||||||
|
own output on its own packs).
|
||||||
|
|
||||||
|
### 2c. Sovereign Bundle (.svb) export (sorcery-go format)
|
||||||
|
|
||||||
|
Kept as its own format key (`svb`) after the qcrows identification
|
||||||
|
was corrected: `ContainerBackend.write_svb()` implements
|
||||||
|
`sorcery-sovereign-bundle-v1` (METADATA.json first entry, optional
|
||||||
|
64-byte ed25519 SIGNATURE.sig over the payload sha256, per-file
|
||||||
|
MANIFEST.txt, then payload) with deterministic `payload_sha`
|
||||||
|
(identical content → identical hash) and optional signing via
|
||||||
|
`AI_LSC_SVB_SIGNING_KEY` + the `cryptography` package. Output:
|
||||||
|
`exports/svb/*.svb`. Cross-verified with a Go verifier mirroring
|
||||||
|
sorcery-go's parse path — including the discovery that sorcery-go's
|
||||||
|
own `VerifySVBSignature` diverges from its builder (it hashes
|
||||||
|
concatenated entry contents instead of the inner tar.gz), so their
|
||||||
|
verifier can't validate their own bundles; the ai-lsc writer follows
|
||||||
|
the builder.
|
||||||
|
|
||||||
|
### 3. Updates actually update now
|
||||||
|
|
||||||
|
The registry's `installer.update_cmd` existed in the schema since
|
||||||
|
v3.0 but nothing executed it. New
|
||||||
|
`InstallerManager.update_tool()`:
|
||||||
|
|
||||||
|
- Registry `update_cmd` runs first, rendered with all path-tree
|
||||||
|
placeholders plus `{install_dir}`, cwd = the *detected* install
|
||||||
|
location (so pre-routing installs update where they live).
|
||||||
|
- Per-type defaults otherwise: `uv tool upgrade` (reinstall
|
||||||
|
step-down preserves `--with` extras), `pipx upgrade`, per-tool
|
||||||
|
venv `pip install -U`, `git pull --ff-only` (+ yarn rebuild for
|
||||||
|
git_node), `npm update`.
|
||||||
|
- `install_with_preflight()` now flows found-tools into
|
||||||
|
`update_tool()` by default — the UI's single Deploy/Update button
|
||||||
|
is a real in-place updater instead of returning "already
|
||||||
|
installed; use force=True".
|
||||||
|
- New `RuntimeExecutor.update_tool()` facade; service rows pass the
|
||||||
|
full registry entry through (`registry_entry=`) so routing,
|
||||||
|
update_cmd, and post_install all flow from one source.
|
||||||
|
|
||||||
|
### 4. Open WebUI install/start fixes
|
||||||
|
|
||||||
|
- **Step-down data loss fixed**: `install_uv` used to fall back to
|
||||||
|
`pipx install open-webui` — silently dropping `--with
|
||||||
|
audioop-lts`, producing an install that later crashed importing
|
||||||
|
`audioop` on Python 3.13. The chain is now: uv (default Python)
|
||||||
|
→ uv with a managed CPython 3.12 (`uv` fetches it automatically —
|
||||||
|
sidesteps wheels not yet rebuilt for newer Pythons) → per-tool pip
|
||||||
|
venv installing the package **plus every `--with` extra**.
|
||||||
|
- Launcher pre-creates `workspaces/openwebui` (DATA_DIR) alongside
|
||||||
|
the existing `configs/auth` bootstrap so first start can't fail on
|
||||||
|
a missing data dir.
|
||||||
|
|
||||||
|
### 5. LibreChat install/start fixes
|
||||||
|
|
||||||
|
- **Fresh-clone start bug**: `yarn backend` aborts without `.env`,
|
||||||
|
and production mode needs `yarn build` assets that plain
|
||||||
|
`yarn install` never produces. The installer entry now carries
|
||||||
|
`post_install: "cp -n .env.example .env 2>/dev/null || true;
|
||||||
|
yarn build"` — runs after clone/update in the project dir.
|
||||||
|
- **Map-aligned paths**: installs to `dashboards/librechat/`
|
||||||
|
(was `tools/librechat`), data → `workspaces/librechat` (the
|
||||||
|
off-map root-level `data/` dir is gone), config stays
|
||||||
|
`configs/librechat`, launcher uses `{dashboards_root}`. Existing
|
||||||
|
`tools/librechat` clones are detected and updated in place.
|
||||||
|
- `agents/librechat_config.py` docstring path corrected to the
|
||||||
|
dashboards location.
|
||||||
|
|
||||||
|
### 6. Housekeeping
|
||||||
|
|
||||||
|
- Version 3.3.0 → 3.4.0 (`pyproject.toml`, `constants.APP_VERSION`).
|
||||||
|
- Container Stacks tab lists qcow2 images (+ manifests, sizes) and
|
||||||
|
excludes the qcow2 dir from the generic directory listing.
|
||||||
|
- Verification tab passes the full registry entry so multi-root
|
||||||
|
detection applies to compliance checks too.
|
||||||
|
|
||||||
## v3.3 — Registry git-URL sync + bandwidth-aware git-pull + ComfyUI
|
## v3.3 — Registry git-URL sync + bandwidth-aware git-pull + ComfyUI
|
||||||
|
|
||||||
Patch release over v3.2. No schema changes, no UI reorganisation, no
|
Patch release over v3.2. No schema changes, no UI reorganisation, no
|
||||||
|
|
@ -123,7 +322,7 @@ bandwidth-saving behaviour the user asked for.
|
||||||
|
|
||||||
- `pyproject.toml`: 3.2.0 → 3.3.0
|
- `pyproject.toml`: 3.2.0 → 3.3.0
|
||||||
- `src/ai_lsc/constants.py`: `APP_VERSION` 3.2.0 → 3.3.0
|
- `src/ai_lsc/constants.py`: `APP_VERSION` 3.2.0 → 3.3.0
|
||||||
(`APP_CODENAME` unchanged: `"Decalogue"`)
|
(`APP_CODENAME` unchanged: `"Ankh of Jah"`)
|
||||||
- New file: `docs/REGISTRY-URLS-v3.3.0.md` (per-file old → new URL
|
- New file: `docs/REGISTRY-URLS-v3.3.0.md` (per-file old → new URL
|
||||||
table, also serves as a release audit trail).
|
table, also serves as a release audit trail).
|
||||||
- `gitcommit` rewritten for v3.3.0.
|
- `gitcommit` rewritten for v3.3.0.
|
||||||
|
|
|
||||||
|
|
@ -187,3 +187,66 @@ WEBUI_DATA_DIR={workspaces_root}/hermes-webui OLLAMA_BASE_URL=http://localhost:1
|
||||||
|
|
||||||
The ai-lsc runtime resolves env-prefix shells via `shlex.split()` on the
|
The ai-lsc runtime resolves env-prefix shells via `shlex.split()` on the
|
||||||
launcher cmd, so the env-assignment form is parsed correctly.
|
launcher cmd, so the env-assignment form is parsed correctly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Open WebUI Python-3.13 audioop fix (follow-up #2)
|
||||||
|
|
||||||
|
After the launcher fix landed, open-webui started successfully but crashed
|
||||||
|
at import time inside the uv-managed tool env:
|
||||||
|
|
||||||
|
```
|
||||||
|
ModuleNotFoundError: No module named 'audioop'
|
||||||
|
…
|
||||||
|
ModuleNotFoundError: No module named 'pyaudioop'
|
||||||
|
```
|
||||||
|
|
||||||
|
Root cause: Python 3.13 removed the `audioop` stdlib module (PEP 594).
|
||||||
|
`pydub` (an open-webui dep) imports `audioop` and falls back to
|
||||||
|
`pyaudioop` on ImportError; neither was present in the uv-managed
|
||||||
|
open-webui env at `~/.local/share/uv/tools/open-webui/`.
|
||||||
|
|
||||||
|
### Fix
|
||||||
|
|
||||||
|
`uv tool install` supports `--with <extra>` for additional dependencies.
|
||||||
|
The `pkg` field in the registry now carries that flag, and `install_uv`
|
||||||
|
was hardened to shlex-split `pkg` so the flag passes through verbatim:
|
||||||
|
|
||||||
|
- `src/ai_lsc/runtime/installer.py` `install_uv` — shlex-splits `pkg`;
|
||||||
|
validates only the first token as the package name; passes remaining
|
||||||
|
tokens as additional argv to `uv tool install`. Falls back to pipx
|
||||||
|
using just the first token.
|
||||||
|
- `src/ai_lsc/runtime/installer.py` `_detect_installation` /
|
||||||
|
`_binary_name` / `_detect_version` — all now take only the first
|
||||||
|
shlex token of `pkg` when looking up the binary on PATH or building
|
||||||
|
version-detection commands. (Without this, preflight would look for
|
||||||
|
a binary literally named `open-webui --with audioop-lts`.)
|
||||||
|
- `src/ai_lsc/registry/layers/user_interfaces.py` — both `openwebui`
|
||||||
|
and `hermes_webui` installer.pkg changed from `"open-webui"` to
|
||||||
|
`"open-webui --with audioop-lts"`. Inline comment explains the
|
||||||
|
Python-3.13 / PEP-594 rationale.
|
||||||
|
- `src/ai_lsc/registry/defaults.py` — openwebui entry updated to match.
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
- 95 Python files AST-parse cleanly.
|
||||||
|
- Registry loads 187 tools; `validate_registry()` reports 0 errors.
|
||||||
|
- `shlex.split("open-webui --with audioop-lts")` yields the expected
|
||||||
|
argv: `["open-webui", "--with", "audioop-lts"]`.
|
||||||
|
- `_binary_name("open-webui --with audioop-lts", "uv")` resolves to
|
||||||
|
`"open_webui"` (the actual on-PATH binary name).
|
||||||
|
|
||||||
|
### Operator workaround for existing installs
|
||||||
|
|
||||||
|
If open-webui was installed before this fix landed, the audioop-lts
|
||||||
|
backport can be added in place:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
~/.local/share/uv/tools/open-webui/bin/python -m pip install audioop-lts
|
||||||
|
```
|
||||||
|
|
||||||
|
Or reinstall via the new pkg spec:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv tool install --force "open-webui --with audioop-lts"
|
||||||
|
```
|
||||||
|
|
|
||||||
|
|
@ -5,12 +5,12 @@
|
||||||
<h1 align="center">AI - Local Stack Control</h1>
|
<h1 align="center">AI - Local Stack Control</h1>
|
||||||
|
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<strong>v3.1.1 — Codename: Ankh of Jah (local-coder-mesh build)</strong><br>
|
<strong>v3.4.0 — Codename: Ankh of Jah</strong><br>
|
||||||
Author: <a href="https://dcos.net">Jeremy Anderson</a> <<a href="mailto:info@dcos.net">info@dcos.net</a>><br>
|
Author: <a href="https://dcos.net">Jeremy Anderson</a> <<a href="mailto:info@dcos.net">info@dcos.net</a>><br>
|
||||||
Web: <a href="https://dcos.net">https://dcos.net</a>
|
Web: <a href="https://dcos.net">https://dcos.net</a>
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
<p align="center"><em>This build includes the local-coder-mesh integration — see <a href="CHANGES.md">CHANGES.md</a> for the full list of changes vs upstream v3.1.</em></p>
|
<p align="center"><em>Includes the local-coder-mesh integration, 10-layer taxonomy migration, MoE QA pass, and openhuman/openwebui hardening — see <a href="CHANGES.md">CHANGES.md</a> for the full version history.</em></p>
|
||||||
|
|
||||||
<p align="center">
|
<p align="center">
|
||||||
A PySide6 desktop application for orchestrating local AI/ML tool stacks across a 10-layer architecture.
|
A PySide6 desktop application for orchestrating local AI/ML tool stacks across a 10-layer architecture.
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""AI Local Stack Control v3.1 — Ankh of Jah
|
"""AI Local Stack Control v3.3.0 — Ankh of Jah
|
||||||
|
|
||||||
Direct launcher. Run from the project root:
|
Direct launcher. Run from the project root:
|
||||||
|
|
||||||
|
|
|
||||||
10
bootstrap.sh
10
bootstrap.sh
|
|
@ -1,6 +1,6 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# ──────────────────────────────────────────────────────────────
|
# ──────────────────────────────────────────────────────────────
|
||||||
# AI Local Stack Control v3.1.1 — Ankh of Jah
|
# AI Local Stack Control v3.3.0 — Ankh of Jah
|
||||||
# Bootstrap Script
|
# Bootstrap Script
|
||||||
#
|
#
|
||||||
# Fully portable: works wherever the tarball lands.
|
# Fully portable: works wherever the tarball lands.
|
||||||
|
|
@ -46,7 +46,7 @@ export AI_LSC_BASE_DIR="$AI_BASE"
|
||||||
|
|
||||||
echo ""
|
echo ""
|
||||||
echo -e "${BOLD}╔══════════════════════════════════════════════════════╗${NC}"
|
echo -e "${BOLD}╔══════════════════════════════════════════════════════╗${NC}"
|
||||||
echo -e "${BOLD}║ AI Local Stack Control v3.1.1 — Ankh of Jah ║${NC}"
|
echo -e "${BOLD}║ AI Local Stack Control v3.3.0 — Ankh of Jah ║${NC}"
|
||||||
echo -e "${BOLD}╚══════════════════════════════════════════════════════╝${NC}"
|
echo -e "${BOLD}╚══════════════════════════════════════════════════════╝${NC}"
|
||||||
echo ""
|
echo ""
|
||||||
echo -e "${CYAN} Project root : ${SCRIPT_DIR}${NC}"
|
echo -e "${CYAN} Project root : ${SCRIPT_DIR}${NC}"
|
||||||
|
|
@ -316,12 +316,16 @@ else
|
||||||
ERRORS=$((ERRORS + 1))
|
ERRORS=$((ERRORS + 1))
|
||||||
fi
|
fi
|
||||||
|
|
||||||
for cmd in ollama podman docker tmux ripgrep fd tree-sitter; do
|
for cmd in ollama podman docker tmux ripgrep fd tree-sitter qemu-img; do
|
||||||
if command -v "$cmd" &>/dev/null; then
|
if command -v "$cmd" &>/dev/null; then
|
||||||
info "${cmd} — found"
|
info "${cmd} — found"
|
||||||
|
else
|
||||||
|
if [ "$cmd" = "qemu-img" ]; then
|
||||||
|
warn "qemu-img — not found (needed for QCOW2 VM-image exports: pacman -S qemu-img / apt install qemu-utils)"
|
||||||
else
|
else
|
||||||
warn "${cmd} — not found (optional)"
|
warn "${cmd} — not found (optional)"
|
||||||
fi
|
fi
|
||||||
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
# ── Summary ───────────────────────────────────────────────────
|
# ── Summary ───────────────────────────────────────────────────
|
||||||
|
|
|
||||||
|
|
@ -92,7 +92,7 @@ Intentionally NOT touched:
|
||||||
- 3 registry-only git tools with real-looking URLs already in
|
- 3 registry-only git tools with real-looking URLs already in
|
||||||
place (goose -> block/goose, nvidia_agent_skills ->
|
place (goose -> block/goose, nvidia_agent_skills ->
|
||||||
NVIDIA/agent-skills, picode -> jasonjmcghee/picode.git).
|
NVIDIA/agent-skills, picode -> jasonjmcghee/picode.git).
|
||||||
- APP_CODENAME ("Decalogue") and all historical codename
|
- APP_CODENAME ("Ankh of Jah") and all historical codename
|
||||||
references in docstrings / UI labels / README banner / bootstrap
|
references in docstrings / UI labels / README banner / bootstrap
|
||||||
banner — left as-is per user's clarification.
|
banner — left as-is per user's clarification.
|
||||||
|
|
||||||
|
|
@ -102,7 +102,7 @@ removed from the registry.
|
||||||
Versioning:
|
Versioning:
|
||||||
- pyproject.toml: 3.2.0 -> 3.3.0
|
- pyproject.toml: 3.2.0 -> 3.3.0
|
||||||
- src/ai_lsc/constants.py: APP_VERSION 3.2.0 -> 3.3.0
|
- src/ai_lsc/constants.py: APP_VERSION 3.2.0 -> 3.3.0
|
||||||
(APP_CODENAME unchanged: "Decalogue")
|
(APP_CODENAME unchanged: "Ankh of Jah")
|
||||||
- docs/REGISTRY-URLS-v3.3.0.md: per-file old -> new URL table.
|
- docs/REGISTRY-URLS-v3.3.0.md: per-file old -> new URL table.
|
||||||
|
|
||||||
Verification:
|
Verification:
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "ai-lsc"
|
name = "ai-lsc"
|
||||||
version = "3.3.0"
|
version = "3.4.0"
|
||||||
description = "AI Local Stack Control — PySide6 desktop app for orchestrating local AI/ML tool stacks"
|
description = "AI Local Stack Control — PySide6 desktop app for orchestrating local AI/ML tool stacks"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
license = {text = "AGPL-3.0-or-later"}
|
license = {text = "AGPL-3.0-or-later"}
|
||||||
|
|
|
||||||
|
|
@ -73,7 +73,7 @@ print(f'Registry loaded: {len(DEFAULT_REGISTRY)} tools')
|
||||||
print(f'Validation errors: {len(errors)}')
|
print(f'Validation errors: {len(errors)}')
|
||||||
"
|
"
|
||||||
|
|
||||||
# Expected output (v3.1):
|
# Expected output (v3.4.0):
|
||||||
# Registry loaded: 125 tools
|
# Registry loaded: 125 tools
|
||||||
# Validation errors: 0
|
# Validation errors: 0
|
||||||
```
|
```
|
||||||
|
|
@ -253,6 +253,6 @@ for f in os.listdir('ai_lsc/registry/layers'):
|
||||||
- Try different **Stack Templates** to find the right combination for your workflow
|
- Try different **Stack Templates** to find the right combination for your workflow
|
||||||
- Set up the **Skills Console** to extend your tool capabilities
|
- Set up the **Skills Console** to extend your tool capabilities
|
||||||
- Use **Code Analysis** to inspect and understand your project dependencies
|
- Use **Code Analysis** to inspect and understand your project dependencies
|
||||||
- Read [CHANGES.md](CHANGES.md) for the full v3.1 changelog
|
- Read [CHANGES.md](CHANGES.md) for the full version history (v3.0 → v3.4.0)
|
||||||
- Read [whatremains.txt](whatremains.txt) for known deferred items (curl|sh installers, etc.)
|
- Read [whatremains.txt](whatremains.txt) for known deferred items (curl|sh installers, etc.)
|
||||||
- Read [docs/ADR-002-pipeline-ticker.md](docs/ADR-002-pipeline-ticker.md) and [docs/ADR-003-workspace-tab.md](docs/ADR-003-workspace-tab.md) for the design rationale behind the two new widgets
|
- Read [docs/ADR-002-pipeline-ticker.md](docs/ADR-002-pipeline-ticker.md) and [docs/ADR-003-workspace-tab.md](docs/ADR-003-workspace-tab.md) for the design rationale behind the two new widgets
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
"""
|
"""
|
||||||
AI Local Stack Control v3.1 — Release codename: Ankh of Jah.
|
AI Local Stack Control v3.3.0 — Release codename: Ankh of Jah.
|
||||||
|
|
||||||
Extracted from the monolithic ``ai_lsc_v11.py`` in incremental phases.
|
Extracted from the monolithic ``ai_lsc_v11.py`` in incremental phases.
|
||||||
Currently contains:
|
Currently contains:
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,7 @@ Usage
|
||||||
config.set_ollama_endpoint(ollama_port=11434)
|
config.set_ollama_endpoint(ollama_port=11434)
|
||||||
config.set_litellm_endpoint(litellm_port=4000)
|
config.set_litellm_endpoint(litellm_port=4000)
|
||||||
config.set_tool_schemas(tool_schemas)
|
config.set_tool_schemas(tool_schemas)
|
||||||
config.save("/mnt/AI/runtime/librechat/config.yaml")
|
config.save("/mnt/AI/dashboards/librechat")
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
|
||||||
|
|
@ -1,9 +1,9 @@
|
||||||
"""
|
"""
|
||||||
AI-LSC v3.1 — Application-wide constants.
|
AI-LSC v3.3.0 — Application-wide constants.
|
||||||
Release codename: Ankh of Jah
|
Release codename: Ankh of Jah
|
||||||
Pure data: file names, schema version, required directories, default ports,
|
Pure data: file names, schema version, required directories, default ports,
|
||||||
status styles, log colours, service licences, tree-skip patterns,
|
status styles, log colours, service licences, tree-skip patterns,
|
||||||
and the reorganized 10-layer navigation layer order.
|
and the 10-layer navigation layer order.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
|
|
@ -15,8 +15,8 @@ BASE_DIR: str = os.environ.get("AI_LSC_BASE_DIR", "/mnt/AI")
|
||||||
CANONICAL_BASE_DIR: str = BASE_DIR
|
CANONICAL_BASE_DIR: str = BASE_DIR
|
||||||
|
|
||||||
# ── Filenames ────────────────────────────────────────────────────────────
|
# ── Filenames ────────────────────────────────────────────────────────────
|
||||||
APP_VERSION: str = "3.3.0"
|
APP_VERSION: str = "3.4.0"
|
||||||
APP_CODENAME: str = "Decalogue"
|
APP_CODENAME: str = "Ankh of Jah"
|
||||||
APP_DISPLAY_NAME: str = f"AI - Local Stack Control v{APP_VERSION} - http://dcos.net"
|
APP_DISPLAY_NAME: str = f"AI - Local Stack Control v{APP_VERSION} - http://dcos.net"
|
||||||
CONFIG_FILE: str = "controller_config.json"
|
CONFIG_FILE: str = "controller_config.json"
|
||||||
APP_ICON_FILE: str = "ai-lsc-logo.png"
|
APP_ICON_FILE: str = "ai-lsc-logo.png"
|
||||||
|
|
@ -51,6 +51,8 @@ REQUIRED_DIRS: list[str] = [
|
||||||
"dashboards", # web UIs and landing pages (Dashy, Open-WebUI, …)
|
"dashboards", # web UIs and landing pages (Dashy, Open-WebUI, …)
|
||||||
"tools", # standalone compiles (built from distfiles)
|
"tools", # standalone compiles (built from distfiles)
|
||||||
"exports/oci-images", # finalized Podman .tar snapshots (-> MinIO registry)
|
"exports/oci-images", # finalized Podman .tar snapshots (-> MinIO registry)
|
||||||
|
"exports/qcow2", # VM disk-image exports (qemu qcow2 / legacy qcow)
|
||||||
|
"exports/qcrows", # QCrows VM container images (.qcrows, spec v0.2)
|
||||||
"scripts", # system admin / maintenance automation for the stack
|
"scripts", # system admin / maintenance automation for the stack
|
||||||
"logs", # system, runtime, and pipeline service logs
|
"logs", # system, runtime, and pipeline service logs
|
||||||
]
|
]
|
||||||
|
|
@ -158,11 +160,14 @@ NAV_LAYER_ORDER: list[str] = [
|
||||||
]
|
]
|
||||||
|
|
||||||
# ── Ollama server candidate paths (probed in order) ────────────────
|
# ── Ollama server candidate paths (probed in order) ────────────────
|
||||||
|
# Map-aligned order: runtime/ (native binaries) is probed first;
|
||||||
|
# the legacy root-level and bin/ locations remain as fallbacks for
|
||||||
|
# pre-v3.4 installs.
|
||||||
OLLAMA_SERVER_CANDIDATES: list[str] = [
|
OLLAMA_SERVER_CANDIDATES: list[str] = [
|
||||||
"ollama", # /mnt/AI/ollama
|
"runtime/ollama", # /mnt/AI/runtime/ollama (canonical)
|
||||||
"tools/ollama", # /mnt/AI/tools/ollama
|
"tools/ollama", # /mnt/AI/tools/ollama
|
||||||
"runtime/ollama", # /mnt/AI/runtime/ollama
|
"ollama", # /mnt/AI/ollama (legacy)
|
||||||
"bin/ollama" # /mnt/AI/bin/ollama
|
"bin/ollama" # /mnt/AI/bin/ollama (legacy)
|
||||||
]
|
]
|
||||||
|
|
||||||
# ── Model tier routing (reserved for v4.0 agentic layer) ──────────
|
# ── Model tier routing (reserved for v4.0 agentic layer) ──────────
|
||||||
|
|
|
||||||
|
|
@ -796,7 +796,10 @@ DEFAULT_REGISTRY: dict = {
|
||||||
},
|
},
|
||||||
"launcher": {
|
"launcher": {
|
||||||
"type": 'tmux',
|
"type": 'tmux',
|
||||||
"cmd": 'OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve',
|
# Force ollama to use /mnt/AI/models/ollama (not ~/.ollama/models)
|
||||||
|
# and put its identity-key + runtime state under /mnt/AI/.ollama.
|
||||||
|
# See inference.py ollama entry for full rationale.
|
||||||
|
"cmd": 'mkdir -p {base_dir}/.ollama && HOME={base_dir} OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve',
|
||||||
"default_port": 11434,
|
"default_port": 11434,
|
||||||
},
|
},
|
||||||
"deps": [
|
"deps": [
|
||||||
|
|
@ -2905,19 +2908,25 @@ DEFAULT_REGISTRY: dict = {
|
||||||
"type": 'git_node',
|
"type": 'git_node',
|
||||||
"pkg": 'https://github.com/danny-avila/LibreChat.git',
|
"pkg": 'https://github.com/danny-avila/LibreChat.git',
|
||||||
"update_cmd": 'git pull --ff-only && yarn install && yarn build',
|
"update_cmd": 'git pull --ff-only && yarn install && yarn build',
|
||||||
|
# Two known fresh-clone start bugs, fixed at install time:
|
||||||
|
# 1. `yarn backend` aborts without an .env file — seed one
|
||||||
|
# from the shipped example (never clobber an existing one).
|
||||||
|
# 2. production mode requires the client assets from
|
||||||
|
# `yarn build` — plain `yarn install` doesn't produce them.
|
||||||
|
"post_install": 'cp -n .env.example .env 2>/dev/null || true; yarn build',
|
||||||
},
|
},
|
||||||
"launcher": {
|
"launcher": {
|
||||||
"type": 'tmux',
|
"type": 'tmux',
|
||||||
"cmd": 'cd {tools_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend',
|
"cmd": 'cd {dashboards_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend',
|
||||||
"default_port": 3080,
|
"default_port": 3080,
|
||||||
},
|
},
|
||||||
"deps": [
|
"deps": [
|
||||||
'ollama',
|
'ollama',
|
||||||
],
|
],
|
||||||
"filesystem": {
|
"filesystem": {
|
||||||
"install": 'tools/librechat',
|
"install": 'dashboards/librechat',
|
||||||
"config": 'configs/librechat',
|
"config": 'configs/librechat',
|
||||||
"data": 'data/librechat',
|
"data": 'workspaces/librechat',
|
||||||
"logs": 'logs/librechat',
|
"logs": 'logs/librechat',
|
||||||
},
|
},
|
||||||
"description": 'Default agent interface with native tool execution, system files parsing, and multi-user configurations.',
|
"description": 'Default agent interface with native tool execution, system files parsing, and multi-user configurations.',
|
||||||
|
|
@ -3125,21 +3134,30 @@ DEFAULT_REGISTRY: dict = {
|
||||||
"category": 'Extensible Interface',
|
"category": 'Extensible Interface',
|
||||||
"installer": {
|
"installer": {
|
||||||
"type": 'uv',
|
"type": 'uv',
|
||||||
"pkg": 'open-webui',
|
# open-webui's pydub dep imports `audioop`, which Python 3.13
|
||||||
|
# removed from the stdlib (PEP 594). The `audioop-lts` backport
|
||||||
|
# restores it; --with pulls it into the tool's isolated env.
|
||||||
|
"pkg": 'open-webui --with audioop-lts',
|
||||||
"env_overrides": {
|
"env_overrides": {
|
||||||
'OPEN_WEBUI_CONFIG_DIR': '{base_dir}/configs/openwebui',
|
'OPEN_WEBUI_CONFIG_DIR': '{base_dir}/configs/openwebui',
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
"launcher": {
|
"launcher": {
|
||||||
"type": 'tmux',
|
"type": 'tmux',
|
||||||
"cmd": 'WEBUI_DATA_DIR={workspaces_root}/openwebui open-webui serve --port {port}',
|
# Minimal launcher — see user_interfaces.py openwebui entry for
|
||||||
|
# the full rationale. Summary: open-webui auto-discovers ollama
|
||||||
|
# at http://localhost:11434 (do NOT set OLLAMA_BASE_URL here).
|
||||||
|
# DATA_DIR replaces the removed --data-dir CLI flag. cd to
|
||||||
|
# configs/auth/ so the auto-generated .webui_secret_key lands
|
||||||
|
# in the canonical private-data dir, not the ai-lsc run dir.
|
||||||
|
"cmd": 'mkdir -p {base_dir}/configs/auth {workspaces_root}/openwebui && chmod 700 {base_dir}/configs/auth && cd {base_dir}/configs/auth && DATA_DIR={workspaces_root}/openwebui open-webui serve --port {port}',
|
||||||
"default_port": 8080,
|
"default_port": 8080,
|
||||||
},
|
},
|
||||||
"deps": [
|
"deps": [
|
||||||
'ollama',
|
'ollama',
|
||||||
],
|
],
|
||||||
"filesystem": {
|
"filesystem": {
|
||||||
"install": 'tools/openwebui',
|
"install": 'dashboards/openwebui',
|
||||||
"config": 'configs/openwebui',
|
"config": 'configs/openwebui',
|
||||||
"data": 'workspaces/openwebui',
|
"data": 'workspaces/openwebui',
|
||||||
"logs": 'logs/openwebui',
|
"logs": 'logs/openwebui',
|
||||||
|
|
|
||||||
|
|
@ -35,7 +35,24 @@ TOOLS: dict[str, dict] = {
|
||||||
},
|
},
|
||||||
"launcher": {
|
"launcher": {
|
||||||
"type": "tmux",
|
"type": "tmux",
|
||||||
"cmd": "OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve",
|
# Force ollama to use /mnt/AI/models/ollama (not ~/.ollama/models)
|
||||||
|
# and put its identity-key + runtime state under /mnt/AI/.ollama
|
||||||
|
# (not ~/.ollama). We set HOME={base_dir} *only for ollama* —
|
||||||
|
# this is safe because ollama doesn't depend on uv-tool/pipx
|
||||||
|
# (which are $HOME-relative). For uv-tool-installed tools like
|
||||||
|
# open-webui, we CANNOT override HOME globally or the tool's
|
||||||
|
# own venv lookup breaks; each tool that needs a HOME redirect
|
||||||
|
# must opt in via its own launcher cmd.
|
||||||
|
#
|
||||||
|
# OLLAMA_HOST=0.0.0.0:{port} — bind to all interfaces so
|
||||||
|
# open-webui and other tools on the box can reach it.
|
||||||
|
# OLLAMA_MODELS — pins the models dir to the canonical ai-lsc
|
||||||
|
# location; this is what makes `ollama list` and open-webui's
|
||||||
|
# model picker see the same set.
|
||||||
|
# mkdir -p {base_dir}/.ollama — pre-creates the dotdir ollama
|
||||||
|
# expects so it doesn't trip on a broken ~/.ollama state on
|
||||||
|
# the host (e.g. a stale file where a dir should be).
|
||||||
|
"cmd": "mkdir -p {base_dir}/.ollama && HOME={base_dir} OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve",
|
||||||
"default_port": 11434
|
"default_port": 11434
|
||||||
},
|
},
|
||||||
"deps": [],
|
"deps": [],
|
||||||
|
|
|
||||||
|
|
@ -20,16 +20,46 @@ TOOLS: dict[str, dict] = {
|
||||||
"category": 'Extensible Interface',
|
"category": 'Extensible Interface',
|
||||||
"installer": {
|
"installer": {
|
||||||
"type": "uv",
|
"type": "uv",
|
||||||
"pkg": "open-webui"
|
# open-webui's pydub dep imports `audioop`, which Python 3.13
|
||||||
|
# removed from the stdlib (PEP 594). The `audioop-lts` backport
|
||||||
|
# restores it; --with pulls it into the tool's isolated env.
|
||||||
|
"pkg": "open-webui --with audioop-lts"
|
||||||
},
|
},
|
||||||
"launcher": {
|
"launcher": {
|
||||||
|
# Minimal launcher — open-webui auto-discovers ollama at
|
||||||
|
# http://localhost:11434 (its built-in default). Do NOT add
|
||||||
|
# OLLAMA_BASE_URL / CORS_ORIGINS / ENABLE_DIRECT_CONNECTIONS /
|
||||||
|
# WEBUI_SECRET_KEY env vars — past iterations proved they break
|
||||||
|
# model discovery. open-webui handles all of these itself when
|
||||||
|
# left alone.
|
||||||
|
#
|
||||||
|
# Two things we DO control:
|
||||||
|
# 1. DATA_DIR — points the sqlite db + uploads at the
|
||||||
|
# canonical workspaces dir. (The old --data-dir CLI flag
|
||||||
|
# was removed upstream; the env var is the replacement.)
|
||||||
|
# 2. cd to {base_dir}/configs/auth/ before launching —
|
||||||
|
# open-webui writes its auto-generated .webui_secret_key
|
||||||
|
# to Path.cwd() (see backend/open_webui/__init__.py
|
||||||
|
# KEY_FILE). Running from configs/auth/ keeps the secret
|
||||||
|
# out of the ai-lsc run dir and in the canonical
|
||||||
|
# private-data location, with no env-var games.
|
||||||
"type": "tmux",
|
"type": "tmux",
|
||||||
"cmd": "WEBUI_DATA_DIR={workspaces_root}/openwebui open-webui serve --port {port}",
|
"cmd": "mkdir -p {base_dir}/configs/auth {workspaces_root}/openwebui && chmod 700 {base_dir}/configs/auth && cd {base_dir}/configs/auth && DATA_DIR={workspaces_root}/openwebui open-webui serve --port {port}",
|
||||||
"default_port": 8080
|
"default_port": 8080
|
||||||
},
|
},
|
||||||
"deps": [
|
"deps": [
|
||||||
"ollama"
|
"ollama"
|
||||||
],
|
],
|
||||||
|
"filesystem": {
|
||||||
|
# Map-aligned per the v3.4 file-system map: the web UI app
|
||||||
|
# marker lives under dashboards/, config under configs/, and
|
||||||
|
# the sqlite/uploads DATA_DIR under workspaces/. The uv-managed
|
||||||
|
# binary itself lives in tools/.uv/bin (on the managed PATH).
|
||||||
|
"install": "dashboards/openwebui",
|
||||||
|
"config": "configs/openwebui",
|
||||||
|
"data": "workspaces/openwebui",
|
||||||
|
"logs": "logs/openwebui"
|
||||||
|
},
|
||||||
"description": "Extensible frontend for LLMs.",
|
"description": "Extensible frontend for LLMs.",
|
||||||
"license": 'MIT',
|
"license": 'MIT',
|
||||||
"flags": {
|
"flags": {
|
||||||
|
|
@ -79,11 +109,17 @@ TOOLS: dict[str, dict] = {
|
||||||
"installer": {
|
"installer": {
|
||||||
"type": "git_node",
|
"type": "git_node",
|
||||||
"pkg": "https://github.com/danny-avila/LibreChat.git",
|
"pkg": "https://github.com/danny-avila/LibreChat.git",
|
||||||
"update_cmd": "git pull --ff-only && yarn install && yarn build"
|
"update_cmd": "git pull --ff-only && yarn install && yarn build",
|
||||||
|
# Two known fresh-clone start bugs, fixed at install time:
|
||||||
|
# 1. `yarn backend` aborts without an .env file — seed one
|
||||||
|
# from the shipped example (never clobber an existing one).
|
||||||
|
# 2. production mode requires the client assets from
|
||||||
|
# `yarn build` — plain `yarn install` doesn't produce them.
|
||||||
|
"post_install": "cp -n .env.example .env 2>/dev/null || true; yarn build"
|
||||||
},
|
},
|
||||||
"launcher": {
|
"launcher": {
|
||||||
"type": "tmux",
|
"type": "tmux",
|
||||||
"cmd": "cd {tools_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend",
|
"cmd": "cd {dashboards_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend",
|
||||||
"default_port": 3080
|
"default_port": 3080
|
||||||
},
|
},
|
||||||
"deps": [
|
"deps": [
|
||||||
|
|
@ -101,9 +137,9 @@ TOOLS: dict[str, dict] = {
|
||||||
"is_skills_collection": False
|
"is_skills_collection": False
|
||||||
},
|
},
|
||||||
"filesystem": {
|
"filesystem": {
|
||||||
"install": "tools/librechat",
|
"install": "dashboards/librechat",
|
||||||
"config": "configs/librechat",
|
"config": "configs/librechat",
|
||||||
"data": "data/librechat",
|
"data": "workspaces/librechat",
|
||||||
"logs": "logs/librechat"
|
"logs": "logs/librechat"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
@ -536,11 +572,21 @@ TOOLS: dict[str, dict] = {
|
||||||
"category": 'Chat Frontend',
|
"category": 'Chat Frontend',
|
||||||
"installer": {
|
"installer": {
|
||||||
"type": "uv",
|
"type": "uv",
|
||||||
"pkg": "open-webui"
|
# Same pydub/audioop Python-3.13 fix as the openwebui entry.
|
||||||
|
"pkg": "open-webui --with audioop-lts"
|
||||||
},
|
},
|
||||||
"launcher": {
|
"launcher": {
|
||||||
"type": "tmux",
|
"type": "tmux",
|
||||||
"cmd": "WEBUI_DATA_DIR={workspaces_root}/hermes-webui OLLAMA_BASE_URL=http://localhost:17051 open-webui serve --port {port}",
|
# Same minimal pattern as the openwebui entry — cd to
|
||||||
|
# configs/auth/ so .webui_secret_key lands there, DATA_DIR
|
||||||
|
# points at the hermes-webui data volume.
|
||||||
|
#
|
||||||
|
# The ONE env var we set here: OLLAMA_BASE_URL. The default is
|
||||||
|
# http://localhost:11434 (ollama direct); hermes_webui needs to
|
||||||
|
# point at hermes_agent (17051) instead so every conversation
|
||||||
|
# flows through the Hermes runtime's tool-use layer. This is
|
||||||
|
# the only difference from the openwebui entry.
|
||||||
|
"cmd": "mkdir -p {base_dir}/configs/auth && chmod 700 {base_dir}/configs/auth && cd {base_dir}/configs/auth && DATA_DIR={workspaces_root}/hermes-webui OLLAMA_BASE_URL=http://127.0.0.1:17051 open-webui serve --port {port}",
|
||||||
"default_port": 8081
|
"default_port": 8081
|
||||||
},
|
},
|
||||||
"deps": [
|
"deps": [
|
||||||
|
|
@ -625,4 +671,54 @@ TOOLS: dict[str, dict] = {
|
||||||
"is_skills_collection": False
|
"is_skills_collection": False
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
'openhuman': {
|
||||||
|
"name": "OpenHuman",
|
||||||
|
"level": 10,
|
||||||
|
"layer": 'Human Interface & System Operations',
|
||||||
|
"role": 'Personal AI Harness',
|
||||||
|
"category": 'Personal AI Harness',
|
||||||
|
"installer": {
|
||||||
|
# Same curl|sh policy category as ollama / meilisearch / fabric
|
||||||
|
# (see whatremains.txt C-05). The openhuman README explicitly
|
||||||
|
# warns this script path is unverified (no detached signature);
|
||||||
|
# the native-package paths (deb / rpm / AppImage / brew cask)
|
||||||
|
# are preferred when the user is ready to move off the script
|
||||||
|
# installer.
|
||||||
|
"type": "script",
|
||||||
|
"cmd": "curl -fsSL https://raw.githubusercontent.com/tinyhumansai/openhuman/main/scripts/install.sh | bash",
|
||||||
|
"pkg": "https://github.com/tinyhumansai/openhuman"
|
||||||
|
},
|
||||||
|
"launcher": {
|
||||||
|
# Desktop GUI app (Rust core + native shell). One-click start
|
||||||
|
# → talks to localhost ollama. OLLAMA_BASE_URL is preset so
|
||||||
|
# openhuman's first-run model-router defaults to the local
|
||||||
|
# ollama instance instead of the managed subscription endpoint.
|
||||||
|
# Wrapped in `bash -c` because launch_desktop uses
|
||||||
|
# subprocess.Popen(argv) directly (no shell), so env-assignment
|
||||||
|
# prefixes must be expanded by an actual shell.
|
||||||
|
"type": "desktop",
|
||||||
|
"cmd": "bash -c 'OLLAMA_BASE_URL=http://127.0.0.1:11434 exec openhuman'",
|
||||||
|
"default_port": None
|
||||||
|
},
|
||||||
|
"deps": [
|
||||||
|
"ollama"
|
||||||
|
],
|
||||||
|
"description": "Personal AI harness: persistent local memory tree, "
|
||||||
|
"agent orchestration on durable graphs, deep-research "
|
||||||
|
"pipeline. Local-first with opt-in Privacy Mode that "
|
||||||
|
"forces all inference to localhost ollama (or another "
|
||||||
|
"managed backend). Native packages (.deb / .rpm / "
|
||||||
|
".AppImage / brew cask) preferred over the script "
|
||||||
|
"installer once the user moves past first boot.",
|
||||||
|
"license": 'GPL-3.0',
|
||||||
|
"flags": {
|
||||||
|
"has_cli": False,
|
||||||
|
"has_gui": True,
|
||||||
|
"has_web": False,
|
||||||
|
"is_ollama": False,
|
||||||
|
"is_passive": False,
|
||||||
|
"is_mcp": False,
|
||||||
|
"is_skills_collection": False
|
||||||
|
}
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -175,7 +175,11 @@ def _check_entry(tool_id: str, entry: dict[str, Any]) -> list[str]:
|
||||||
# Warn if script cmd doesn't contain {tools_root}
|
# Warn if script cmd doesn't contain {tools_root}
|
||||||
if itype == "script" and inst.get("cmd"):
|
if itype == "script" and inst.get("cmd"):
|
||||||
cmd_str = inst["cmd"]
|
cmd_str = inst["cmd"]
|
||||||
if "{tools_root}" not in cmd_str and tool_id != "ollama":
|
# carve-out: ollama and openhuman ship native installers
|
||||||
|
# (apt/dpkg/AppImage/brew cask) that target system paths,
|
||||||
|
# not tools_root. Their script installers are accepted as-is
|
||||||
|
# per the C-05 curl|sh policy (see whatremains.txt).
|
||||||
|
if "{tools_root}" not in cmd_str and tool_id not in {"ollama", "openhuman"}:
|
||||||
errors.append(
|
errors.append(
|
||||||
f"{tool_id}: script installer cmd should reference "
|
f"{tool_id}: script installer cmd should reference "
|
||||||
f"{{{{tools_root}}}} to avoid polluting system dirs"
|
f"{{{{tools_root}}}} to avoid polluting system dirs"
|
||||||
|
|
|
||||||
|
|
@ -104,16 +104,28 @@ class RuntimeExecutor:
|
||||||
port: str = "",
|
port: str = "",
|
||||||
model_arg: str = "",
|
model_arg: str = "",
|
||||||
) -> dict[str, str]:
|
) -> dict[str, str]:
|
||||||
"""Build the ``{placeholders}`` dict used by launcher commands."""
|
"""Build the ``{placeholders}`` dict used by launcher commands.
|
||||||
|
|
||||||
|
Every key of the path tree is exposed (``{tools_root}``,
|
||||||
|
``{dashboards_root}``, ``{runtime_root}``, ``{configs_root}``,
|
||||||
|
``{datasets_root}``, …) so registry launcher templates can
|
||||||
|
reference any canonical /mnt/AI location.
|
||||||
|
"""
|
||||||
from ai_lsc.constants import BASE_DIR
|
from ai_lsc.constants import BASE_DIR
|
||||||
return {
|
from ai_lsc.utils.paths import build_path_tree
|
||||||
|
ctx = {
|
||||||
|
str(k): str(v)
|
||||||
|
for k, v in build_path_tree(BASE_DIR).items()
|
||||||
|
}
|
||||||
|
ctx.update({
|
||||||
"base_dir": BASE_DIR,
|
"base_dir": BASE_DIR,
|
||||||
"tools_root": self.tools_root,
|
"tools_root": self.tools_root,
|
||||||
"models_root": self.models_root,
|
"models_root": self.models_root,
|
||||||
"workspaces_root": self.workspaces_root,
|
"workspaces_root": self.workspaces_root,
|
||||||
"port": port,
|
"port": port,
|
||||||
"model_arg": model_arg,
|
"model_arg": model_arg,
|
||||||
}
|
})
|
||||||
|
return ctx
|
||||||
|
|
||||||
# -- service lifecycle -----------------------------------------------
|
# -- service lifecycle -----------------------------------------------
|
||||||
|
|
||||||
|
|
@ -224,15 +236,26 @@ class RuntimeExecutor:
|
||||||
env_overrides: dict[str, str] | None = None,
|
env_overrides: dict[str, str] | None = None,
|
||||||
filesystem: dict[str, str] | None = None,
|
filesystem: dict[str, str] | None = None,
|
||||||
license_spdx: str | None = None,
|
license_spdx: str | None = None,
|
||||||
|
registry_entry: dict | None = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Dispatch tool installation to the correct installer.
|
"""Dispatch tool installation to the correct installer.
|
||||||
|
|
||||||
If *tool_id* is provided, the installer uses preflight detection
|
If *tool_id* is provided, the installer uses preflight detection.
|
||||||
and routes artifacts to ``tools_root/<tool_id>/``.
|
When the tool is already installed, the call becomes an in-place
|
||||||
|
**update** (honouring the registry's ``installer.update_cmd``)
|
||||||
|
unless *force* is set or the tool is missing.
|
||||||
|
|
||||||
If *force* is True, skips preflight and installs unconditionally.
|
If *force* is True, skips preflight and installs unconditionally.
|
||||||
|
|
||||||
*post_install* runs a shell command inside ``tools_root/<tool_id>``
|
*registry_entry* (the tool's full registry metadata dict) routes
|
||||||
after clone (e.g. ``pip install -r requirements.txt``, ``make``).
|
artifacts to the canonical install root declared by
|
||||||
|
``filesystem.install`` / the v3.4 router (web UIs →
|
||||||
|
``dashboards/<id>/``, standalone compiles → ``tools/<id>/``)
|
||||||
|
and supplies ``update_cmd`` / ``post_install`` defaults.
|
||||||
|
|
||||||
|
*post_install* runs a shell command inside the tool's install
|
||||||
|
directory after clone (e.g. ``pip install -r requirements.txt``,
|
||||||
|
``make``).
|
||||||
|
|
||||||
*env_overrides* remaps upstream environment variables (HF_HOME,
|
*env_overrides* remaps upstream environment variables (HF_HOME,
|
||||||
TRANSFORMERS_CACHE, etc.) into ``/mnt/AI/`` paths.
|
TRANSFORMERS_CACHE, etc.) into ``/mnt/AI/`` paths.
|
||||||
|
|
@ -259,6 +282,7 @@ class RuntimeExecutor:
|
||||||
post_install=post_install,
|
post_install=post_install,
|
||||||
env_overrides=env_overrides,
|
env_overrides=env_overrides,
|
||||||
license_spdx=license_spdx,
|
license_spdx=license_spdx,
|
||||||
|
entry=registry_entry,
|
||||||
)
|
)
|
||||||
return self._installer.run(
|
return self._installer.run(
|
||||||
inst_type=inst_type,
|
inst_type=inst_type,
|
||||||
|
|
@ -269,6 +293,40 @@ class RuntimeExecutor:
|
||||||
post_install=post_install,
|
post_install=post_install,
|
||||||
env_overrides=env_overrides,
|
env_overrides=env_overrides,
|
||||||
license_spdx=license_spdx,
|
license_spdx=license_spdx,
|
||||||
|
entry=registry_entry,
|
||||||
|
)
|
||||||
|
|
||||||
|
def update_tool(
|
||||||
|
self,
|
||||||
|
tool_id: str,
|
||||||
|
inst_type: str,
|
||||||
|
pkg: str,
|
||||||
|
cmd: str = "",
|
||||||
|
update_cmd: str = "",
|
||||||
|
registry_entry: dict | None = None,
|
||||||
|
env_overrides: dict[str, str] | None = None,
|
||||||
|
post_install: str | None = None,
|
||||||
|
) -> str:
|
||||||
|
"""Update an already-installed tool in place.
|
||||||
|
|
||||||
|
Honours the registry's ``installer.update_cmd`` when provided;
|
||||||
|
otherwise falls back to per-type defaults (``uv tool upgrade``,
|
||||||
|
``pipx upgrade``, ``git pull --ff-only`` + node rebuild, …).
|
||||||
|
Runs in the *detected* install location so tools installed
|
||||||
|
before the v3.4 layout change update where they live.
|
||||||
|
|
||||||
|
Returns a description of the result.
|
||||||
|
"""
|
||||||
|
_validate_tool_id(tool_id)
|
||||||
|
return self._installer.update_tool(
|
||||||
|
tool_id=tool_id,
|
||||||
|
inst_type=inst_type,
|
||||||
|
pkg=pkg,
|
||||||
|
cmd=cmd,
|
||||||
|
update_cmd=update_cmd,
|
||||||
|
entry=registry_entry,
|
||||||
|
env_overrides=env_overrides,
|
||||||
|
post_install=post_install,
|
||||||
)
|
)
|
||||||
|
|
||||||
# -- verification ---------------------------------------------------
|
# -- verification ---------------------------------------------------
|
||||||
|
|
@ -280,6 +338,7 @@ class RuntimeExecutor:
|
||||||
pkg: str,
|
pkg: str,
|
||||||
cmd: str = "",
|
cmd: str = "",
|
||||||
filesystem: dict[str, str] | None = None,
|
filesystem: dict[str, str] | None = None,
|
||||||
|
registry_entry: dict | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Run the installation compliance checklist for a tool.
|
"""Run the installation compliance checklist for a tool.
|
||||||
|
|
||||||
|
|
@ -291,6 +350,7 @@ class RuntimeExecutor:
|
||||||
pkg=pkg,
|
pkg=pkg,
|
||||||
cmd=cmd,
|
cmd=cmd,
|
||||||
filesystem=filesystem,
|
filesystem=filesystem,
|
||||||
|
entry=registry_entry,
|
||||||
)
|
)
|
||||||
|
|
||||||
# -- model management ------------------------------------------------
|
# -- model management ------------------------------------------------
|
||||||
|
|
|
||||||
|
|
@ -53,6 +53,11 @@ from typing import Any
|
||||||
from urllib.parse import urlparse
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
from ai_lsc.utils.logging import get_logger
|
from ai_lsc.utils.logging import get_logger
|
||||||
|
from ai_lsc.utils.paths import (
|
||||||
|
build_path_tree,
|
||||||
|
candidate_install_dirs,
|
||||||
|
install_dir_for,
|
||||||
|
)
|
||||||
from ai_lsc.utils.process import enriched_env
|
from ai_lsc.utils.process import enriched_env
|
||||||
|
|
||||||
logger = get_logger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
@ -92,6 +97,21 @@ def _validate_url(url: str, *, allow_schemes: tuple[str, ...] = ("http", "https"
|
||||||
raise ValueError(f"unsafe URL rejected: {url!r}")
|
raise ValueError(f"unsafe URL rejected: {url!r}")
|
||||||
return url
|
return url
|
||||||
|
|
||||||
|
|
||||||
|
def _uv_with_extras(tokens: list[str]) -> list[str]:
|
||||||
|
"""Extract the package names pulled in by ``--with`` flags.
|
||||||
|
|
||||||
|
``["open-webui", "--with", "audioop-lts"]`` → ``["audioop-lts"]``.
|
||||||
|
Multiple ``--with`` flags each contribute one package; unknown
|
||||||
|
flags contribute nothing. Used by the uv→pip step-down so the
|
||||||
|
fallback venv keeps every extra the registry declared.
|
||||||
|
"""
|
||||||
|
extras: list[str] = []
|
||||||
|
for i, tok in enumerate(tokens[:-1]):
|
||||||
|
if tok == "--with":
|
||||||
|
extras.append(tokens[i + 1])
|
||||||
|
return extras
|
||||||
|
|
||||||
# Step-down containment order (most isolated first)
|
# Step-down containment order (most isolated first)
|
||||||
STEP_DOWN_ORDER: list[str] = [
|
STEP_DOWN_ORDER: list[str] = [
|
||||||
"ollama", "uv", "pipx", "pip",
|
"ollama", "uv", "pipx", "pip",
|
||||||
|
|
@ -152,6 +172,43 @@ class InstallerManager:
|
||||||
# e.g. in the UI layer).
|
# e.g. in the UI layer).
|
||||||
self.license_gate = license_gate
|
self.license_gate = license_gate
|
||||||
|
|
||||||
|
# ── Install-root routing ─────────────────────────────────────────
|
||||||
|
|
||||||
|
def _dest_for(
|
||||||
|
self,
|
||||||
|
tool_id: str,
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
|
) -> str:
|
||||||
|
"""Resolve the on-disk destination directory for *tool_id*.
|
||||||
|
|
||||||
|
Honours the registry entry's ``filesystem.install`` and the
|
||||||
|
v3.4 install-root router (web UIs → ``dashboards/``, everything
|
||||||
|
else → ``tools/``). Falls back to the historical
|
||||||
|
``tools_root/<tool_id>`` when no entry is supplied, so
|
||||||
|
pre-v3.4 call sites keep working unchanged.
|
||||||
|
"""
|
||||||
|
if entry:
|
||||||
|
return str(install_dir_for(entry, tool_id, self.base_dir))
|
||||||
|
return os.path.join(self.tools_root, tool_id)
|
||||||
|
|
||||||
|
def _existing_dir_for(
|
||||||
|
self,
|
||||||
|
tool_id: str,
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
|
) -> str:
|
||||||
|
"""Like :meth:`_dest_for`, but prefer an existing install.
|
||||||
|
|
||||||
|
Scans the candidate roots in router order and returns the first
|
||||||
|
directory that already exists; only when none exists is the
|
||||||
|
canonical (fresh-install) destination returned. This keeps
|
||||||
|
updates flowing to wherever a tool actually lives today
|
||||||
|
instead of forking a second copy under the new canonical root.
|
||||||
|
"""
|
||||||
|
for cand in candidate_install_dirs(tool_id, entry, self.base_dir):
|
||||||
|
if cand.is_dir():
|
||||||
|
return str(cand)
|
||||||
|
return self._dest_for(tool_id, entry)
|
||||||
|
|
||||||
# ── Environment construction ─────────────────────────────────────
|
# ── Environment construction ─────────────────────────────────────
|
||||||
|
|
||||||
def _env(
|
def _env(
|
||||||
|
|
@ -198,14 +255,20 @@ class InstallerManager:
|
||||||
env["PIPX_HOME"] = os.path.join(self.tools_root, ".pipx")
|
env["PIPX_HOME"] = os.path.join(self.tools_root, ".pipx")
|
||||||
|
|
||||||
# ── Per-tool env overrides from registry ────────────────────────
|
# ── Per-tool env overrides from registry ────────────────────────
|
||||||
# Keys may contain {tools_root}, {base_dir} placeholders.
|
# Values may contain any {path_tree_key} placeholder
|
||||||
|
# ({tools_root}, {base_dir}, {dashboards_root}, …).
|
||||||
if env_overrides:
|
if env_overrides:
|
||||||
|
tree = build_path_tree(self.base_dir)
|
||||||
for key, raw_val in env_overrides.items():
|
for key, raw_val in env_overrides.items():
|
||||||
expanded = raw_val.replace(
|
expanded = raw_val.replace(
|
||||||
"{tools_root}", self.tools_root,
|
"{tools_root}", self.tools_root,
|
||||||
).replace(
|
).replace(
|
||||||
"{base_dir}", self.base_dir,
|
"{base_dir}", self.base_dir,
|
||||||
)
|
)
|
||||||
|
for tree_key, tree_val in tree.items():
|
||||||
|
expanded = expanded.replace(
|
||||||
|
"{" + tree_key + "}", str(tree_val),
|
||||||
|
)
|
||||||
env[key] = expanded
|
env[key] = expanded
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"env override: %s=%s (tool %s)", key, expanded, tool_id,
|
"env override: %s=%s (tool %s)", key, expanded, tool_id,
|
||||||
|
|
@ -230,6 +293,7 @@ class InstallerManager:
|
||||||
inst_type: str,
|
inst_type: str,
|
||||||
pkg: str,
|
pkg: str,
|
||||||
cmd: str = "",
|
cmd: str = "",
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Check whether a tool is already installed before installing.
|
"""Check whether a tool is already installed before installing.
|
||||||
|
|
||||||
|
|
@ -246,7 +310,7 @@ class InstallerManager:
|
||||||
}
|
}
|
||||||
|
|
||||||
location, version = self._detect_installation(
|
location, version = self._detect_installation(
|
||||||
tool_id, inst_type, pkg, cmd,
|
tool_id, inst_type, pkg, cmd, entry,
|
||||||
)
|
)
|
||||||
if location:
|
if location:
|
||||||
result["found"] = True
|
result["found"] = True
|
||||||
|
|
@ -262,39 +326,54 @@ class InstallerManager:
|
||||||
inst_type: str,
|
inst_type: str,
|
||||||
pkg: str,
|
pkg: str,
|
||||||
cmd: str = "",
|
cmd: str = "",
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
) -> tuple[str, str]:
|
) -> tuple[str, str]:
|
||||||
"""Detect existing installation. Returns (location, version)."""
|
"""Detect existing installation. Returns (location, version).
|
||||||
|
|
||||||
# 1. Check tools_root/<tool_id> directory existence
|
Probes every candidate install root (router dir, dashboards/,
|
||||||
tool_dir = os.path.join(self.tools_root, tool_id)
|
tools/) so tools installed before the v3.4 routing change are
|
||||||
|
still found and offered an update instead of a reinstall.
|
||||||
|
"""
|
||||||
|
|
||||||
|
# `pkg` may carry extra uv/flags tokens (e.g.
|
||||||
|
# "open-webui --with audioop-lts"); for preflight we only care
|
||||||
|
# about the first token — the actual package name.
|
||||||
|
pkg_token = shlex.split(pkg)[0] if pkg else pkg
|
||||||
|
|
||||||
|
# 1. Check candidate install directories (router order)
|
||||||
|
for tool_dir in candidate_install_dirs(
|
||||||
|
tool_id, entry, self.base_dir,
|
||||||
|
):
|
||||||
if os.path.isdir(tool_dir):
|
if os.path.isdir(tool_dir):
|
||||||
ver = self._detect_version(inst_type, pkg, cmd, tool_dir)
|
ver = self._detect_version(
|
||||||
return tool_dir, ver
|
inst_type, pkg_token, cmd, str(tool_dir),
|
||||||
|
)
|
||||||
|
return str(tool_dir), ver
|
||||||
|
|
||||||
# 2. Check tools_root/.pipx, tools_root/.uv, tools_root/.local
|
# 2. Check tools_root/.pipx, tools_root/.uv, tools_root/.local
|
||||||
for subdir in [".pipx", ".uv", ".local"]:
|
for subdir in [".pipx", ".uv", ".local"]:
|
||||||
check = os.path.join(self.tools_root, subdir, "bin", pkg)
|
check = os.path.join(self.tools_root, subdir, "bin", pkg_token)
|
||||||
if os.path.exists(check):
|
if os.path.exists(check):
|
||||||
return os.path.dirname(check), ""
|
return os.path.dirname(check), ""
|
||||||
|
|
||||||
# 3. Check tools_root/bin
|
# 3. Check tools_root/bin
|
||||||
bin_check = os.path.join(self.tools_root, "bin", pkg)
|
bin_check = os.path.join(self.tools_root, "bin", pkg_token)
|
||||||
if os.path.exists(bin_check):
|
if os.path.exists(bin_check):
|
||||||
return os.path.dirname(bin_check), ""
|
return os.path.dirname(bin_check), ""
|
||||||
|
|
||||||
# 4. Check system PATH via shutil.which
|
# 4. Check system PATH via shutil.which
|
||||||
binary_name = self._binary_name(pkg, inst_type)
|
binary_name = self._binary_name(pkg_token, inst_type)
|
||||||
system_path = shutil.which(binary_name)
|
system_path = shutil.which(binary_name)
|
||||||
if system_path:
|
if system_path:
|
||||||
ver = self._detect_version(inst_type, pkg, cmd)
|
ver = self._detect_version(inst_type, pkg_token, cmd)
|
||||||
return system_path, ver
|
return system_path, ver
|
||||||
|
|
||||||
# 5. OS package manager query (pacman / dnf / apt) — list-form
|
# 5. OS package manager query (pacman / dnf / apt) — list-form
|
||||||
# subprocess calls, no shell, no interpolation.
|
# subprocess calls, no shell, no interpolation.
|
||||||
_PKG_MGR_QUERIES: dict[str, list[str]] = {
|
_PKG_MGR_QUERIES: dict[str, list[str]] = {
|
||||||
"pacman": ["pacman", "-Qi", pkg],
|
"pacman": ["pacman", "-Qi", pkg_token],
|
||||||
"dnf": ["dnf", "info", pkg],
|
"dnf": ["dnf", "info", pkg_token],
|
||||||
"apt": ["dpkg", "-s", pkg],
|
"apt": ["dpkg", "-s", pkg_token],
|
||||||
}
|
}
|
||||||
if inst_type in _PKG_MGR_QUERIES:
|
if inst_type in _PKG_MGR_QUERIES:
|
||||||
try:
|
try:
|
||||||
|
|
@ -306,19 +385,24 @@ class InstallerManager:
|
||||||
for line in proc.stdout.splitlines():
|
for line in proc.stdout.splitlines():
|
||||||
if line.strip().startswith("Version"):
|
if line.strip().startswith("Version"):
|
||||||
ver = line.split(":", 1)[-1].strip()
|
ver = line.split(":", 1)[-1].strip()
|
||||||
return f"{inst_type}:{pkg}", ver
|
return f"{inst_type}:{pkg_token}", ver
|
||||||
except (OSError, subprocess.SubprocessError):
|
except (OSError, subprocess.SubprocessError):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
return "", ""
|
return "", ""
|
||||||
|
|
||||||
def _binary_name(self, pkg: str, inst_type: str) -> str:
|
def _binary_name(self, pkg: str, inst_type: str) -> str:
|
||||||
"""Map a package name to its likely binary name."""
|
"""Map a package name to its likely binary name.
|
||||||
|
|
||||||
|
Takes the first shlex token of *pkg* so a spec like
|
||||||
|
``"open-webui --with audioop-lts"`` resolves to ``open_webui``.
|
||||||
|
"""
|
||||||
|
first = shlex.split(pkg)[0] if pkg else pkg
|
||||||
if inst_type == "npm":
|
if inst_type == "npm":
|
||||||
return pkg if "/" not in pkg else pkg.split("/")[-1]
|
return first if "/" not in first else first.split("/")[-1]
|
||||||
if inst_type in ("uv", "pip"):
|
if inst_type in ("uv", "pip"):
|
||||||
return pkg.replace("-", "_").replace(".", "_")
|
return first.replace("-", "_").replace(".", "_")
|
||||||
return pkg
|
return first
|
||||||
|
|
||||||
def _detect_version(
|
def _detect_version(
|
||||||
self,
|
self,
|
||||||
|
|
@ -329,9 +413,19 @@ class InstallerManager:
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Try to extract the installed version."""
|
"""Try to extract the installed version."""
|
||||||
if inst_type == "git":
|
if inst_type == "git":
|
||||||
git_dir = os.path.join(self.tools_root, pkg.split("/")[-1]
|
# Use only the first token of `pkg` for the dir-name lookup
|
||||||
.replace(".git", ""))
|
# so a spec like "repo --branch foo" doesn't break the
|
||||||
if os.path.isdir(os.path.join(git_dir, ".git")):
|
# rsplit("/"). (Git pkg specs are normally bare URLs, but
|
||||||
|
# be defensive.)
|
||||||
|
pkg_first = shlex.split(pkg)[0] if pkg else pkg
|
||||||
|
repo_name = pkg_first.split("/")[-1].replace(".git", "")
|
||||||
|
git_dirs = [
|
||||||
|
str(d) for d in candidate_install_dirs(
|
||||||
|
repo_name, None, self.base_dir,
|
||||||
|
)
|
||||||
|
if os.path.isdir(os.path.join(d, ".git"))
|
||||||
|
]
|
||||||
|
for git_dir in git_dirs:
|
||||||
for argv in (
|
for argv in (
|
||||||
["git", "describe", "--tags", "--abbrev=0"],
|
["git", "describe", "--tags", "--abbrev=0"],
|
||||||
["git", "rev-parse", "--short", "HEAD"],
|
["git", "rev-parse", "--short", "HEAD"],
|
||||||
|
|
@ -355,7 +449,11 @@ class InstallerManager:
|
||||||
else:
|
else:
|
||||||
tmpl = _VERSION_CMDS.get(inst_type, "")
|
tmpl = _VERSION_CMDS.get(inst_type, "")
|
||||||
if tmpl:
|
if tmpl:
|
||||||
ver_argv = shlex.split(tmpl.format(pkg=pkg, cmd=pkg))
|
# Use only the first token of `pkg` so a spec like
|
||||||
|
# "open-webui --with audioop-lts" renders as just the
|
||||||
|
# binary name in the version-detection command.
|
||||||
|
pkg_first = shlex.split(pkg)[0] if pkg else pkg
|
||||||
|
ver_argv = shlex.split(tmpl.format(pkg=pkg_first, cmd=pkg_first))
|
||||||
|
|
||||||
if not ver_argv:
|
if not ver_argv:
|
||||||
return ""
|
return ""
|
||||||
|
|
@ -378,14 +476,16 @@ class InstallerManager:
|
||||||
self,
|
self,
|
||||||
tool_id: str,
|
tool_id: str,
|
||||||
post_install_cmd: str,
|
post_install_cmd: str,
|
||||||
|
dest: str = "",
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Run a post-install hook inside ``tools_root/<tool_id>``."""
|
"""Run a post-install hook inside the tool's install directory."""
|
||||||
if not post_install_cmd:
|
if not post_install_cmd:
|
||||||
return ""
|
return ""
|
||||||
dest = os.path.join(self.tools_root, tool_id)
|
cwd = dest or os.path.join(self.tools_root, tool_id)
|
||||||
env = self._env(tool_id)
|
env = self._env(tool_id)
|
||||||
# Replace {tools_root} in the command
|
# Resolve {tools_root} / {base_dir} / {install_dir} and every
|
||||||
cmd = post_install_cmd.replace("{tools_root}", self.tools_root)
|
# path-tree placeholder in the command.
|
||||||
|
cmd = self._render_cmd(post_install_cmd, cwd)
|
||||||
logger.info("Running post-install for %s: %s", tool_id, cmd)
|
logger.info("Running post-install for %s: %s", tool_id, cmd)
|
||||||
try:
|
try:
|
||||||
# Post-install commands are arbitrary shell snippets supplied by
|
# Post-install commands are arbitrary shell snippets supplied by
|
||||||
|
|
@ -395,7 +495,7 @@ class InstallerManager:
|
||||||
# break out of the subprocess call itself.
|
# break out of the subprocess call itself.
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
["bash", "-c", cmd], check=True, env=env,
|
["bash", "-c", cmd], check=True, env=env,
|
||||||
timeout=300, cwd=dest,
|
timeout=600, cwd=cwd,
|
||||||
)
|
)
|
||||||
return f"Post-install completed for {tool_id}."
|
return f"Post-install completed for {tool_id}."
|
||||||
except (subprocess.CalledProcessError, OSError) as exc:
|
except (subprocess.CalledProcessError, OSError) as exc:
|
||||||
|
|
@ -404,6 +504,20 @@ class InstallerManager:
|
||||||
)
|
)
|
||||||
return f"Post-install FAILED for {tool_id}: {exc}"
|
return f"Post-install FAILED for {tool_id}: {exc}"
|
||||||
|
|
||||||
|
def _render_cmd(self, cmd: str, install_dir: str = "") -> str:
|
||||||
|
"""Resolve path placeholders in a registry shell snippet.
|
||||||
|
|
||||||
|
Supports ``{tools_root}``, ``{base_dir}``, ``{install_dir}``
|
||||||
|
and every key of the path tree (``{dashboards_root}``, …).
|
||||||
|
"""
|
||||||
|
rendered = cmd.replace("{tools_root}", self.tools_root)
|
||||||
|
rendered = rendered.replace("{base_dir}", self.base_dir)
|
||||||
|
if install_dir:
|
||||||
|
rendered = rendered.replace("{install_dir}", install_dir)
|
||||||
|
for key, value in build_path_tree(self.base_dir).items():
|
||||||
|
rendered = rendered.replace("{" + key + "}", str(value))
|
||||||
|
return rendered
|
||||||
|
|
||||||
# ── Strategy methods ────────────────────────────────────────────
|
# ── Strategy methods ────────────────────────────────────────────
|
||||||
|
|
||||||
def install_ollama(self, pkg: str, tool_id: str) -> str:
|
def install_ollama(self, pkg: str, tool_id: str) -> str:
|
||||||
|
|
@ -438,26 +552,84 @@ class InstallerManager:
|
||||||
return f"Ollama model '{pkg}' queued for pull."
|
return f"Ollama model '{pkg}' queued for pull."
|
||||||
|
|
||||||
def install_uv(self, pkg: str, tool_id: str,
|
def install_uv(self, pkg: str, tool_id: str,
|
||||||
env_overrides: dict[str, str] | None = None) -> str:
|
env_overrides: dict[str, str] | None = None,
|
||||||
"""Install a Python tool via ``uv tool install`` pinned to tools_root."""
|
entry: dict[str, Any] | None = None) -> str:
|
||||||
dest = os.path.join(self.tools_root, tool_id)
|
"""Install a Python tool via ``uv tool install`` pinned to tools_root.
|
||||||
|
|
||||||
|
The ``pkg`` string may carry extra ``uv`` flags after the package
|
||||||
|
name — e.g. ``"open-webui --with audioop-lts"``. The first token
|
||||||
|
is validated as the package name; the remaining tokens are passed
|
||||||
|
through to ``uv tool install`` verbatim. This lets registry
|
||||||
|
entries pin Python-3.13-deps backports (audioop-lts for pydub,
|
||||||
|
etc.) without needing new schema fields or threading new params
|
||||||
|
through every caller.
|
||||||
|
|
||||||
|
Step-down containment (fix for the open-webui install bug):
|
||||||
|
|
||||||
|
1. ``uv tool install <pkg> [flags]`` with the default Python.
|
||||||
|
2. If that fails, retry with ``--python 3.12`` — uv fetches a
|
||||||
|
managed CPython 3.12 on the fly, which sidesteps wheels that
|
||||||
|
have not been rebuilt for the host's newer Python (the
|
||||||
|
pydub/audioop class of failure).
|
||||||
|
3. If uv fails entirely, fall back to a per-tool pip venv that
|
||||||
|
installs the package **plus every ``--with`` extra** — the
|
||||||
|
historical pipx fallback silently dropped the extras, which
|
||||||
|
produced installs that broke at import time.
|
||||||
|
"""
|
||||||
|
dest = self._dest_for(tool_id, entry)
|
||||||
os.makedirs(dest, exist_ok=True)
|
os.makedirs(dest, exist_ok=True)
|
||||||
env = self._env(tool_id, env_overrides)
|
env = self._env(tool_id, env_overrides)
|
||||||
|
# Shlex-split so registry authors can write
|
||||||
|
# `pkg: "open-webui --with audioop-lts"` and have it land as
|
||||||
|
# `["uv", "tool", "install", "open-webui", "--with", "audioop-lts"]`.
|
||||||
|
tokens = shlex.split(pkg)
|
||||||
|
if not tokens:
|
||||||
|
raise ValueError(f"invalid pkg spec for {tool_id}: {pkg!r}")
|
||||||
|
_validate_pkg(tokens[0])
|
||||||
|
extras = _uv_with_extras(tokens)
|
||||||
try:
|
try:
|
||||||
_validate_pkg(pkg)
|
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
["uv", "tool", "install", pkg],
|
["uv", "tool", "install", *tokens],
|
||||||
check=True, env=env, timeout=300,
|
check=True, env=env, timeout=300,
|
||||||
)
|
)
|
||||||
return f"UV tool '{pkg}' installed to {env['UV_TOOL_DIR']}."
|
return f"UV tool '{tokens[0]}' installed to {env['UV_TOOL_DIR']}."
|
||||||
except subprocess.CalledProcessError:
|
except subprocess.CalledProcessError:
|
||||||
logger.info("uv install failed for %s, stepping down to pipx", pkg)
|
logger.info(
|
||||||
return self.install_pipx(pkg, tool_id, env_overrides)
|
"uv install failed for %s on the default Python; "
|
||||||
|
"retrying with a managed CPython 3.12", tokens[0],
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
["uv", "tool", "install", "--python", "3.12", *tokens],
|
||||||
|
check=True, env=env, timeout=600,
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
f"UV tool '{tokens[0]}' installed to "
|
||||||
|
f"{env['UV_TOOL_DIR']} (managed CPython 3.12)."
|
||||||
|
)
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
if extras:
|
||||||
|
logger.info(
|
||||||
|
"uv failed for %s even on 3.12; falling back to a "
|
||||||
|
"pip venv WITH the %d --with extra(s) intact",
|
||||||
|
tokens[0], len(extras),
|
||||||
|
)
|
||||||
|
pip_spec = " ".join([tokens[0], *extras])
|
||||||
|
return self.install_pip(
|
||||||
|
pip_spec, tool_id, env_overrides, entry,
|
||||||
|
)
|
||||||
|
logger.info(
|
||||||
|
"uv failed for %s; stepping down to pipx", tokens[0],
|
||||||
|
)
|
||||||
|
return self.install_pipx(
|
||||||
|
tokens[0], tool_id, env_overrides, entry,
|
||||||
|
)
|
||||||
|
|
||||||
def install_pipx(self, pkg: str, tool_id: str,
|
def install_pipx(self, pkg: str, tool_id: str,
|
||||||
env_overrides: dict[str, str] | None = None) -> str:
|
env_overrides: dict[str, str] | None = None,
|
||||||
|
entry: dict[str, Any] | None = None) -> str:
|
||||||
"""Install a Python CLI tool via ``pipx`` pinned to tools_root."""
|
"""Install a Python CLI tool via ``pipx`` pinned to tools_root."""
|
||||||
dest = os.path.join(self.tools_root, tool_id)
|
dest = self._dest_for(tool_id, entry)
|
||||||
os.makedirs(dest, exist_ok=True)
|
os.makedirs(dest, exist_ok=True)
|
||||||
env = self._env(tool_id, env_overrides)
|
env = self._env(tool_id, env_overrides)
|
||||||
try:
|
try:
|
||||||
|
|
@ -469,12 +641,18 @@ class InstallerManager:
|
||||||
return f"pipx '{pkg}' installed to {env['PIPX_HOME']}."
|
return f"pipx '{pkg}' installed to {env['PIPX_HOME']}."
|
||||||
except subprocess.CalledProcessError:
|
except subprocess.CalledProcessError:
|
||||||
logger.info("pipx install failed for %s, stepping down to pip", pkg)
|
logger.info("pipx install failed for %s, stepping down to pip", pkg)
|
||||||
return self.install_pip(pkg, tool_id, env_overrides)
|
return self.install_pip(pkg, tool_id, env_overrides, entry)
|
||||||
|
|
||||||
def install_pip(self, pkg: str, tool_id: str,
|
def install_pip(self, pkg: str, tool_id: str,
|
||||||
env_overrides: dict[str, str] | None = None) -> str:
|
env_overrides: dict[str, str] | None = None,
|
||||||
"""Install a Python tool via ``pip`` into a per-tool venv."""
|
entry: dict[str, Any] | None = None) -> str:
|
||||||
dest = os.path.join(self.tools_root, tool_id)
|
"""Install a Python tool via ``pip`` into a per-tool venv.
|
||||||
|
|
||||||
|
*pkg* may be a space-separated spec naming several distributions
|
||||||
|
(``"open-webui audioop-lts"``) — the uv step-down path uses this
|
||||||
|
to keep ``--with`` extras alive in the fallback venv.
|
||||||
|
"""
|
||||||
|
dest = self._dest_for(tool_id, entry)
|
||||||
venv_dir = os.path.join(dest, ".venv")
|
venv_dir = os.path.join(dest, ".venv")
|
||||||
os.makedirs(dest, exist_ok=True)
|
os.makedirs(dest, exist_ok=True)
|
||||||
env = self._env(tool_id, env_overrides)
|
env = self._env(tool_id, env_overrides)
|
||||||
|
|
@ -484,16 +662,20 @@ class InstallerManager:
|
||||||
check=True, env=env, timeout=60,
|
check=True, env=env, timeout=60,
|
||||||
)
|
)
|
||||||
pip_bin = os.path.join(venv_dir, "bin", "pip")
|
pip_bin = os.path.join(venv_dir, "bin", "pip")
|
||||||
|
pip_tokens = shlex.split(pkg) if pkg else []
|
||||||
|
if not pip_tokens:
|
||||||
|
raise ValueError(f"invalid pkg spec for {tool_id}: {pkg!r}")
|
||||||
try:
|
try:
|
||||||
_validate_pkg(pkg)
|
for tok in pip_tokens:
|
||||||
|
_validate_pkg(tok)
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
[pip_bin, "install", pkg],
|
[pip_bin, "install", *pip_tokens],
|
||||||
check=True, env=env, timeout=300,
|
check=True, env=env, timeout=300,
|
||||||
)
|
)
|
||||||
except subprocess.CalledProcessError as exc:
|
except subprocess.CalledProcessError as exc:
|
||||||
logger.warning("pip install failed for %s: %s", pkg, exc)
|
logger.warning("pip install failed for %s: %s", pkg, exc)
|
||||||
raise
|
raise
|
||||||
self._symlink_venv_bin(tool_id, venv_dir, pkg)
|
self._symlink_venv_bin(tool_id, venv_dir, pip_tokens[0])
|
||||||
return f"pip '{pkg}' installed to {venv_dir}."
|
return f"pip '{pkg}' installed to {venv_dir}."
|
||||||
|
|
||||||
def _symlink_venv_bin(
|
def _symlink_venv_bin(
|
||||||
|
|
@ -544,9 +726,10 @@ class InstallerManager:
|
||||||
return f"Dispatched apt for {pkg}."
|
return f"Dispatched apt for {pkg}."
|
||||||
|
|
||||||
def install_npm(self, pkg: str, tool_id: str,
|
def install_npm(self, pkg: str, tool_id: str,
|
||||||
env_overrides: dict[str, str] | None = None) -> str:
|
env_overrides: dict[str, str] | None = None,
|
||||||
|
entry: dict[str, Any] | None = None) -> str:
|
||||||
"""Install an npm package to an isolated prefix under tools_root."""
|
"""Install an npm package to an isolated prefix under tools_root."""
|
||||||
dest = os.path.join(self.tools_root, tool_id)
|
dest = self._dest_for(tool_id, entry)
|
||||||
os.makedirs(dest, exist_ok=True)
|
os.makedirs(dest, exist_ok=True)
|
||||||
env = self._env(tool_id, env_overrides)
|
env = self._env(tool_id, env_overrides)
|
||||||
_validate_pkg(pkg)
|
_validate_pkg(pkg)
|
||||||
|
|
@ -562,6 +745,7 @@ class InstallerManager:
|
||||||
tool_id: str,
|
tool_id: str,
|
||||||
post_install: str | None = None,
|
post_install: str | None = None,
|
||||||
env_overrides: dict[str, str] | None = None,
|
env_overrides: dict[str, str] | None = None,
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Clone or update a git repository at ``tools_root/<tool_id>``.
|
"""Clone or update a git repository at ``tools_root/<tool_id>``.
|
||||||
|
|
||||||
|
|
@ -571,8 +755,14 @@ class InstallerManager:
|
||||||
fails for any reason — diverged branches, network errors,
|
fails for any reason — diverged branches, network errors,
|
||||||
corrupted index, etc.), move the old dir aside and re-clone
|
corrupted index, etc.), move the old dir aside and re-clone
|
||||||
fresh, so the install always ends in a usable state.
|
fresh, so the install always ends in a usable state.
|
||||||
|
|
||||||
|
The destination follows the v3.4 install-root router when
|
||||||
|
*entry* is supplied (web UIs → ``dashboards/<tool_id>/``).
|
||||||
|
A pre-existing clone under ``tools/<tool_id>/`` is still
|
||||||
|
detected and updated in place — the install never forks a
|
||||||
|
second copy just because the routing changed.
|
||||||
"""
|
"""
|
||||||
dest = os.path.join(self.tools_root, tool_id)
|
dest = self._existing_dir_for(tool_id, entry)
|
||||||
git_dir = os.path.join(dest, ".git")
|
git_dir = os.path.join(dest, ".git")
|
||||||
|
|
||||||
# Step-down: each branch resolves one situation and assigns msg.
|
# Step-down: each branch resolves one situation and assigns msg.
|
||||||
|
|
@ -607,7 +797,7 @@ class InstallerManager:
|
||||||
msg = f"Git source cloned: {dest}"
|
msg = f"Git source cloned: {dest}"
|
||||||
|
|
||||||
if post_install:
|
if post_install:
|
||||||
self._run_post_install(tool_id, post_install)
|
self._run_post_install(tool_id, post_install, dest)
|
||||||
return msg
|
return msg
|
||||||
|
|
||||||
def install_git_node(
|
def install_git_node(
|
||||||
|
|
@ -615,6 +805,7 @@ class InstallerManager:
|
||||||
pkg: str,
|
pkg: str,
|
||||||
tool_id: str,
|
tool_id: str,
|
||||||
post_install: str | None = None,
|
post_install: str | None = None,
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Clone or update a git repo and run ``yarn install``.
|
"""Clone or update a git repo and run ``yarn install``.
|
||||||
|
|
||||||
|
|
@ -623,8 +814,11 @@ class InstallerManager:
|
||||||
then ``yarn install`` to pick up any changed dependencies. If
|
then ``yarn install`` to pick up any changed dependencies. If
|
||||||
the destination exists but is not a git repo, or if the pull
|
the destination exists but is not a git repo, or if the pull
|
||||||
fails, move the old dir aside and re-clone fresh.
|
fails, move the old dir aside and re-clone fresh.
|
||||||
|
|
||||||
|
Destination routing matches :meth:`install_git` (existing
|
||||||
|
clones are updated in place wherever they live).
|
||||||
"""
|
"""
|
||||||
dest = os.path.join(self.tools_root, tool_id)
|
dest = self._existing_dir_for(tool_id, entry)
|
||||||
git_dir = os.path.join(dest, ".git")
|
git_dir = os.path.join(dest, ".git")
|
||||||
|
|
||||||
# Step-down: each branch resolves one situation, runs the
|
# Step-down: each branch resolves one situation, runs the
|
||||||
|
|
@ -671,7 +865,7 @@ class InstallerManager:
|
||||||
msg = f"Git+Node source synchronized: {dest}"
|
msg = f"Git+Node source synchronized: {dest}"
|
||||||
|
|
||||||
if post_install:
|
if post_install:
|
||||||
self._run_post_install(tool_id, post_install)
|
self._run_post_install(tool_id, post_install, dest)
|
||||||
return msg
|
return msg
|
||||||
|
|
||||||
# ── git install helpers ────────────────────────────────────────
|
# ── git install helpers ────────────────────────────────────────
|
||||||
|
|
@ -767,6 +961,7 @@ class InstallerManager:
|
||||||
post_install: str | None = None,
|
post_install: str | None = None,
|
||||||
env_overrides: dict[str, str] | None = None,
|
env_overrides: dict[str, str] | None = None,
|
||||||
license_spdx: str | None = None,
|
license_spdx: str | None = None,
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Dispatch to the correct installer strategy.
|
"""Dispatch to the correct installer strategy.
|
||||||
|
|
||||||
|
|
@ -774,6 +969,11 @@ class InstallerManager:
|
||||||
|
|
||||||
Parameters
|
Parameters
|
||||||
----------
|
----------
|
||||||
|
entry :
|
||||||
|
The tool's full registry entry. When supplied, artifacts
|
||||||
|
are routed to the install-root destination declared by
|
||||||
|
``filesystem.install`` / the v3.4 router instead of the
|
||||||
|
flat ``tools_root/<tool_id>``.
|
||||||
license_spdx :
|
license_spdx :
|
||||||
SPDX ID for the tool's license. If provided AND a
|
SPDX ID for the tool's license. If provided AND a
|
||||||
``license_gate`` was passed to the InstallerManager
|
``license_gate`` was passed to the InstallerManager
|
||||||
|
|
@ -806,21 +1006,29 @@ class InstallerManager:
|
||||||
|
|
||||||
strategies: dict[str, Any] = {
|
strategies: dict[str, Any] = {
|
||||||
"ollama": lambda: self.install_ollama(pkg, tool_id),
|
"ollama": lambda: self.install_ollama(pkg, tool_id),
|
||||||
"uv": lambda: self.install_uv(pkg, tool_id, env_overrides),
|
"uv": lambda: self.install_uv(
|
||||||
"pipx": lambda: self.install_pipx(pkg, tool_id, env_overrides),
|
pkg, tool_id, env_overrides, entry,
|
||||||
"pip": lambda: self.install_pip(pkg, tool_id, env_overrides),
|
),
|
||||||
|
"pipx": lambda: self.install_pipx(
|
||||||
|
pkg, tool_id, env_overrides, entry,
|
||||||
|
),
|
||||||
|
"pip": lambda: self.install_pip(
|
||||||
|
pkg, tool_id, env_overrides, entry,
|
||||||
|
),
|
||||||
"script": lambda: self.install_script(
|
"script": lambda: self.install_script(
|
||||||
cmd, ctx, tool_id, env_overrides,
|
cmd, ctx, tool_id, env_overrides,
|
||||||
),
|
),
|
||||||
"pacman": lambda: self.install_pacman(pkg),
|
"pacman": lambda: self.install_pacman(pkg),
|
||||||
"dnf": lambda: self.install_dnf(pkg),
|
"dnf": lambda: self.install_dnf(pkg),
|
||||||
"apt": lambda: self.install_apt(pkg),
|
"apt": lambda: self.install_apt(pkg),
|
||||||
"npm": lambda: self.install_npm(pkg, tool_id, env_overrides),
|
"npm": lambda: self.install_npm(
|
||||||
|
pkg, tool_id, env_overrides, entry,
|
||||||
|
),
|
||||||
"git": lambda: self.install_git(
|
"git": lambda: self.install_git(
|
||||||
pkg, tool_id, post_install, env_overrides,
|
pkg, tool_id, post_install, env_overrides, entry,
|
||||||
),
|
),
|
||||||
"git_node": lambda: self.install_git_node(
|
"git_node": lambda: self.install_git_node(
|
||||||
pkg, tool_id, post_install,
|
pkg, tool_id, post_install, entry,
|
||||||
),
|
),
|
||||||
"custom": lambda: self.install_custom(pkg, tool_id),
|
"custom": lambda: self.install_custom(pkg, tool_id),
|
||||||
}
|
}
|
||||||
|
|
@ -852,6 +1060,7 @@ class InstallerManager:
|
||||||
inst_type=meta.get("installer", {}).get("type", "pacman"),
|
inst_type=meta.get("installer", {}).get("type", "pacman"),
|
||||||
pkg=meta.get("installer", {}).get("pkg", ""),
|
pkg=meta.get("installer", {}).get("pkg", ""),
|
||||||
cmd=meta.get("installer", {}).get("cmd", ""),
|
cmd=meta.get("installer", {}).get("cmd", ""),
|
||||||
|
entry=meta,
|
||||||
)
|
)
|
||||||
for tid, meta in tools.items()
|
for tid, meta in tools.items()
|
||||||
}
|
}
|
||||||
|
|
@ -867,15 +1076,28 @@ class InstallerManager:
|
||||||
post_install: str | None = None,
|
post_install: str | None = None,
|
||||||
env_overrides: dict[str, str] | None = None,
|
env_overrides: dict[str, str] | None = None,
|
||||||
license_spdx: str | None = None,
|
license_spdx: str | None = None,
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
|
update_if_found: bool = True,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Install a tool with preflight detection.
|
"""Install a tool with preflight detection.
|
||||||
|
|
||||||
If the tool is already installed and *force* is False, returns
|
If the tool is already installed and *force* is False:
|
||||||
a message saying the tool exists and suggesting an update.
|
|
||||||
|
* ``update_if_found=True`` (default) — run the update path
|
||||||
|
(:meth:`update_tool`), honouring the registry's
|
||||||
|
``installer.update_cmd`` when declared. This is what makes
|
||||||
|
the UI's single Deploy/Update button behave as a smooth
|
||||||
|
in-place updater.
|
||||||
|
* ``update_if_found=False`` — return the historical
|
||||||
|
"already installed" notice without touching anything.
|
||||||
|
|
||||||
If *force* is True, proceeds with installation regardless.
|
If *force* is True, proceeds with installation regardless.
|
||||||
|
|
||||||
Parameters
|
Parameters
|
||||||
----------
|
----------
|
||||||
|
entry :
|
||||||
|
The tool's full registry entry (routes artifacts + enables
|
||||||
|
update_cmd).
|
||||||
license_spdx :
|
license_spdx :
|
||||||
SPDX ID for the tool's license. Forwarded to ``run()``
|
SPDX ID for the tool's license. Forwarded to ``run()``
|
||||||
for gate checking.
|
for gate checking.
|
||||||
|
|
@ -884,16 +1106,152 @@ class InstallerManager:
|
||||||
# subprocess call on a blocked tool.
|
# subprocess call on a blocked tool.
|
||||||
self._check_license(tool_id, license_spdx)
|
self._check_license(tool_id, license_spdx)
|
||||||
|
|
||||||
check = self.preflight(tool_id, inst_type, pkg, cmd)
|
check = self.preflight(tool_id, inst_type, pkg, cmd, entry)
|
||||||
if check["found"] and not force:
|
if check["found"] and not force:
|
||||||
|
if not update_if_found:
|
||||||
return (
|
return (
|
||||||
f"Tool '{tool_id}' already installed at {check['location']}. "
|
f"Tool '{tool_id}' already installed at "
|
||||||
|
f"{check['location']}. "
|
||||||
f"Version: {check['version'] or 'unknown'}. "
|
f"Version: {check['version'] or 'unknown'}. "
|
||||||
f"Use force=True to update."
|
f"Use force=True to update."
|
||||||
)
|
)
|
||||||
|
update_cmd = (entry or {}).get("installer", {}).get(
|
||||||
|
"update_cmd", "",
|
||||||
|
) if entry else ""
|
||||||
|
return self.update_tool(
|
||||||
|
tool_id=tool_id,
|
||||||
|
inst_type=inst_type,
|
||||||
|
pkg=pkg,
|
||||||
|
cmd=cmd,
|
||||||
|
update_cmd=update_cmd or "",
|
||||||
|
entry=entry,
|
||||||
|
env_overrides=env_overrides,
|
||||||
|
post_install=post_install,
|
||||||
|
detected=check,
|
||||||
|
)
|
||||||
return self.run(
|
return self.run(
|
||||||
inst_type, pkg, cmd, ctx, tool_id,
|
inst_type, pkg, cmd, ctx, tool_id,
|
||||||
post_install, env_overrides,
|
post_install, env_overrides, license_spdx, entry,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ── Updates ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def update_tool(
|
||||||
|
self,
|
||||||
|
tool_id: str,
|
||||||
|
inst_type: str,
|
||||||
|
pkg: str,
|
||||||
|
cmd: str = "",
|
||||||
|
update_cmd: str = "",
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
|
env_overrides: dict[str, str] | None = None,
|
||||||
|
post_install: str | None = None,
|
||||||
|
detected: dict[str, Any] | None = None,
|
||||||
|
) -> str:
|
||||||
|
"""Update an already-installed tool in place.
|
||||||
|
|
||||||
|
Resolution order:
|
||||||
|
|
||||||
|
1. **Registry ``update_cmd``** — rendered with the path-tree
|
||||||
|
placeholders plus ``{install_dir}`` and executed in the
|
||||||
|
tool's install directory (e.g. LibreChat's
|
||||||
|
``git pull --ff-only && yarn install && yarn build``).
|
||||||
|
2. **Per-type defaults** — ``uv tool upgrade``, ``pipx
|
||||||
|
upgrade``, per-tool venv ``pip install -U``, ``git pull
|
||||||
|
--ff-only`` (+ node rebuild for ``git_node``), ``npm
|
||||||
|
update``.
|
||||||
|
3. **System packages** (pacman / dnf / apt) and ``custom``
|
||||||
|
installs return guidance instead of shelling out.
|
||||||
|
|
||||||
|
The working directory is the *detected* install location when
|
||||||
|
known, else the router destination — so updates land where the
|
||||||
|
tool actually lives, never forking a second copy.
|
||||||
|
"""
|
||||||
|
_validate_tool_id(tool_id)
|
||||||
|
dest = (
|
||||||
|
detected.get("location")
|
||||||
|
if detected and detected.get("location")
|
||||||
|
else self._existing_dir_for(tool_id, entry)
|
||||||
|
)
|
||||||
|
env = self._env(tool_id, env_overrides)
|
||||||
|
|
||||||
|
# 1. Registry-declared update command wins.
|
||||||
|
if update_cmd:
|
||||||
|
rendered = self._render_cmd(update_cmd, dest)
|
||||||
|
logger.info("Updating %s via update_cmd: %s", tool_id, rendered)
|
||||||
|
subprocess.run(
|
||||||
|
["bash", "-c", rendered], check=True, env=env,
|
||||||
|
cwd=dest if os.path.isdir(dest) else None,
|
||||||
|
timeout=1800,
|
||||||
|
)
|
||||||
|
return f"Update completed for {tool_id} (update_cmd) in {dest}."
|
||||||
|
|
||||||
|
# 2. Per-type defaults.
|
||||||
|
pkg_first = shlex.split(pkg)[0] if pkg else pkg
|
||||||
|
if inst_type == "uv":
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
["uv", "tool", "upgrade", pkg_first],
|
||||||
|
check=True, env=env, timeout=600,
|
||||||
|
)
|
||||||
|
return f"UV tool '{pkg_first}' upgraded."
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
# upgrade failed (e.g. broken env) — reinstall keeps
|
||||||
|
# --with extras alive via install_uv's step-down.
|
||||||
|
logger.info(
|
||||||
|
"uv upgrade failed for %s; reinstalling", pkg_first,
|
||||||
|
)
|
||||||
|
return self.install_uv(
|
||||||
|
pkg, tool_id, env_overrides, entry,
|
||||||
|
)
|
||||||
|
if inst_type == "pipx":
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
["pipx", "upgrade", pkg_first],
|
||||||
|
check=True, env=env, timeout=600,
|
||||||
|
)
|
||||||
|
return f"pipx '{pkg_first}' upgraded."
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
logger.info(
|
||||||
|
"pipx upgrade failed for %s; reinstalling", pkg_first,
|
||||||
|
)
|
||||||
|
return self.install_pipx(
|
||||||
|
pkg, tool_id, env_overrides, entry,
|
||||||
|
)
|
||||||
|
if inst_type == "pip":
|
||||||
|
venv_pip = os.path.join(dest, ".venv", "bin", "pip")
|
||||||
|
if os.path.isfile(venv_pip):
|
||||||
|
_validate_pkg(pkg_first)
|
||||||
|
subprocess.run(
|
||||||
|
[venv_pip, "install", "--upgrade", pkg_first],
|
||||||
|
check=True, env=env, timeout=600,
|
||||||
|
)
|
||||||
|
return f"pip '{pkg_first}' upgraded in {dest}/.venv."
|
||||||
|
return (
|
||||||
|
f"No managed venv for {tool_id}; re-run with force to "
|
||||||
|
f"reinstall."
|
||||||
|
)
|
||||||
|
if inst_type in ("git", "git_node"):
|
||||||
|
# install_git / install_git_node already implement
|
||||||
|
# pull-ff-only-with-fallback + node rebuild semantics.
|
||||||
|
handler = (self.install_git_node if inst_type == "git_node"
|
||||||
|
else self.install_git)
|
||||||
|
msg = handler(
|
||||||
|
pkg, tool_id, post_install, env_overrides, entry,
|
||||||
|
)
|
||||||
|
return f"Update: {msg}"
|
||||||
|
if inst_type == "npm":
|
||||||
|
_validate_pkg(pkg_first)
|
||||||
|
subprocess.run(
|
||||||
|
["npm", "update", "--prefix", dest, pkg_first],
|
||||||
|
check=True, env=env, timeout=600,
|
||||||
|
)
|
||||||
|
return f"NPM '{pkg_first}' updated in {dest}."
|
||||||
|
|
||||||
|
# 3. Types we deliberately don't auto-update.
|
||||||
|
return (
|
||||||
|
f"'{tool_id}' is a {inst_type}-managed install — update it "
|
||||||
|
f"with the system package manager or its upstream installer."
|
||||||
)
|
)
|
||||||
|
|
||||||
# ── Installation verification ───────────────────────────────────
|
# ── Installation verification ───────────────────────────────────
|
||||||
|
|
@ -905,6 +1263,7 @@ class InstallerManager:
|
||||||
pkg: str,
|
pkg: str,
|
||||||
cmd: str = "",
|
cmd: str = "",
|
||||||
filesystem: dict[str, str] | None = None,
|
filesystem: dict[str, str] | None = None,
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Run a compliance checklist against a single tool installation.
|
"""Run a compliance checklist against a single tool installation.
|
||||||
|
|
||||||
|
|
@ -927,9 +1286,10 @@ class InstallerManager:
|
||||||
fs = filesystem or {}
|
fs = filesystem or {}
|
||||||
tool_dir = os.path.join(self.tools_root, tool_id)
|
tool_dir = os.path.join(self.tools_root, tool_id)
|
||||||
|
|
||||||
# 1. Native install detected
|
# 1. Native install detected (multi-root: router + dashboards
|
||||||
|
# + tools, so pre-v3.4 installs still verify)
|
||||||
location, version = self._detect_installation(
|
location, version = self._detect_installation(
|
||||||
tool_id, inst_type, pkg, cmd,
|
tool_id, inst_type, pkg, cmd, entry,
|
||||||
)
|
)
|
||||||
checks.append(VerifyCheck(
|
checks.append(VerifyCheck(
|
||||||
name="Native Install",
|
name="Native Install",
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -219,6 +219,7 @@ if _HAS_QT:
|
||||||
self._setup_environment_hierarchy()
|
self._setup_environment_hierarchy()
|
||||||
self._migrate_legacy_state_files()
|
self._migrate_legacy_state_files()
|
||||||
|
|
||||||
|
self._migrate_registry_dir()
|
||||||
self.dtach_bin: str | None = find_binary("dtach-ng", "dtach")
|
self.dtach_bin: str | None = find_binary("dtach-ng", "dtach")
|
||||||
# License gate — checks every tool's license before the
|
# License gate — checks every tool's license before the
|
||||||
# installer dispatches to a subprocess. SaaS-blocked
|
# installer dispatches to a subprocess. SaaS-blocked
|
||||||
|
|
@ -245,7 +246,7 @@ if _HAS_QT:
|
||||||
self.txt_base_dir.setReadOnly(True)
|
self.txt_base_dir.setReadOnly(True)
|
||||||
|
|
||||||
self.registry_mgr = RegistryManager(
|
self.registry_mgr = RegistryManager(
|
||||||
os.path.join(self.base_dir, "registry")
|
os.path.join(self.base_dir, "configs", "registry")
|
||||||
)
|
)
|
||||||
self.skill_resolver = (
|
self.skill_resolver = (
|
||||||
SkillRuntimeResolver(self.skills_root)
|
SkillRuntimeResolver(self.skills_root)
|
||||||
|
|
@ -360,6 +361,40 @@ if _HAS_QT:
|
||||||
filter(None, [uv_bin, npm_bin])
|
filter(None, [uv_bin, npm_bin])
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def _migrate_registry_dir(self) -> None:
|
||||||
|
"""One-time move of the app registry into configs/registry.
|
||||||
|
|
||||||
|
v3.4 aligned the base dir with the canonical file-system
|
||||||
|
map: the app-internal ecosystem registry (ecosystem.json +
|
||||||
|
user tweaks) is app state and belongs under ``configs/``.
|
||||||
|
Pre-v3.4 installs kept it at ``<base_dir>/registry/`` —
|
||||||
|
move it when the legacy dir exists and the new one does
|
||||||
|
not; remove the legacy dir when it is left empty. Never
|
||||||
|
raises.
|
||||||
|
"""
|
||||||
|
import shutil
|
||||||
|
from ai_lsc.utils.paths import legacy_registry_dir
|
||||||
|
|
||||||
|
old = legacy_registry_dir(self.base_dir)
|
||||||
|
new = os.path.join(self.base_dir, "configs", "registry")
|
||||||
|
try:
|
||||||
|
if os.path.isdir(old):
|
||||||
|
os.makedirs(new, exist_ok=True)
|
||||||
|
for fname in os.listdir(old):
|
||||||
|
src = os.path.join(old, fname)
|
||||||
|
dst = os.path.join(new, fname)
|
||||||
|
if not os.path.exists(dst):
|
||||||
|
shutil.move(src, dst)
|
||||||
|
if not os.listdir(old):
|
||||||
|
os.rmdir(old)
|
||||||
|
from ai_lsc.utils.logging import get_logger
|
||||||
|
get_logger(__name__).info(
|
||||||
|
"Migrated tool registry to %s (legacy %s "
|
||||||
|
"removed)", new, old,
|
||||||
|
)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
# ───────────────────────────────────────────────────────────────
|
# ───────────────────────────────────────────────────────────────
|
||||||
# LEFT NAVIGATION RACK LAYOUT
|
# LEFT NAVIGATION RACK LAYOUT
|
||||||
# ───────────────────────────────────────────────────────────────
|
# ───────────────────────────────────────────────────────────────
|
||||||
|
|
@ -964,7 +999,7 @@ if _HAS_QT:
|
||||||
layout.setContentsMargins(30, 20, 30, 20)
|
layout.setContentsMargins(30, 20, 30, 20)
|
||||||
|
|
||||||
header = QHBoxLayout()
|
header = QHBoxLayout()
|
||||||
lbl_title = QLabel("<b>AI-LSC v3.1 — Ankh of Jah</b>")
|
lbl_title = QLabel("<b>AI-LSC v3.3.0 — Ankh of Jah</b>")
|
||||||
lbl_title.setFont(QFont("Segoe UI", 16))
|
lbl_title.setFont(QFont("Segoe UI", 16))
|
||||||
header.addWidget(lbl_title)
|
header.addWidget(lbl_title)
|
||||||
header.addStretch()
|
header.addStretch()
|
||||||
|
|
@ -1368,11 +1403,14 @@ if _HAS_QT:
|
||||||
if handler:
|
if handler:
|
||||||
handler()
|
handler()
|
||||||
|
|
||||||
# Drift detection
|
# Drift detection (multi-root: checks the router dir,
|
||||||
|
# dashboards/, and tools/ so pre-v3.4 installs are not
|
||||||
|
# false-flagged)
|
||||||
state_path = self._get_active_state_file()
|
state_path = self._get_active_state_file()
|
||||||
drift: list[str] = []
|
drift: list[str] = []
|
||||||
if state_path:
|
if state_path:
|
||||||
try:
|
try:
|
||||||
|
from ai_lsc.utils.paths import candidate_install_dirs
|
||||||
with open(state_path) as f:
|
with open(state_path) as f:
|
||||||
state = json.load(f)
|
state = json.load(f)
|
||||||
git_types = {"git", "git_node"}
|
git_types = {"git", "git_node"}
|
||||||
|
|
@ -1383,8 +1421,12 @@ if _HAS_QT:
|
||||||
and self.registry_mgr.get_tool(tid)
|
and self.registry_mgr.get_tool(tid)
|
||||||
.get("installer", {})
|
.get("installer", {})
|
||||||
.get("type") in git_types
|
.get("type") in git_types
|
||||||
and not os.path.exists(
|
and not any(
|
||||||
os.path.join(self.tools_root, tid)
|
d.is_dir() for d in candidate_install_dirs(
|
||||||
|
tid,
|
||||||
|
self.registry_mgr.get_tool(tid),
|
||||||
|
self.base_dir,
|
||||||
|
)
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
except Exception:
|
except Exception:
|
||||||
|
|
@ -1449,6 +1491,11 @@ if _HAS_QT:
|
||||||
label = {
|
label = {
|
||||||
"lxc": "LXC configs + launch script",
|
"lxc": "LXC configs + launch script",
|
||||||
"firecracker": "Firecracker microVM configs + launch script",
|
"firecracker": "Firecracker microVM configs + launch script",
|
||||||
|
"qcow2": "QCOW2 VM disk image",
|
||||||
|
"qcow": "QCOW (legacy) VM disk image",
|
||||||
|
"raw": "Raw VM disk image",
|
||||||
|
"qcrows": "QCrows VM container image (.qcrows)",
|
||||||
|
"svb": "Sovereign Bundle (.svb)",
|
||||||
}.get(backend_type, f"{backend_type.capitalize()} Compose")
|
}.get(backend_type, f"{backend_type.capitalize()} Compose")
|
||||||
self.log(
|
self.log(
|
||||||
f"{label} generated: {out_file}",
|
f"{label} generated: {out_file}",
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,8 @@
|
||||||
"""ContainerStacksTab widget — lists exported stack files.
|
"""ContainerStacksTab widget — lists exported stack files.
|
||||||
|
|
||||||
Displays available stack snapshots and provides export buttons for
|
Displays available stack snapshots and provides export buttons for
|
||||||
Podman Compose and Docker Compose outputs.
|
Podman Compose, Docker Compose, LXC, Firecracker, and QCOW2/QCOW VM
|
||||||
|
disk-image outputs.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
|
|
@ -60,6 +61,32 @@ if _HAS_QT:
|
||||||
lambda: self.main.finalize_stack_export("firecracker")
|
lambda: self.main.finalize_stack_export("firecracker")
|
||||||
)
|
)
|
||||||
header.addWidget(btn_firecracker)
|
header.addWidget(btn_firecracker)
|
||||||
|
|
||||||
|
btn_qcow2 = QPushButton("Export -> QCOW2 Image")
|
||||||
|
btn_qcow2.setStyleSheet("background-color: #d35400;")
|
||||||
|
btn_qcow2.clicked.connect(
|
||||||
|
lambda: self.main.finalize_stack_export("qcow2")
|
||||||
|
)
|
||||||
|
header.addWidget(btn_qcow2)
|
||||||
|
|
||||||
|
btn_qcrows = QPushButton("Export -> QCrows Image (.qcrows)")
|
||||||
|
btn_qcrows.setStyleSheet("background-color: #7f8c8d;")
|
||||||
|
btn_qcrows.clicked.connect(
|
||||||
|
lambda: self.main.finalize_stack_export("qcrows")
|
||||||
|
)
|
||||||
|
header.addWidget(btn_qcrows)
|
||||||
|
|
||||||
|
btn_svb = QPushButton("Export -> Sovereign .svb")
|
||||||
|
btn_svb.clicked.connect(
|
||||||
|
lambda: self.main.finalize_stack_export("svb")
|
||||||
|
)
|
||||||
|
header.addWidget(btn_svb)
|
||||||
|
|
||||||
|
btn_qcow = QPushButton("Export -> QCOW (legacy)")
|
||||||
|
btn_qcow.clicked.connect(
|
||||||
|
lambda: self.main.finalize_stack_export("qcow")
|
||||||
|
)
|
||||||
|
header.addWidget(btn_qcow)
|
||||||
layout.addLayout(header)
|
layout.addLayout(header)
|
||||||
|
|
||||||
self.file_list = QListWidget()
|
self.file_list = QListWidget()
|
||||||
|
|
@ -75,12 +102,27 @@ if _HAS_QT:
|
||||||
# the LXC and Firecracker backends.
|
# the LXC and Firecracker backends.
|
||||||
if fname.endswith((".yml", ".yaml", ".json", ".sh")):
|
if fname.endswith((".yml", ".yaml", ".json", ".sh")):
|
||||||
self.file_list.addItem(fname)
|
self.file_list.addItem(fname)
|
||||||
# Also surface subdirectories that hold per-container /
|
# Image-format dirs (qcow2/, and any future format like the
|
||||||
# per-VM configs (lxc/, firecracker/).
|
# in-house qcrows/) list their images + sidecar manifests
|
||||||
|
# with sizes; config dirs (lxc/, firecracker/) get a plain
|
||||||
|
# directory entry.
|
||||||
for fname in sorted(os.listdir(self.main.exports_root)):
|
for fname in sorted(os.listdir(self.main.exports_root)):
|
||||||
|
if fname.startswith("."):
|
||||||
|
# transient .staging-* dirs from in-flight exports
|
||||||
|
continue
|
||||||
fpath = os.path.join(self.main.exports_root, fname)
|
fpath = os.path.join(self.main.exports_root, fname)
|
||||||
if os.path.isdir(fpath):
|
if not os.path.isdir(fpath):
|
||||||
|
continue
|
||||||
|
if fname in ("lxc", "firecracker"):
|
||||||
self.file_list.addItem(f"{fname}/ (directory)")
|
self.file_list.addItem(f"{fname}/ (directory)")
|
||||||
|
continue
|
||||||
|
for sub in sorted(os.listdir(fpath)):
|
||||||
|
sub_path = os.path.join(fpath, sub)
|
||||||
|
if os.path.isfile(sub_path) and not sub.startswith("."):
|
||||||
|
size_mb = os.path.getsize(sub_path) // (1024 * 1024)
|
||||||
|
self.file_list.addItem(
|
||||||
|
f"{fname}/{sub} ({size_mb} MiB)"
|
||||||
|
)
|
||||||
|
|
||||||
else:
|
else:
|
||||||
ContainerStacksTab = None
|
ContainerStacksTab = None
|
||||||
|
|
|
||||||
|
|
@ -612,13 +612,17 @@ if _HAS_QT:
|
||||||
mgr = InstallerManager(
|
mgr = InstallerManager(
|
||||||
tools_root=self.main.tools_root,
|
tools_root=self.main.tools_root,
|
||||||
base_bin_dir=self.main.base_bin_dir,
|
base_bin_dir=self.main.base_bin_dir,
|
||||||
|
base_dir=self.main.base_dir,
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
if inst_type == "git_node":
|
if inst_type == "git_node":
|
||||||
msg = mgr.install_git_node(pkg, tool_id, post_install)
|
msg = mgr.install_git_node(
|
||||||
|
pkg, tool_id, post_install, entry=meta,
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
msg = mgr.install_git(
|
msg = mgr.install_git(
|
||||||
pkg, tool_id, post_install, env_overrides or None,
|
pkg, tool_id, post_install, env_overrides or None,
|
||||||
|
entry=meta,
|
||||||
)
|
)
|
||||||
self.main.log(msg, "GitRepo")
|
self.main.log(msg, "GitRepo")
|
||||||
self.refresh()
|
self.refresh()
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,11 @@ Renders one tool (or skill:-prefixed behavior binding) inside the
|
||||||
Tools/Services page. Each row shows the service name, live status,
|
Tools/Services page. Each row shows the service name, live status,
|
||||||
CPU load, port input, model selector (for engine/LLM-runtime services),
|
CPU load, port input, model selector (for engine/LLM-runtime services),
|
||||||
Ollama pull controls, launcher buttons (CLI/GUI/Web), and
|
Ollama pull controls, launcher buttons (CLI/GUI/Web), and
|
||||||
Install/Sync + Start/Stop action buttons.
|
Install/Update + Start/Stop action buttons. The install button runs
|
||||||
|
preflight first: missing tools install to their map-aligned root
|
||||||
|
(dashboards/ for web UIs, tools/ otherwise); already-installed tools
|
||||||
|
update in place (registry ``update_cmd`` when declared, per-type
|
||||||
|
default otherwise).
|
||||||
|
|
||||||
All process management is delegated to
|
All process management is delegated to
|
||||||
:class:`~ai_lsc.runtime.executor.RuntimeExecutor` -- this widget
|
:class:`~ai_lsc.runtime.executor.RuntimeExecutor` -- this widget
|
||||||
|
|
@ -222,7 +226,7 @@ if _HAS_QT:
|
||||||
btn_api.clicked.connect(self._open_api_dialog)
|
btn_api.clicked.connect(self._open_api_dialog)
|
||||||
layout.addWidget(btn_api)
|
layout.addWidget(btn_api)
|
||||||
|
|
||||||
self.btn_update = QPushButton("Install / Sync")
|
self.btn_update = QPushButton("Install / Update")
|
||||||
self.btn_update.setStyleSheet("background-color: #8e44ad;")
|
self.btn_update.setStyleSheet("background-color: #8e44ad;")
|
||||||
self.btn_update.clicked.connect(self.smart_install)
|
self.btn_update.clicked.connect(self.smart_install)
|
||||||
self._install_btn_original_text = self.btn_update.text()
|
self._install_btn_original_text = self.btn_update.text()
|
||||||
|
|
@ -331,6 +335,7 @@ if _HAS_QT:
|
||||||
tool_id=self.tool_id,
|
tool_id=self.tool_id,
|
||||||
ctx=ctx,
|
ctx=ctx,
|
||||||
license_spdx=license_spdx,
|
license_spdx=license_spdx,
|
||||||
|
registry_entry=self.meta,
|
||||||
)
|
)
|
||||||
QTimer.singleShot(
|
QTimer.singleShot(
|
||||||
0, lambda d=desc, t=self.tool_id: self._on_install_done(d, t)
|
0, lambda d=desc, t=self.tool_id: self._on_install_done(d, t)
|
||||||
|
|
|
||||||
|
|
@ -85,6 +85,7 @@ class VerificationWorker(QThread if _HAS_QT else object): # type: ignore[misc]
|
||||||
pkg=installer.get("pkg", ""),
|
pkg=installer.get("pkg", ""),
|
||||||
cmd=installer.get("cmd", ""),
|
cmd=installer.get("cmd", ""),
|
||||||
filesystem=fs,
|
filesystem=fs,
|
||||||
|
entry=meta,
|
||||||
)
|
)
|
||||||
self.tool_done.emit(tool_id, result)
|
self.tool_done.emit(tool_id, result)
|
||||||
completed += 1
|
completed += 1
|
||||||
|
|
|
||||||
|
|
@ -4,14 +4,26 @@ AI-LSC — Centralised path definitions.
|
||||||
Every path in the application is derived from ``BASE_DIR``
|
Every path in the application is derived from ``BASE_DIR``
|
||||||
using :mod:`pathlib`. Import these in UI and orchestration code
|
using :mod:`pathlib`. Import these in UI and orchestration code
|
||||||
instead of constructing paths with ``os.path.join`` ad-hoc.
|
instead of constructing paths with ``os.path.join`` ad-hoc.
|
||||||
|
|
||||||
|
This module also owns the **install-root router** — the mapping that
|
||||||
|
decides where a tool's artifacts live under the canonical /mnt/AI
|
||||||
|
layout (``tools/`` for standalone compiles, ``dashboards/`` for web
|
||||||
|
UIs, ``runtime/`` for native binaries) — plus the multi-root
|
||||||
|
candidate list used for detection of pre-existing installs.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
from ai_lsc.constants import BASE_DIR, REQUIRED_DIRS
|
from ai_lsc.constants import BASE_DIR, REQUIRED_DIRS
|
||||||
|
|
||||||
|
# Only path-tree keys are auto-resolvable as launcher placeholders.
|
||||||
|
# `{port}` / `{model_arg}` are runtime values resolved separately.
|
||||||
|
_TOOL_ID_RE = re.compile(r"^[A-Za-z0-9_.:\-]+$")
|
||||||
|
|
||||||
|
|
||||||
def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
|
def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
|
||||||
"""Return a dict of well-known absolute paths under *base_dir*.
|
"""Return a dict of well-known absolute paths under *base_dir*.
|
||||||
|
|
@ -41,19 +53,21 @@ def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
|
||||||
"datasets_root": Path("/mnt/AI/datasets"), # parent of wordlists/, huggingface/, github/
|
"datasets_root": Path("/mnt/AI/datasets"), # parent of wordlists/, huggingface/, github/
|
||||||
"pipelines_root": Path("/mnt/AI/pipelines"), # ETL / chunking / routing scripts
|
"pipelines_root": Path("/mnt/AI/pipelines"), # ETL / chunking / routing scripts
|
||||||
"configs_root": Path("/mnt/AI/configs"), # app state + templated configs
|
"configs_root": Path("/mnt/AI/configs"), # app state + templated configs
|
||||||
"registry_root": Path("/mnt/AI/registry"), # app-internal: ecosystem.json + manifests/
|
"registry_root": Path("/mnt/AI/configs/registry"), # app-internal: ecosystem.json
|
||||||
"agents_root": Path("/mnt/AI/agents"), # configs and chains for autonomous actors
|
"agents_root": Path("/mnt/AI/agents"), # configs and chains for autonomous actors
|
||||||
"skills_root": Path("/mnt/AI/skills"), # 3rd-party integrations and tool wrappers
|
"skills_root": Path("/mnt/AI/skills"), # 3rd-party integrations and tool wrappers
|
||||||
"projects_root": Path("/mnt/AI/projects"), # parent of active/, labs/, vault/
|
"projects_root": Path("/mnt/AI/projects"), # parent of active/, labs/, vault/
|
||||||
"blueprints_root": Path("/mnt/AI/blueprints"), # Dockerfiles / build contexts for Podman exports
|
"blueprints_root": Path("/mnt/AI/blueprints"), # Dockerfiles / build contexts for Podman exports
|
||||||
"workspaces_root": Path("/mnt/AI/workspaces"), # Jupyter, OpenNotebook, etc.
|
"workspaces_root": Path("/mnt/AI/workspaces"), # Jupyter, OpenNotebook, etc.
|
||||||
"dashboards_root": Path("/mnt/AI/dashboards"), # web UIs (Dashy, Open-WebUI, Hermes WebUI, etc.)
|
"dashboards_root": Path("/mnt/AI/dashboards"), # web UIs (Dashy, Homepage, Open-WebUI, etc.)
|
||||||
"exports_root": Path("/mnt/AI/exports"), # parent of oci-images/
|
"exports_root": Path("/mnt/AI/exports"), # parent of oci-images/, qcow2/, qcrows/
|
||||||
|
"exports_qcow2": Path("/mnt/AI/exports/qcow2"), # VM disk images (qcow2 / legacy qcow)
|
||||||
|
"exports_qcrows": Path("/mnt/AI/exports/qcrows"),# QCrows VM container images (.qcrows)
|
||||||
"scripts_root": Path("/mnt/AI/scripts"), # system admin / maintenance automation
|
"scripts_root": Path("/mnt/AI/scripts"), # system admin / maintenance automation
|
||||||
"logs_root": Path("/mnt/AI/logs"),
|
"logs_root": Path("/mnt/AI/logs"),
|
||||||
"backends_root": Path("/mnt/AI/backends"), # S3/MinIO/Ceph connection profiles
|
"backends_root": Path("/mnt/AI/backends"), # S3/MinIO/Ceph connection profiles
|
||||||
"distfiles_root": Path("/mnt/AI/distfiles"), # permanent local mirror of source tarballs
|
"distfiles_root": Path("/mnt/AI/distfiles"), # permanent local mirror of source tarballs
|
||||||
"config_root": Path("/mnt/AI/configs"), # app state + templated configs
|
"config_root": Path("/mnt/AI/configs"), # alias of configs_root
|
||||||
}
|
}
|
||||||
"""
|
"""
|
||||||
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
|
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
|
||||||
|
|
@ -67,7 +81,11 @@ def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
|
||||||
"corpus_root": root / "corpus",
|
"corpus_root": root / "corpus",
|
||||||
"datasets_root": root / "datasets",
|
"datasets_root": root / "datasets",
|
||||||
"pipelines_root": root / "pipelines",
|
"pipelines_root": root / "pipelines",
|
||||||
"registry_root": root / "registry",
|
# App-internal registry state (ecosystem.json + user tweaks)
|
||||||
|
# lives under configs/ per the v3.4 file-system map. Pre-v3.4
|
||||||
|
# installs used <base>/registry — main_window migrates that
|
||||||
|
# directory on startup.
|
||||||
|
"registry_root": root / "configs" / "registry",
|
||||||
"agents_root": root / "agents",
|
"agents_root": root / "agents",
|
||||||
"skills_root": root / "skills",
|
"skills_root": root / "skills",
|
||||||
"projects_root": root / "projects",
|
"projects_root": root / "projects",
|
||||||
|
|
@ -75,6 +93,8 @@ def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
|
||||||
"workspaces_root": root / "workspaces",
|
"workspaces_root": root / "workspaces",
|
||||||
"dashboards_root": root / "dashboards",
|
"dashboards_root": root / "dashboards",
|
||||||
"exports_root": root / "exports",
|
"exports_root": root / "exports",
|
||||||
|
"exports_qcow2": root / "exports" / "qcow2",
|
||||||
|
"exports_qcrows": root / "exports" / "qcrows",
|
||||||
"scripts_root": root / "scripts",
|
"scripts_root": root / "scripts",
|
||||||
"logs_root": root / "logs",
|
"logs_root": root / "logs",
|
||||||
"backends_root": root / "backends",
|
"backends_root": root / "backends",
|
||||||
|
|
@ -89,6 +109,121 @@ def build_path_tree(base_dir: str | Path | None = None) -> dict[str, Path]:
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── Install-root router ─────────────────────────────────────────────
|
||||||
|
#
|
||||||
|
# The v3.4 file-system map gives every top-level directory a purpose:
|
||||||
|
# tools/ — standalone compiles (built from distfiles)
|
||||||
|
# runtime/ — native compiled binaries (ollama, llama.cpp, MinIO…)
|
||||||
|
# dashboards/ — web UIs and landing pages (Open-WebUI, LibreChat…)
|
||||||
|
#
|
||||||
|
# A tool's declared ``filesystem.install`` (a path relative to the base
|
||||||
|
# dir) is the single source of truth. When absent, the router picks a
|
||||||
|
# root by tool kind — but only when the launcher does not already pin
|
||||||
|
# the tool to ``{tools_root}`` via a ``cd {tools_root}/<id>`` command
|
||||||
|
# (a pinned launcher would break if the artifacts moved).
|
||||||
|
INSTALL_ROOT_DEFAULT = "tools"
|
||||||
|
INSTALL_ROOT_ROUTES: dict[str, str] = {
|
||||||
|
"web": "dashboards", # flags.has_web → web UI apps
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def install_dir_for(
|
||||||
|
entry: dict[str, Any] | None,
|
||||||
|
tool_id: str = "",
|
||||||
|
base_dir: str | Path | None = None,
|
||||||
|
) -> Path:
|
||||||
|
"""Return the canonical install directory for a registry *entry*.
|
||||||
|
|
||||||
|
Resolution order:
|
||||||
|
|
||||||
|
1. ``filesystem.install`` declared on the entry (relative to the
|
||||||
|
base dir) — honoured verbatim.
|
||||||
|
2. ``flags.has_web`` is set AND the launcher does not reference
|
||||||
|
``{tools_root}`` → ``<base>/dashboards/<tool_id>/``.
|
||||||
|
3. Otherwise → ``<base>/tools/<tool_id>/`` (the historical
|
||||||
|
default, still correct for standalone compiles).
|
||||||
|
|
||||||
|
Falls back to ``tools/<tool_id>`` for empty / malformed entries.
|
||||||
|
"""
|
||||||
|
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
|
||||||
|
entry = entry or {}
|
||||||
|
|
||||||
|
rel = (entry.get("filesystem") or {}).get("install", "")
|
||||||
|
if rel:
|
||||||
|
# Normalise without allowing traversal outside the base dir.
|
||||||
|
parts = [p for p in Path(rel).parts if p not in (".", "..")]
|
||||||
|
clean = Path(*parts) if parts else Path("")
|
||||||
|
if str(clean) not in ("", "."):
|
||||||
|
return root / clean
|
||||||
|
|
||||||
|
if tool_id and not _TOOL_ID_RE.fullmatch(tool_id):
|
||||||
|
tool_id = ""
|
||||||
|
|
||||||
|
launcher_cmd = (entry.get("launcher") or {}).get("cmd", "") or ""
|
||||||
|
pins_tools_root = "{tools_root}" in launcher_cmd
|
||||||
|
flags = entry.get("flags") or {}
|
||||||
|
if (tool_id and flags.get("has_web")
|
||||||
|
and INSTALL_ROOT_ROUTES.get("web")
|
||||||
|
and not pins_tools_root):
|
||||||
|
return root / INSTALL_ROOT_ROUTES["web"] / tool_id
|
||||||
|
return root / INSTALL_ROOT_DEFAULT / (tool_id or "")
|
||||||
|
|
||||||
|
|
||||||
|
def candidate_install_dirs(
|
||||||
|
tool_id: str,
|
||||||
|
entry: dict[str, Any] | None = None,
|
||||||
|
base_dir: str | Path | None = None,
|
||||||
|
) -> list[Path]:
|
||||||
|
"""Return every directory an install of *tool_id* might live in.
|
||||||
|
|
||||||
|
Ordered best-candidate-first: the router's canonical dir, then the
|
||||||
|
other plausible roots (dashboards/, tools/), de-duplicated. Used
|
||||||
|
by installer preflight / verification / drift detection so tools
|
||||||
|
installed before the v3.4 routing change are still detected.
|
||||||
|
"""
|
||||||
|
if not tool_id or not _TOOL_ID_RE.fullmatch(tool_id):
|
||||||
|
return []
|
||||||
|
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
|
||||||
|
entry = entry or {}
|
||||||
|
|
||||||
|
cands: list[Path] = [install_dir_for(entry, tool_id, root)]
|
||||||
|
for dirname in (INSTALL_ROOT_ROUTES.get("web", "dashboards"),
|
||||||
|
INSTALL_ROOT_DEFAULT):
|
||||||
|
cands.append(root / dirname / tool_id)
|
||||||
|
# Pre-v3.4 git tools could land in a dir named after the repo
|
||||||
|
# (not the tool_id) — cover the common LibreChat-style casing too.
|
||||||
|
repo_hint = (entry.get("installer", {}) or {}).get("pkg", "")
|
||||||
|
if isinstance(repo_hint, str) and repo_hint.rstrip("/").endswith(".git"):
|
||||||
|
repo_name = repo_hint.rstrip("/").rsplit("/", 1)[-1][:-4]
|
||||||
|
if repo_name and _TOOL_ID_RE.fullmatch(repo_name):
|
||||||
|
cands.append(root / INSTALL_ROOT_DEFAULT / repo_name)
|
||||||
|
|
||||||
|
seen: set[Path] = set()
|
||||||
|
ordered: list[Path] = []
|
||||||
|
for c in cands:
|
||||||
|
if c not in seen:
|
||||||
|
seen.add(c)
|
||||||
|
ordered.append(c)
|
||||||
|
return ordered
|
||||||
|
|
||||||
|
|
||||||
|
def legacy_registry_dir(base_dir: str | Path | None = None) -> Path:
|
||||||
|
"""Pre-v3.4 location of the app-internal registry dir."""
|
||||||
|
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
|
||||||
|
return root / "registry"
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_required_dirs(base_dir: str | Path | None = None) -> list[Path]:
|
||||||
|
"""Create every REQUIRED_DIRS entry under *base_dir*; return the list."""
|
||||||
|
root = Path(base_dir) if base_dir is not None else Path(BASE_DIR)
|
||||||
|
created: list[Path] = []
|
||||||
|
for rel in REQUIRED_DIRS:
|
||||||
|
p = root / rel
|
||||||
|
p.mkdir(parents=True, exist_ok=True)
|
||||||
|
created.append(p)
|
||||||
|
return created
|
||||||
|
|
||||||
|
|
||||||
def resolve_launcher_cmd(
|
def resolve_launcher_cmd(
|
||||||
cmd_template: str,
|
cmd_template: str,
|
||||||
base_dir: str | Path | None = None,
|
base_dir: str | Path | None = None,
|
||||||
|
|
@ -97,17 +232,30 @@ def resolve_launcher_cmd(
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Resolve ``{placeholder}`` tokens in a launcher command template.
|
"""Resolve ``{placeholder}`` tokens in a launcher command template.
|
||||||
|
|
||||||
Recognised placeholders (case-sensitive):
|
Every key of :func:`build_path_tree` is recognised as a
|
||||||
``{port}``, ``{base_dir}``, ``{tools_root}``,
|
placeholder (``{base_dir}``, ``{tools_root}``, ``{dashboards_root}``,
|
||||||
``{models_root}``, ``{workspaces_root}``, ``{model_arg}``
|
``{runtime_root}``, ``{configs_root}``, …), plus the runtime values
|
||||||
|
``{port}`` and ``{model_arg}``. Unknown placeholders are left
|
||||||
|
untouched so callers can chain their own substitution.
|
||||||
"""
|
"""
|
||||||
paths = build_path_tree(base_dir)
|
paths = build_path_tree(base_dir)
|
||||||
|
resolved = cmd_template
|
||||||
|
for key, value in paths.items():
|
||||||
|
resolved = resolved.replace("{" + key + "}", str(value))
|
||||||
return (
|
return (
|
||||||
cmd_template
|
resolved
|
||||||
.replace("{port}", str(port or ""))
|
.replace("{port}", str(port or ""))
|
||||||
.replace("{base_dir}", str(paths["base_dir"]))
|
|
||||||
.replace("{tools_root}", str(paths["tools_root"]))
|
|
||||||
.replace("{models_root}", str(paths["models_root"]))
|
|
||||||
.replace("{workspaces_root}", str(paths["workspaces_root"]))
|
|
||||||
.replace("{model_arg}", model_arg)
|
.replace("{model_arg}", model_arg)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"build_path_tree",
|
||||||
|
"install_dir_for",
|
||||||
|
"candidate_install_dirs",
|
||||||
|
"legacy_registry_dir",
|
||||||
|
"ensure_required_dirs",
|
||||||
|
"resolve_launcher_cmd",
|
||||||
|
"INSTALL_ROOT_DEFAULT",
|
||||||
|
"INSTALL_ROOT_ROUTES",
|
||||||
|
]
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue