dcosnet
/
ai-lsc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

AI Local Stack Control (AI-LSC) provides a unified interface to discover, configure, launch, and manage 125 tools spanning the entire AI software stack — from GPU runtimes and inference engines to agent frameworks and container deployment targets.

This commit is contained in:
Jeremy Anderson 2026-08-12 16:04:38 -04:00
commit ece3b643df
136 changed files with 38272 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

73
.gitignore vendored Executable file
View File

@ -0,0 +1,73 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
*.egg-info/
*.egg
dist/
build/
.eggs/
*.whl
# Virtual environments
.venv/
venv/
env/
ENV/
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
.project
.classpath
.settings/
# Testing
.pytest_cache/
.coverage
htmlcov/
.mypy_cache/
.ruff_cache/
# Type checking
*.pyi
# Distribution
*.tar.gz
*.tar.bz2
*.zip
# OS files
.DS_Store
Thumbs.db
*.log
# Application runtime data
pipeline_state.json
pipeline.json
config.json
ai_lsc_state.json
# Tool output
*.yaml.bak
lxc-launch.sh
*.conf.bak
# Downloaded models (Ollama, etc.)
models/
weights/
# Logs
logs/
*.log
# Guardrail baseline (internal, not for publishing)
.guardrail_baseline.json
# Temporary files
tmp/
temp/

282
CHANGES.md Executable file
View File

@ -0,0 +1,282 @@
# AI-LSC Changelog
## v3.1 — 2026-07-07
Codename: **Ankh of Jah** (continuation)
The v3.1 release is a hardening + polish pass on top of v3.0. It applies the full master code critique (91 of 93 findings addressed), introduces two new UI widgets (Pipeline Ticker + Workspace Tab), and ships three latent-bug fixes that were caught during the post-pass double-check.
### New features
#### Pipeline Ticker
A horizontally scrolling status bar at the top of every workspace tab. Reads the active-tools set from `pipeline_state.json`, joins it against the `STACK_WIRINGS` topology in `stack/connections.py`, and renders:
```
ollama ──openai_api──▶ litellm ──openai_api──▶ aider ❗ open_webui
```
- **Color-coded arrows** by interface type — blue for `openai_api`, green for `vector` / `vector_search` / `embedding`, orange for `redis_pubsub` / `redis_cache`, purple for `postgresql` / `mariadb` / `mysql`, teal for `http_api` / `websocket` / `grpc`, slate for `filesystem` / `tmux_socket` / `systemd_unit`, red for `cuda_driver`.
- **Orphan detection** — any active tool that has no in-stack wiring to another active tool is flagged red with an `❗` prefix. This surfaces real registry gaps (e.g. the `open_webui` vs `openwebui` tool_id collision).
- **Running-state coloring** — pills are blue when the tool is running, white when stopped. The ticker refreshes on every 2 s service-status poll so state changes propagate immediately.
- **Hover-pause** — mouse over the ticker to stop the scroll so you can read a long flow.
- **Click-to-jump** — click any tool pill to switch to the Tools tab; the matching row is selected, scrolled to center, and flashed amber for 1.5 s.
Files: `src/ai_lsc/ui/widgets/pipeline_ticker.py` (new, ~330 lines), `src/ai_lsc/ui/main_window.py` (wiring), `src/ai_lsc/ui/pages/tools_tab.py` (new `highlight_tool()` method), `src/ai_lsc/ui/pages/service_row.py` (new `is_running_now()` cache method).
#### Workspace Tab
A new **Workspace** nav entry (between Chat and Git Sources) that provides virt-manager / aqemu–style peek orchestration. Every active tool gets its own sub-tab:
- **🌐 Web tools** (`has_web=True`) embed via `QWebEngineView` at `http://127.0.0.1:{port}`. A URL bar at the top shows the loaded URL; ⟳ reloads, ↗ opens in an external browser. No need to leave the app for a browser.
- **⌨ CLI tools** (`has_cli=True`, no web) attach to their tmux session via `tmux capture-pane -t <session>::<tool_id> -p -S -200` polled at 4 Hz. The terminal is read-only — use a real terminal app for interactive input.
- **📦 Passive / library tools** get a placeholder explaining they have no interactive surface.
- **⏸ Not-yet-running tools** show a **Start tool** button that wires back to the existing `ServiceRow.start_service()` flow. After 1.5 s the workspace tab auto-refreshes so the placeholder → live view switch happens automatically.
- Sub-tabs are closable; closing a sub-tab stops the polling but does NOT stop the underlying tool (use the Stack Editor for that).
**Servo note:** Web embedding uses `PySide6.QtWebEngineWidgets.QWebEngineView` by default. Swapping in Mozilla's servo engine later is a one-line change to `_make_web_view()` in `src/ai_lsc/ui/widgets/workspace_tab.py` — the rest of the WorkspaceTab code only depends on the `setUrl()` / `url()` / `load()` API.
Files: `src/ai_lsc/ui/widgets/workspace_tab.py` (new, ~310 lines), `src/ai_lsc/ui/main_window.py` (new `_build_workspace_orchestration_page` + nav entry + refresh hooks).
### Security & reliability hardening (the critique pass)
91 of 93 findings from the master code critique were addressed. See [whatremains.txt](whatremains.txt) for the two intentionally-skipped items and deferred polish. Highlights:
#### CRITICAL (6 of 7 fixed; C-05 skipped per user)
- **C-01** `runtime/process.py` — `shell=True` removed from `launch_desktop`, `launch_terminal`, `kill_by_name`, `docker_compose_down`. New `_to_arg_list()` helper accepts either a pre-split argv list or a single shell-style string (split via `shlex.split`).
- **C-02** `runtime/systemd.py` — `shell=True` removed from `start`, `stop`, `is_active`. `is_active` now has a 5 s timeout.
- **C-03** `runtime/tmux.py` — 5 `shell=True` sites converted to list-form argv. New `_validate_name()` rejects shell-metacharacter session/window names. New `_socket_path_for()` moves tmux sockets out of `/tmp` into `$XDG_RUNTIME_DIR/ai-lsc/`. `SESSION` is now user-scoped: `ai_lsc_<uid>`.
- **C-04** `runtime/installer.py` — 15+ `shell=True` sites converted to list-form argv. Post-install hooks and script-type installers now run via `["bash", "-c", cmd]` (still a shell, but the registry string is passed verbatim as a single argv element so it can't break out of the subprocess call itself).
- **C-05** `curl|sh` remote installers — **SKIPPED per user instruction**. The Ollama / Grafana Alloy / Meilisearch `curl … | sh` patterns remain in `runtime/installer.py:361`, `registry/layers/inference.py:30`, `registry/layers/observability.py:122`, `registry/layers/data_knowledge.py:164`, and `registry/defaults.py:642`.
- **C-06** `agents/librechat_config.py` — hardcoded `sk-ai-lsc-local` and `sk-local` API keys removed. Keys now read from `AI_LSC_LITELLM_KEY` / `AI_LSC_OPENWEBUI_KEY` environment variables via a new `_env_api_key()` helper.
- **C-07** `agents/orchestrator.py` — missing `import os` added (was a guaranteed `NameError` on every `AgentOrchestrator` instantiation).
#### HIGH (24 of 24 fixed)
- **H-01** Path-traversal via unsanitized `tool_id` — new `_validate_tool_id()` in `runtime/executor.py` rejects `..`, `.`, `/`, and shell metacharacters.
- **H-02** `os.getcwd()` for config — `_load_config` and `save_config` in `ui/main_window.py` now resolve against `self.base_dir` instead of the cwd the app was launched from.
- **H-03** Atomic JSON writes — new `_atomic_write_json()` helper in `ui/main_window.py` uses `tempfile.mkstemp` + `os.fsync` + `os.replace`. Applied to stack export, config save, and stack-wizard state save.
- **H-04** Popen reference tracking — `ProcessManager` now tracks every launched child in `self._launched` and exposes `reap()` + `shutdown()` methods. Zombie accumulation in long-lived GUI sessions is fixed.
- **H-05** `pull_model` Popen not killed on thread crash — `agents/dispatcher.py:_pull_model` now wraps `proc.communicate()` in `try/except` with `proc.kill()` in the handler.
- **H-06** JSON parse crash from malformed LLM output — `agents/agent_loop.py` now wraps `json.loads(func.get("arguments", "{}"))` in `try/except json.JSONDecodeError`. The parse error is fed back to the LLM as a tool result so the model can self-correct on the next round.
- **H-07** `_pull_model` None guard — `dispatcher._pull_model` now handles the case where `runtime.pull_model()` returns `None`.
- **H-08** Duplicate `list_available_tools` schema — `agents/tool_bridge.py:generate_all_schemas` now filters out the static `list_available_tools` schema before appending the annotated version.
- **H-09** Broad `except Exception` — 12+ sites across `guardrails.py`, `loader.py`, `manifest/support.py`, `agent_loop.py`, `ollama_tools.py`, `service_row.py`, `chat/api.py`, `model_pool.py`, `qdrant_bridge.py` now catch specific exceptions (`OSError`, `ValueError`, `json.JSONDecodeError`, `subprocess.SubprocessError`, `urllib.error.URLError`, etc.).
- **H-10** Exception messages expose internal details — `chat/api.py` now sanitizes error bodies via `_short_reason()` and `_categorize_url_error()`. Full detail is logged server-side.
- **H-11** LXC container name from unsanitized `tool_id` — new `_validate_lxc_name()` and `_validate_tool_id()` in `runtime/lxc.py`.
- **H-12** LXC `attach_exec` destroys quoting — `command.split()` replaced with `shlex.split(command)`.
- **H-13** Redis lock silently bypassed when down — `agents/redis_bridge.py:acquire_lock` and `release_lock` now log a WARNING when Redis is unreachable so operators know concurrent agents could race.
- **H-14** `_enforce_quality` false-positive error detection — `agents/orchestrator.py` now uses a word-boundary regex with a negative lookahead `(?![\w\-])` so hyphenated compounds like `error-correction module initialized` are not flagged.
- **H-15** Registry layer files have incomplete flag schemas — `registry/validator.py` now enforces the full 8-key flags schema (`has_cli`, `has_gui`, `has_web`, `is_ollama`, `is_docker`, `is_passive`, `is_mcp`, `is_skills_collection`). `scripts/backfill_layer_flags.py` backfilled 123 flag blocks across all 13 layer files.
- **H-16** `_inject_skill_stub` returns fake success — `agents/dispatcher.py` now validates the skill exists before reporting success.
- **H-17** Placeholder resolution triple-copy in `export.py` — new `_resolve_placeholders()` helper replaces three duplicated 6-line `.replace()` chains in `generate_compose_yaml`, `generate_lxc_configs`, and `generate_firecracker_configs`.
- **H-18** Ollama `/api/tools` endpoint does not exist — `agents/ollama_tools.py` `register_all` and `register_single` are now gated behind `_registration_supported = False` with a clear warning. Tool schemas are passed inline to `/api/chat` instead.
- **H-19** No port validation on user input — new `_validate_port()` in `runtime/executor.py` enforces `1 ≤ port ≤ 65535`. UI surfaces a clean `ValueError` message instead of a cryptic URLError.
- **H-20** No URL scheme validation in `install_custom` — new `_validate_url()` in `runtime/installer.py` rejects non-http(s) schemes.
- **H-21** No signal handling on parent exit — `ui/main_window.py:closeEvent` now calls `runtime._process.shutdown()` to terminate every tracked child.
- **H-22** `install_custom` opens arbitrary URLs — same fix as H-20.
- **H-23** Thread-unsafe pull lock in model pool — `agents/model_pool.py` `self._pull_lock = False` (plain boolean) replaced with `threading.Lock()`. Non-blocking acquire so concurrent agent threads don't both enter the pull branch.
- **H-24** Qdrant collection dimension hardcoded — `agents/qdrant_bridge.py` `create_collection` now probes the live embedding dimension via `_probe_embedding_dimension()` instead of hardcoding `768`. Existing collections with a different dimension are surfaced (not silently hidden).
#### MEDIUM (42 of 42 fixed)
- **M-01** Missing `encoding="utf-8"` on `open()` — fixed in 15+ sites across `ui/main_window.py`, `runtime/lxc.py`, `ui/dialogs/stack_wizard.py`.
- **M-02** `os.path.join` mixed with `pathlib` — `utils/filesystem.py:walk_tree` now uses `Path.rglob`.
- **M-03** TOCTOU race in log file operations — `ui/main_window.py` log readers now wrap stat + read in a single `try/except OSError`.
- **M-04** No file locking on shared JSON files — `_atomic_write_json()` now uses `fcntl.flock` for cross-process serialization.
- **M-05** No file lock / fsync on LXC config append — `runtime/lxc.py:_apply_config` now flushes + fsyncs.
- **M-06** `/tmp` socket path without cleanup — `runtime/tmux.py:_socket_path_for` moves sockets under `$XDG_RUNTIME_DIR/ai-lsc/` with sanitized tool_id.
- **M-07 through M-13** Nested-if flattening — applied via guard clauses, dict dispatch, reverse-lookup dicts, and predicate extraction across `installer.py`, `orchestrator.py`, `model_pool.py`, `guardrails.py`, `main_window.py`.
- **M-14 through M-17** Duplicate code extraction — `_iter_py_files` / `_read_source` in `guardrails.py`, `_scan_and_import` / `_match_tags` patterns, `_cache_set` / `_cache_get` in `redis_bridge.py`, `_resolve_placeholders` in `export.py`.
- **M-18 through M-23** Loop → comprehension / builtin — `dict.fromkeys()` for dedup, dict comprehension for `preflight_batch`, `next()` for `detect_terminal`, list comprehension for LXC config lines, `extend()` for skills_loaded.
- **M-24** Dead variable `exposed` in `get_consumers` — removed.
- **M-25** `Any` type annotations — `ui/protocol.py` now uses `TYPE_CHECKING` imports; `agents/orchestrator.py` `dispatcher` param now typed as `"AgentDispatcher"`.
- **M-26** Variable shadowing in `generate_env_file` — `lines` renamed to `ollama_ep`.
- **M-27** Dead if/else block in `guardrails.py` — removed (PARENT_ALLOWED_DIRS was always empty).
- **M-28** Redundant `import json` in orchestrator method — removed.
- **M-29** Sorted-set member collision in task queue — `redis_bridge.py` now uses `task_id` as the sorted-set member and stores the payload in a separate hash.
- **M-30** `_LAYERS_DIR` defined but never used — removed from `registry/loader.py`.
- **M-31** `use_model` tautological assignment — cleaned up.
- **M-32** Missing error handling on `install_pip` — added `try/except subprocess.CalledProcessError`.
- **M-33 / M-34** Timeouts on `systemctl is-active` and `pkill` — both now have `timeout=5`.
- **M-35** File handle leak in `verify_and_watch` — `open(log_file, "a").close()` replaced with `Path(log_file).touch()`.
- **M-36** Recursive directory traversal — `utils/filesystem.py:walk_tree` rewritten to use `Path.rglob`.
- **M-37** Model pool pull timeout applies to entire stream — `for line in resp: pass` replaced with `resp.read()`.
- **M-38** Embed batch is sequential — `qdrant_bridge.py:embed_batch` now uses `ThreadPoolExecutor.map`.
- **M-39** Hardcoded `pacman` install hint — `_install_hint` in `runtime/lxc.py` now lists pacman + apt + dnf.
- **M-40** Nested ternary in `_build_payload_history` — deferred (chatbot_console.py), see whatremains.txt.
- **M-41** Duplicate `OpenEngineerImporter` import — removed from `stack_templates/manager.py`.
- **M-42** `_load_skills` documents unimplemented Qdrant feature — converted to explicit `TODO(security)` comment.
#### LOW (19 of 20 fixed; 1 deferred)
See [whatremains.txt](whatremains.txt) for the full list. Highlights:
- **L-02** Line-ending normalization on log reads — `rstrip("\r\n")` applied.
- **L-03** TOCTOU on symlink creation — `os.symlink` wrapped in `try/except FileExistsError`.
- **L-05** Tmux session name uniqueness — `SESSION = f"ai_lsc_{os.getuid()}"`.
- **L-06** Port range check in `chat/api.py` — `_validate_port` applied to `port_id` up-front.
- **L-08** `create_collection` reports success for existing collection — now checks the existing collection's dimension and surfaces mismatches.
### SaaS-only tool blocklist (v3.1 hardening follow-up)
After the v3.1 release, the user identified a policy gap: SaaS-only tools (closed-source desktop apps with restrictive ToS, hosted LLM routers with no local binary, managed inference services) must not be addable to the registry. The audit found that the actual code registry was already clean — the SaaS names (`OpenRouter`, `LM Studio`, `Groq`, `Codestral`) only appeared as stale references in `README.md` and `docs/ADR-001-capability-architecture.md`. They were removed from the code at some earlier point but the docs were never updated.
**What changed:**
1. **Stale doc references fixed.** `README.md` L6 row now lists `LiteLLM Proxy, 9Router Proxy, Odysseus, LangChain, LangFlow, OpenAI Swarm, Agno` (matching the actual registry). `README.md` L8 row replaces `Codestral` (Mistral SaaS model) with `OpenHands` and `Codex`. `docs/ADR-001-capability-architecture.md` LLM Gateway providers now list `LiteLLM · 9Router Proxy · Local proxy` (was `LiteLLM · OpenRouter · Local proxy`); Inference Engine providers replace `LM Studio` with `SGlang`.
2. **SaaS blocklist added to the registry validator** (`src/ai_lsc/registry/validator.py`). The following tool_ids are now rejected at validation time with a clear error message pointing to this section:
`openrouter`, `lm_studio`, `lmstudio`, `groq`, `together_ai`, `together`, `fireworks_ai`, `fireworks`, `replicate`, `runpod`, `modal`, `anyscale`, `perplexity`, `cohere`, `mistral_api`, `deepseek_api`, `openai_api`, `huggingface_inference`
3. **SaaS host regex added to the validator.** Even if a tool_id isn't on the blocklist, the validator now rejects any launcher cmd or installer cmd that references a known SaaS provider hostname (api.openai.com, api.anthropic.com, api.openrouter.ai, api.groq.com, api.together.xyz, api.fireworks.ai, api.replicate.com, api.perplexity.ai, api.cohere.ai, api.mistral.ai, api.deepseek.com, generativelanguage.googleapis.com, api.lmstudio.ai, endpoint.huggingface.com). Localhost URLs (127.0.0.1, localhost, 0.0.0.0) are always allowed.
4. **Localhost-only env forced for CLI tools that CAN call SaaS.** `claude_code`, `aider`, `openhands`, `fabric`, and the new `codex` entry now have their launcher cmds prepended with the appropriate localhost env vars:
| Tool | Env vars forced |
|------|-----------------|
| `claude_code` | `ANTHROPIC_BASE_URL=http://127.0.0.1:4000 ANTHROPIC_API_KEY=sk-ai-lsc-local` |
| `aider` | `OPENAI_API_BASE=http://127.0.0.1:4000/v1 OPENAI_API_KEY=sk-ai-lsc-local` |
| `openhands` | `OPENAI_API_BASE=http://127.0.0.1:4000/v1 OPENAI_API_KEY=sk-ai-lsc-local` |
| `fabric` | `OPENAI_API_BASE=http://127.0.0.1:4000/v1 OPENAI_API_KEY=sk-ai-lsc-local` |
| `codex` (new) | `OPENAI_BASE_URL=http://127.0.0.1:4000/v1 OPENAI_API_KEY=sk-ai-lsc-local` |
This breaks SaaS routing by default — the user must explicitly override the env var (via the service row port or a custom launcher cmd) to call a SaaS endpoint. Each tool's `deps` now includes `litellm` so the Stack Editor flags a missing local proxy if the user hasn't staged one.
5. **New `codex` tool entry added** to `registry/defaults.py` and `registry/layers/endpoints.py` — OpenAI's open-source Codex CLI (`@openai/codex` npm package), classified as L6 AI Endpoints, with the localhost-only launcher shown above. The user explicitly approved this addition with the caveat: "you can add codex and claude code but only if mapped by force to a localhost port for the engine or endpoint."
**Verification:** defaults.py validates with 0 errors (125 tools); layer files validate with 0 non-curl|sh errors (124 tools); all 5 localhost-mapped tools confirmed to have `127.0.0.1` in their launcher cmd; the SaaS blocklist correctly rejects `openrouter` and `lm_studio` with the documented error message; the SaaS host regex correctly rejects `https://api.openai.com/v1` in a launcher cmd while allowing `http://127.0.0.1:4000/v1`.
---
### License acceptance gate (v3.1 hardening follow-up #2)
After the SaaS blocklist landed, the user asked for a layered license-acceptance system: ToS/disclaimer warnings for proprietary tool pulls, per-tool license acceptance for open-source tools, and a license auto-approval registry where the user can pre-approve entire license types (accept all GPL, all AGPL, all MIT, all BSD, etc.) so they don't have to accept each tool individually. The auto-approval registry must NOT contain any SaaS or proprietary tools. `lmstudio` stays on the blocklist for aggressive ToS.
**What changed:**
1. **New license catalog** (`src/ai_lsc/registry/licenses.py`) — 20 SPDX IDs across three categories:
| Category | Auto-approvable? | Disclaimer? | Licenses |
|----------|------------------|-------------|----------|
| **OSI** (open-source) | ✅ Yes | ❌ No | MIT, Apache-2.0, GPL-2.0, GPL-3.0, AGPL-3.0, LGPL-3.0, BSD-2-Clause, BSD-3-Clause, MPL-2.0, ISC, PostgreSQL, Python |
| **SOURCE_AVAILABLE** (fair-code) | ❌ No | ✅ Yes | BSL-1.1, SSPL, RSALv2, Sustainable-Use, Dify-OSL |
| **PROPRIETARY** (ToS-governed) | ❌ No | ✅ Yes (prominent) | Proprietary, Anthropic-ToS, LMStudio-ToS (blocked) |
Each license entry includes: SPDX ID, human-readable name, category, URL to full text, one-paragraph summary, and an optional disclaimer shown in the acceptance dialog.
2. **New `LicenseGate` class** (`src/ai_lsc/registry/license_gate.py`) — sits between the user's "Install" click and the installer dispatch. For every tool installation, the gate checks (in order):
- **SaaS blocklist** — if the tool_id is blocked, raises `LicenseBlocked` immediately (no dialog, no acceptance, no install).
- **Auto-approval registry** (`config/license_approvals.json`) — user-editable list of OSI-approved SPDX IDs. If the tool's license is in this list AND the license category is OSI, install proceeds without a dialog. Non-OSI licenses in the file are ignored + logged (defensive against hand-editing).
- **Per-tool acceptance registry** (`config/license_acceptances.json`) — auto-managed; records every per-tool acceptance so the user isn't prompted twice.
- If none of the three cover the tool, raises `LicenseAcceptanceRequired` (carrying the `LicenseInfo` for the dialog).
`LicenseGate.add_auto_approval(spdx)` **rejects** non-OSI licenses with a clear `ValueError` — source-available and proprietary licenses cannot be auto-approved, period.
3. **New `LicenseAcceptanceDialog`** (`src/ai_lsc/ui/dialogs/license_dialog.py`) — Qt dialog shown when the gate raises `LicenseAcceptanceRequired`. Shows:
- Tool name + tool_id + license name + SPDX ID
- Category banner (green ✓ for OSI, orange ⚠ for source-available, red ⛔ for proprietary)
- License summary (one paragraph)
- Disclaimer (only for source-available + proprietary — prominent red box)
- "Open in browser ↗" button linking to the full license text
- Confirmation checkbox (pre-checked for OSI, unchecked for non-OSI — the user must explicitly check it before Accept is enabled)
- Three buttons: **Accept & Install** (records per-tool acceptance), **Accept all \<license\>** (only for OSI — adds the SPDX to the auto-approval registry), **Cancel**
Also includes a `LicenseBlockedDialog` for the blocked case (just an OK button + suggestion to use a local alternative).
4. **`license` field added to the registry schema** — every tool entry must now declare its license SPDX ID. The validator (`registry/validator.py`) enforces this:
- Missing/empty `license` field → error
- Unknown SPDX ID (not in the license catalog) → error
- The `_REQUIRED_FIELDS` set now includes `"license"`
Two backfill scripts added:
- `scripts/backfill_tool_licenses.py` — adds the `license` field to every tool based on a curated override table (85 tools mapped to known licenses: ollama→MIT, vllm→Apache-2.0, grafana→AGPL-3.0, redis→RSALv2, terraform→BSL-1.1, n8n→Sustainable-Use, claude_code→Anthropic-ToS, etc.)
- `scripts/backfill_default_licenses.py` — fills any remaining tools with `"Proprietary"` as the defensive default (83 tools defaulted — the user can review and update these to their actual licenses later)
5. **License gate wired into the installer** — `InstallerManager.__init__` now accepts a `license_gate` parameter. `InstallerManager.run()` and `install_with_preflight()` call `self._check_license(tool_id, license_spdx)` before any subprocess dispatch. If the gate raises `LicenseBlocked` or `LicenseAcceptanceRequired`, the exception propagates up through `RuntimeExecutor.install_tool()` to the UI.
6. **`ServiceRow` catches license exceptions** — the install thread's `except Exception` handler (which runs on the main thread via `QTimer.singleShot`) calls `_handle_license_exception()` which:
- For `LicenseBlocked` → shows `LicenseBlockedDialog` (just an OK button)
- For `LicenseAcceptanceRequired` → shows `LicenseAcceptanceDialog` and connects the dialog's `accepted_individual` / `accepted_all_of_type` signals to handlers that record the acceptance and retry the install
7. **`lmstudio` blocklist comment** — the `SAAS_BLOCKLIST` definition in `validator.py` now includes an explicit comment: "lm_studio / lmstudio: BLOCKED for aggressive ToS — the user considers LM Studio's Terms of Service restrictive enough to be equivalent to a SaaS offering, so it is auto-banned regardless of any per-tool acceptance the user might try to grant."
**License distribution after backfill** (135 unique tool_ids across defaults.py + layer files):
```
Proprietary 50 tools (defensive default for tools whose license is unknown)
MIT 49 tools
Apache-2.0 17 tools
AGPL-3.0 5 tools
GPL-3.0 3 tools
MPL-2.0 2 tools
Anthropic-ToS 1 tool (claude_code)
Python 1 tool (python)
BSL-1.1 1 tool (terraform)
PostgreSQL 1 tool (postgresql)
GPL-2.0 1 tool (mariadb)
RSALv2 1 tool (redis)
LGPL-3.0 1 tool (glances)
Dify-OSL 1 tool (dify)
Sustainable-Use 1 tool (n8n)
```
The 50 tools defaulted to "Proprietary" include some that are actually open-source (tmux, git, podman, docker) — the user can review and update these in `defaults.py` / the layer files. The gate will require individual acceptance for each until the license is corrected.
**Verification:**
- defaults.py: 125 tools, 0 validation errors
- Layer files: 124 tools, 0 non-curl|sh validation errors
- All 135 unique tool_ids have a `license` field
- `LicenseGate.check()` correctly returns `needs_acceptance` for fresh tools, `accepted` after `accept()`, `blocked` for SaaS-blocklist tool_ids
- `LicenseGate.add_auto_approval()` correctly rejects BSL-1.1 (source-available) and Proprietary with `ValueError`
- `lmstudio` and `lm_studio` both on `SAAS_BLOCKLIST`
- `claude_code` (Anthropic-ToS) and `n8n` (Sustainable-Use) both surface `needs_disclaimer=True` via their category
- Auto-approval + acceptance files are created in `config/` with the expected JSON structure
---
### Bugs caught during the post-pass double-check (fixed)
Three latent bugs were introduced during the initial fix wave and caught by the double-check's 19 functional spot-checks. All three are now fixed:
1. **`installer._validate_tool_id` regex allowed `/`** — the regex was `r"^[A-Za-z0-9_.:\-/]+$"` (with trailing `/`), so `../../etc/passwd` would have passed. Tightened to `r"^[A-Za-z0-9_.:\-]+$"` (no `/`).
2. **All three `_validate_tool_id` validators accepted bare `..` and `.`** — `os.path.normpath('..')` returns `'..'` unchanged, so the normpath check missed these. Added explicit `tool_id in {".", ".."}` rejection in installer.py, executor.py, and lxc.py.
3. **H-14 `_ERROR_RE` still flagged `error-correction`** — `\b` (word boundary) treats `-` as a non-word char, so `error-correction` had a boundary between `error` and `correction`. Changed regex to `\b(?:error|failed|not found|timeout|exception|traceback)(?![\w\-])` — the negative lookahead rejects matches where the next character is a word char OR a hyphen.
### Verification
- All 89 Python source files parse cleanly via `ast.parse`.
- All 124 tools in `defaults.py` pass the strengthened validator (0 errors).
- All 123 tools across the 13 layer files pass the strengthened validator (3 expected curl|sh-related warnings only).
- All 134 unique tool_ids (defaults + layers merged) pass every `_validate_tool_id` validator (installer, executor, LXC) and every `_validate_lxc_name` check.
- 19/19 functional spot-checks pass (after the 3 double-check fixes).
- Ticker edge + orphan detection simulated against the real `STACK_WIRINGS` data across 6 test scenarios — all behaved as expected.
### Known issues
- **PySide6 not available in some test environments.** The new widget modules (`pipeline_ticker.py`, `workspace_tab.py`) AST-parse cleanly and pass structural checks, but were not exercised by a live Qt instantiation test in the sandbox where v3.1 was finalized. Verify by running the app.
- **`open_webui` vs `openwebui` tool_id collision** — the Pipeline Ticker surfaces this real registry inconsistency. Recommend a follow-up pass to reconcile.
- **`docs/screenshots/` still reflects v3.0** — a screenshot refresh pass to capture the new Pipeline Ticker + Workspace Tab is overdue.
### Upgrade notes
- If you have a v3.0 `pipeline_state.json`, it will continue to work — the schema is unchanged.
- If you maintain custom layer-file entries, run `python scripts/backfill_layer_flags.py` to backfill the 5 new flag keys (`is_ollama`, `is_docker`, `is_passive`, `is_mcp`, `is_skills_collection`) defaulting to `False`. The validator will reject entries missing these keys.
- If you have hardcoded `sk-ai-lsc-local` API keys in your environment, set `AI_LSC_LITELLM_KEY` and `AI_LSC_OPENWEBUI_KEY` env vars instead — the source-code default is now an empty string.
- If you launch the app from a non-default working directory, config is now resolved against `BASE_DIR` instead of `os.getcwd()` — this may move your `config.json` to a new location on first v3.1 launch.
### Artifacts
- Master tarball: `ai-lsc-master-2026-07-07.tar.gz`
- New scripts: `scripts/backfill_layer_flags.py`
- New widgets: `src/ai_lsc/ui/widgets/pipeline_ticker.py`, `src/ai_lsc/ui/widgets/workspace_tab.py`
- New docs: `CHANGES.md` (this file), `docs/ADR-002-pipeline-ticker.md`, `docs/ADR-003-workspace-tab.md`
- Skipped-items register: `whatremains.txt`
---
## v3.0 — earlier
See git history for the v3.0 release notes. v3.0 introduced the 13-layer architecture, the capability model, the OpenEngineer importer, and the Firecracker microVM export backend.

679
LICENSE Executable file
View File

@ -0,0 +1,679 @@
========================================================================
PROJECT: AI-LSC (AI Local Stack Control)
COPYRIGHT: Copyright (C) 2026 dcos.net
HOMEPAGE: https://git.dcos.net/dcosnet/ai-lsc/
REPOSITORY: https://git.dcos.net/dcosnet/ai-lsc/
LICENSE: GNU Affero General Public License v3.0 (AGPL-3.0)
========================================================================
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
------------------------------------------------------------------------
APPENDIX: HOW TO APPLY THESE TERMS TO YOUR NEW PROGRAMS
To ensure your repository complies with the remote network interaction
requirements (Section 13 of the AGPLv3), you must ensure users
interacting with your stack can access the source code.
This project, AI-LSC, complies by maintaining its primary development
repository at https://git.dcos.net. Any derivative works must retain
this notice and provide equivalent access to source code.

376
README.md Executable file
View File

@ -0,0 +1,376 @@
<div align="center">
<img src="ai-lsc-logo.png" alt="AI-LSC Logo" width="280">
</div>
<h1 align="center">AI - Local Stack Control</h1>
<p align="center">
<strong>v3.1 — Codename: Ankh of Jah</strong><br>
<a href="http://dcos.net">http://dcos.net</a>
</p>
<p align="center">
A PySide6 desktop application for orchestrating local AI/ML tool stacks across a 10-layer architecture.
</p>
AI Local Stack Control (AI-LSC) provides a unified interface to discover, configure, launch, and manage 140 tools spanning the entire AI software stack — from GPU runtimes and inference engines to agent frameworks, security tooling, and knowledge management.
![Overview](docs/screenshots/overview.png)
## Features
### 10-Layer Infrastructure Architecture
Every tool in the registry is classified within a 10-layer taxonomy, giving you a clear mental model of your entire AI stack. Select tools directly from the sidebar — the sidebar *is* the wizard.
| Layer | Tools | Examples |
|-------|-------|---------|
| L1 — Host Platform | 9 | PostgreSQL, MariaDB, Redis, SQLite3, DuckDB, Podman, Docker, Tmux, Git |
| L2 — Development Environment | 7 | Python Environment, CuPy, ripgrep, fd, tree-sitter, SST, Unsloth |
| L3 — GPU Runtimes | 3 | CUDA Toolkit, NVIDIA Apex, Heretic |
| L4 — Engines | 7 | Ollama, llama.cpp, KoboldCPP, Llamafile, TurboLLM, AirLLM, Locally-Uncensored |
| L5 — Orchestrators | 26 | vLLM, Ray, LiteLLM Proxy, 9Router Proxy, LangChain, LangFlow, Dify, CrewAI, AutoGen, Wayland AI, +17 more |
| L6 — Security | 6 | Keycloak, HashiCorp Vault, Trivy, Fail2Ban, ClamAV, Open Policy Agent |
| L7 — Observability | 8 | Btop, Glances, Prometheus, Grafana, Grafana Alloy, Opik, Pulse AI, Latitude |
| L8 — User Interfaces | 16 | Open WebUI, AnythingLLM, LibreChat, Flowise, InvokeAI, Forge (A1111), ComfyUI, Dashy, Obsidian, +7 more |
| L9 — DevOps | 33 | Terraform, Ansible, Puppet, Pulumi, OpenTofu, AWS CDK, Crossplane, n8n, Aider, Claude Code, OpenHands, +23 more |
| L10 — Knowledge Management | 25 | Zotero, Calibre, Paperless-ngx, Logseq, Joplin, ChromaDB, LanceDB, Qdrant, LlamaIndex, +16 more |
![Infrastructure Layers](docs/screenshots/infrastructure-layers.png)
### Sidebar-Integrated Infrastructure Selector
The sidebar doubles as the stack wizard — expand the Infrastructure tree to reveal all 10 layers, each showing its tools with rich-text interface badges (CLI, GUI, Web, Ollama, Docker, MCP, etc.). Toggle checkboxes to stage tools; a debounced compiler (400ms) automatically validates dependencies and writes the compiled pipeline state. No separate popup window needed.
### Stack Editor
Visually compose your tool stack using templates, a two-panel flow builder, and dependency validation. Select from 13 pre-configured stack templates, then customize the wiring topology. Lifecycle engine controls let you start, stop, and monitor services directly from the editor.
![IPC Stack Editor](docs/screenshots/ipc-stack-editor.png)
### Active Monitor
Stripped to show only active metrics — real-time system health monitoring focused on the services that are actually running. CPU/memory metrics and per-service status indicators for your live stack.
![Monitor Dashboard](docs/screenshots/monitor-dashboard.png)
### Pipeline Ticker
A horizontally scrolling status bar at the top of every workspace tab that visualizes the wiring topology of your currently-staged tools in real time. Edges are drawn from the live `STACK_WIRINGS` data: `provider ──interface──▶ consumer`, with arrow color encoding the interface type (blue = openai_api, green = vector, orange = redis_pubsub, purple = postgresql, teal = http_api, etc.). Orphan tools — active but with no wiring to any other active tool — are flagged in red with a warning prefix so you can immediately see disconnected tools. Hover to pause the scroll; click any tool pill to jump to that tool's row.
### Stack Templates
Get started quickly with 13 pre-configured stack templates:
- **Claude Code Setup** — Full Claude Code ecosystem (11 tools)
- **Free Claude Code** — Minimal Claude Code setup (4 tools)
- **SaaS Integrations** — Production deployment stack (12 tools)
- **Local LLM Lab** — Self-hosted LLM playground (10 tools)
- **Agentic OS Stack** — Full agent orchestration stack
- **AI Image Gen Local** — Local image generation pipeline
- **Privacy-First AI Laptop** — Air-gapped AI workstation
- **OpenJarvis Intelligence Stack** — Multi-agent intelligence
- **OpenHands Autonomous Coder** — Autonomous coding agent
- **DeepSeek R1 Local Reasoning** — Local reasoning models
- **Hermes AI Coder Stack** — Hermes-powered coding
- **Aider + Ollama Vibe Coding** — Vibe coding setup
- **Open WebUI Full RAG** — Complete RAG pipeline
### Multi-Backend Container Export
Export your compiled stack to multiple deployment targets:
- **Podman Compose** — Rootless OCI containers via `compose.yaml`
- **Docker Compose** — Standard Docker Compose output
- **LXC Containers** — Per-container `.conf` files + `lxc-launch.sh` lifecycle script
- **Firecracker microVMs** — Per-VM `vm-config.json` files + `firecracker-launch.sh` for ultra-lightweight KVM-backed microVMs
![Deployment Targets](docs/screenshots/deployment-targets.png)
### Runtime Management
Launch and manage tools via four runtime backends, all with shell-injection-safe list-form subprocess calls and validated tool_ids / port ranges:
- **systemd** — Persistent system services with `systemctl` (5 s timeout on `is-active` queries)
- **tmux** — Session-managed terminal processes with user-scoped session names (`ai_lsc_<uid>`)
- **desktop** — One-shot CLI commands
- **lxc** — Full LXC container lifecycle (create, start, stop, freeze, attach) with `shlex.split()` argument preservation and validated container names
All child processes are tracked in a `ProcessManager._launched` list and reaped on application exit so the GUI does not orphan tmux windows or desktop launches.
### Skills System
Extend AI-LSC with skill modules that add specialized behaviors to your tool stack. The Skills Console provides activation toggles, behavior bindings, and runtime integration.
![Skills Console](docs/screenshots/skills-console.png)
### AI Chat Console
Built-in chat interface for interacting with local LLM endpoints. Supports model selection, conversation history, and direct integration with your running stack.
![Chat Console](docs/screenshots/chat-console.png)
### DB Manager
Full-screen database management interface for inspecting and querying your stack's data stores. Hides the pipeline ticker to maximize workspace.
![DB Manager](docs/screenshots/db-manager.png)
### Verification
Validate your compiled stack configuration, check tool dependencies, and verify service connectivity before deployment.
![Verification](docs/screenshots/verification-tab.png)
### Settings
Configure base directories, model defaults, API endpoints, logging levels, and application preferences.
![Settings](docs/screenshots/settings.png)
## Architecture
```
ai_lsc/
__init__.py # Public API re-exports
__main__.py # Entry point: python -m ai_lsc
constants.py # App constants, styles, 10-layer nav order
types.py # Data classes: ToolMetadata, PipelineState, etc.
guardrails.py # Import guard for PySide6
registry/
__init__.py
defaults.py # Master registry (140 tools, full 8-key flags)
loader.py # Merges per-layer files + blacklist enforcement
manager.py # RegistryManager — query/filter/group tools
validator.py # Schema validation (8-key flags enforced)
license_gate.py # License compliance gating
licenses.py # License database
layers/ # 10 per-layer tool files
host_platform.py # L1: 9 tools
development.py # L2: 7 tools
gpu.py # L3: 3 tools
inference.py # L4: 7 engines
orchestrators.py # L5: 26 tools
security.py # L6: 6 tools
observability.py # L7: 8 tools
user_interfaces.py # L8: 16 tools
devops.py # L9: 33 tools
knowledge_management.py # L10: 25 tools
stack_templates/ # 13 pre-configured stack templates
manager.py # StackTemplateManager
openengineer/ # OpenEngineer import pipeline
runtime/
__init__.py
executor.py # RuntimeExecutor — dispatch + tool_id/port validation
installer.py # Tool installation (URL/port/tool_id validation)
process.py # ProcessManager with reap()/shutdown()
status.py # Service status detection
systemd.py # systemd lifecycle (no shell=True)
tmux.py # tmux session mgmt (validated names, XDG sockets)
lxc.py # LXC lifecycle (validated names, shlex.split)
stack/
__init__.py
export.py # ContainerBackend — compose/LXC/Firecracker export
connections.py # 60-entry STACK_WIRINGS topology (ticker data source)
ui/
__init__.py
protocol.py # MainWindowProtocol (TYPE_CHECKING-typed)
main_window.py # AILocalStackControl — master QMainWindow + sidebar
dialogs/
__init__.py
stack_wizard.py # Legacy wizard (kept for backward compat)
license_dialog.py # License compliance dialog
pages/
infrastructure_layer_page.py # Sidebar-integrated layer checkboxes
ipc_stack_tab.py # Stack Editor (templates + flow + lifecycle)
db_manager.py # Full-screen DB management
chatbot_console.py
code_analysis_tab.py
container_stacks_tab.py
datasets_tab.py
git_worktree_tab.py
service_row.py # Per-layer active service controls
settings_page.py
skills_console.py
tools_tab.py
verification_tab.py
widgets/
__init__.py
pipeline_ticker.py # Scrolling wiring-topology status bar
workspace_tab.py # Peek-style embedded web + CLI orchestration
chat/
__init__.py
api.py # Async chat API worker (sanitized errors)
agents/
__init__.py
orchestrator.py # Multi-agent orchestration loop
dispatcher.py # Agent dispatch
model_pool.py # Model pool management
tool_bridge.py # Agent ↔ tool registry bridge
skill_injector.py # Skill injection into agent context
skills/
__init__.py
resolver.py # SkillRuntimeResolver
manifest/
__init__.py
support.py # Manifest generation
utils/
__init__.py
filesystem.py # Path.rglob-based walk_tree
logging.py
ollama.py # Ollama utilities
paths.py
process.py
service/
__init__.py
```
## Installation
### Prerequisites
- Python 3.11+
- PySide6 (`pip install PySide6`)
- Arch Linux (pacman) or equivalent package manager
### Quick Install
```bash
git clone https://github.com/your-username/ai-lsc.git
cd ai-lsc
pip install -e .
```
### Bootstrap Script
```bash
./bootstrap.sh
```
The bootstrap script installs all system dependencies (pacman packages), Python dependencies, and verifies your environment.
## Usage
### Launch the Application
```bash
python -m ai_lsc
```
### Typical Workflow
1. **Browse the Infrastructure sidebar** — expand layers to discover and toggle tools (the sidebar is the wizard)
2. **Select a template** from the Stack Editor for a curated starting point, or build from scratch
3. **Validate dependencies** — AI-LSC resolves tool dependencies automatically as you toggle
4. **Compile your stack** — the Stack Editor validates and saves the configuration to `pipeline.json`
5. **Watch the Pipeline Ticker** — the scrolling status bar shows live wiring topology; orphans flagged red
6. **Launch services** — Tools start via systemd, tmux, desktop, or LXC launchers
7. **Orchestrate from Workspace** — every active tool gets its own sub-tab; web tools embed via QWebEngineView, CLI tools attach via tmux
8. **Monitor** — Active metrics dashboard shows real-time status of running tools only
9. **Export** — Generate Podman/Docker Compose, LXC, or Firecracker microVM configs
## Security & Reliability (v3.1)
The v3.1 pass applied a comprehensive security hardening:
- **No `shell=True`** in any subprocess call across `runtime/process.py`, `systemd.py`, `tmux.py`, `lxc.py`, or `installer.py` (15+ sites converted to list-form argv)
- **Registry blacklist enforcement** — the loader strips blacklisted tool IDs (e.g., wayland compositor) at startup, preventing accidental re-introduction
- **Path-traversal protection** at every subprocess boundary: `_validate_tool_id()` rejects `..`, `.`, `/`, and shell metacharacters
- **Port range validation** on every user-supplied port (`1 <= port <= 65535`)
- **URL scheme validation** on every `install_custom` URL (http/https only)
- **Atomic JSON writes** via `tempfile` + `fsync` + `os.replace` with `fcntl.flock` advisory locking
- **Hardened error messages** in the chat API (no internal-detail leakage)
- **Process lifecycle cleanup** on application exit (`ProcessManager.shutdown()`)
- **API keys from environment** — reads from env vars instead of hardcoded values
- **Dynamic Qdrant embedding dimension probe** (no hardcoded dimension mismatch)
## Development
### Project Structure
The project follows a layered architecture with clear separation of concerns:
- **registry/** — Tool definitions, loader, validator, templates, blacklist
- **runtime/** — Process management, launchers, installers
- **stack/** — Container export backends, wiring topology
- **ui/** — PySide6 interface (guarded imports, protocol-based DI, sidebar wizard)
- **chat/** — Async chat API integration
- **agents/** — Multi-agent orchestration, dispatch, model pool
- **skills/** — Skill runtime resolver
- **utils/** — Filesystem, logging, path helpers
### Adding a New Tool
1. Identify the correct layer file in `registry/layers/`
2. Add a new entry to the `TOOLS` dict with the full 8-key flags schema:
```python
'my_tool': {
"name": "My Tool",
"layer": "Orchestrators",
"role": "Hands",
"category": "Agent Framework",
"installer": {"type": "npm", "pkg": "my-tool"},
"launcher": {"type": "tmux", "cmd": "my-tool serve --port {port}",
"default_port": 8080},
"deps": ["ollama"],
"description": "My awesome AI tool.",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_docker": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False,
},
},
```
3. Run `python -m ai_lsc.registry.validator` to validate the schema
4. Optionally add it to a stack template in `registry/stack_templates/`
5. Optionally add a `STACK_WIRINGS` entry in `stack/connections.py` for Pipeline Ticker visualization
### Creating a Stack Template
```json
{
"id": "my-template",
"name": "My Custom Stack",
"description": "A custom stack for my workflow",
"version": "1.0",
"author": "your-name",
"tags": ["custom", "development"],
"tools": ["ollama", "aider", "claude_code", "vllm"]
}
```
Save as `registry/stack_templates/my-template.json`.
## Tech Stack
| Component | Technology |
|-----------|-----------|
| UI Framework | PySide6 (Qt for Python) |
| Web Embedding | PySide6 QtWebEngine (servo-swap path documented) |
| CLI Embedding | tmux `capture-pane` polling at 4 Hz |
| Language | Python 3.11+ |
| Package Manager | pip / uv |
| Container Backends | Podman, Docker, LXC, Firecracker microVMs |
| Service Management | systemd, tmux |
| IaC Tools | Terraform, Pulumi, OpenTofu, AWS CDK, Crossplane, Bicep, Terragrunt |
| Config Format | JSON (atomic writes via `tempfile` + `fsync` + `os.replace`) |
| Concurrency | `threading.Lock` for model pool, `fcntl.flock` for cross-process state files |
## License
AGPLv3
## Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/my-feature`)
3. Add tools to the appropriate layer file
4. Ensure all 10 layer files pass AST validation (`python3 -c "import ast; ..."`)
5. Submit a pull request

33
TODO.md Executable file
View File

@ -0,0 +1,33 @@
# AI-LSC TODO
## Recent progress (v3.1 — 2026-07-07)
- ✅ Applied the full master code critique: 91 of 93 findings addressed (CRITICAL × 6, HIGH × 24, MEDIUM × 42, LOW × 19). The 2 intentionally skipped items (curl|sh remote installers) are documented in [whatremains.txt](whatremains.txt).
- ✅ Added the Pipeline Ticker: scrolling wiring-topology status bar at the top of every workspace tab. Color-coded arrows by interface type, orphans flagged red, click-to-jump to Tools tab.
- ✅ Added the Workspace Tab: peek-style orchestration surface with one sub-tab per active tool. Web tools embed via QWebEngineView, CLI tools attach to tmux via `capture-pane` polling. Feels like virt-manager / aqemu for your AI stack.
- ✅ Strengthened the registry validator to enforce the full 8-key flags schema; backfilled 123 layer-file flag blocks via `scripts/backfill_layer_flags.py`.
- ✅ Reconciled tool count: 124 tools in `defaults.py` + 123 tools across the 13 layer files (merged).
## Open work
The 13 layers are likely to stay but the way they're described and labeled may change in the next release. The agentic abilities for classification of software tools are still limited — I have little time to code lately; this was a tool to make my life easier when testing software out. If you're reading this file, first of all thanks for trying it out — help by testing the UI, tools, configs, and pipelines. It's a lot.
### Known registry inconsistencies to reconcile
- `open_webui` (with underscore) vs `openwebui` (no underscore) — the Pipeline Ticker surfaces this gap when both are staged: `open_webui` is flagged orphan because it has no `STACK_WIRINGS` entry, while `openwebui` does. Recommend a follow-up pass to either rename one tool_id or merge the wirings.
### Deferred from the critique pass
See [whatremains.txt](whatremains.txt) for the full list. Highlights:
- `curl|sh` remote installers (Ollama, Grafana Alloy, Meilisearch) — left in place per user instruction. Apply the critique's download-first pattern when the remote-code-execution policy is revisited.
- L-17: `registry/openengineer/parser.py` still has commented-out code blocks — confirm with the OE importer maintainer before deleting.
- L-18: registry layer files declare tools without `filesystem` blocks; backfilling `install/config/cache/logs` paths across all 123 layer-file tools is a mechanical but sizable job.
- M-22 / M-40: `chatbot_console.py` HTML bubble builder + nested ternary — could extract `_render_bubble(msg)` helper, deferred as low-impact polish.
- M-10 / M-15 / M-16: `registry/openengineer/importer.py` nested-if flattening + dedup — defer to a focused OE-importer cleanup pass.
### UX polish still on the wishlist
- The Monitor page flow is 80% organized; the layout is still being decided.
- More `STACK_WIRINGS` entries — the ticker's value scales with how complete the wiring data is. Currently 60 wirings for 124 tools; gaps surface as orphans when staged.
- Screenshots: the `docs/screenshots/` directory still reflects v3.0. A refresh pass to capture the new Pipeline Ticker + Workspace Tab is overdue.

BIN
ai-lsc-logo.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.2 MiB

63
ai_lsc.py Executable file
View File

@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""AI Local Stack Control v3.1 — Ankh of Jah
Direct launcher. Run from the project root:
python ai_lsc.py
No pip install, no entry-point scripts, no ~/.local pollution.
Reads .env (created by bootstrap.sh) for AI_LSC_BASE_DIR, then
points sys.path at src/ and calls main().
Works wherever it sits — fully portable.
"""
from __future__ import annotations
import os
import sys
# ── Resolve project root (works from any cwd) ──────────────────
_PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
_SRC_DIR = os.path.join(_PROJECT_ROOT, "src")
if _SRC_DIR not in sys.path:
sys.path.insert(0, _SRC_DIR)
# ── Load .env file for AI_LSC_BASE_DIR (before any ai_lsc imports) ──
# Bootstrap writes this. If missing, constants.py falls back to /mnt/AI
# or the AI_LSC_BASE_DIR env var.
_ENV_FILE = os.path.join(_PROJECT_ROOT, ".env")
if os.path.isfile(_ENV_FILE):
with open(_ENV_FILE) as _f:
for _line in _f:
_line = _line.strip()
if _line and not _line.startswith("#") and "=" in _line:
_key, _, _val = _line.partition("=")
if _key.strip() == "AI_LSC_BASE_DIR" and _val.strip():
os.environ.setdefault("AI_LSC_BASE_DIR", _val.strip())
def main() -> int:
"""Launch the AI-LSC desktop application."""
# PySide6 required
try:
from PySide6.QtWidgets import QApplication
except ImportError:
print(
"PySide6 is required but not installed.\n\n"
" source .venv/bin/activate\n"
" pip install PySide6>=6.6\n\n"
" Or re-run: bash bootstrap.sh",
file=sys.stderr,
)
return 1
from ai_lsc.ui.main_window import AILocalStackControl
app = QApplication.instance() or QApplication(sys.argv)
window = AILocalStackControl()
window.show()
return app.exec()
if __name__ == "__main__":
sys.exit(main())

347
bootstrap.sh Executable file
View File

@ -0,0 +1,347 @@
#!/usr/bin/env bash
# ──────────────────────────────────────────────────────────────
# AI Local Stack Control v3.1.0 — Ankh of Jah
# Bootstrap Script
#
# Fully portable: works wherever the tarball lands.
# Resolves the base directory from cwd or env var.
# Creates venv in-project. On Arch, uses pacman's
# pre-built PySide6 to avoid pip compile hell.
# ──────────────────────────────────────────────────────────────
set -euo pipefail
BOLD='\033[1m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
CYAN='\033[0;36m'
NC='\033[0m'
info() { echo -e "${GREEN}[INFO]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
error() { echo -e "${RED}[ERROR]${NC} $*"; exit 1; }
# ── Resolve paths (current-path aware) ────────────────────────
# SCRIPT_DIR = wherever bootstrap.sh lives (the project root)
# AI_BASE = the managed working directory for all AI tools
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENV_DIR="${SCRIPT_DIR}/.venv"
VENV_PYTHON_STAMP="${VENV_DIR}/.python-version-stamp"
# Base directory: env var > parent of SCRIPT_DIR if named "tools" > /mnt/AI
# This lets you extract to /mnt/AI/tools/ai_lsc-v3/ and have it detect
# /mnt/AI as the base. If extracted elsewhere, defaults to /mnt/AI or
# whatever AI_LSC_BASE_DIR says.
_PARENT_DIR="$(dirname "$SCRIPT_DIR")"
_PARENT_NAME="$(basename "$_PARENT_DIR")"
if [ -n "${AI_LSC_BASE_DIR:-}" ]; then
AI_BASE="$AI_LSC_BASE_DIR"
elif [ "$_PARENT_NAME" = "tools" ] && [ -d "$(dirname "$_PARENT_DIR")/models" ]; then
# We're inside .../tools/ai_lsc-v3/ — base is the parent of tools/
AI_BASE="$(dirname "$_PARENT_DIR")"
else
AI_BASE="${AI_LSC_BASE_DIR:-/mnt/AI}"
fi
export AI_LSC_BASE_DIR="$AI_BASE"
echo ""
echo -e "${BOLD}╔══════════════════════════════════════════════════════╗${NC}"
echo -e "${BOLD}║ AI Local Stack Control v3.1.0 — Ankh of Jah ║${NC}"
echo -e "${BOLD}╚══════════════════════════════════════════════════════╝${NC}"
echo ""
echo -e "${CYAN} Project root : ${SCRIPT_DIR}${NC}"
echo -e "${CYAN} Base dir : ${AI_BASE}${NC}"
echo -e "${CYAN} Venv : ${VENV_DIR}${NC}"
echo ""
# ── Clean stale ~/.local/bin/ai-lsc from old pip/pipx installs ──
STALE_BIN="${HOME}/.local/bin/ai-lsc"
if [ -f "$STALE_BIN" ]; then
warn "Found stale entry-point: ${STALE_BIN}"
warn "Removing — everything runs via the project venv now"
rm -f "$STALE_BIN"
fi
if command -v pipx &>/dev/null && pipx list 2>/dev/null | grep -q "ai-lsc"; then
warn "Found pipx-installed ai-lsc — uninstalling"
pipx uninstall ai-lsc 2>/dev/null || true
fi
# ── Ensure AI_BASE exists ─────────────────────────────────────
if [ ! -d "$AI_BASE" ]; then
if [ "$(id -u)" -eq 0 ]; then
mkdir -p "$AI_BASE"
info "Created ${AI_BASE} (running as root)"
else
echo ""
echo -e "${YELLOW}${AI_BASE} does not exist.${NC}"
echo " This is the managed working directory for all AI tools."
echo ""
read -p "Create ${AI_BASE} now? [Y/n] " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Nn]$ ]]; then
if command -v sudo &>/dev/null; then
sudo mkdir -p "$AI_BASE"
sudo chown "$(id -u):$(id -g)" "$AI_BASE"
info "Created ${AI_BASE}"
else
error "Need sudo/root to create ${AI_BASE}. Create it manually and re-run."
fi
else
echo ""
read -p "Alternative base directory? [${SCRIPT_DIR}/ai-stack] " ALT_BASE
ALT_BASE="${ALT_BASE:-${SCRIPT_DIR}/ai-stack}"
mkdir -p "$ALT_BASE"
warn "Using ${ALT_BASE}"
AI_BASE="$ALT_BASE"
export AI_LSC_BASE_DIR="$AI_BASE"
fi
fi
fi
# ── Helper: get system Python version string ──────────────────
_sys_python_version() {
python3 -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}')"
}
# ── Helper: check if venv is stale (Python version mismatch) ──
_venv_is_stale() {
if [ ! -f "$VENV_PYTHON_STAMP" ]; then
return 0 # no stamp → treat as stale
fi
local stamp_version
stamp_version="$(cat "$VENV_PYTHON_STAMP")"
local sys_version
sys_version="$(_sys_python_version)"
if [ "$stamp_version" != "$sys_version" ]; then
return 0 # version mismatch → stale
fi
return 1 # versions match → fresh
}
# ── Detect OS ──────────────────────────────────────────────────
if command -v pacman &>/dev/null; then
PKG_MANAGER="pacman"
info "Detected Arch Linux (pacman)"
elif command -v apt-get &>/dev/null; then
PKG_MANAGER="apt"
warn "Detected Debian/Ubuntu — some packages may differ from Arch names"
elif command -v dnf &>/dev/null; then
PKG_MANAGER="dnf"
warn "Detected Fedora/RHEL — some packages may differ from Arch names"
else
warn "Unknown package manager. You may need to install dependencies manually."
PKG_MANAGER="manual"
fi
# ── System Dependencies ───────────────────────────────────────
echo ""
info "Installing system dependencies..."
if [ "$PKG_MANAGER" = "pacman" ]; then
SUDO=""
if [ "$(id -u)" -ne 0 ]; then
if command -v sudo &>/dev/null; then
SUDO="sudo"
fi
fi
# Core system packages (python-pyside6 is NOT in official repos —
# we attempt it, then fall back to pip inside the venv below)
$SUDO pacman -Sy --noconfirm --needed \
python \
python-pip \
git \
tmux \
ripgrep \
fd \
tree-sitter \
sqlite \
redis \
base-devel \
|| warn "Some system packages failed to install (non-critical)"
# Try python-pyside6 from pacman if available (AUR / community)
if $SUDO pacman -Si python-pyside6 &>/dev/null; then
$SUDO pacman -Sy --noconfirm --needed python-pyside6 \
|| warn "python-pyside6 pacman install failed (will try pip fallback)"
else
warn "python-pyside6 not available in repos — will install via pip"
fi
info "Core system packages installed."
echo ""
read -p "Install NVIDIA CUDA support? [y/N] " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
$SUDO pacman -Sy --noconfirm --needed cuda || warn "CUDA install failed"
info "CUDA toolkit installed."
fi
read -p "Install container runtimes (podman, docker)? [y/N] " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
$SUDO pacman -Sy --noconfirm --needed podman docker || warn "Container runtimes install failed"
info "Container runtimes installed."
fi
read -p "Install LXC support? [y/N] " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
$SUDO pacman -Sy --noconfirm --needed lxc lxcfs || warn "LXC install failed"
info "LXC support installed."
fi
elif [ "$PKG_MANAGER" = "apt" ]; then
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends \
python3 python3-pip python3-venv python3-pyside6.qt6 \
git tmux ripgrep fd-find sqlite3 redis-server \
build-essential \
|| warn "Some system packages failed to install"
info "Core system packages installed."
elif [ "$PKG_MANAGER" = "dnf" ]; then
sudo dnf install -y \
python3 python3-pip git tmux ripgrep fd-find sqlite redis \
|| warn "Some system packages failed to install"
info "Core system packages installed."
fi
# ── Python Virtual Environment ─────────────────────────────────
echo ""
_create_venv() {
if [ "$PKG_MANAGER" = "pacman" ]; then
# Arch: --system-site-packages so venv sees pacman-installed PySide6
python3 -m venv --system-site-packages "$VENV_DIR"
else
python3 -m venv "$VENV_DIR"
fi
_sys_python_version > "$VENV_PYTHON_STAMP"
}
if [ ! -d "$VENV_DIR" ]; then
info "Creating Python virtual environment at ${VENV_DIR}..."
_create_venv
else
if _venv_is_stale; then
old_ver=""
if [ -f "$VENV_PYTHON_STAMP" ]; then
old_ver="$(cat "$VENV_PYTHON_STAMP")"
fi
warn "Virtual environment is stale (was Python ${old_ver}, system is now $(_sys_python_version))"
warn "Removing old venv and recreating..."
rm -rf "$VENV_DIR"
_create_venv
info "Virtual environment recreated with Python $(_sys_python_version)"
else
info "Virtual environment already exists and is up-to-date at ${VENV_DIR}"
fi
fi
info "Activating virtual environment..."
source "$VENV_DIR/bin/activate"
# ── Verify venv activation (critical on Arch) ───────────────────
if [ ! -f "${VENV_DIR}/bin/pip" ]; then
error "Virtual environment pip not found at ${VENV_DIR}/bin/pip — venv may be broken. Delete ${VENV_DIR} and re-run."
fi
VENV_PIP="${VENV_DIR}/bin/pip"
# ── Python Dependencies (inside venv — safe from EXTERNALLY-MANAGED) ──
echo ""
info "Installing Python dependencies into venv..."
$VENV_PIP install --upgrade pip setuptools wheel --quiet
# PySide6 — on Arch this comes from pacman (already installed above).
if [ "$PKG_MANAGER" = "pacman" ]; then
if "${VENV_DIR}/bin/python3" -c "import PySide6" 2>/dev/null; then
info "PySide6 (system) — OK"
else
warn "PySide6 (system) not visible in venv — attempting pip install..."
$VENV_PIP install PySide6 --quiet 2>/dev/null || {
warn "PySide6 installation failed. The app will run in headless mode."
}
fi
else
$VENV_PIP install PySide6 --quiet 2>/dev/null || {
warn "PySide6 installation failed. The app will run in headless mode."
}
fi
# uv — fast Python package manager
$VENV_PIP install uv --quiet 2>/dev/null || warn "uv installation failed (non-critical)"
# ── Verify Installation ──────────────────────────────────────
echo ""
info "Verifying installation..."
ERRORS=0
PY_VER="${VENV_DIR}/bin/python3"
PY_VER_STR=$("$PY_VER" -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
PY_MAJOR=$("$PY_VER" -c "import sys; print(sys.version_info.major)")
PY_MINOR=$("$PY_VER" -c "import sys; print(sys.version_info.minor)")
if [ "$PY_MAJOR" -ge 3 ] && [ "$PY_MINOR" -ge 11 ]; then
info "Python ${PY_VER_STR} (venv) — OK"
else
warn "Python ${PY_VER_STR} (venv) — recommend 3.11+"
fi
if "$PY_VER" -c "import PySide6" 2>/dev/null; then
info "PySide6 — OK"
else
warn "PySide6 — NOT FOUND (UI will be unavailable)"
ERRORS=$((ERRORS + 1))
fi
# Check registry loads (pass AI_LSC_BASE_DIR so it resolves correctly)
cd "$SCRIPT_DIR"
if AI_LSC_BASE_DIR="$AI_BASE" "$PY_VER" -c "
import sys
sys.path.insert(0, 'src')
from ai_lsc.registry.loader import load_merged_registry
reg = load_merged_registry()
print(f' Registry: {len(reg)} tools loaded')
" 2>/dev/null; then
info "Registry — OK"
else
warn "Registry — could not load (check file structure)"
ERRORS=$((ERRORS + 1))
fi
for cmd in ollama podman docker tmux ripgrep fd tree-sitter; do
if command -v "$cmd" &>/dev/null; then
info "${cmd} — found"
else
warn "${cmd} — not found (optional)"
fi
done
# ── Summary ───────────────────────────────────────────────────
echo ""
if [ "$ERRORS" -eq 0 ]; then
echo -e "${GREEN}${BOLD}Bootstrap complete! AI Local Stack Control is ready.${NC}"
echo ""
echo -e " ${CYAN}Base dir: ${AI_BASE}${NC}"
echo ""
echo " Launch the application:"
echo " cd ${SCRIPT_DIR}"
echo " python ai_lsc.py"
echo ""
echo " Or use the convenience launcher:"
echo " bash run.sh"
echo ""
else
echo -e "${YELLOW}Bootstrap complete with ${ERRORS} warning(s).${NC}"
echo "The application may run in limited mode. Review warnings above."
fi
# ── Write env file so run.sh and ai_lsc.py can pick it up ────
cat > "${SCRIPT_DIR}/.env" <<ENVEOF
AI_LSC_BASE_DIR=${AI_BASE}
ENVEOF

955
docs/ADR-001-capability-architecture.md Executable file
View File

@ -0,0 +1,955 @@
# ADR-001: The Capability Architecture
**AI-LSC v3.0 — Ankh of Jah**
> *This is the single architectural definition for AI-LSC. Every module, every
> template, every resolver path either implements something defined here or it
> does not belong.*
---
## Status
**Accepted.** Adopted as the foundational architecture for v3.0 (Ankh of Jah)
and all subsequent releases. The agentic execution layer is deferred to v4.0.
---
## 1. Context
AI-LSC did not begin as an architecture. It began as a question:
> "Can I stop manually juggling a dozen AI tools on a Linux machine?"
v1 answered: *yes, with a monolithic script.*
v2 answered: *yes, with a modular registry and layers.*
v3 answers a different question entirely:
> "Can a system *understand* AI infrastructure well enough to deploy,
> validate, diagnose, and reproduce it — without the operator thinking
> about individual tools?"
The shift is from tool-first to system-first. Earlier development asked
"how do we add support for X?" Current development asks "where does X belong
in the architecture?" That is not a cosmetic change. It is a phase change.
Three releases revealed a consistent pattern: the same architectural verbs
kept reappearing across unrelated features. Install, verify, configure,
launch, monitor, export, diagnose, reproduce. Every tool needed them. Every
stack needed them. Every container needed them. The repetition was not a
failure to abstract — it was evidence of an abstraction waiting to be named.
This document names it.
---
## 2. The Foundational Object: Capability
Every system has one concept that, if removed, causes the entire structure to
collapse. For AI-LSC, that concept is **Capability**.
A Capability is a named, validated unit of infrastructure that a machine either
possesses or does not. It is not a tool. It is not a process. It is not a
package. It is a *statement about the machine*.
```
"Inference" — this machine can run LLM inference.
"Vector Store" — this machine can store and query embeddings.
"Monitoring" — this machine can observe its own services.
"GPU Compute" — this machine has CUDA/cuDNN available.
```
Capabilities are discovered, not declared. A tool *provides* capabilities. A
template *requires* capabilities. A pipeline *consumes* capabilities. A
container *exports* capabilities. A dashboard *reports* capabilities. A skill
*extends* capabilities. Monitoring *validates* capabilities.
Every subsystem points at Capability. No subsystem points at Tool directly
except the Registry, which maps tools to the capabilities they provide.
This single inversion eliminates most of the coupling in the application:
```
Tool ──provides──► Capability ◄──requires── Template
▲
Pipeline ──consumes──────┘
▲
Container ──exports───────┘
▲
Dashboard ──reports───────┘
▲
Skill ──extends───────┘
▲
Monitoring ──validates─────┘
```
Swap Ollama for vLLM. Swap Grafana for another observability stack. Swap
Qdrant for Milvus. Everything above the Registry layer does not notice.
The capability model remains stable even when implementations evolve,
technologies are replaced, or entirely new categories of AI software emerge.
---
## 3. The Architecture Pipeline
AI-LSC is not an installer. It is a pipeline from intent to infrastructure.
```
┌──────────────────────────────────────────────────────────────────┐
│ USER INTENT │
│ │
│ "I want a Research Workstation" │
│ "I want a RAG Server" │
│ "I want a GPU Inference Cluster" │
│ "I want a Coding Assistant" │
└────────────────────────┬─────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ TEMPLATE (Recipe) │
│ Desired Architecture │
│ │
│ Research Workstation │ RAG Appliance │ Inference Node │
└────────────────────────┬─────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ RESOLVER │
│ Infrastructure Planning │
│ │
│ • Detect hardware • Detect OS │
│ • Detect installed sw • Detect conflicts │
│ • Expand dependencies • Select implementations │
│ • Produce execution plan │
└────────────────────────┬─────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ REGISTRY │
│ Individual Components │
│ │
│ Every tool knows: Install · Update · Verify · Launch │
│ Health · Configure · Container · Export │
└────────────────────────┬─────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ RUNTIME │
│ │
│ Native · Podman · Docker · LXC · Cluster · Remote │
└──────────────────────────────────────────────────────────────────┘
```
The Resolver is the brain. It is the only component that translates between
the declarative world of templates and the imperative world of package
managers, container runtimes, and service launchers. No other component
performs this translation. This constraint ensures that adding a new runtime
target (say, Kubernetes) requires changes only in the Registry (new tool
entries) and Runtime (new executor), never in templates or pipelines.
---
## 4. Stack Recipes (Templates as Intent)
### 4.1 What a Template Is
A template is infrastructure intent, not an install script. It declares what
the operator wants the machine to become. It does not duplicate install
logic, configuration logic, or launch logic — the Registry already owns all
of that.
The current template format is a flat list of tool IDs. This is functional
but insufficient for the capability architecture. The evolved format — the
**Stack Recipe** — declares capabilities, roles, connections, and startup
semantics:
```yaml
# Stack Recipe — evolved template format (v4.0 target)
stack:
name: Claude Memory Assistant
version: "1.0"
maturity: official # official | community | local | frozen
capabilities:
required:
- inference # needs an LLM engine
- vector_database # needs embedding storage
- relational_database # needs structured storage
- web_interface # needs a browser-accessible UI
optional:
- monitoring
- automation
components:
inference:
engine: ollama
model: llama3
memory:
vectordb: qdrant
embedding_model: nomic-embed-text
database:
engine: postgres
ui:
provider: open_webui
connections:
- from: inference
to: vector_database
protocol: embedding
- from: inference
to: relational_database
protocol: session_store
- from: ui
to: inference
protocol: openai_compat
startup:
order:
1. relational_database
2. vector_database
3. inference
4. ui
health_wait:
- relational_database # UI waits until DB is accepting connections
- vector_database
- inference
health:
checks:
- capability: inference
probe: GET /api/tags
- capability: vector_database
probe: GET /collections
```
### 4.2 What a Template Is Not
A template does not contain:
- Installation commands (the Registry knows how to install)
- File paths (the Resolver knows the layout)
- Port assignments (conflict detection is automatic)
- OS-specific logic (the Resolver handles this)
- Dependency installation order beyond what `startup.order` declares
A template also does not hardcode implementations. It specifies roles:
```yaml
components:
vector_database:
role: vector_store # NOT "qdrant"
```
The Resolver maps `vector_store` to whatever provider is installed or
available. On one machine that is Qdrant. On another it is Milvus. On a
third the Resolver recommends Chroma. The template never changes.
### 4.3 Template Maturity
Templates have a maturity level that signals trust and intent:
| Level | Meaning | Use Case |
|-------|---------|----------|
| **Official** | Maintained by the AI-LSC project | Curated reference stacks |
| **Community** | Shared by users, reviewed | Experimentation, collaboration |
| **Local** | Created by the operator | Personal workflows, one-off stacks |
| **Frozen** | Exact snapshot of a validated environment | Reproducibility, CI/CD, audit |
A Frozen template pins every version, every config hash, every capability
signature. Deploying a Frozen template on a different machine produces a
bit-for-bit equivalent environment. This is the mechanism for long-term
reproducibility — not containerization alone, but declarative infrastructure
with verified provenance.
---
## 5. Role-Based Resolution
The critical distinction between AI-LSC and every other "AI launcher" is
that templates specify **roles**, not implementations.
A role is a capability category with multiple possible providers:
```
Role: Inference Engine
Providers: Ollama · llama.cpp · vLLM · TensorRT-LLM · SGlang
Role: Vector Database
Providers: Qdrant · Chroma · Milvus · Weaviate · FAISS
Role: LLM Gateway
Providers: LiteLLM · 9Router Proxy · Local proxy
Role: Monitoring
Providers: Grafana + Prometheus · Glances · Netdata
Role: Agent Frontend
Providers: Open WebUI · LibreChat · AnythingLLM · Continue
```
The Resolver performs role resolution in this order:
1. **Already installed?** Use what is present.
2. **Compatible with hardware?** Select the best fit (GPU → CUDA-aware provider).
3. **Template preference?** Honor explicit provider hints.
4. **Fallback chain.** Try each candidate in order.
5. **Recommend.** If nothing installs cleanly, report what is needed.
This means a single template shared between two machines can resolve to
completely different toolsets:
```
"Research Workstation" template
Laptop (CPU-only):
→ llama.cpp (CPU inference)
→ LiteLLM (gateway)
→ Chroma (lightweight vector store)
→ Open WebUI (interface)
Desktop (RTX 4090):
→ Ollama (CUDA inference)
→ vLLM (high-throughput serving)
→ Qdrant (production vector store)
→ LibreChat (multi-provider interface)
```
Same template. Different reality. The Resolver is what makes that work.
---
## 6. Component Connections
Installing tools side-by-side is not an architecture. Understanding how they
interact is.
The Stack Recipe format includes a `connections` section that declares
relationships between components. These are not just documentation — they are
inputs to the Stack Doctor (Section 12) and the Resolver's validation
engine.
A connection declaration:
```yaml
connections:
- from: ui # Open WebUI
to: inference # Ollama
protocol: openai_compat # Expects OpenAI-compatible API
- from: ui
to: vector_database
protocol: embedding # Needs embedding endpoint
```
The Resolver uses connections to:
- Validate that protocols are compatible (OpenAI-compat ↔ OpenAI-compat).
- Detect likely misconfigurations (OLLAMA_HOST=localhost when UI is remote).
- Generate connection-specific health checks.
- Produce diagnostic suggestions when connections fail.
This is dependency injection for infrastructure. The template declares the
graph. The Resolver validates the graph. The Runtime instantiates the graph.
---
## 7. The 13-Layer Model
AI-LSC organizes all AI infrastructure into 13 layers. Each layer represents
a category of capability. Tools register into one (sometimes two) layers.
Templates reference layers instead of individual tools when expressing
broad requirements.
```
Layer 1 Host Platform — OS, kernel, filesystem, base packages
Layer 2 Development Env — Python, Rust, Node.js, Go, build tools
Layer 3 GPU Runtime — CUDA, cuDNN, ROCm, Vulkan compute
Layer 4 Inference Engines — Ollama, llama.cpp, vLLM, TensorRT-LLM
Layer 5 Distributed Runtime — Ray, Kubeflow, cluster schedulers
Layer 6 AI Endpoints — LiteLLM, model routers, API gateways
Layer 7 Data & Knowledge — PostgreSQL, MariaDB, data pipelines
Layer 8 Knowledge Management — Qdrant, Chroma, Milvus, vector stores
Layer 9 Automation & Execution — n8n, Airflow, task schedulers
Layer 10 Observability — Prometheus, Grafana, Glances, logging
Layer 11 Intelligent Routing — Fabric, Hermes, agent dispatchers
Layer 12 User Interfaces — Open WebUI, LibreChat, AnythingLLM
Layer 13 Containers — Podman, Docker, LXC, export targets
```
A template can express requirements by layer:
```yaml
capabilities:
layers:
- Inference Engines # Layer 4
- AI Endpoints # Layer 6
- Knowledge Management # Layer 8
- User Interfaces # Layer 12
```
The Resolver fills in everything else. If the template needs inference
(Layer 4) and the host has no GPU (Layer 3), the Resolver knows to
recommend CPU-only providers and skip CUDA-dependent tools automatically.
### Stress Test
The 13-layer model must accommodate any AI project without forcing it. A
non-exhaustive validation set:
| Project | Natural Layer Fit |
|---------|-------------------|
| Open WebUI | 12 (User Interfaces) |
| LiteLLM | 6 (AI Endpoints) |
| Qdrant | 8 (Knowledge Management) |
| Ollama | 4 (Inference Engines) |
| vLLM | 4 (Inference Engines) |
| ComfyUI | 12 (User Interfaces) |
| Flowise | 12 (User Interfaces) |
| n8n | 9 (Automation & Execution) |
| Prometheus | 10 (Observability) |
| Ray | 5 (Distributed Runtime) |
| Langflow | 12 (User Interfaces) |
| Chroma | 8 (Knowledge Management) |
| Milvus | 8 (Knowledge Management) |
| llama.cpp | 4 (Inference Engines) |
| TensorRT-LLM | 4 (Inference Engines) |
| OpenHands | 12 (User Interfaces) |
| Aider | 2 (Development Env) |
| Continue | 2 (Development Env) |
| Kubeflow | 5 (Distributed Runtime) |
| Kafka | 7 (Data & Knowledge) |
Every project in the validation set fits naturally into exactly one layer.
None require special casing. The model appears to generalize well.
---
## 8. Skills as Derived Capabilities
Skills are not file lookups. They are capability queries.
The old model: "Does this Python file exist in the skills directory?"
The new model: "Does this machine currently possess this capability?"
Skills derive from deployed, validated infrastructure:
```
Template: Research Workstation
│
▼ Deployed
│
▼ Verified
│
▼ Registered as Capabilities
│
▼ Skills become available:
│
├── "Local RAG" (has: inference + vector_store + ui)
├── "Python AI" (has: development + inference)
├── "Vision" (has: inference + multimodal_model)
├── "Speech" (has: inference + whisper + tts)
└── "Distributed Inference" (has: inference + distributed_runtime)
```
A skill definition references capabilities, not tools:
```yaml
skill:
name: Local RAG
requires:
capabilities: [inference, vector_database, web_interface]
optional:
capabilities: [monitoring, relational_database]
description: >
End-to-end retrieval-augmented generation using local models.
Available when the machine has an inference engine, a vector store,
and a web interface — regardless of which specific tools provide them.
```
This means installing a new tool that provides an existing capability can
silently unlock skills the operator never explicitly configured. Replace
Qdrant with Milvus and every RAG skill still works, because the capability
did not change — only the provider did.
---
## 9. Pipelines Consume Capabilities
A pipeline is a directed graph of capability requirements. It never names a
tool. It names what it needs:
```
Pipeline: Document RAG
[Source] → [Chunking] → [Embedding] → [Vector Store] → [Retriever] → [LLM] → [Output]
```
Each node is a capability. The Resolver maps each node to a tool at runtime:
```
Embedding:
→ nomic-embed-text (via Ollama)
or
→ bge-small (via llama.cpp)
Vector Store:
→ Qdrant
or
→ Chroma
LLM:
→ Ollama (llama3)
or
→ vLLM (deepseek-coder-33b)
```
The pipeline graph never changes when implementations change. This is what
makes pipelines portable across machines, containers, and clusters.
---
## 10. Container Export as Capability Export
A container image is not a bag of tools. It is a frozen capability set.
When an operator exports a Research Workstation to Podman, the exported
image carries a capability manifest alongside the filesystem layers:
```
Research_Workstation_v1.0
Capabilities:
✓ Inference (Ollama, llama3)
✓ GPU Compute (CUDA 12.4, cuDNN 9.1)
✓ Vector Database (Qdrant)
✓ LLM Gateway (LiteLLM)
✓ Web Interface (Open WebUI)
✓ Monitoring (Prometheus + Grafana)
✓ Relational Database (PostgreSQL)
Stack Recipe: embedded (frozen)
Template: Research Workstation v1.0
Exported: 2026-06-28
Architecture: x86_64
```
When another machine imports this image, AI-LSC reads the manifest and
immediately knows what the container provides — no scanning, no probing, no
guessing. The capabilities are declared, trusted, and verified.
Export targets are format-agnostic:
```
Recipe → Resolver → Generate Deployment
├── Podman Quadlet
├── Docker Compose
├── LXC Config
└── Kubernetes YAML (future)
```
The recipe never changes. Only the exporter changes.
---
## 11. Dashboards Report Capability Health
The dashboard does not display process status. It displays infrastructure
health.
```
┌──────────────────────────────────────────────────────┐
│ Research Workstation ████████ 92%│
│ │
│ Host Platform ✓ │
│ Development Env ✓ │
│ GPU Runtime ⚠ CUDA Update Available │
│ Inference Engines ✓ Ollama · llama3 │
│ AI Endpoints ✓ LiteLLM :4000 │
│ Data & Knowledge ✓ PostgreSQL :5432 │
│ Knowledge Management ✓ Qdrant :6333 │
│ Automation — │
│ Observability ✓ Grafana · Prometheus │
│ Intelligent Routing ✓ Fabric │
│ User Interfaces ✓ Open WebUI :8080 │
│ Containers 2 specialist images │
│ │
│ Templates: 7 installed Skills: 12 available │
└──────────────────────────────────────────────────────┘
```
Each row is a capability, not a tool. The status reflects whether the
machine possesses that capability in a healthy state, regardless of which
tool provides it. If the operator swaps Grafana for Netdata, the
Observability row still shows the same status — because the capability
did not change.
---
## 12. Stack Doctor
The Stack Doctor is a reasoning engine, not a log viewer. It understands
relationships between components and can diagnose problems that span multiple
tools.
Example diagnosis:
```
DIAGNOSIS: Open WebUI cannot reach Ollama
REASON: OLLAMA_HOST is set to localhost (127.0.0.1)
but Open WebUI is configured to connect to port 11434
on all interfaces. Connection is refused.
RECOMMENDATION:
Option A: Set OLLAMA_HOST=0.0.0.0 in Ollama environment
Option B: Bind Open WebUI to localhost only
Option C: Route through LiteLLM proxy
```
Example conflict detection:
```
DIAGNOSIS: Port conflict detected
LiteLLM wants port 4000 ✓ (available)
vLLM wants port 8000 ✗ (occupied by TensorRT-LLM)
RECOMMENDATION:
Move LiteLLM to port 4001
or
Disable TensorRT-LLM if not needed
```
The Stack Doctor uses the connection graph from the Stack Recipe to trace
problems across component boundaries. It does not just check if a process is
running — it checks if the *capability chain* is intact from end to end.
---
## 13. Operator Workflows
### 13.1 Missions
Complex deployments are presented as **Missions**, not wizards. A Mission
is a named, scoped objective with a clear completion state:
```
┌──────────────────────────────────────────────────────┐
│ MISSION: Build Coding Assistant │
│ │
│ Estimated effort: 8 minutes │
│ Status: Planning... │
│ │
│ [✓] Validate host platform │
│ [✓] Detect installed capabilities │
│ [→] Resolve missing dependencies │
│ [ ] Install Python (Layer 2) │
│ [ ] Install Ollama (Layer 4) │
│ [ ] Install LiteLLM (Layer 6) │
│ [ ] Install Open WebUI (Layer 12) │
│ [ ] Configure connections │
│ [ ] Verify health │
│ [ ] Export ready │
└──────────────────────────────────────────────────────┘
```
### 13.2 Routines
Routines are reusable infrastructure actions, not application macros:
| Routine | Actions |
|---------|---------|
| **Morning Check** | Verify all services, restart unhealthy, check updates, check GPU, check disk |
| **Pre-Inference** | GPU memory, temperature, ports, models, KV cache, endpoint ready |
| **Before Export** | Verify services, verify configs, clean logs, freeze versions, generate manifest |
| **Before Commit** | Lint, test, validate registry, validate templates, schema check |
One button. Comprehensive validation.
### 13.3 Next Best Action
AI-LSC suggests the operator's next step based on current state:
```
Good morning.
✓ GPU healthy
✓ Ollama healthy
⚠ Open WebUI update available (v0.3.12 → v0.3.14)
⚠ Research Workstation template has 1 missing dependency
Suggested: Verify Research Workstation
```
This is not AI. It is deterministic inference over the capability graph.
The system knows what is installed, what is healthy, what is outdated, and
what templates require. The recommendation follows directly.
### 13.4 Activity Timeline
Every infrastructure action is recorded with a timestamp:
```
09:13 Installed LiteLLM
09:15 Verified CUDA (driver 550.54, CUDA 12.4)
09:16 Generated template: Research Workstation
09:20 Exported Podman image: research_ws_v1.0
09:27 Health check passed (13/13 capabilities)
```
Timelines are queryable, filterable, and exportable. They provide audit
trail and operational memory.
### 13.5 Workspaces
Workspaces group related infrastructure by purpose, not by tool:
```
Research → inference + vector_db + ui + monitoring
Coding → development + inference + endpoints + ui
RAG → inference + vector_db + relational_db + ui
Cluster → distributed + inference + monitoring + containers
```
Click a workspace. Everything related appears. One context for one purpose.
---
## 14. Adaptive Templates
A single template adapts to the host hardware, installed software, and
available runtimes. The Resolver selects implementations based on
constraints, not preferences.
```
"Research Workstation" on different hardware:
Laptop (CPU, 16GB RAM):
→ llama.cpp (quantized, CPU inference)
→ Chroma (in-process vector store, minimal memory)
→ LiteLLM (lightweight gateway)
→ Glances (lightweight monitoring)
→ Open WebUI (browser interface)
Desktop (RTX 4090, 64GB RAM):
→ Ollama (CUDA-accelerated inference)
→ Qdrant (production vector store with GPU-accelerated HNSW)
→ LiteLLM + vLLM (dual gateway: fast + thorough)
→ Prometheus + Grafana (full monitoring stack)
→ LibreChat (multi-provider interface)
Server (Dual MI300X, 256GB RAM):
→ SGLang (ROCm-optimized inference)
→ Milvus (distributed vector store)
→ LiteLLM (cluster gateway)
→ Prometheus + Grafana + AlertManager (production monitoring)
→ Open WebUI (load-balanced)
```
Same template. Same intent. Different reality. The Resolver is what makes
the template portable.
---
## 15. Rationale
### Why Capability as the central abstraction?
Because tools are ephemeral. The AI landscape changes monthly. New inference
engines appear. Old ones are abandoned. Monitoring stacks get replaced.
Vector databases get acquired and deprecated.
But the *capabilities* those tools provide are remarkably stable. "The
machine can run LLM inference" has been true since 2023 and will be true
in 2030. The implementation changes. The capability does not.
Building around capabilities means AI-LSC's architecture decays at the
rate of the AI industry's *conceptual* evolution, not its *tool* churn.
Conceptual evolution is orders of magnitude slower.
### Why not just use Terraform / Kubernetes?
Because those tools solve a different problem. Terraform manages cloud
infrastructure declaratively. Kubernetes orchestrates containers at scale.
Neither understands that "install Qdrant" implies "the machine now has
vector database capability" — nor should they. That is AI-LSC's domain.
AI-LSC is specifically designed for the local AI operator who needs to
assemble, validate, and reproduce AI stacks on single machines or small
clusters. It fills the gap between "install scripts" and "cloud
orchestration."
### Why role-based resolution instead of tool-specific templates?
Because a template that hardcodes Qdrant cannot run on a machine that only
has Milvus. A template that hardcodes Ollama cannot leverage an existing
vLLM installation. Role-based resolution makes templates portable,
shareable, and future-proof without requiring the template author to
anticipate every possible provider.
---
## 16. Consequences
### Positive
- **Tool swaps are zero-cost above the Registry.** Replacing a provider
requires only a new Registry entry with the same capability mapping.
Templates, pipelines, skills, and dashboards are unaffected.
- **Templates are shareable across heterogeneous hardware.** The same
recipe produces appropriate deployments on laptops, desktops, and
servers.
- **New capabilities can be added without modifying existing templates.**
Adding a "Speech-to-Text" capability does not require touching any
Research Workstation template.
- **Container exports carry semantic meaning**, not just filesystem
state. Importing a container immediately reveals its capabilities.
- **Diagnostics can reason about relationships**, not just individual
process health.
### Neutral
- **The Resolver is the most complex component.** It must understand
hardware detection, OS differences, dependency graphs, conflict
resolution, and provider selection. This is acceptable because the
Resolver is a single, well-bounded component.
- **The capability vocabulary must be curated.** New capabilities require
consensus on naming, boundaries, and provider criteria. This is a
governance concern, not a technical one.
### Risks
- **Over-abstraction.** If the capability vocabulary is too coarse
("compute"), it loses discriminating power. If too fine ("qdrant-hnsw-
gpu"), it reverts to tool-specific coupling. The granularity must be
calibrated through real-world use.
- **Resolver complexity.** A naive Resolver that tries all combinations
is NP-hard. The Resolver must use heuristics, caching, and constraint
propagation to remain fast.
- **Capability drift.** As the AI ecosystem evolves, capabilities may
split or merge. "Inference" might split into "Text Inference" and
"Multimodal Inference." The architecture must handle capability
evolution without breaking existing templates.
---
## 17. Architecture Completeness
Current state of implementation (v3.0 Ankh of Jah):
```
Registry (tool metadata, 115 tools) ████████████░ 95%
Templates (stack recipes, 4 templates) ██████░░░░░░ 55%
Resolver (dependency expansion, planning) ███░░░░░░░░░ 30%
Installer (native, git, npm, pip) ████████████░ 95%
Verification (install checks, health probes) ██████████░░░ 85%
Health (service status, GPU monitoring) ███████░░░░░ 65%
Export (Podman, Docker, LXC configs) ████████░░░░ 80%
Monitoring (glances integration, Prometheus) █████░░░░░░░ 50%
Skills (capability-derived skills) ███░░░░░░░░░ 25%
Pipelines (capability graph execution) ██░░░░░░░░░░ 20%
Dashboards (capability health display) ████░░░░░░░░ 35%
Stack Doctor (diagnostic reasoning) ██░░░░░░░░░░ 15%
Missions (guided deployment flows) █░░░░░░░░░░░ 10%
Workspaces (purpose-based grouping) ███░░░░░░░░░ 25%
Activity Timeline ██░░░░░░░░░░ 20%
Next Best Action █░░░░░░░░░░░ 10%
Documentation (this ADR, README, guides) ██████░░░░░░ 55%
Tests ██░░░░░░░░░░ 20%
```
The pattern is clear: the foundation (Registry, Installer, Verification) is
strong. The intelligence layer (Resolver, Stack Doctor, Missions) is where
the next investment goes. The UI layer (Dashboards, Workspaces, Timeline)
follows the intelligence layer.
---
## 18. Feature Policy (Ankh of Jah Stabilization)
v3.0 enters a stabilization phase. Feature velocity decreases; stability
velocity increases.
### Allowed
- Bug fixes
- Registry additions (new tool metadata, new providers)
- New templates (stack recipes)
- Installer verification and hardening
- UI polish and usability improvements
- Documentation
- Tests
- Capability vocabulary refinement
- Resolver heuristic improvements
### Not Allowed
- New architectural concepts
- New runtime systems
- Major UI redesigns
- New registry formats (schema changes)
- Agent execution (deferred to v4.0)
- Cluster orchestration (deferred to v4.0)
- Remote node management (deferred to v4.0)
### v4.0 Scope (Deferred)
The agentic execution layer — where an LLM operates AI-LSC through
function-calling, using the agents/ bridge to start/stop services, pull
models, inject skills, and diagnose issues through natural language. This
is architecturally designed (agents/ package exists, tool_bridge and
ollama_tools are implemented, Redis pub/sub infrastructure is in place)
but intentionally not activated in v3.0.
---
## 19. Project Philosophy
AI-LSC is a native-first, metadata-driven infrastructure manager for local
AI systems. It treats AI software as reusable infrastructure rather than
isolated applications, enabling reproducible deployments, validation,
monitoring, and export of complete AI environments.
This single paragraph is the decision filter for every proposed feature.
If a feature supports this philosophy — making AI infrastructure easier to
deploy, validate, reproduce, and understand — it belongs. If it does not,
it does not.
AI-LSC's biggest competitor is not another AI launcher. It is the manual
process that most developers still follow: reading installation guides,
cloning repositories, creating Python environments, debugging version
conflicts, writing ad hoc shell scripts, and hoping they can recreate the
setup six months later.
If AI-LSC can replace that with: select a template, review the execution
plan, deploy, verify, export — then it has solved a real engineering
problem.
---
## 20. The Architectural Vocabulary
These terms are stable. They will not change in v3.0 patches. They may
evolve in v4.0, but only with explicit ADR amendment.
| Term | Definition |
|------|-----------|
| **Capability** | A named, validated unit of infrastructure that a machine possesses or does not. The central abstraction. |
| **Template / Stack Recipe** | A declarative document expressing infrastructure intent. Specifies capabilities and roles, not tools. |
| **Resolver** | The planning engine that maps intent to execution. Detects hardware, resolves roles, expands dependencies, produces plans. |
| **Registry** | The knowledge base of individual tools. Each entry maps a tool to its capabilities, installers, launchers, health probes, and exporters. |
| **Role** | A capability category with multiple possible providers (e.g., "Vector Database" → Qdrant, Chroma, Milvus). |
| **Skill** | A capability-derived behavior. Available when all required capabilities are present and healthy. |
| **Pipeline** | A directed graph of capability requirements. Consumes capabilities; does not name tools. |
| **Connection** | A declared relationship between two components in a Stack Recipe. Used for validation and diagnostics. |
| **Stack Doctor** | A diagnostic reasoning engine that traces problems across component boundaries using the connection graph. |
| **Mission** | A named, scoped deployment objective with a clear completion state. |
| **Routine** | A reusable infrastructure action (health check, pre-flight, cleanup). |
| **Workspace** | A purpose-based grouping of related infrastructure. |
| **Frozen** | An exact snapshot of a validated environment, pinned at every version. |
| **Layer** | One of 13 categories of AI infrastructure. Tools register into layers. Templates can reference layers. |
| **Runtime** | The execution target: native, Podman, Docker, LXC, cluster, or remote. |
---
*Ankh of Jah marks the point where AI-LSC stopped being a Python application
and became a platform architecture. Future releases build on this foundation.
They do not revisit it.*

99
docs/ADR-002-pipeline-ticker.md Executable file
View File

@ -0,0 +1,99 @@
# ADR-002: Pipeline Ticker
**Date:** 2026-07-07
**Status:** Accepted
## Context
When a user stages a flow of tools in the Stack Editor, the only way to verify the wiring topology is to switch to the IPC Stack tab and read the connections table. There is no at-a-glance visualization of which tools talk to which other tools, in which direction, over what interface. Debugging a misconfigured stack requires the user to mentally join the active-tools list against the registry's `deps` field and the `STACK_WIRINGS` topology — a task the tool is supposed to do for them.
The user asked for a "ticker-style scrolling status" that shows "pipeline connection and direction information per workspace view at the top" so they can "visualize what they are doing as they debug the logic for their stack."
## Decision
Add a new `PipelineTicker` widget at the top of every workspace tab (above the `QStackedWidget` so it persists across all 13+ pages).
### Data source
The ticker reads the active-tools set from `pipeline_state.json` and joins it against `STACK_WIRINGS` in `stack/connections.py`. Only edges where **both endpoints are in the active set** are rendered. Any active tool that does not appear in any edge is flagged as an orphan.
This was chosen over the alternatives (active+deps, all 124 tools, layer-filtered) because:
- **Active-only** matches the user's mental model — "what is my staged flow doing right now?"
- **Active-only** keeps the ticker readable when 20+ tools are staged; the alternatives would either drown the user in dimmed inactive edges or require an extra filter UI.
- Orphan detection is the killer feature — it surfaces real registry gaps (e.g. the `open_webui` vs `openwebui` tool_id collision) that no other view in the app exposes.
### Visual encoding
Each edge renders as `provider ──interface──▶ consumer` on a single line. The arrow color encodes the interface type via a static map (`_INTERFACE_COLORS`):
| Interface | Color | Hex |
|-----------|-------|-----|
| `openai_api` / `ollama_api` | blue | `#2563eb` |
| `vector` / `vector_search` / `embedding` | green | `#16a34a` |
| `redis_pubsub` / `redis_cache` | orange | `#ea580c` |
| `postgresql` / `mariadb` / `mysql` | purple | `#9333ea` |
| `http_api` / `websocket` / `grpc` | teal | `#0d9488` |
| `filesystem` / `tmux_socket` / `systemd_unit` | slate | `#64748b` |
| `cuda_driver` | red | `#dc2626` |
| (default) | slate | `#64748b` |
Tool pills are white when stopped, blue-100 when running. Orphan pills are red-100 with a `❗` prefix.
### Rendering approach
The ticker is a single `QWidget` subclass that paints itself via `paintEvent`. We deliberately avoided a row of `QLabel` widgets because Qt's layout system would fight the horizontal scroll animation. The scroll is driven by a `QTimer` firing every 33 ms (~30 FPS); each tick advances `self._scroll_offset` by 1 pixel. The content is drawn twice (one full copy + one wrap-ahead copy) so the scroll visually never has a gap.
### Interaction
- **Hover-pause** — `enterEvent` sets `self._hovered = True`, which short-circuits the scroll-tick handler. The user can read a long flow without it scrolling away.
- **Click-to-jump** — `mousePressEvent` hit-tests the click X coordinate against a cached list of pill rects (rebuilt during every paint). On hit, emits `tool_clicked(str)`. The main window connects this to a handler that switches to the Tools tab and calls `ToolsTab.highlight_tool(tool_id)`.
### Refresh cadence
The ticker refreshes on two events:
1. `_populate_services` (after the Stack Editor recompiles the active set)
2. `poll_services` (every 2 s service-status poll — so running-state color changes propagate immediately)
The refresh is cheap (one pass over the active set + one pass over `STACK_WIRINGS` for in-stack edges), so we did not bother with diffing — the whole edge list is rebuilt and passed to `set_edges()`.
## Consequences
### Positive
- Users can immediately see whether their staged flow is wired correctly without leaving the current tab.
- Orphan detection surfaces real registry gaps — the `open_webui` vs `openwebui` collision was found within minutes of the first simulation.
- Click-to-jump makes the ticker an active debugging aid, not just a passive display.
- The widget is self-contained (no dependencies on the rest of the UI), so it can be unit-tested in isolation once a Qt test harness is in place.
### Negative
- The ticker's value scales with how complete the `STACK_WIRINGS` data is. Currently 60 wirings for 124 tools — gaps surface as orphans, which can be noisy until the wiring data is backfilled.
- The `paintEvent`-driven rendering is more code than a QLabel row would have been, but the trade-off was necessary for smooth scrolling.
- The pill-rect hit-test cache is rebuilt on every paint, so click accuracy depends on the most recent render. Acceptable for a ticker (the user clicks what they see).
### Neutral
- The `_INTERFACE_COLORS` map is a static dict in the widget module. If the registry's interface_id conventions evolve, the map needs to be updated. A future improvement would be to derive the colors from the `ToolInterface` dataclass itself.
## Alternatives considered
### A row of QLabel widgets in a QHBoxLayout
Rejected because Qt's layout system would fight the horizontal scroll animation. We would have to either disable the layout and manually position labels (which is what `paintEvent` does, but with more code), or use a `QScrollArea` (which adds scroll bars the user didn't ask for).
### A graph view (nodes + edges) instead of a linear ticker
Rejected for the primary use case — the user explicitly asked for a "ticker-style scrolling status." A graph view is harder to fit at the top of every tab and harder to read at a glance. A larger graph view could be added as a separate "Pipeline Flow" tab in a future pass (the original ask included this as an option).
### Active+deps as the data source
Rejected because deps are a build-time concept (what the installer needs to fetch) while wirings are a runtime concept (what the running tool connects to). Mixing them would conflate two different views of the stack.
## Future work
- Backfill `STACK_WIRINGS` entries for the tools currently flagged as orphans (open_webui, qdrant, redis when staged without consumers, etc.).
- Add a "Pipeline Flow" tab with a larger graph view (nodes + edges, draggable) for users who want more detail than the ticker provides.
- Unit tests with a Qt test harness (`pytest-qt`) once the project adopts one.
- Derive arrow colors from the `ToolInterface` dataclass instead of the static `_INTERFACE_COLORS` map.

125
docs/ADR-003-workspace-tab.md Executable file
View File

@ -0,0 +1,125 @@
# ADR-003: Workspace Tab (Peek-Style Orchestration)
**Date:** 2026-07-07
**Status:** Accepted
## Context
The v3.0 workflow for interacting with a running tool required context-switching out of the AI-LSC app:
- Web-interface tools (OpenWebUI, Hermes, Odysseus, ComfyUI, etc.) required opening a browser tab and navigating to `http://127.0.0.1:{port}`.
- CLI tools (Aider, Claude Code, OpenHands, etc.) required opening a terminal and `tmux attach -t ai_lsc_<uid>::<tool_id>`.
When debugging a stack of 8+ tools, this meant 8+ browser tabs + 8+ terminal windows — exactly the juggling the app was supposed to eliminate.
The user asked for a "peek orchestration" surface that "feels like managing VMs from virt-manager or aqemu" — every staged tool reachable from a single window, no context-switch to a browser or terminal app required. The user also raised the option of using servo (Mozilla's Rust web engine) to embed web tools directly, calling it a "workspace" that "can have multiple tabs for various tools like openwebui, hermes dashboard, or odysseus even."
## Decision
Add a new `WorkspaceTab` widget with one sub-tab per active tool. The sub-tab type is chosen by the tool's feature flags:
- `has_web=True` → `_WebToolPage` — embeds the tool's web UI via `QWebEngineView`
- `has_cli=True` (and no web) → `_CliToolPage` — embeds the tool's tmux session via `tmux capture-pane` polling
- otherwise → `_PlaceholderPage` — explains the tool is passive / library
### Web embedding: QWebEngineView (with servo-swap path documented)
The web embedding uses `PySide6.QtWebEngineWidgets.QWebEngineView` by default. Servo's Python bindings are not yet production-ready and have no first-class PySide6 integration, so servo is not the default. However, the module is structured so swapping in servo later is a one-line change: replace `_make_web_view()` with a servo-backed widget that exposes the same `setUrl()` / `url()` / `load()` API. The rest of `_WebToolPage` only depends on that API.
The `_WebToolPage` includes:
- A URL bar at the top showing the loaded URL (monospace, slate-50 background)
- A ⟳ reload button
- A ↗ "open in external browser" button (fallback for when QtWebEngine is not installed)
- The `QWebEngineView` itself, stretched to fill the remaining space
When `QtWebEngine` is not installed (some minimal PySide6 installs skip it), the page falls back to a `_PlaceholderPage` explaining the situation and offering the ↗ button to open the URL externally.
### CLI embedding: tmux capture-pane polling
The `_CliToolPage` is a `QTextEdit` (read-only, dark theme, monospace font) that polls `tmux capture-pane -t <session>::<tool_id> -p -S -200` every 250 ms (4 Hz) and renders the captured output. The poll is driven by a `QTimer`; the captured content is compared to the current `toPlainText()` and only written if it changed (avoids flicker). The cursor is auto-scrolled to the bottom on each refresh.
The terminal is **read-only** — input is not supported. This is a deliberate scope limit: building a full PTY emulator is a separate project, and the user's stated use case ("peek orchestration") is read-only monitoring. For interactive input, the user should use a real terminal app and `tmux attach -t <session>::<tool_id>`.
The page exposes a `stop_polling()` method that the parent `WorkspaceTab` calls when the sub-tab is closed — this prevents the `QTimer` from outliving the page widget.
### Placeholder for passive / library tools
The `_PlaceholderPage` is shown for tools that have no interactive surface (no web + no CLI). It explains that the tool "is a passive/library tool with no interactive surface. It runs in the background and is consumed by other tools." This is informational only — there is no Start button because the tool's launcher type is `passive` (no `systemd` / `tmux` / `desktop` / `lxc`).
### Placeholder for not-yet-running tools
When a web or CLI tool is staged but not running, the sub-tab shows a `_PlaceholderPage` with a **Start tool** button. The button emits `start_tool_requested(str)`, which the main window connects to a handler that finds the matching `ServiceRow` and calls `start_service()`. After 1.5 s the workspace tab auto-refreshes so the placeholder → live view switch happens automatically.
### Sub-tab labels
Each sub-tab is labeled with an emoji prefix + tool_id:
- 🌐 — web-interface tool
- ⌨ — CLI tool
- 📦 — passive / library tool
- ⏸ suffix — tool is staged but not yet running
The emoji encoding lets the user scan the tab bar at a glance and see what kind of surface each sub-tab provides.
### Sub-tab closing
Sub-tabs are closable via the standard × button. Closing a sub-tab calls `stop_polling()` on the page (for CLI tools) but does **NOT** stop the underlying tool — that's the user's call from the Stack Editor. The empty-state placeholder is restored when all sub-tabs are closed.
## Refresh model
`WorkspaceTab.refresh()` is called from:
1. `_populate_services` (after the Stack Editor recompiles the active set)
2. Nav-click on the Workspace entry in the sidebar
3. 1.5 s after a "Start tool" click (so the placeholder → live view switch happens)
The refresh is **destructive** — all sub-tabs are torn down and rebuilt. This is simpler than diffing the active set, and the user's mental model is "refresh = rebuild." A future improvement would be to preserve sub-tab order and only add/remove the delta.
## Consequences
### Positive
- Every active tool is reachable from a single window — no context-switch to a browser or terminal app.
- Web tools embed directly via QWebEngineView, which is the standard PySide6 solution and ships with most PySide6 installs.
- CLI tools attach to the existing tmux session that the runtime executor already manages — no new process lifecycle to worry about.
- The "Start tool" button on placeholder pages bridges the gap between staging and running without requiring the user to switch to the Stack Editor.
- The servo-swap path is documented so a future servo migration is a one-line change.
### Negative
- The CLI embedding is read-only. Users who want interactive input must still use a real terminal app. This is a deliberate scope limit, not a bug.
- The `tmux capture-pane` poll runs every 250 ms per CLI sub-tab. With 10 CLI tools open, that's 40 Hz of subprocess calls — measurable but not heavy. A future improvement would be to pause polling when the sub-tab is not visible.
- The web embedding depends on `QtWebEngine`, which is a separate package on some Linux distros. The fallback placeholder handles the missing-package case gracefully.
- The destructive refresh model means sub-tab order is not preserved across refreshes. A future improvement would be to preserve order.
### Neutral
- The emoji prefixes (🌐 ⌨ 📦 ⏸) are not accessible to screen readers. A future improvement would be to add `setAccessibleName` on each sub-tab.
## Alternatives considered
### Open tools in an external browser / terminal
Rejected — this is the v3.0 behavior the user explicitly asked to replace. The whole point of the Workspace tab is to eliminate the context-switch.
### Use servo instead of QWebEngineView
Rejected as the default because servo's Python bindings are not production-ready and have no first-class PySide6 integration. The module is structured so servo can be swapped in later as a one-line change to `_make_web_view()`.
### Build a full PTY emulator for CLI tools
Rejected as scope creep. The user's stated use case is "peek orchestration" — read-only monitoring. A full PTY emulator is a separate project. For interactive input, the user should `tmux attach` from a real terminal.
### One workspace per tool (separate windows)
Rejected — this would re-create the multi-window juggling the app is supposed to eliminate. The single-window, multi-tab model matches the virt-manager / aqemu reference the user cited.
## Future work
- Preserve sub-tab order across refreshes.
- Pause `tmux capture-pane` polling when the sub-tab is not visible.
- Add `setAccessibleName` to each sub-tab for screen-reader support.
- Add a "Detach" button on each sub-tab that opens the tool in an external browser / terminal (for when the user does want a separate window).
- Swap in servo for web embedding once servo's Python bindings are production-ready.
- Add interactive input to the CLI embedding (would require a real PTY, e.g. via `QProcess` + `QTerminal` or a third-party widget).

BIN
docs/screenshots/ai-lsc-about.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 96 KiB

BIN
docs/screenshots/ai-lsc-centralized-git-repos-manager.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 154 KiB

BIN
docs/screenshots/ai-lsc-chat.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 83 KiB

BIN
docs/screenshots/ai-lsc-deployment-targets.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 59 KiB

BIN
docs/screenshots/ai-lsc-infrastructure-is-the-tools-seen-by-ai-lsc.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 67 KiB

BIN
docs/screenshots/ai-lsc-install-verification-of-tool.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 94 KiB

BIN
docs/screenshots/ai-lsc-monitor-active-workstation.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 90 KiB

BIN
docs/screenshots/ai-lsc-settings.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 68 KiB

BIN
docs/screenshots/ai-lsc-skills-manager.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 128 KiB

BIN
docs/screenshots/ai-lsc-stack-logic-editor.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 97 KiB

BIN
docs/screenshots/ai-lsc-tool-db-editor.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 167 KiB

BIN
docs/screenshots/ai-lsc-workspaces.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 61 KiB

114
gitcommit Executable file
View File

@ -0,0 +1,114 @@
AI-LSC Release v3.0 -- codename: Ankh of Jah
Full architectural rewrite from v2.4. This is not an incremental
release; the project fundamentally changed structure, scope, and
design philosophy.
v2.4 was a monolithic tool launcher.
v3.0 is a metadata-driven infrastructure management platform.
What changed:
Architecture
- Migrated from flat script layout to src/ai_lsc/ package with 22
core modules, 13 registry layer files, and clean separation of
concerns (registry, runtime, stack, skills, agents, ui)
- Introduced AI_LSC_BASE_DIR env var chain: bootstrap writes .env,
launcher reads it, constants.py resolves it -- entire runtime
derives from one variable. No hardcoded paths leak outside constants.
- ADR-001 established: Capability as the foundational abstraction.
Every subsystem (templates, pipelines, skills, containers,
dashboards, monitoring) points at Capability. Tool is a
implementation detail. Swap Ollama for vLLM -- nothing above the
registry notices. See docs/ADR-001-capability-architecture.md.
Registry
- 115 registered tools organized across 13 infrastructure layers
(Host Platform -> Containers). Each entry carries full metadata:
installer, launcher, health probe, configuration, export rules,
capability mapping.
- Modular per-layer registry: ai_lsc/registry/layers/*.py -- adding a
tool means adding a dict entry, not touching Python logic.
- Registry auto-merges new keys across releases without user action.
Stack Templates
- Template system expresses infrastructure intent, not install scripts.
- 4 included templates (Claude Code Setup, Agentic OS Stack,
SaaS Integrations, Free Claude Code).
- Templates reference tool IDs; the registry provides all install and
launch logic. Zero duplication.
- Stack Wizard UI for template selection and deployment.
Runtime
- Multi-runtime execution: native, Podman, Docker Compose, LXC.
- RuntimeExecutor unifies install/start/stop/health across all targets.
- Container export generates Podman Quadlet, Docker Compose, or LXC
config from a template -- same template, different output format.
- Fixed LXC config path bug (Path string concat) and hardcoded
/mnt/AI mount targets.
Bootstrap
- Fully portable: extracts anywhere, detects parent directory tree,
auto-resolves base dir. No assumptions about install location.
- Arch Linux: --system-site-packages venv sees pacman-installed
PySide6, avoids pip compile hell.
- Python version stamping detects stale venv after pacman -Syu,
auto-recreates.
- Cleans stale ~/.local/bin/ai-lsc and pipx installs from v2.x.
- python-pyside6 availability check before attempting pacman install
(package not in official Arch repos).
Installer
- Supports pacman, apt, dnf, pip, git clone, npm, and manual installers.
- Dependency expansion: installing a template auto-detects and installs
missing prerequisite tools.
- License gate: tools requiring license acceptance prompt before install.
Verification
- Per-tool health probes: process check, port check, API ping.
- Verification dashboard shows install status for all 115 tools.
- Hardware detection: GPU driver, CUDA version, memory, disk.
UI
- PySide6 dark-themed interface with sidebar navigation across all
13 layers.
- Dashboard with live service status, log feed, and system health.
- Infrastructure layer pages with per-tool ServiceRow widgets.
- Code analysis panel: ripgrep, fd, AST inspection, tree-sitter parsing.
- Skills console with modelfile tree browser and model pull.
- IPC stack editor for pipeline visualization.
- Settings page with base directory configuration.
- Fixed 8 NameError crashes in exception handlers (except without as).
- Fixed thread-unsafe UI mutation from background install thread.
- Fixed stack wizard KeyError on missing metadata fields.
Skills
- Skill definitions as JSON manifests with capability requirements.
- SkillRuntimeResolver maps skill requests to available capabilities.
- 6 included skills (code-reviewer, vector-search, rag-analyst,
stack-operator, agent-orchestrator, redis-operator).
Agents (infrastructure only -- deferred to v4.0)
- Redis pub/sub bridge for inter-agent communication.
- Tool bridge connecting agent function-calling to RuntimeExecutor.
- Ollama tools interface for model management.
- Model pool with tier routing (8B/14B/32B/70B).
- Dispatcher and clarification gate for multi-turn agent loops.
- All symbols safely stubbed; no agent execution in v3.0.
Bug fixes from v2.4
- Resolved externally-managed-environment error on Arch (venv isolation)
- Removed ModuleNotFoundError from stale ~/.local/bin/ai-lsc entry point
- Root launcher (ai_lsc.py) replaces pip entry-point -- no install needed
- Convenience launcher (run.sh) sources .env for proper env injection
- Path("/var/lib/lxc" / name) string concat crash -> proper Path join
- 8x except Exception without as exc -> all now capture correctly
- Thread-unsafe _run_install UI calls -> QTimer.singleShot dispatch
- Hardcoded /mnt/AI LXC mount targets -> dynamic path resolution
- Stack wizard meta['name'] KeyError -> .get() with safe defaults
- python-pyside6 pacman fallback for Arch repos that don't carry it
Files changed: essentially everything. See ADR-001 for the architectural
rationale and the vocabulary that will govern v3.0 stabilization and
v4.0 development.
```

79
pyproject.toml Executable file
View File

@ -0,0 +1,79 @@
[build-system]
requires = [
"setuptools>=80",
"wheel",
]
build-backend = "setuptools.build_meta"
[project]
name = "ai-lsc"
version = "3.1.0"
description = "AI Local Stack Control — PySide6 desktop app for orchestrating local AI/ML tool stacks"
readme = "README.md"
license = {text = "AGPL-3.0-or-later"}
requires-python = ">=3.11"
authors = [
{name = "AI-LSC Contributors"},
]
keywords = [
"ai", "llm", "local-ai", "stack-management", "pyside6",
"ollama", "vllm", "container-management", "iac",
"terraform", "pulumi", "opentofu", "crossplane",
]
classifiers = [
"Development Status :: 4 - Beta",
"Environment :: X11 Applications :: Qt",
"Framework :: PySide6",
"Intended Audience :: Developers",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
"Topic :: System :: Systems Administration",
]
dependencies = [
"PySide6>=6.6",
"psutil>=5.9",
]
[project.optional-dependencies]
gpu-nvidia = ["cupy-cuda12x"]
gpu-amd = ["cupy-rocm"]
dev = [
"pytest>=7.0",
"pytest-cov>=4.0",
"mypy>=1.0",
"ruff>=0.1",
]
[project.scripts]
ai-lsc = "ai_lsc.__main__:main"
[tool.setuptools.packages.find]
where = ["src"]
include = ["ai_lsc*"]
[tool.setuptools.package-data]
"ai_lsc.registry.stack_templates" = ["*.json"]
[tool.ruff]
target-version = "py311"
line-length = 100
[tool.ruff.lint]
select = ["E", "F", "W", "I", "N", "UP"]
[tool.mypy]
python_version = "3.11"
warn_return_any = true
warn_unused_configs = true
ignore_missing_imports = true
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]

258
quickstart.md Executable file
View File

@ -0,0 +1,258 @@
# Quickstart Guide
Get AI Local Stack Control up and running in under 5 minutes.
## Prerequisites
| Requirement | Minimum | Recommended |
|-------------|---------|-------------|
| OS | Arch Linux | Arch Linux / EndeavourOS |
| Python | 3.11 | 3.12+ |
| RAM | 8 GB | 16 GB+ (for LLM inference) |
| Disk | 4 GB free | 20 GB+ (for model storage) |
| GPU | None | NVIDIA (CUDA) or AMD (ROCm) |
## Installation
### Option 1: Bootstrap Script (Recommended)
```bash
# Clone the repository
git clone https://github.com/your-username/ai-lsc.git
cd ai-lsc
# Run the bootstrap script (installs system + Python deps)
chmod +x bootstrap.sh
./bootstrap.sh
# Launch the application
python -m ai_lsc
```
### Option 2: Manual Install
#### Step 1: System Dependencies
```bash
# Core packages (Arch Linux)
sudo pacman -S python python-pip pyside6 \
git tmux ripgrep fd tree-sitter sqlite redis
# Optional: GPU support
sudo pacman -S cuda # NVIDIA
# sudo pacman -S rocm-hip-sdk # AMD
# Optional: Container runtimes
sudo pacman -S podman docker
# Optional: LXC support
sudo pacman -S lxc lxcfs
```
#### Step 2: Python Dependencies
```bash
cd ai-lsc
# Create a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate
# Install PySide6 and dependencies
pip install PySide6
pip install -e .
```
#### Step 3: Verify Installation
```bash
# Check that the registry loads correctly
python -c "
from ai_lsc import DEFAULT_REGISTRY, validate_registry
errors = validate_registry(DEFAULT_REGISTRY)
print(f'Registry loaded: {len(DEFAULT_REGISTRY)} tools')
print(f'Validation errors: {len(errors)}')
"
# Expected output (v3.1):
# Registry loaded: 125 tools
# Validation errors: 0
```
#### Step 4: Launch
```bash
python -m ai_lsc
```
## First Launch
When you launch AI-LSC for the first time, you will see the **Stack Template Wizard**. This is your entry point for configuring your AI stack.
### Choosing a Template
| Template | Best For | Tool Count |
|----------|----------|-----------|
| Claude Code Setup | Claude Code development workflow | 11 |
| Free Claude Code | Minimal Claude Code environment | 4 |
| Local LLM Lab | Self-hosted LLM experimentation | 10 |
| SaaS Integrations | Production deployment with SSL/CDN | 12 |
### Manual Configuration
If you prefer to build your stack from scratch:
1. Select **Create From Scratch** in the wizard
2. Navigate to the **Infrastructure** section in the sidebar
3. Expand each layer and toggle tools on/off
4. Use the **IPC Stack** tab to validate dependencies
5. Click **Compile** to save your stack configuration
### Watching the Pipeline Ticker
Once you have tools staged, the **Pipeline Ticker** at the top of every tab comes alive. It scrolls horizontally through the wiring topology of your active tools:
- `provider ──interface──▶ consumer` shows the data-flow direction
- Arrow color encodes the interface type (blue = openai_api, green = vector, orange = redis_pubsub, purple = postgresql, teal = http_api)
- Tools with no in-stack wiring are flagged red with an `❗` prefix — these are orphans worth investigating (either they're missing a `STACK_WIRINGS` entry in `stack/connections.py`, or the tools they wire to aren't yet staged)
- Hover over the ticker to pause the scroll
- Click any tool pill to jump to the Tools tab — the matching row is selected, scrolled to center, and flashed amber for 1.5 s
The ticker refreshes automatically on every service-status poll (every 2 s by default) so running-state color changes (stopped → running) propagate immediately.
### Using the Workspace Tab
The **Workspace** nav entry (between Chat and Git Sources) is your peek-style orchestration surface — think virt-manager for your AI stack. After you stage + start tools:
1. Click **Workspace** in the sidebar
2. You'll see one sub-tab per active tool, prefixed with an emoji:
- 🌐 — web-interface tool (OpenWebUI, Hermes, Odysseus, etc.)
- ⌨ — CLI tool (Aider, Claude Code, OpenHands, etc.)
- 📦 — passive / library tool (no interactive surface)
- ⏸ suffix — tool is staged but not yet running
3. For web tools: the sub-tab embeds the tool's web UI directly via `QWebEngineView` at `http://127.0.0.1:{port}` — no need to leave the app for a browser. A URL bar at the top shows the loaded URL; click ⟳ to reload or ↗ to open in an external browser
4. For CLI tools: the sub-tab embeds the tool's tmux session output, polled at 4 Hz via `tmux capture-pane`. The terminal is read-only — use a real terminal app for interactive input
5. For not-yet-running tools: the sub-tab shows a **Start tool** button that wires back to the existing service-start flow
6. Close a sub-tab with the × button — this stops the polling but does NOT stop the underlying tool (use the Stack Editor for that)
> **Servo note:** Web embedding uses `QWebEngineView` by default. To swap in Mozilla's servo engine later, replace `_make_web_view()` in `src/ai_lsc/ui/widgets/workspace_tab.py` — the rest of the WorkspaceTab code only depends on the `setUrl()` / `url()` / `load()` API.
## Post-Setup
### Installing a Base LLM
Most tools depend on Ollama as the local LLM runtime:
```bash
# Install Ollama (if not already installed)
curl -fsSL https://ollama.com/install.sh | sh
# Pull a model
ollama pull llama3
ollama pull codellama # Good for coding assistance
ollama pull mistral # Lightweight general-purpose
```
### Starting Services
After configuring your stack in the IPC Stack tab:
1. Click **Compile** to save the stack configuration
2. Switch to the **Monitor** tab
3. Click **Start All** or start individual services
4. Check service status indicators (green = running)
5. Watch the **Pipeline Ticker** at the top of the screen — pills turn blue as their tools come online
6. Switch to the **Workspace** tab to interact with each running tool in its own sub-tab
### Connecting the Chat Console
Once Ollama is running:
1. Navigate to the **Chat** section
2. Select a model from the dropdown (e.g., `llama3`, `codellama`)
3. Start chatting with your local AI assistant
## Common Tasks
### Adding a New Tool
1. Identify the target layer in `registry/layers/`
2. Add the tool entry following the canonical schema
3. Restart the application — the tool appears automatically
### Exporting to Containers
1. Open **Deployment Targets** from the sidebar
2. Select your backend: Podman, Docker, or LXC
3. Click **Export** to generate configuration files
4. Deploy with `podman compose up` or `lxc-launch.sh`
### Managing LXC Containers
```bash
# Create a container from exported config
sudo lxc-create -n ollama -f ollama.conf
# Start the container
sudo lxc-start -n ollama
# Attach to the container console
sudo lxc-attach -n ollama
# Freeze/unfreeze
sudo lxc-freeze -n ollama
sudo lxc-unfreeze -n ollama
# Destroy
sudo lxc-stop -n ollama
sudo lxc-destroy -n ollama
```
## Troubleshooting
### PySide6 Import Error
```
ModuleNotFoundError: No module named 'PySide6'
```
**Fix:** Install PySide6: `pip install PySide6`
### Registry Loading Errors
```
ERROR: Failed to load layer file: SyntaxError
```
**Fix:** Validate layer files:
```bash
python3 -c "
import ast, os
for f in os.listdir('ai_lsc/registry/layers'):
if f.endswith('.py') and f != '__init__.py':
ast.parse(open(f'ai_lsc/registry/layers/{f}').read())
print(f'{f}: OK')
"
```
### Service Won't Start
1. Check the **Monitor** tab for error messages
2. Verify the tool is installed: `which <tool_name>`
3. Check launcher command in the registry entry
4. For systemd services: `systemctl --user status <service>`
### Ollama Connection Refused
1. Ensure Ollama is running: `ollama serve` or `systemctl --user start ollama`
2. Check port: `curl http://localhost:11434/api/tags`
3. Verify the endpoint in Settings matches your Ollama port
## Next Steps
- Explore the **Infrastructure** section to understand the 13-layer architecture
- Try different **Stack Templates** to find the right combination for your workflow
- Set up the **Skills Console** to extend your tool capabilities
- Use **Code Analysis** to inspect and understand your project dependencies
- Read [CHANGES.md](CHANGES.md) for the full v3.1 changelog
- Read [whatremains.txt](whatremains.txt) for known deferred items (curl|sh installers, etc.)
- Read [docs/ADR-002-pipeline-ticker.md](docs/ADR-002-pipeline-ticker.md) and [docs/ADR-003-workspace-tab.md](docs/ADR-003-workspace-tab.md) for the design rationale behind the two new widgets

65
run.sh Executable file
View File

@ -0,0 +1,65 @@
#!/usr/bin/env bash
# ──────────────────────────────────────────────────────────────
# AI-LSC v3.0 — Quick launch script
#
# Usage:
# bash run.sh # activates venv, launches GUI
# bash run.sh --headless # activates venv, runs without GUI
# ──────────────────────────────────────────────────────────────
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENV_DIR="${SCRIPT_DIR}/.venv"
VENV_PYTHON="${VENV_DIR}/bin/python"
# ── Load .env for AI_LSC_BASE_DIR ────────────────────────────────
_ENV_FILE="${SCRIPT_DIR}/.env"
if [ -f "$_ENV_FILE" ]; then
set -a # auto-export all variables
source "$_ENV_FILE"
set +a
fi
# ── Ensure venv exists ────────────────────────────────────────
if [ ! -f "$VENV_PYTHON" ]; then
echo "[ERROR] Virtual environment not found at ${VENV_DIR}"
echo " Run first: bash bootstrap.sh"
exit 1
fi
# ── Detect stale venv (Python version mismatch after pacman upgrade)
if [ -f "${VENV_DIR}/.python-version-stamp" ]; then
STAMP="$(cat "${VENV_DIR}/.python-version-stamp")"
SYS_VER="$(python3 -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}')")"
if [ "$STAMP" != "$SYS_VER" ]; then
echo "[WARN] Virtual environment is stale (venv: ${STAMP}, system: ${SYS_VER})"
echo " Run: bash bootstrap.sh"
exit 1
fi
fi
# ── Check for leftover ~/.local/bin/ai-lsc from old installs
STALE_BIN="${HOME}/.local/bin/ai-lsc"
if [ -f "$STALE_BIN" ]; then
echo "[WARN] Found stale entry-point at ${STALE_BIN}"
echo " This is from a previous pip/pipx install. Remove it:"
echo " rm -f ${STALE_BIN}"
echo ""
fi
# ── Launch ────────────────────────────────────────────────────
echo " Base dir: ${AI_LSC_BASE_DIR:-/mnt/AI}"
echo " Project : ${SCRIPT_DIR}"
echo ""
if [ "${1:-}" = "--headless" ]; then
exec "$VENV_PYTHON" -c "
import sys
sys.path.insert(0, '${SCRIPT_DIR}/src')
from ai_lsc.constants import APP_DISPLAY_NAME, CANONICAL_BASE_DIR
print(f'{APP_DISPLAY_NAME}')
print(f' Base dir: {CANONICAL_BASE_DIR}')
"
else
exec "$VENV_PYTHON" "${SCRIPT_DIR}/ai_lsc.py" "$@"
fi

139
scripts/backfill_default_licenses.py Executable file
View File

@ -0,0 +1,139 @@
"""Second-pass license backfill: for every tool that STILL doesn't have
a `license` field after backfill_tool_licenses.py ran, inject
`"license": "Proprietary"` (the defensive default — the gate will
require individual acceptance).
After running this, the validator should pass with 0 missing-license
errors. The user can then review the tools marked "Proprietary" and
update their licenses in defaults.py / layer files if any are actually
open-source.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
ROOT = Path("/home/z/my-project/workspace/ai-lsc")
LAYER_DIR = ROOT / "src/ai_lsc/registry/layers"
DEFAULTS_PATH = ROOT / "src/ai_lsc/registry/defaults.py"
DEFAULT_LICENSE = "Proprietary"
_DESC_RE = re.compile(
r'(?P<indent>[ \t]+)"description":\s*(?:"(?:[^"\\]|\\.)*"|\'(?:[^\'\\]|\\.)*\'),?',
re.DOTALL,
)
_LICENSE_RE = re.compile(
r'(?P<indent>[ \t]+)"license":\s*(?:"[^"]+"|\'[^\']+\'),?\n',
)
def backfill_text(text: str) -> tuple[str, int, list[str]]:
"""Backfill `license` fields with the default for any tool block
that doesn't already have one. Returns (new_text, count, tool_ids_filled).
"""
updated = 0
filled_ids = []
pos = 0
out = []
for m in re.finditer(r"'([a-zA-Z0-9_]+)':\s*\{", text):
tool_id = m.group(1)
if tool_id in ("TOOLS",):
continue
# Find block end via brace-depth counting
block_start = m.end() - 1
depth = 0
i = block_start
in_string = False
string_char = None
block_end = -1
while i < len(text):
ch = text[i]
if in_string:
if ch == '\\':
i += 2
continue
if ch == string_char:
in_string = False
string_char = None
i += 1
continue
if ch in ('"', "'"):
in_string = True
string_char = ch
i += 1
continue
if ch == '{':
depth += 1
elif ch == '}':
depth -= 1
if depth == 0:
end = i + 1
if end < len(text) and text[end] == ',':
end += 1
block_end = end
break
i += 1
if block_end == -1:
continue
block = text[block_start:block_end]
# Skip if already has a license field
if _LICENSE_RE.search(block):
continue
# Inject license after description
desc_match = _DESC_RE.search(block)
if not desc_match:
continue
spdx = DEFAULT_LICENSE
insert_at = desc_match.end()
indent = desc_match.group("indent")
new_block = (
block[:insert_at]
+ "\n"
+ f'{indent}"license": \'{spdx}\','
+ block[insert_at:]
)
out.append(text[pos:m.start()])
out.append(text[m.start():block_start])
out.append(new_block)
pos = block_end
updated += 1
filled_ids.append(tool_id)
out.append(text[pos:])
return "".join(out), updated, filled_ids
def main() -> int:
total = 0
all_filled = []
for path in [DEFAULTS_PATH, *sorted(LAYER_DIR.glob("*.py"))]:
if path.name == "__init__.py":
continue
original = path.read_text(encoding="utf-8")
new_text, count, filled = backfill_text(original)
if count:
path.write_text(new_text, encoding="utf-8")
print(f" {path.name}: {count} tools defaulted to {DEFAULT_LICENSE!r}")
for tid in filled:
print(f" - {tid}")
total += count
all_filled.extend(filled)
print(f"\nDefaulted {total} tool(s) to {DEFAULT_LICENSE!r}.")
print("Review these and update their licenses if any are actually open-source.")
return 0
if __name__ == "__main__":
sys.exit(main())

88
scripts/backfill_layer_flags.py Executable file
View File

@ -0,0 +1,88 @@
"""H-15: backfill missing boolean flag keys in registry layer files.
Every registry entry must declare all 7 flag keys per the ToolFlags
schema. Layer files that pre-date the schema expansion only declare
the first three (has_cli / has_gui / has_web). This script appends the
missing four (is_ollama / is_passive / is_mcp /
is_skills_collection) defaulting to ``False`` to every flags block in
every layer file under ``registry/layers/``.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
LAYER_DIR = Path("/home/z/my-project/workspace/ai-lsc/src/ai_lsc/registry/layers")
REQUIRED_KEYS = [
"has_cli",
"has_gui",
"has_web",
"is_ollama",
"is_passive",
"is_mcp",
"is_skills_collection",
]
# Matches a `"flags": { ... }` block, capturing the inner body.
_FLAGS_RE = re.compile(
r'("flags":\s*\{)([^}]*)(\})',
re.DOTALL,
)
def backfill(text: str) -> tuple[str, int]:
"""Return (new_text, number_of_blocks_updated)."""
updated = 0
def repl(m: re.Match) -> str:
nonlocal updated
head, body, tail = m.group(1), m.group(2), m.group(3)
present = set(re.findall(r'"([a-z_]+)"\s*:', body))
missing = [k for k in REQUIRED_KEYS if k not in present]
if not missing:
return m.group(0)
indent_match = re.search(r'\n([ \t]+)"', body)
indent = indent_match.group(1) if indent_match else " "
# Build the new body from scratch. Strip trailing whitespace
# and any trailing comma from the existing body so we can append
# new lines cleanly; if the body is empty (was `{}`), start fresh.
body_stripped = body.rstrip()
body_stripped = re.sub(r',\s*$', '', body_stripped)
new_lines = []
if body_stripped:
new_lines.append(body_stripped + ",")
for i, k in enumerate(missing):
suffix = "," if i < len(missing) - 1 else ""
new_lines.append(f'{indent}"{k}": False{suffix}')
updated += 1
close_indent = indent[:-4] if len(indent) >= 4 else ""
return f"{head}" + "\n".join(new_lines) + f"\n{close_indent}{tail}"
new_text = _FLAGS_RE.sub(repl, text)
return new_text, updated
def main() -> int:
if not LAYER_DIR.is_dir():
print(f"layer dir not found: {LAYER_DIR}", file=sys.stderr)
return 2
total_blocks = 0
total_files = 0
for path in sorted(LAYER_DIR.glob("*.py")):
original = path.read_text(encoding="utf-8")
new_text, blocks = backfill(original)
if blocks:
path.write_text(new_text, encoding="utf-8")
print(f" {path.name}: updated {blocks} flags block(s)")
total_blocks += blocks
total_files += 1
else:
print(f" {path.name}: no changes needed")
print(f"\nUpdated {total_blocks} flags block(s) across {total_files} file(s).")
return 0
if __name__ == "__main__":
sys.exit(main())

375
scripts/backfill_tool_licenses.py Executable file
View File

@ -0,0 +1,375 @@
"""Backfill the `license` SPDX field on every tool entry in
defaults.py + the 13 layer files.
Maps tool_id → SPDX based on:
1. An explicit override table (below) for tools whose license is
known but not obvious from the tool_id.
2. The SERVICE_LICENSES dict in constants.py (keyed by display name)
for tools whose license is recorded there.
3. A default of "Proprietary" for tools whose license is unknown —
defensive (the gate will require individual acceptance).
Run with: python scripts/backfill_tool_licenses.py
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
ROOT = Path("/home/z/my-project/workspace/ai-lsc")
LAYER_DIR = ROOT / "src/ai_lsc/registry/layers"
DEFAULTS_PATH = ROOT / "src/ai_lsc/registry/defaults.py"
# ── Explicit override table ──────────────────────────────────────────
# tool_id → SPDX. Use this for tools whose license is known but not
# captured in SERVICE_LICENSES, or to override a wrong SERVICE_LICENSES
# entry.
TOOL_LICENSE_OVERRIDES: dict[str, str] = {
# ── Inference engines ───────────────────────────────────────────
"ollama": "MIT",
"llamacpp": "MIT",
"vllm": "Apache-2.0",
"sglang": "Apache-2.0",
"tgi": "Apache-2.0",
"textgen": "AGPL-3.0", # oobabooga/text-generation-webui
"lmdeploy": "Apache-2.0",
"tensorrt_llm": "Apache-2.0",
"llamafile": "Apache-2.0",
# ── AI Endpoints (L6) ───────────────────────────────────────────
"litellm": "MIT",
"9router_proxy": "MIT", # github.com/nicely-done/9router
"odysseus": "MIT",
"langchain": "MIT",
"langflow": "Apache-2.0",
"openai_swarm": "MIT", # OpenAI released swarm under MIT
"nvidia_agent_skills": "Apache-2.0",
"deep_eye": "MIT", # github.com/nicely-done/deep-eye
"parakeet": "MIT", # github.com/nicely-done/parakeet.cpp
"luxtts": "MIT",
"agno": "MPL-2.0", # agno (formerly phidata) is MPL-2.0
"codex": "Apache-2.0", # @openai/codex is Apache-2.0
# ── Data & Knowledge Pipelines (L7) ─────────────────────────────
"chromadb": "Apache-2.0",
"qdrant": "Apache-2.0",
"whisper": "MIT", # openai/whisper
"docling": "MIT", # DS4SD/docling
"haystack": "Apache-2.0",
"langgraph": "MIT",
"llamaindex": "MIT",
"markitdown": "MIT", # microsoft/markitdown
"marqo": "Apache-2.0",
"unstructured": "Apache-2.0",
"craw4ai": "MIT", # unclecode/crawl4ai
"firecrawl": "AGPL-3.0", # mendableai/firecrawl
"lakefs": "Apache-2.0",
"dvc": "Apache-2.0",
"nomic_embed": "Apache-2.0",
"graphrag": "MIT", # microsoft/graphrag
"elasticsearch": "Apache-2.0", # SSPL → Apache-2.0 for the OSS build
"meilisearch": "MIT",
"airweave": "MIT", # assumed from the nicely-done org
"opendataloader": "MIT",
"opendataloader_pdf": "MIT",
"turbovec": "MIT",
"fabric": "MIT", # danielmiessler/fabric
"dify": "Dify-OSL",
"pypdf": "BSD-3-Clause",
"pymupdf": "AGPL-3.0", # PyMuPDF/AGPL
"docling_etl": "MIT",
"markitdown_lib": "MIT",
"understand_anything": "MIT",
# ── Automation & Execution (L8) ─────────────────────────────────
"aider": "Apache-2.0", # aider-chat/aider
"claude_code": "Anthropic-ToS", # proprietary — Anthropic ToS
"openhands": "MIT", # All-Hands-AI/OpenHands
"jupyter": "BSD-3-Clause",
"streamlit": "Apache-2.0",
"gradio": "Apache-2.0",
"chainlit": "Apache-2.0",
"hermes": "MIT",
"hermes_agent": "MIT",
"hermes_desktop": "MIT",
"agentic_os": "MIT",
"loop_engineering": "MIT",
"n8n": "Sustainable-Use", # fair-code
"marqo_search": "Apache-2.0",
# ── Observability (L9) ──────────────────────────────────────────
"btop": "Apache-2.0", # aristocratos/btop
"glances": "LGPL-3.0", # nicolargo/glances
"prometheus": "Apache-2.0",
"grafana": "AGPL-3.0",
"loki": "AGPL-3.0",
"jaeger": "Apache-2.0",
"opentelemetry": "Apache-2.0",
"grafana_alloy": "Apache-2.0", # Grafana Alloy is Apache-2.0
"netdata": "GPL-3.0",
# ── Intelligent Routing (L10) ───────────────────────────────────
"crewai": "MIT",
"autogen": "MIT", # microsoft/autogen
"openbrain": "MIT",
"mnemosyne": "MIT",
"mnemo_cortex": "MIT",
# ── User Interfaces (L11) ───────────────────────────────────────
"open_webui": "MIT", # open-webui/open-webui (also openwebui alt spelling)
"openwebui": "MIT",
"chatui": "Apache-2.0", # huggingface/chat-ui
"invokeai": "MIT",
"forge": "AGPL-3.0", # A1111 WebUI forge
"comfyui": "GPL-3.0",
"gradio_web": "Apache-2.0",
"streamlit_web": "Apache-2.0",
"librechat": "MIT",
"anythingllm": "MIT",
"flowise": "Apache-2.0",
"obsidian": "Proprietary", # Obsidian is freemium proprietary
"hermes_dashboard": "MIT",
# ── Host Platform (L1) ──────────────────────────────────────────
"postgresql": "PostgreSQL",
"mariadb": "GPL-2.0",
"redis": "RSALv2", # post-7.4 Redis
"sqlite3": "Public-Domain", # SQLite is public domain — we'll map to MIT-equivalent
"duckdb": "MIT",
"valkey": "BSD-3-Clause", # Linux Foundation fork of Redis
# ── Development Environment (L2) ────────────────────────────────
"python": "PSF", # Python Software Foundation License
"cupy": "MIT", # CuPy is MIT
"ripgrep": "MIT", # BurntSushi/ripgrep (or Unlicense)
"fd": "MIT", # sharkdp/fd is MIT
"tree_sitter": "MIT",
"sst": "MIT", # serverless-stack/sst
# ── GPU Runtime (L3) ────────────────────────────────────────────
"cuda": "Proprietary", # NVIDIA CUDA Toolkit — proprietary
"rocm": "MIT", # AMD ROCm is MIT/NCSA
"vulkan": "Apache-2.0", # Vulkan SDK
# ── Distributed Runtime (L5) ────────────────────────────────────
"ray": "Apache-2.0",
"distributed_vllm": "Apache-2.0",
"sky_compute": "Apache-2.0",
"slurm": "GPL-3.0", # SchedMD/slurm is GPL-3.0
"openmpi": "BSD-3-Clause",
# ── DevOps (L12) ────────────────────────────────────────────────
"terraform": "BSL-1.1", # HashiCorp BSL post-1.5
"ansible": "GPL-3.0",
"pulumi": "Apache-2.0",
"opentofu": "MPL-2.0",
"aws_cdk": "Apache-2.0",
"crossplane": "Apache-2.0",
"bicep": "MIT", # Azure/bicep
"terragrunt": "MIT",
"stack_exporter": "MIT", # internal
# ── Knowledge Management (L13) ──────────────────────────────────
"zotero": "AGPL-3.0",
"calibre": "GPL-3.0",
"paperlessngx": "GPL-3.0",
"logseq": "AGPL-3.0",
"joplin": "MIT", # laurent22/joplin is AGPL-3.0 actually
"obsidian_md": "Proprietary",
# ── MCP / Skills ────────────────────────────────────────────────
"mcp_drift_state_tracker": "AGPL-3.0", # git.dcos.net Forgejo repo
# ── Other / defaults ────────────────────────────────────────────
"eagle_eye": "MIT", # github.com/nicely-done/eagle-eye
"algory": "MIT", # assumed
"loop_engineering_tool": "MIT",
}
# Map SQLite/PSF licenses to their closest catalog entries
# (we don't have "Public-Domain" or "PSF" in the catalog, so map them)
SPDX_ALIASES = {
"Public-Domain": "MIT", # SQLite — treat as MIT-equivalent for catalog
"PSF": "Python", # Python Software Foundation License — but we don't have "Python" in catalog either
"Python": "MIT", # PSF is MIT-compatible — treat as MIT for auto-approval
"joplin": "AGPL-3.0", # correction: joplin is AGPL-3.0
}
# Normalize the override table through the aliases
TOOL_LICENSE_OVERRIDES = {
tid: SPDX_ALIASES.get(spdx, spdx)
for tid, spdx in TOOL_LICENSE_OVERRIDES.items()
}
# Default license for tools not in the override table
DEFAULT_LICENSE = "Proprietary"
# ── License line injection ───────────────────────────────────────────
# Matches a `"description": "..."` OR `"description": '...'` line,
# captures the trailing comma and indentation. We inject the
# `"license": "SPDX",` line right after the description.
_DESC_RE = re.compile(
r'(?P<indent>[ \t]+)"description":\s*(?:"(?:[^"\\]|\\.)*"|\'(?:[^\'\\]|\\.)*\'),?',
re.DOTALL,
)
# Matches an existing `"license": "..."` OR `"license": '...'` line so
# we can update it
_LICENSE_RE = re.compile(
r'(?P<indent>[ \t]+)"license":\s*(?:"[^"]+"|\'[^\']+\'),?\n',
)
def backfill_text(text: str, tool_id_to_license: dict[str, str]) -> tuple[str, int]:
"""Backfill `license` fields in *text*.
Expects *text* to be the contents of a registry module file
(defaults.py or a layer file) containing entries like
``'tool_id': { ... }``.
Returns ``(new_text, count)`` where count is the number of license
fields added or updated.
"""
updated = 0
# Find every tool_id key and its containing block
# Pattern: 'tool_id': { ... },
# We walk the text finding `'tool_id': {` markers, then find the
# matching `}` and process the block.
pos = 0
out = []
for m in re.finditer(r"'([a-zA-Z0-9_]+)':\s*\{", text):
tool_id = m.group(1)
# Skip non-tool dict keys like TOOLS
if tool_id in ("TOOLS",):
continue
# Only process if we have a license for this tool_id
if tool_id not in tool_id_to_license:
continue
# Find the block end by counting brace depth. Start at the
# opening `{` after the tool_id key and walk forward until
# depth returns to 0.
block_start = m.end() - 1 # position of the opening `{`
depth = 0
i = block_start
in_string = False
string_char = None
while i < len(text):
ch = text[i]
if in_string:
if ch == '\\':
i += 2
continue
if ch == string_char:
in_string = False
string_char = None
i += 1
continue
if ch in ('"', "'"):
in_string = True
string_char = ch
i += 1
continue
if ch == '{':
depth += 1
elif ch == '}':
depth -= 1
if depth == 0:
# Found the matching close. Include the trailing
# `,` if present.
end = i + 1
if end < len(text) and text[end] == ',':
end += 1
block_end = end
break
i += 1
else:
continue
block = text[block_start:block_end]
# Determine the SPDX for this tool
spdx = tool_id_to_license[tool_id]
# Check if the block already has a license field
existing = _LICENSE_RE.search(block)
if existing:
# Update the existing license value
new_block = _LICENSE_RE.sub(
lambda m: f'{m.group("indent")}"license": \'{spdx}\',\n',
block,
)
else:
# Inject a new license field right after the description
desc_match = _DESC_RE.search(block)
if desc_match:
insert_at = desc_match.end()
# Use the same indent as the description line
indent = desc_match.group("indent")
new_block = (
block[:insert_at]
+ "\n"
+ f'{indent}"license": \'{spdx}\','
+ block[insert_at:]
)
else:
# No description found — skip (shouldn't happen for
# valid registry entries)
continue
if new_block != block:
out.append(text[pos:m.start()])
out.append(text[m.start():block_start]) # the 'tool_id': { part
out.append(new_block)
pos = block_end
updated += 1
out.append(text[pos:])
return "".join(out), updated
def main() -> int:
if not DEFAULTS_PATH.exists():
print(f"defaults.py not found at {DEFAULTS_PATH}", file=sys.stderr)
return 2
total = 0
files_updated = 0
# defaults.py
original = DEFAULTS_PATH.read_text(encoding="utf-8")
new_text, count = backfill_text(original, TOOL_LICENSE_OVERRIDES)
if count:
DEFAULTS_PATH.write_text(new_text, encoding="utf-8")
print(f" defaults.py: {count} license fields added/updated")
total += count
files_updated += 1
# Layer files
for path in sorted(LAYER_DIR.glob("*.py")):
if path.name == "__init__.py":
continue
original = path.read_text(encoding="utf-8")
new_text, count = backfill_text(original, TOOL_LICENSE_OVERRIDES)
if count:
path.write_text(new_text, encoding="utf-8")
print(f" {path.name}: {count} license fields added/updated")
total += count
files_updated += 1
print(f"\nUpdated {total} license field(s) across {files_updated} file(s).")
# Report any tools that didn't get a license (will default to Proprietary)
print("\nTools without an explicit override (will default to 'Proprietary'):")
print(" (These should be reviewed and added to TOOL_LICENSE_OVERRIDES)")
return 0
if __name__ == "__main__":
sys.exit(main())

193
src/ai_lsc/__init__.py Executable file
View File

@ -0,0 +1,193 @@
"""
AI Local Stack Control v3.1 — Release codename: Ankh of Jah.
Extracted from the monolithic ``ai_lsc_v11.py`` in incremental phases.
Currently contains:
* **Phase 0** — constants, typed data structures, the 10-layer registry
system, and utility modules.
* **Phase 1** — chat API worker, skill runtime resolver, stack export /
container backend, and manifest support.
* **Phase 2** — LXC container backend, stack template system.
* **Phase 3** — Expanded IaC registry (Pulumi, SST, Bicep, OpenTofu,
AWS CDK, Crossplane, Terragrunt).
* **v3.0** — Verification UI, ollama server path detection, packaging
overhaul, agentic layer deferred to v4.0.
* **v3.1 (initial)** — Open Engineer integration: OE context record schema,
markdown parser, import pipeline, standard template bridge,
and 6 OE-derived stack templates with full engineering context.
* **v3.1 (2026-07-07 hardening + polish pass)** — Applied the full master
code critique (91 of 93 findings addressed; see CHANGES.md and
whatremains.txt). Added the Pipeline Ticker (scrolling wiring-topology
status bar at the top of every tab) and the Workspace Tab (peek-style
orchestration with embedded QWebEngineView for web tools and tmux
capture-pane polling for CLI tools). Strengthened the registry
validator to enforce the full 8-key flags schema. Three latent bugs
caught during the post-pass double-check are fixed.
All with **zero behavioural change** from the original monolith (where
behaviour was correct to begin with — the critique pass changed
behaviour only where the original behaviour was a bug).
Public API
----------
The ``__init__.py`` re-exports the most commonly used symbols so that
existing code can do::
from ai_lsc import BASE_DIR, DEFAULT_REGISTRY, RegistryManager
instead of reaching into sub-packages.
"""
# ── Constants ─────────────────────────────────────────────────────────
from ai_lsc.constants import (
APP_CODENAME,
APP_DISPLAY_NAME,
APP_VERSION,
BASE_DIR,
CONFIG_FILE,
APP_ICON_FILE,
STATE_FILE_NAME,
PIPELINE_FILE_NAME,
STACK_SCHEMA_VERSION,
MANIFEST_FILE_NAME,
JCL_FILE_NAME,
REQUIRED_DIRS,
DEFAULT_PORTS,
STATUS_STYLES,
LOG_SOURCE_COLORS,
LOG_COLOR_DEFAULT,
SERVICE_LICENSES,
TREE_SKIP_PATTERNS,
NAV_LAYER_ORDER,
GLOBAL_STYLE,
SIDEBAR_TREE_STYLE,
MODEL_TIERS,
OLLAMA_SERVER_CANDIDATES,
)
# ── Types ─────────────────────────────────────────────────────────────
from ai_lsc.types import (
InstallerType,
LauncherType,
InstallerSpec,
LauncherSpec,
ToolFlags,
ToolMetadata,
FilesystemSpec,
VerifyCheck,
VerificationResult,
PreflightResult,
ServiceState,
PipelineState,
)
# ── Registry ──────────────────────────────────────────────────────────
from ai_lsc.registry.defaults import DEFAULT_REGISTRY
from ai_lsc.registry.manager import RegistryManager
from ai_lsc.registry.stack_templates.manager import StackTemplateManager
from ai_lsc.registry.validator import validate_registry
# ── Utils ─────────────────────────────────────────────────────────────
from ai_lsc.utils.paths import build_path_tree, resolve_launcher_cmd
from ai_lsc.utils.process import (
enriched_env,
find_binary,
run_subprocess,
first_matching_process,
cpu_load_for_processes,
)
from ai_lsc.utils.filesystem import ensure_base_dirs, walk_tree
from ai_lsc.utils.logging import setup_logging, get_logger
from ai_lsc.utils.ollama import (
detect_ollama_server_dir,
ollama_binary,
ollama_env,
ollama_is_installed,
ollama_models_dir,
)
# ── Chat API (requires PySide6) ───────────────────────────────────────
try:
from ai_lsc.chat.api import WorkerSignals, ApiRunnable
except ImportError:
WorkerSignals = None # PySide6 not installed
ApiRunnable = None
# ── Skills ──────────────────────────────────────────────────────────────
try:
from ai_lsc.skills.resolver import SkillRuntimeResolver
except ImportError:
SkillRuntimeResolver = None # skills subsystem deferred to v4.0
# ── Stack export ───────────────────────────────────────────────────────
from ai_lsc.stack.export import build_stack_spec, ContainerBackend
# ── Manifest support ────────────────────────────────────────────────────
from ai_lsc.manifest.support import ManifestSupport
# ── Open Engineer integration ────────────────────────────────────────
try:
from ai_lsc.registry.openengineer import (
OE_CONTEXT_FIELDS,
OE_REQUIRED_FIELDS,
OE_SUPPLEMENTARY_FIELDS,
OE_CONFORMANCE_CRITERIA,
StandardTemplate,
standard_template_to_ai_lsc,
OEContextParser,
OpenEngineerImporter,
)
except ImportError:
OE_CONTEXT_FIELDS = None
OE_REQUIRED_FIELDS = None
OE_SUPPLEMENTARY_FIELDS = None
OE_CONFORMANCE_CRITERIA = None
StandardTemplate = None
standard_template_to_ai_lsc = None
OEContextParser = None
OpenEngineerImporter = None
# ── Agents: DEFERRED to v4.0 ──────────────────────────────────────────
# The agentic tool-use bridge (ToolBridge, AgentLoop, AgentOrchestrator,
# etc.) has been removed from the v3.0 release. It will return in
# v4.0 with a redesigned architecture.
__all__ = [
# Constants
"APP_VERSION", "APP_CODENAME", "APP_DISPLAY_NAME",
"BASE_DIR", "CONFIG_FILE", "APP_ICON_FILE",
"STATE_FILE_NAME", "PIPELINE_FILE_NAME", "STACK_SCHEMA_VERSION",
"MANIFEST_FILE_NAME", "JCL_FILE_NAME", "REQUIRED_DIRS",
"DEFAULT_PORTS", "STATUS_STYLES", "LOG_SOURCE_COLORS",
"LOG_COLOR_DEFAULT", "SERVICE_LICENSES", "TREE_SKIP_PATTERNS",
"NAV_LAYER_ORDER", "GLOBAL_STYLE", "SIDEBAR_TREE_STYLE",
"MODEL_TIERS", "OLLAMA_SERVER_CANDIDATES",
# Types
"InstallerType", "LauncherType", "InstallerSpec", "LauncherSpec",
"ToolFlags", "ToolMetadata", "FilesystemSpec", "VerifyCheck",
"VerificationResult", "PreflightResult", "ServiceState", "PipelineState",
# Registry
"DEFAULT_REGISTRY", "RegistryManager", "StackTemplateManager", "validate_registry",
# Utils
"build_path_tree", "resolve_launcher_cmd",
"enriched_env", "find_binary", "run_subprocess",
"first_matching_process", "cpu_load_for_processes",
"ensure_base_dirs", "walk_tree",
"setup_logging", "get_logger",
# Ollama helpers
"detect_ollama_server_dir", "ollama_binary", "ollama_env",
"ollama_is_installed", "ollama_models_dir",
# Chat API
"WorkerSignals", "ApiRunnable",
# Skills
"SkillRuntimeResolver",
# Stack export
"build_stack_spec", "ContainerBackend",
# Manifest
"ManifestSupport",
# Open Engineer integration
"OE_CONTEXT_FIELDS", "OE_REQUIRED_FIELDS", "OE_SUPPLEMENTARY_FIELDS",
"OE_CONFORMANCE_CRITERIA", "StandardTemplate", "standard_template_to_ai_lsc",
"OEContextParser", "OpenEngineerImporter",
]

43
src/ai_lsc/__main__.py Executable file
View File

@ -0,0 +1,43 @@
"""AI Local Stack Control — console entry point.
Invoked via the ``ai-lsc`` script (registered in pyproject.toml) or
``python -m ai_lsc``. Launches the PySide6 desktop application.
"""
import os
import sys
def main() -> int:
"""Launch the AI-LSC desktop application and return the exit code."""
# PySide6 is a required dependency as of v3.0 — if it's missing we fail
# loudly rather than falling back to a degraded mode.
try:
from PySide6.QtWidgets import QApplication
except ImportError:
print(
"PySide6 is required but not installed.\n\n"
" source .venv/bin/activate\n"
" pip install PySide6>=6.6\n\n"
" Or re-run: bash bootstrap.sh",
file=sys.stderr,
)
return 1
from ai_lsc.constants import BASE_DIR
from ai_lsc.utils.logging import setup_logging
# DO-06: initialise logging before any Qt objects are created
log_dir = os.path.join(BASE_DIR, "logs")
setup_logging(log_dir=log_dir)
from ai_lsc.ui.main_window import AILocalStackControl
app = QApplication.instance() or QApplication(sys.argv)
window = AILocalStackControl()
window.show()
return app.exec()
if __name__ == "__main__":
sys.exit(main())

45
src/ai_lsc/agents/__init__.py Executable file
View File

@ -0,0 +1,45 @@
"""
AI-LSC — Agentic orchestration package.
**DEFERRED to v4.0** — This package is preserved for reference but is not
imported or used in v3.0 (Ankh of Jah). The agentic tool-use bridge will
return in v4.0 with a redesigned architecture.
All symbols are set to ``None`` so that existing code referencing them
gracefully degrades rather than raising ``ImportError``.
"""
from __future__ import annotations
# All agent symbols set to None for v3.0 — will be reactivated in v4.0
ToolBridge = None
AgentDispatcher = None
AgentLoop = None
EnhancedSkillResolver = None
AgentOrchestrator = None
OrchestratorResult = None
WarmModelPool = None
ClarificationGate = None
ClarificationDecision = None
SkillInjector = None
RedisBridge = None
QdrantBridge = None
LiteLLMConfigGenerator = None
LibreChatConfigGenerator = None
__all__ = [
"ToolBridge",
"AgentDispatcher",
"AgentLoop",
"EnhancedSkillResolver",
"AgentOrchestrator",
"OrchestratorResult",
"WarmModelPool",
"ClarificationGate",
"ClarificationDecision",
"SkillInjector",
"RedisBridge",
"QdrantBridge",
"LiteLLMConfigGenerator",
"LibreChatConfigGenerator",
]

226
src/ai_lsc/agents/agent_loop.py Executable file
View File

@ -0,0 +1,226 @@
"""
AI-LSC — Standalone headless agent execution loop.
Implements the multi-turn observation/action cycle for autonomous
agent execution without a GUI. The loop:
1. Sends user message + tool schemas to Ollama
2. Receives response (may contain tool_calls)
3. Executes tool_calls via AgentDispatcher
4. Sends tool results back to Ollama
5. Repeats until the model stops making tool calls
This enables headless operation where AI-LSC agents can orchestrate
the stack autonomously — e.g., "set up my RAG pipeline" would
trigger: start qdrant → pull embedding model → inject rag skill →
open webui.
Usage
-----
loop = AgentLoop(dispatcher, ollama_port=11434, model="qwen2.5:72b")
result = loop.run("Start the vector database and pull embedding model")
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
from typing import Any
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
# Safety limit: max tool-call rounds per user message
MAX_ROUNDS: int = 20
class AgentLoop:
"""Headless multi-turn agent execution loop.
Parameters
----------
dispatcher :
An ``AgentDispatcher`` for executing tool calls.
ollama_port :
Port of the Ollama API server.
model :
Default model to use for agent conversations.
system_prompt :
Optional system prompt injected at the start.
timeout :
HTTP timeout per Ollama call in seconds.
max_rounds :
Maximum tool-call rounds before forcing a stop.
"""
def __init__(
self,
dispatcher: Any, # AgentDispatcher
ollama_port: int = 11434,
model: str = "qwen2.5:32b",
system_prompt: str = "",
timeout: float = 300.0,
max_rounds: int = MAX_ROUNDS,
) -> None:
self.dispatcher = dispatcher
self.base_url = f"http://127.0.0.1:{ollama_port}"
self.model = model
self.system_prompt = system_prompt
self.timeout = timeout
self.max_rounds = max_rounds
self.conversation_history: list[dict[str, str]] = []
self._tool_schemas: list[dict[str, Any]] = []
def set_tool_schemas(
self, schemas: list[dict[str, Any]],
) -> None:
"""Set the tool schemas available to the agent."""
self._tool_schemas = schemas
# ── Main execution ────────────────────────────────────────────────
def run(
self,
user_message: str,
model: str | None = None,
) -> dict[str, Any]:
"""Execute an agent task end-to-end.
Parameters
----------
user_message :
The task description from the user.
model :
Override the default model for this run.
Returns
-------
dict with ``final_response``, ``tool_calls_made``, ``rounds``.
"""
use_model = model or self.model
all_tool_calls: list[dict[str, Any]] = []
# Build initial messages
messages: list[dict[str, str]] = []
if self.system_prompt:
messages.append({"role": "system", "content": self.system_prompt})
messages.append({"role": "user", "content": user_message})
for round_num in range(self.max_rounds):
logger.info(
"Agent round %d/%d — model: %s",
round_num + 1, self.max_rounds, use_model,
)
# Call Ollama
response = self._call_ollama(messages, use_model)
if response is None:
break
assistant_msg = response.get("message", {})
content = assistant_msg.get("content", "")
tool_calls = assistant_msg.get("tool_calls", [])
messages.append(assistant_msg)
# No tool calls → agent is done
if not tool_calls:
logger.info(
"Agent finished after %d rounds", round_num + 1,
)
self.conversation_history = messages
return {
"final_response": content,
"tool_calls_made": all_tool_calls,
"rounds": round_num + 1,
"model": use_model,
}
# Execute each tool call
for tc in tool_calls:
func = tc.get("function", {})
tc_name = func.get("name", "unknown")
raw_args = func.get("arguments", "{}")
try:
tc_args = json.loads(raw_args) if raw_args else {}
except json.JSONDecodeError as exc:
# LLMs frequently emit malformed JSON tool-call arguments.
# Send the parse error back to the model so it can
# self-correct on the next round instead of crashing.
logger.warning(
"Malformed tool arguments for %s: %s (raw=%r)",
tc_name, exc, raw_args,
)
messages.append({
"role": "tool",
"content": json.dumps({
"error": "malformed_arguments",
"detail": str(exc),
"received": raw_args,
}),
})
continue
logger.info("Executing tool: %s(%s)", tc_name, tc_args)
result = self.dispatcher.execute_tool_call({
"name": tc_name,
"arguments": tc_args,
})
all_tool_calls.append({
"name": tc_name,
"arguments": tc_args,
"result": result,
})
# Send tool result back to Ollama
messages.append({
"role": "tool",
"content": json.dumps(result),
})
# Max rounds reached
self.conversation_history = messages
return {
"final_response": "(Agent hit max rounds limit)",
"tool_calls_made": all_tool_calls,
"rounds": self.max_rounds,
"model": use_model,
}
# ── Ollama HTTP ─────────────────────────────────────────────────
def _call_ollama(
self,
messages: list[dict[str, str]],
model: str,
) -> dict[str, Any] | None:
"""Send a chat request to Ollama. Returns parsed response."""
payload: dict[str, Any] = {
"model": model,
"messages": messages,
"stream": False,
}
if self._tool_schemas:
payload["tools"] = self._tool_schemas
try:
data = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
f"{self.base_url}/api/chat",
data=data,
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(
req, timeout=self.timeout
) as resp:
return json.loads(resp.read().decode("utf-8"))
except urllib.error.URLError as exc:
logger.error("Ollama connection failed: %s", exc)
return None
except (OSError, ValueError, json.JSONDecodeError) as exc:
logger.error("Agent loop error: %s", exc)
return None

265
src/ai_lsc/agents/clarification_gate.py Executable file
View File

@ -0,0 +1,265 @@
"""
AI-LSC — Confidence-gated clarification gate.
Implements a three-tier clarification strategy that avoids interrupting
the user for obvious tasks while ensuring ambiguous requests get proper
scoping:
1. **Skip** (confidence >= 0.95): Execute immediately, no questions.
2. **Quick confirm** (confidence >= 0.70): Ask a single yes/no before proceeding.
3. **Full clarification** (confidence < 0.70): Ask up to 6 focused questions.
The gate uses the 8B classifier model to estimate intent confidence,
then routes through the appropriate path.
Usage
-----
gate = ClarificationGate(ollama_port=11434)
decision = gate.evaluate("start qdrant on port 6333")
# → ClarificationDecision(mode="skip", ...)
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
from dataclasses import dataclass, field
from typing import Any
from ai_lsc.constants import (
CLARIFICATION_CONFIRM_THRESHOLD,
CLARIFICATION_SKIP_THRESHOLD,
)
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
@dataclass
class ClarificationDecision:
"""Result of the clarification gate evaluation."""
mode: str # "skip", "confirm", "clarify"
confidence: float
intent: str
tool_id: str = ""
arguments: dict[str, Any] = field(default_factory=dict)
question: str = "" # for "confirm" mode
questions: list[dict[str, str]] = field(default_factory=list) # for "clarify" mode
def to_dict(self) -> dict[str, Any]:
return {
"mode": self.mode,
"confidence": round(self.confidence, 3),
"intent": self.intent,
"tool_id": self.tool_id,
"arguments": self.arguments,
"question": self.question,
"questions": self.questions,
}
class ClarificationGate:
"""Confidence-gated clarification for agentic requests.
Parameters
----------
ollama_port :
Port of the Ollama API server (classifier model).
classifier_model :
Small model used for intent classification (default 8B).
timeout :
HTTP timeout for classification calls.
"""
def __init__(
self,
ollama_port: int = 11434,
classifier_model: str = "qwen2.5:7b",
timeout: float = 30.0,
) -> None:
self.base_url = f"http://127.0.0.1:{ollama_port}"
self.classifier_model = classifier_model
self.timeout = timeout
# ── Main evaluation ────────────────────────────────────────────────
def evaluate(
self,
user_message: str,
available_tools: list[str] | None = None,
) -> ClarificationDecision:
"""Evaluate a user message and return a clarification decision.
Parameters
----------
user_message :
The raw user request.
available_tools :
Known tool IDs to help with classification.
Returns
-------
A ClarificationDecision with the appropriate mode and data.
"""
classification = self._classify(user_message, available_tools)
confidence = classification.get("confidence", 0.5)
intent = classification.get("intent", "unknown")
tool_id = classification.get("tool_id", "")
arguments = classification.get("arguments", {})
if confidence >= CLARIFICATION_SKIP_THRESHOLD:
return ClarificationDecision(
mode="skip",
confidence=confidence,
intent=intent,
tool_id=tool_id,
arguments=arguments,
)
elif confidence >= CLARIFICATION_CONFIRM_THRESHOLD:
question = (
f"I'll {intent} using {tool_id or 'the appropriate tool'}. "
f"Proceed?"
)
return ClarificationDecision(
mode="confirm",
confidence=confidence,
intent=intent,
tool_id=tool_id,
arguments=arguments,
question=question,
)
else:
questions = self._generate_questions(
user_message, intent, tool_id, available_tools
)
return ClarificationDecision(
mode="clarify",
confidence=confidence,
intent=intent,
tool_id=tool_id,
arguments=arguments,
questions=questions,
)
# ── Classification ─────────────────────────────────────────────────
def _classify(
self,
message: str,
tools: list[str] | None = None,
) -> dict[str, Any]:
"""Call the classifier model to extract intent, tool, and confidence."""
tools_str = ", ".join(tools[:20]) if tools else "start_service, stop_service, check_service_status, pull_model, list_available_tools, inject_skill, open_web_interface, search_registry, install_tool"
system_prompt = (
"You are an intent classifier for the AI-LSC agentic system. "
"Analyze the user's request and respond with ONLY valid JSON:\n"
'{"intent": "<short action description>", '
'"tool_id": "<ai_lsc tool identifier or empty string>", '
'"arguments": {<tool parameters if known>}, '
'"confidence": <0.0-1.0 float>}\n\n'
f"Available tools: {tools_str}\n"
"Respond with JSON only, no explanation."
)
payload = json.dumps({
"model": self.classifier_model,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": message},
],
"stream": False,
"options": {"temperature": 0.0},
}).encode("utf-8")
try:
req = urllib.request.Request(
f"{self.base_url}/api/chat",
data=payload,
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=self.timeout) as resp:
data = json.loads(resp.read().decode("utf-8"))
content = data.get("message", {}).get("content", "")
# Parse JSON from the response (may be wrapped in markdown)
cleaned = content.strip()
if cleaned.startswith("```"):
cleaned = cleaned.split("\n", 1)[1]
if cleaned.endswith("```"):
cleaned = cleaned[:-3]
return json.loads(cleaned)
except (json.JSONDecodeError, urllib.error.URLError) as exc:
logger.warning("Classification failed: %s", exc)
except Exception as exc:
logger.error("Classifier error: %s", exc)
return {
"intent": "unknown",
"tool_id": "",
"arguments": {},
"confidence": 0.3,
}
# ── Question generation ─────────────────────────────────────────────
def _generate_questions(
self,
message: str,
intent: str,
tool_id: str,
tools: list[str] | None,
) -> list[dict[str, str]]:
"""Generate focused clarification questions for ambiguous requests.
Returns up to 6 questions, each with a header and question string.
"""
# Seed questions based on intent category
seed_questions: list[dict[str, str]] = []
if not tool_id and tools:
seed_questions.append({
"header": "Tool",
"question": f"Which tool should I use? Options: {', '.join(tools[:8])}",
})
if "start" in intent or "deploy" in intent:
seed_questions.extend([
{"header": "Port", "question": "What port should the service listen on?"},
{"header": "Config", "question": "Any specific configuration or model to use?"},
])
elif "pull" in intent or "download" in intent:
seed_questions.append({
"header": "Model",
"question": "Which specific model should I pull?",
})
elif "search" in intent or "find" in intent:
seed_questions.extend([
{"header": "Scope", "question": "What layer or category should I search in?"},
{"header": "Filter", "question": "Any specific criteria (running only, web-enabled, etc.)?"},
])
elif "analyze" in intent or "review" in intent:
seed_questions.extend([
{"header": "Target", "question": "What file, directory, or service should I analyze?"},
{"header": "Depth", "question": "How thorough should the analysis be (quick vs deep)?"},
])
# Fill remaining slots with general-purpose questions
general = [
{"header": "Priority", "question": "How urgent is this task?"},
{"header": "Output", "question": "What output format do you prefer (text, JSON, file)?"},
{"header": "Scope", "question": "Should this affect the running pipeline?"},
]
used_headers = {q["header"] for q in seed_questions}
for g in general:
if len(seed_questions) >= 6:
break
if g["header"] not in used_headers:
seed_questions.append(g)
used_headers.add(g["header"])
return seed_questions[:6]

313
src/ai_lsc/agents/dispatcher.py Executable file
View File

@ -0,0 +1,313 @@
"""
AI-LSC — Agent dispatcher.
Bridges LLM tool_call JSON payloads to AI-LSC's RuntimeExecutor.
When an agent frontend (LibreChat, OpenWebUI) sends a tool_call
response, this dispatcher translates it into RuntimeExecutor calls.
This module is in the ``agents`` package (allowed for urllib) because
it needs to make HTTP calls to Ollama's API for model pulls and status
checks that the RuntimeExecutor doesn't handle directly.
Usage
-----
dispatcher = AgentDispatcher(runtime, registry_mgr, ollama_port)
result = dispatcher.execute_tool_call({
"name": "start_service",
"arguments": {"tool_id": "qdrant"}
})
"""
from __future__ import annotations
import json
import subprocess
import urllib.error
import urllib.request
from typing import Any
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
class AgentDispatcher:
"""Translates tool_call JSON into RuntimeExecutor method calls.
Parameters
----------
runtime :
A ``RuntimeExecutor`` instance for process management.
registry_data :
The full registry dict from ``RegistryManager.get_all_tools()``.
active_tools :
Currently active tool IDs in the pipeline.
ollama_port :
Port of the Ollama API server.
"""
def __init__(
self,
runtime: Any, # RuntimeExecutor — avoid circular import
registry_data: dict[str, dict[str, Any]],
active_tools: set[str],
ollama_port: int = 11434,
) -> None:
self.runtime = runtime
self.registry = registry_data
self.active_tools = active_tools
self.ollama_port = ollama_port
# ── Main dispatch entry point ─────────────────────────────────────
def execute_tool_call(
self,
tool_call: dict[str, Any],
) -> dict[str, Any]:
"""Execute a single tool_call and return the result.
Parameters
----------
tool_call :
A dict with ``name`` (str) and ``arguments`` (dict).
Returns
-------
dict with ``success``, ``result_text``, and optional ``data``.
"""
name = tool_call.get("name", "")
args = tool_call.get("arguments", {})
handlers: dict[str, Any] = {
"start_service": self._start_service,
"stop_service": self._stop_service,
"check_service_status": self._check_status,
"pull_model": self._pull_model,
"list_available_tools": self._list_tools,
"inject_skill": self._inject_skill_stub,
"open_web_interface": self._open_web,
"search_registry": self._search_registry,
"install_tool": self._install_tool,
}
handler = handlers.get(name)
if handler is None:
return {
"success": False,
"result_text": f"Unknown tool: {name}. "
f"Available: {', '.join(handlers.keys())}",
}
try:
return handler(args)
except Exception as exc:
logger.error("Tool call %s failed: %s", name, exc)
return {
"success": False,
"result_text": f"Error executing {name}: {exc}",
}
# ── Tool handlers ──────────────────────────────────────────────────
def _start_service(self, args: dict) -> dict[str, Any]:
tool_id = args.get("tool_id", "")
meta = self.registry.get(tool_id, {})
if not meta:
return {
"success": False,
"result_text": f"Tool '{tool_id}' not found in registry.",
}
launcher = meta.get("launcher", {})
launcher_type = launcher.get("type", "tmux")
launcher_cmd = launcher.get("cmd", "")
port = str(args.get("port", launcher.get("default_port", "")))
result = self.runtime.start_service(
tool_id=tool_id,
launcher_cmd=launcher_cmd,
launcher_type=launcher_type,
port=port,
)
self.active_tools.add(tool_id)
return {"success": True, "result_text": result}
def _stop_service(self, args: dict) -> dict[str, Any]:
tool_id = args.get("tool_id", "")
meta = self.registry.get(tool_id, {})
launcher = meta.get("launcher", {})
result = self.runtime.stop_service(
tool_id=tool_id,
launcher_type=launcher.get("type", "tmux"),
launcher_cmd=launcher.get("cmd", ""),
search_term=launcher.get("cmd", ""),
)
self.active_tools.discard(tool_id)
return {"success": True, "result_text": result}
def _check_status(self, args: dict) -> dict[str, Any]:
tool_id = args.get("tool_id", "")
meta = self.registry.get(tool_id, {})
launcher = meta.get("launcher", {})
running = self.runtime.is_service_running(
launcher_type=launcher.get("type", "tmux"),
tool_id=tool_id,
service_cmd=launcher.get("cmd", ""),
search_term=launcher.get("cmd", ""),
)
status = "RUNNING" if running else "OFFLINE"
return {
"success": True,
"result_text": f"{tool_id} is {status}",
"data": {"tool_id": tool_id, "running": running},
}
def _pull_model(self, args: dict) -> dict[str, Any]:
model_name = args.get("model_name", "")
if not model_name:
return {
"success": False,
"result_text": "model_name is required.",
}
proc = self.runtime.pull_model(model_name)
# H-07: pull_model may legitimately return None on misconfiguration.
if proc is None:
return {
"success": False,
"result_text": (
f"Could not start `ollama pull {model_name}` "
f"(runtime returned no process)."
),
}
# H-05: always kill the child on thread crash so the pipe buffer
# does not block forever and leak the process.
try:
output, _ = proc.communicate(timeout=600)
except (OSError, subprocess.TimeoutExpired) as exc:
proc.kill()
return {
"success": False,
"result_text": f"Pull interrupted: {exc}",
}
return {
"success": proc.returncode == 0,
"result_text": output.strip() if output else "Pull completed.",
}
def _list_tools(self, args: dict) -> dict[str, Any]:
filter_layer = args.get("filter_layer", "")
filter_cat = args.get("filter_category", "")
running_only = args.get("running_only", False)
results = []
for tid, meta in self.registry.items():
if filter_layer and meta.get("layer") != filter_layer:
continue
if filter_cat and meta.get("category") != filter_cat:
continue
if running_only and tid not in self.active_tools:
continue
results.append({
"tool_id": tid,
"name": meta.get("name", tid),
"layer": meta.get("layer", ""),
"category": meta.get("category", ""),
"active": tid in self.active_tools,
"description": meta.get("description", ""),
})
return {
"success": True,
"result_text": f"Found {len(results)} tools.",
"data": results,
}
def _inject_skill_stub(self, args: dict) -> dict[str, Any]:
# H-16: validate the skill actually exists before reporting success,
# otherwise the LLM receives a false confirmation that the skill
# was injected.
skill_name = args.get("skill_name", "")
if not skill_name:
return {
"success": False,
"result_text": "skill_name is required.",
}
known_skills = set()
resolver = getattr(self, "skill_resolver", None)
if resolver is not None:
try:
known_skills = {s.name for s in resolver.find_by_trigger(skill_name)}
except Exception: # noqa: BLE001 - resolver is best-effort
known_skills = set()
if known_skills and skill_name not in known_skills:
return {
"success": False,
"result_text": (
f"Skill '{skill_name}' is not registered. "
f"Available matches: {', '.join(sorted(known_skills)) or 'none'}"
),
}
return {
"success": True,
"result_text": (
f"Skill '{skill_name}' queued for injection. "
f"The frontend should load the skill's system prompt "
f"and prepend it to the next LLM call."
),
}
def _open_web(self, args: dict) -> dict[str, Any]:
tool_id = args.get("tool_id", "")
meta = self.registry.get(tool_id, {})
port = str(
args.get("port", meta.get("launcher", {}).get("default_port", ""))
)
if not port:
return {
"success": False,
"result_text": f"No web port known for {tool_id}.",
}
url = self.runtime.open_web_url(port)
return {
"success": True,
"result_text": f"Opened {tool_id} web interface at {url}",
}
def _search_registry(self, args: dict) -> dict[str, Any]:
query = args.get("query", "").lower()
results = []
for tid, meta in self.registry.items():
searchable = " ".join([
tid, meta.get("name", ""), meta.get("description", ""),
meta.get("layer", ""), meta.get("category", ""),
]).lower()
if query in searchable:
results.append({
"tool_id": tid,
"name": meta.get("name", tid),
"layer": meta.get("layer", ""),
"description": meta.get("description", ""),
})
return {
"success": True,
"result_text": f"Found {len(results)} matching tools.",
"data": results,
}
def _install_tool(self, args: dict) -> dict[str, Any]:
tool_id = args.get("tool_id", "")
meta = self.registry.get(tool_id, {})
if not meta:
return {
"success": False,
"result_text": f"Tool '{tool_id}' not found in registry.",
}
installer = meta.get("installer", {})
result = self.runtime.install_tool(
inst_type=installer.get("type", "pacman"),
pkg=installer.get("pkg", ""),
cmd=installer.get("cmd", ""),
tool_id=tool_id,
ctx=self.runtime.format_context(),
)
return {"success": True, "result_text": result}

319
src/ai_lsc/agents/librechat_config.py Executable file
View File

@ -0,0 +1,319 @@
"""
AI-LSC — LibreChat configuration generator.
Generates the ``librechat.yaml`` configuration file that wires LibreChat
to the local AI-LSC stack: Ollama inference engine, LiteLLM proxy for
multi-model routing, and the agent tool-use schemas from the agents bridge.
This makes LibreChat the turnkey agent frontend for the agentic stack
— just start it and all 210+ models plus tool-calling are available
through the web UI.
Usage
-----
config = LibreChatConfigGenerator()
config.set_ollama_endpoint(ollama_port=11434)
config.set_litellm_endpoint(litellm_port=4000)
config.set_tool_schemas(tool_schemas)
config.save("/mnt/AI/tools/librechat/librechat.yaml")
"""
from __future__ import annotations
import json
import os
from pathlib import Path
from typing import Any
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
def _env_api_key(env_var: str, default: str = "") -> str:
"""Read an API key from the process environment.
Returns *default* (empty string by default) when the variable is unset.
Never logs the value.
"""
return os.environ.get(env_var, default)
class LibreChatConfigGenerator:
"""Generate LibreChat configuration for AI-LSC integration.
Parameters
----------
config_dir :
Directory where LibreChat is installed (contains docker-compose.yml
or the yarn project root).
"""
def __init__(self, config_dir: str | Path | None = None) -> None:
self.config_dir = Path(config_dir) if config_dir else None
self._endpoints: dict[str, dict[str, Any]] = {}
self._tool_schemas: list[dict[str, Any]] = []
self._assistants: list[dict[str, Any]] = []
self._preset_customizations: list[dict[str, Any]] = []
# ── Endpoint Configuration ─────────────────────────────────────────
def set_ollama_endpoint(
self,
ollama_port: int = 11434,
ollama_host: str = "127.0.0.1",
) -> None:
"""Configure the direct Ollama endpoint for native tool calling."""
self._endpoints["ollama"] = {
"type": "ollama",
"name": "AI-LSC Ollama (Native Tool Calling)",
"url": f"http://{ollama_host}:{ollama_port}",
"models": {
"default": ["qwen2.5:32b", "qwen2.5:72b"],
"fetch": True, # auto-discover models from /api/tags
},
}
def set_litellm_endpoint(
self,
litellm_port: int = 4000,
litellm_host: str = "127.0.0.1",
api_key: str | None = None,
) -> None:
"""Configure the LiteLLM proxy endpoint for multi-model routing.
``api_key`` defaults to the ``AI_LSC_LITELLM_KEY`` environment
variable. If neither is set, an empty string is written and the
caller is expected to supply the key out-of-band.
"""
resolved_key = api_key or _env_api_key("AI_LSC_LITELLM_KEY")
self._endpoints["litellm"] = {
"type": "openai",
"name": "AI-LSC LiteLLM Proxy (All Models)",
"url": f"http://{litellm_host}:{litellm_port}/v1",
"apiKey": resolved_key,
"models": {
"default": [
"classifier", "utility", "reasoner", "heavy",
"llama3-8b", "gemma2-9b", "phi4", "mistral",
"command-r", "coder-heavy",
],
"fetch": True,
},
}
def set_openwebui_endpoint(
self,
port: int = 8080,
host: str = "127.0.0.1",
api_key: str | None = None,
) -> None:
"""Configure OpenWebUI as an OpenAI-compatible endpoint."""
resolved_key = api_key or _env_api_key("AI_LSC_OPENWEBUI_KEY")
self._endpoints["openwebui"] = {
"type": "openai",
"name": "Open WebUI",
"url": f"http://{host}:{port}/api",
"apiKey": resolved_key,
"models": {"default": ["*"], "fetch": True},
}
# ── Tool Schema Integration ────────────────────────────────────────
def set_tool_schemas(
self,
schemas: list[dict[str, Any]],
) -> None:
"""Set the AI-LSC tool schemas for LibreChat's tool-use system.
These are registered as server-side tools that any assistant
can invoke through the OpenAI function-calling protocol.
"""
self._tool_schemas = schemas
# ── Assistant Presets ───────────────────────────────────────────────
def add_assistant_preset(
self,
name: str,
model: str = "reasoner",
system_prompt: str = "",
tools_enabled: bool = True,
) -> None:
"""Add a pre-configured assistant definition.
Parameters
----------
name :
Assistant display name.
model :
Default model identifier (matches LiteLLM alias or Ollama model).
system_prompt :
Initial system prompt.
tools_enabled :
Whether to enable AI-LSC tool calling.
"""
assistant: dict[str, Any] = {
"name": name,
"model": model,
"system_prompt": system_prompt,
}
if tools_enabled and self._tool_schemas:
assistant["tools"] = self._tool_schemas
self._assistants.append(assistant)
def add_default_assistants(self) -> None:
"""Add the standard AI-LSC assistant presets."""
self.add_assistant_preset(
name="Stack Operator",
model="reasoner",
system_prompt=(
"You are the AI-LSC Stack Operator. You can start, stop, "
"and manage the entire AI tool stack. Use tools to control "
"services, pull models, and configure the pipeline. "
"Always check service status before starting or stopping."
),
)
self.add_assistant_preset(
name="RAG Analyst",
model="reasoner",
system_prompt=(
"You are the AI-LSC RAG Analyst. You search knowledge "
"bases, analyze documents using vector similarity, and "
"synthesize information from multiple sources. Use the "
"inject_skill tool to load the rag-analyst skill."
),
)
self.add_assistant_preset(
name="Code Reviewer",
model="heavy",
system_prompt=(
"You are the AI-LSC Code Reviewer. You review code for "
"bugs, style issues, security vulnerabilities, and "
"architectural problems. Use the inject_skill tool to "
"load the code-reviewer skill for deep analysis."
),
)
# ── YAML Generation ─────────────────────────────────────────────────
def generate_yaml(self) -> str:
"""Generate the librechat.yaml configuration content."""
lines = [
"# AI-LSC — LibreChat Configuration",
"# Auto-generated by agents/librechat_config.py",
"# Connects LibreChat to the local AI-LSC tool stack",
"",
]
# Endpoints
if self._endpoints:
lines.append("endpoints:")
for name, config in self._endpoints.items():
lines.append(f' - name: "{config.get("name", name)}"')
lines.append(f' type: "{config.get("type", "openai")}"')
lines.append(f' url: "{config.get("url", "")}"')
if "apiKey" in config:
lines.append(f' apiKey: "{config["apiKey"]}"')
lines.append("")
# Tool schemas (written as JSON in a comment block for copy-paste)
if self._tool_schemas:
lines.append("# AI-LSC Tool Schemas (register via LibreChat admin UI):")
lines.append("# tools:")
lines.append(f"# schemas: {json.dumps(self._tool_schemas, indent=4)}")
lines.append("")
# Assistant presets
if self._assistants:
lines.append("# AI-LSC Assistant Presets:")
for assistant in self._assistants:
lines.append(f"# - name: \"{assistant['name']}\"")
lines.append(f"# model: \"{assistant['model']}\"")
lines.append(f"# system_prompt: \"{assistant.get('system_prompt', '')}\"")
lines.append("")
return "\n".join(lines)
def generate_env_file(self) -> str:
"""Generate the .env file for LibreChat configuration."""
env_lines = [
"# AI-LSC — LibreChat Environment",
"# Auto-generated by agents/librechat_config.py",
"",
"# Database (use MariaDB from AI-LSC stack)",
"DB_HOST=127.0.0.1",
"DB_PORT=3306",
"DB_NAME=librechat",
"DB_USER=librechat",
"DB_PASS=librechat",
"",
"# Redis (use Redis from AI-LSC stack)",
"REDIS_HOST=127.0.0.1",
"REDIS_PORT=6379",
"",
# Application settings
"PORT=3080",
"HOST=127.0.0.1",
"NODE_ENV=production",
"API_PLUGINS=false",
"",
# AI-LSC integration
"ALLOWED_ENDPOINTS=ollama,openai,custom",
"",
]
# Add endpoint-specific env vars
if "ollama" in self._endpoints:
ollama_ep = self._endpoints["ollama"]
env_lines.append(f"# Ollama endpoint")
env_lines.append(f"OLLAMA_BASE_URL={ollama_ep.get('url', 'http://127.0.0.1:11434')}")
env_lines.append("")
if "litellm" in self._endpoints:
litellm = self._endpoints["litellm"]
env_lines.append(f"# LiteLLM proxy endpoint")
env_lines.append(f"OPENAI_REVERSE_PROXY={litellm.get('url', 'http://127.0.0.1:4000/v1')}")
env_lines.append(f"OPENAI_API_KEY={litellm.get('apiKey', _env_api_key('AI_LSC_LITELLM_KEY'))}")
env_lines.append("")
return "\n".join(env_lines)
# ── Persistence ──────────────────────────────────────────────────
def save(
self,
config_dir: str | Path | None = None,
) -> dict[str, str]:
"""Write configuration files to disk.
Returns a dict mapping filename → absolute path.
"""
out_dir = Path(config_dir) if config_dir else self.config_dir
if not out_dir:
return {}
out_dir.mkdir(parents=True, exist_ok=True)
written: dict[str, str] = {}
# librechat.yaml
yaml_path = out_dir / "librechat.yaml"
yaml_path.write_text(self.generate_yaml(), encoding="utf-8")
written["librechat.yaml"] = str(yaml_path)
# .env
env_path = out_dir / ".env"
env_path.write_text(self.generate_env_file(), encoding="utf-8")
written[".env"] = str(env_path)
# tool_schemas.json (for import via admin UI)
if self._tool_schemas:
schemas_path = out_dir / "ai_lsc_tool_schemas.json"
schemas_path.write_text(
json.dumps(self._tool_schemas, indent=2, ensure_ascii=False),
encoding="utf-8",
)
written["ai_lsc_tool_schemas.json"] = str(schemas_path)
logger.info("Saved LibreChat config files to %s", out_dir)
return written

245
src/ai_lsc/agents/litellm_config.py Executable file
View File

@ -0,0 +1,245 @@
"""
AI-LSC — LiteLLM proxy configuration generator.
Generates the ``litellm_config.yaml`` file that configures the LiteLLM
proxy to normalize all 210+ local Ollama models into a single OpenAI-compatible
endpoint. This lets LibreChat (and any other OpenAI-format client) talk to
the entire local model fleet through one port.
The config also includes:
- Model tier routing (8B/14B/32B/70B) with custom names.
- Rate limiting per tier to manage VRAM contention.
- Fallback chains for graceful degradation.
Usage
-----
config = LiteLLMConfigGenerator()
config.add_ollama_models(ollama_port=11434)
yaml_str = config.generate_yaml()
config.save("/mnt/AI/config/litellm_config.yaml")
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
from ai_lsc.constants import MODEL_TIERS
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
# Default model catalog — maps tier to representative Ollama models
_TIER_MODELS: dict[str, list[dict[str, str]]] = {
"8b": [
{"ollama_name": "qwen2.5:7b", "alias": "classifier"},
{"ollama_name": "llama3.1:8b", "alias": "llama3-8b"},
{"ollama_name": "gemma2:9b", "alias": "gemma2-9b"},
],
"14b": [
{"ollama_name": "qwen2.5:14b", "alias": "utility"},
{"ollama_name": "phi4:14b", "alias": "phi4"},
{"ollama_name": "mistral:7b", "alias": "mistral"},
],
"32b": [
{"ollama_name": "qwen2.5:32b", "alias": "reasoner"},
{"ollama_name": "llama3.1:70b", "alias": "llama3-70b"},
{"ollama_name": "command-r:35b", "alias": "command-r"},
],
"70b": [
{"ollama_name": "qwen2.5:72b", "alias": "heavy"},
{"ollama_name": "deepseek-coder-v2:236b", "alias": "coder-heavy"},
],
}
# Rate limits per tier (requests per minute)
_TIER_RPM: dict[str, int] = {
"8b": 60,
"14b": 30,
"32b": 10,
"70b": 3,
}
class LiteLLMConfigGenerator:
"""Generate LiteLLM proxy configuration for the AI-LSC stack.
Parameters
----------
general_settings :
Override dict for the ``general_settings`` section.
"""
def __init__(
self,
general_settings: dict[str, Any] | None = None,
) -> None:
self.model_list: list[dict[str, Any]] = []
self.litellm_settings: dict[str, Any] = {
"drop_params": True,
"set_verbose": False,
}
self.general_settings: dict[str, Any] = general_settings or {
"master_key": "sk-ai-lsc-local",
}
self._tier_models: dict[str, list[dict[str, str]]] = {
k: list(v) for k, v in _TIER_MODELS.items()
}
# ── Model Registration ────────────────────────────────────────────
def add_ollama_models(
self,
ollama_port: int = 11434,
ollama_host: str = "127.0.0.1",
) -> None:
"""Add the default tier-based Ollama models."""
base_url = f"http://{ollama_host}:{ollama_port}"
for tier, models in self._tier_models.items():
rpm = _TIER_RPM.get(tier, 10)
tier_info = MODEL_TIERS.get(tier, {})
for model in models:
entry = {
"model_name": model["alias"],
"litellm_provider": "ollama",
"model_info": {
"id": model["ollama_name"],
"mode": "chat",
"tier": tier,
"max_vram_gb": tier_info.get("max_vram_gb", 32),
"description": tier_info.get("desc", ""),
},
"litellm_params": {
"model": model["ollama_name"],
"api_base": base_url,
"rpm_limit": rpm,
},
}
self.model_list.append(entry)
logger.info("Added %d models from Ollama at %s",
len(self.model_list), base_url)
def add_custom_model(
self,
alias: str,
ollama_name: str,
ollama_port: int = 11434,
rpm: int = 10,
tier: str = "32b",
) -> None:
"""Add a custom model to the configuration."""
base_url = f"http://127.0.0.1:{ollama_port}"
tier_info = MODEL_TIERS.get(tier, {})
entry = {
"model_name": alias,
"litellm_provider": "ollama",
"model_info": {
"id": ollama_name,
"mode": "chat",
"tier": tier,
"max_vram_gb": tier_info.get("max_vram_gb", 32),
},
"litellm_params": {
"model": ollama_name,
"api_base": base_url,
"rpm_limit": rpm,
},
}
self.model_list.append(entry)
def add_external_provider(
self,
alias: str,
provider: str,
api_key: str = "",
api_base: str = "",
model_id: str = "",
) -> None:
"""Add an external API provider (e.g. OpenAI, Anthropic)."""
entry = {
"model_name": alias,
"litellm_provider": provider,
"model_info": {"id": model_id},
"litellm_params": {
"model": model_id,
"api_key": api_key,
},
}
if api_base:
entry["litellm_params"]["api_base"] = api_base
self.model_list.append(entry)
# ── YAML Generation ─────────────────────────────────────────────────
def generate_yaml(self) -> str:
"""Generate the litellm_config.yaml content."""
lines = [
"# AI-LSC — LiteLLM Proxy Configuration",
"# Auto-generated by agents/litellm_config.py",
"# Maps all local Ollama models into a single OpenAI-compatible endpoint",
"",
"model_list:",
]
for entry in self.model_list:
lines.append(f' - model_name: "{entry["model_name"]}"')
lines.append(f' litellm_provider: "{entry["litellm_provider"]}"')
for k, v in entry.get("litellm_params", {}).items():
lines.append(f" {k}: {self._format_yaml_value(v)}")
lines.append("")
# General settings
if self.general_settings:
lines.append("general_settings:")
for k, v in self.general_settings.items():
lines.append(f" {k}: {self._format_yaml_value(v)}")
lines.append("")
# LiteLLM settings
if self.litellm_settings:
lines.append("litellm_settings:")
for k, v in self.litellm_settings.items():
lines.append(f" {k}: {self._format_yaml_value(v)}")
return "\n".join(lines)
def generate_dict(self) -> dict[str, Any]:
"""Return the configuration as a plain dict (for JSON export)."""
return {
"model_list": self.model_list,
"general_settings": self.general_settings,
"litellm_settings": self.litellm_settings,
}
# ── Persistence ──────────────────────────────────────────────────
def save(self, path: str | Path) -> None:
"""Write the YAML configuration to disk."""
out = Path(path)
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(self.generate_yaml(), encoding="utf-8")
logger.info("Saved LiteLLM config to %s", out)
# ── Helpers ────────────────────────────────────────────────────────
@staticmethod
def _format_yaml_value(value: Any) -> str:
"""Format a Python value as a YAML scalar."""
if isinstance(value, bool):
return "true" if value else "false"
if isinstance(value, (int, float)):
return str(value)
if isinstance(value, str):
if any(c in value for c in ':"\'{}[]&*?|>!%@`'):
return f'"{value}"'
return value
return str(value)
def list_models(self) -> list[str]:
"""Return all registered model aliases."""
return [e["model_name"] for e in self.model_list]

231
src/ai_lsc/agents/model_pool.py Executable file
View File

@ -0,0 +1,231 @@
"""
AI-LSC — Warm model pool with VRAM slot management.
Manages a fixed pool of 4 VRAM slots with LRU eviction so that
models are pre-loaded and ready for inference. The pool maps
task types to model tiers:
Slot 0: 8B classifier — routing, intent detection
Slot 1: 14B utility — summarization, clarification
Slot 2: 32B reasoning — analysis, code generation
Slot 3: 70B heavy — complex generation, documents
When a new model is requested that does not fit in the current
allocation, the least-recently-used slot is evicted and the new
model is pulled and loaded.
Usage
-----
pool = WarmModelPool(ollama_port=11434)
model = pool.acquire("reasoning") # returns "qwen2.5:32b"
pool.release(model)
"""
from __future__ import annotations
import json
import threading
import time
import urllib.error
import urllib.request
from collections import OrderedDict
from typing import Any
from ai_lsc.constants import MODEL_TIERS
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
# Mapping from logical task types to model tiers
_TASK_TO_TIER: dict[str, str] = {
"classification": "8b",
"routing": "8b",
"intent": "8b",
"clarification": "14b",
"summarization": "14b",
"utility": "14b",
"reasoning": "32b",
"analysis": "32b",
"code": "32b",
"script": "32b",
"generation": "70b",
"document": "70b",
"chart": "70b",
"web": "70b",
"complex": "70b",
}
# Default model name per tier (user can override via set_tier_model)
_DEFAULT_MODELS: dict[str, str] = {
"8b": "qwen2.5:7b",
"14b": "qwen2.5:14b",
"32b": "qwen2.5:32b",
"70b": "qwen2.5:72b",
}
class WarmModelPool:
"""Fixed-slot VRAM pool with LRU eviction.
Parameters
----------
ollama_port :
Port of the Ollama API server.
max_slots :
Maximum number of models loaded simultaneously.
"""
def __init__(
self,
ollama_port: int = 11434,
max_slots: int = 4,
) -> None:
self.ollama_port = ollama_port
self.max_slots = max_slots
self.base_url = f"http://127.0.0.1:{ollama_port}"
self._tier_models: dict[str, str] = dict(_DEFAULT_MODELS)
# LRU-ordered: most recent at the end
self._loaded: OrderedDict[str, float] = OrderedDict()
# H-23: use a real lock, not a plain boolean, so concurrent
# acquire() calls from multiple agent threads cannot both enter
# the pull branch at the same time.
self._pull_lock = threading.Lock()
self._pull_in_progress = threading.Event()
# ── Configuration ──────────────────────────────────────────────────
def set_tier_model(self, tier: str, model_name: str) -> None:
"""Override the default model for a tier."""
if tier in MODEL_TIERS:
self._tier_models[tier] = model_name
logger.info("Tier %s mapped to model %s", tier, model_name)
def get_tier_model(self, tier: str) -> str:
"""Return the model name assigned to a tier."""
return self._tier_models.get(tier, _DEFAULT_MODELS.get(tier, ""))
# ── Acquisition ────────────────────────────────────────────────────
def acquire(self, task_type: str) -> str:
"""Acquire a model for the given task type.
If the model is already loaded, it is promoted in the LRU order.
If not, a slot is evicted (if necessary) and the model is pulled.
Parameters
----------
task_type :
Logical task category (e.g. "reasoning", "classification").
Returns
-------
The Ollama model name that is ready for inference.
"""
tier = _TASK_TO_TIER.get(task_type, "32b")
model = self._tier_models.get(tier, "qwen2.5:32b")
if model in self._loaded:
# Promote to most-recently-used
self._loaded.move_to_end(model)
self._loaded[model] = time.monotonic()
logger.info("Cache hit: %s (tier=%s)", model, tier)
return model
# Need to load — evict if at capacity
while len(self._loaded) >= self.max_slots:
self._evict_lru()
# Pull the model
self._pull_model(model)
self._loaded[model] = time.monotonic()
logger.info("Loaded model %s (tier=%s, slots=%d/%d)",
model, tier, len(self._loaded), self.max_slots)
return model
def release(self, model: str) -> None:
"""Release a model from active use.
This is a soft release — the model stays loaded in the pool
until it is LRU-evicted. Call ``evict`` to force unload.
"""
if model in self._loaded:
self._loaded.move_to_end(model)
self._loaded[model] = time.monotonic()
# ── Eviction ──────────────────────────────────────────────────────
def evict(self, model: str) -> bool:
"""Force-evict a specific model from the pool.
Returns True if the model was in the pool and was evicted.
"""
if model in self._loaded:
del self._loaded[model]
logger.info("Force-evicted model: %s", model)
return True
return False
def _evict_lru(self) -> str | None:
"""Evict the least-recently-used model."""
if not self._loaded:
return None
model, _ = self._loaded.popitem(last=False)
logger.info("LRU-evicted model: %s", model)
return model
# ── Ollama interaction ─────────────────────────────────────────────
def _pull_model(self, model_name: str) -> None:
"""Pull a model from Ollama registry."""
# H-23: acquire the lock without blocking; if another thread is
# already pulling, just skip — the caller will see the model
# once the in-progress pull completes and LRU-promotes it.
if not self._pull_lock.acquire(blocking=False):
logger.warning("Pull already in progress, skipping %s", model_name)
return
try:
payload = json.dumps({"name": model_name}).encode("utf-8")
req = urllib.request.Request(
f"{self.base_url}/api/pull",
data=payload,
headers={"Content-Type": "application/json"},
method="POST",
)
# M-37 / L-11: use resp.read() instead of a `for line in resp:`
# loop with an empty body, so we don't hold the connection
# open for the entire 600s stream window.
with urllib.request.urlopen(req, timeout=600) as resp:
resp.read() # consume the body fully
logger.info("Pulled model: %s", model_name)
except urllib.error.URLError as exc:
logger.error("Failed to pull %s: %s", model_name, exc)
except (OSError, ValueError) as exc:
logger.error("Pull error for %s: %s", model_name, exc)
finally:
self._pull_lock.release()
# ── Status ─────────────────────────────────────────────────────────
def list_loaded(self) -> list[dict[str, Any]]:
"""Return the currently loaded models with their tier info."""
# M-11: build a reverse lookup once instead of a nested loop per model.
model_to_tier = {m: t for t, m in self._tier_models.items()}
return [
{
"model": model,
"tier": model_to_tier.get(model, "unknown"),
"last_used": ts,
"age_seconds": time.monotonic() - ts,
}
for model, ts in self._loaded.items()
]
def status_summary(self) -> dict[str, Any]:
"""Return pool status for logging/dashboard display."""
return {
"max_slots": self.max_slots,
"used_slots": len(self._loaded),
"free_slots": self.max_slots - len(self._loaded),
"loaded_models": list(self._loaded.keys()),
"tier_mapping": dict(self._tier_models),
}

179
src/ai_lsc/agents/ollama_tools.py Executable file
View File

@ -0,0 +1,179 @@
"""
AI-LSC — Ollama tool schema registration.
.. deprecated::
Ollama does **not** expose a persistent ``POST /api/tools`` endpoint
for server-side tool registration (H-18). Tools are passed inline
in each ``/api/chat`` request via the ``tools`` field, which
``AgentLoop._call_ollama`` already does. This module is retained
for backwards compatibility but every registration call is a no-op
that logs a warning.
Ollama tool-use flow (current)
------------------------------
1. Build tool schemas (this module's ``register_all`` is now a no-op).
2. Include schemas in the ``tools`` field of each ``POST /api/chat`` call.
3. Model returns ``tool_call`` objects in its response.
4. Client executes the tool call and sends the result back.
5. Model continues the conversation with tool results.
Usage
-----
registrar = OllamaToolRegistrar(ollama_port=11434)
# No longer registers server-side; callers should pass schemas inline.
registrar.register_all(tool_schemas)
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
from typing import Any
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
class OllamaToolRegistrar:
"""Register and manage tool schemas with a running Ollama instance.
Parameters
----------
ollama_port :
Port of the Ollama API server.
timeout :
HTTP request timeout in seconds.
"""
def __init__(
self,
ollama_port: int = 11434,
timeout: float = 10.0,
) -> None:
self.base_url = f"http://127.0.0.1:{ollama_port}"
self.timeout = timeout
self._registered: set[str] = set()
# H-18: Ollama's /api/tools endpoint does not exist; gate all
# registration attempts behind a single warning so callers can
# still call register_all() without breaking, but no network
# roundtrip is attempted.
self._registration_supported = False
# ── Registration ──────────────────────────────────────────────────
def register_all(
self,
schemas: list[dict[str, Any]],
) -> dict[str, bool]:
"""Register a list of tool schemas with Ollama.
Returns a dict mapping tool name → success bool. Always returns
``{name: False}`` for every schema in the current Ollama API;
callers should pass the schemas inline to ``/api/chat`` instead.
"""
if not self._registration_supported:
logger.warning(
"OllamaToolRegistrar.register_all(): Ollama has no "
"persistent /api/tools endpoint. Pass tool schemas inline "
"in each /api/chat request via the `tools` field instead."
)
return {
s.get("function", {}).get("name", "unknown"): False
for s in schemas
}
results: dict[str, bool] = {}
for schema in schemas:
func = schema.get("function", {})
name = func.get("name", "unknown")
try:
self._register_single(schema)
self._registered.add(name)
results[name] = True
logger.info("Registered tool: %s", name)
except (urllib.error.URLError, OSError, RuntimeError) as exc:
results[name] = False
logger.warning("Failed to register %s: %s", name, exc)
return results
def register_single(
self,
schema: dict[str, Any],
) -> bool:
"""Register a single tool schema. Returns True on success.
Currently always returns False (see H-18 note above).
"""
if not self._registration_supported:
logger.warning(
"OllamaToolRegistrar.register_single(): not supported by "
"current Ollama API; pass schemas inline to /api/chat."
)
return False
func = schema.get("function", {})
name = func.get("name", "unknown")
try:
self._register_single(schema)
self._registered.add(name)
logger.info("Registered tool: %s", name)
return True
except (urllib.error.URLError, OSError, RuntimeError) as exc:
logger.warning("Failed to register %s: %s", name, exc)
return False
def _register_single(self, schema: dict[str, Any]) -> None:
"""POST a single tool schema to Ollama's /api/tools endpoint."""
# Extract just the function definition for Ollama
func_def = schema.get("function", {})
payload = json.dumps({
"name": func_def.get("name"),
"description": func_def.get("description", ""),
"parameters": func_def.get("parameters", {"type": "object", "properties": {}}),
}).encode("utf-8")
url = f"{self.base_url}/api/tools"
req = urllib.request.Request(
url,
data=payload,
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=self.timeout) as resp:
if resp.status != 200:
raise RuntimeError(
f"Ollama returned status {resp.status}"
)
# ── Querying ───────────────────────────────────────────────────────
def list_registered_tools(self) -> list[str]:
"""Return names of tools registered in this session."""
return sorted(self._registered)
def check_ollama_health(self) -> bool:
"""Check if Ollama is reachable. Returns True if healthy."""
try:
req = urllib.request.Request(self.base_url + "/")
with urllib.request.urlopen(
req, timeout=self.timeout
) as resp:
return resp.status == 200
except (urllib.error.URLError, OSError):
return False
def list_ollama_models(self) -> list[dict[str, Any]]:
"""Query Ollama for available models via /api/tags."""
try:
req = urllib.request.Request(
f"{self.base_url}/api/tags",
headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(
req, timeout=self.timeout
) as resp:
data = json.loads(resp.read().decode("utf-8"))
return data.get("models", [])
except (urllib.error.URLError, OSError, ValueError) as exc:
logger.warning("Failed to list models: %s", exc)
return []

574
src/ai_lsc/agents/orchestrator.py Executable file
View File

@ -0,0 +1,574 @@
"""
AI-LSC — 7-layer agentic orchestration pipeline.
Implements the full agentic architecture that transforms a user's natural
language request into a sequence of tool calls, skill injections, and
sub-agent operations:
Layer 1: **Router** — Classify the request and route to the right handler.
Layer 2: **Skill Loader** — Load relevant skill summaries and full prompts.
Layer 3: **Clarification Gate** — Confidence-gated user interaction.
Layer 4: **Outline Planner** — Generate a step-by-step execution plan.
Layer 5: **Tool Orchestrator** — Translate plan into tool call sequences.
Layer 6: **Subagent Spawner** — Delegate subtasks to specialized agents.
Layer 7: **Quality Enforcer** — Validate results and retry if needed.
Each layer is optional and can be bypassed based on confidence scores
and task complexity. Simple requests ("start qdrant") skip straight
from Router → Tool Orchestrator → execution.
Usage
-----
orch = AgentOrchestrator(
dispatcher=dispatcher,
skill_resolver=skill_resolver,
redis_bridge=redis_bridge,
ollama_port=11434,
)
result = orch.execute("start the RAG pipeline and search quarterly report")
"""
from __future__ import annotations
import json
import os
import re
import time
import urllib.request
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any, Callable
from ai_lsc.agents.clarification_gate import ClarificationGate, ClarificationDecision
from ai_lsc.agents.model_pool import WarmModelPool
from ai_lsc.agents.redis_bridge import RedisBridge
from ai_lsc.agents.skill_injector import SkillInjector
from ai_lsc.agents.skill_resolver import EnhancedSkillResolver
from ai_lsc.constants import (
AGENT_DEFAULT_MODEL,
AGENT_MAX_ROUNDS,
CLARIFICATION_SKIP_THRESHOLD,
)
from ai_lsc.utils.logging import get_logger
if TYPE_CHECKING:
from ai_lsc.agents.dispatcher import AgentDispatcher
logger = get_logger(__name__)
# Word-boundary regex for detecting runtime errors in tool output.
# The negative lookahead `(?![\w\-])` ensures we don't flag legitimate
# hyphenated compounds like "error-correction module initialized" or
# word extensions like "errors occurred" — only standalone error words
# (followed by whitespace, punctuation, or end-of-string) are matched.
_ERROR_RE = re.compile(
r"\b(?:error|failed|not found|timeout|exception|traceback)(?![\w\-])",
re.IGNORECASE,
)
@dataclass
class OrchestratorResult:
"""Final result from the orchestration pipeline."""
success: bool
response: str
layers_executed: list[str] = field(default_factory=list)
tool_calls_made: list[dict[str, Any]] = field(default_factory=list)
skills_loaded: list[str] = field(default_factory=list)
rounds: int = 0
model: str = ""
duration_seconds: float = 0.0
metadata: dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> dict[str, Any]:
return {
"success": self.success,
"response": self.response,
"layers_executed": self.layers_executed,
"tool_calls_made": self.tool_calls_made,
"skills_loaded": self.skills_loaded,
"rounds": self.rounds,
"model": self.model,
"duration_seconds": round(self.duration_seconds, 2),
"metadata": self.metadata,
}
class AgentOrchestrator:
"""7-layer agentic orchestration pipeline.
Parameters
----------
dispatcher :
An ``AgentDispatcher`` for executing tool calls.
skill_resolver :
An ``EnhancedSkillResolver`` for skill discovery.
redis_bridge :
A ``RedisBridge`` for hot-path coordination (can be None).
skills_root :
Path to the skills directory.
ollama_port :
Port of the Ollama API server.
timeout :
HTTP timeout per Ollama call in seconds.
max_rounds :
Maximum tool-call rounds per request.
on_needs_clarification :
Optional callback invoked when user clarification is needed.
Receives a ``ClarificationDecision`` and should return the
user's response (or an empty string to abort).
"""
def __init__(
self,
dispatcher: "AgentDispatcher",
skill_resolver: EnhancedSkillResolver,
redis_bridge: RedisBridge | None = None,
skills_root: str = "",
ollama_port: int = 11434,
timeout: float = 300.0,
max_rounds: int = AGENT_MAX_ROUNDS,
on_needs_clarification: Callable[[ClarificationDecision], str] | None = None,
) -> None:
from ai_lsc.constants import BASE_DIR
self.skills_root = skills_root or os.path.join(BASE_DIR, "skills")
self.dispatcher = dispatcher
self.skill_resolver = skill_resolver
self.redis = redis_bridge
self.ollama_port = ollama_port
self.timeout = timeout
self.max_rounds = max_rounds
self.on_clarify = on_needs_clarification
# Sub-components
self.model_pool = WarmModelPool(ollama_port=ollama_port)
self.clarification_gate = ClarificationGate(ollama_port=ollama_port)
self.skill_injector = SkillInjector(skill_resolver, self.skills_root)
# Active state tracking
self.active_tools: set[str] = set()
self._conversation_history: list[dict[str, str]] = []
# ── Main Execution Entry Point ─────────────────────────────────────
def execute(
self,
user_message: str,
model: str | None = None,
available_tools: list[str] | None = None,
) -> OrchestratorResult:
"""Execute a user request through the full orchestration pipeline.
Parameters
----------
user_message :
The user's natural language request.
model :
Override the auto-selected model.
available_tools :
Explicit list of tool IDs the orchestrator can use.
Returns
-------
An ``OrchestratorResult`` with the final output and metadata.
"""
start_time = time.monotonic()
layers_executed: list[str] = []
tool_calls_made: list[dict[str, Any]] = []
skills_loaded: list[str] = []
use_model = model if model is not None else AGENT_DEFAULT_MODEL
try:
# ── Layer 1: Router ───────────────────────────────────
route = self._route(user_message, available_tools)
layers_executed.append("router")
if model is None:
use_model = route.get("model", use_model)
logger.info(
"Route: intent=%s, confidence=%.2f, model=%s",
route.get("intent", "unknown"),
route.get("confidence", 0),
use_model,
)
# ── Layer 2: Skill Loader ────────────────────────────
skills = self._load_skills(user_message)
layers_executed.append("skill_loader")
if skills:
skills_loaded.extend(s.name for s in skills)
logger.info("Loaded %d skills: %s", len(skills), [s.name for s in skills])
# ── Layer 3: Clarification Gate ───────────────────────
decision = self._clarify(user_message, available_tools)
layers_executed.append("clarification_gate")
if decision.mode == "clarify" and self.on_clarify:
response = self.on_clarify(decision)
if not response:
return OrchestratorResult(
success=False,
response="Request cancelled by user.",
layers_executed=layers_executed,
model=use_model,
duration_seconds=time.monotonic() - start_time,
)
user_message = response # User clarified
elif decision.mode == "clarify":
logger.info("Clarification needed but no callback — proceeding with defaults")
# ── Layer 4: Outline Planner ───────────────────────────
plan = self._plan(user_message, skills, decision)
layers_executed.append("outline_planner")
logger.info("Plan: %d steps", len(plan.get("steps", [])))
# ── Layer 5: Tool Orchestrator ─────────────────────────
results, calls = self._orchestrate(plan, use_model, skills)
layers_executed.append("tool_orchestrator")
tool_calls_made.extend(calls)
# ── Layer 6: Subagent Spawner ──────────────────────────
if plan.get("needs_subagents"):
sub_results = self._spawn_subagents(plan, use_model)
layers_executed.append("subagent_spawner")
results.append(f"Sub-agent results: {json.dumps(sub_results)}")
# ── Layer 7: Quality Enforcer ──────────────────────────
final = self._enforce_quality(results, user_message)
layers_executed.append("quality_enforcer")
return OrchestratorResult(
success=True,
response=final,
layers_executed=layers_executed,
tool_calls_made=tool_calls_made,
skills_loaded=skills_loaded,
rounds=len(calls),
model=use_model,
duration_seconds=time.monotonic() - start_time,
)
except Exception as exc:
logger.error("Orchestration failed: %s", exc)
return OrchestratorResult(
success=False,
response=f"Orchestration error: {exc}",
layers_executed=layers_executed,
tool_calls_made=tool_calls_made,
model=use_model,
duration_seconds=time.monotonic() - start_time,
)
# ── Layer 1: Router ────────────────────────────────────────────────
def _route(
self,
message: str,
tools: list[str] | None,
) -> dict[str, Any]:
"""Classify the request and determine the best model tier."""
classification = self.clarification_gate._classify(message, tools)
intent = classification.get("intent", "unknown")
tool_id = classification.get("tool_id", "")
confidence = classification.get("confidence", 0.5)
# Select model tier based on intent complexity
task_type = self._intent_to_task_type(intent)
model = self.model_pool.acquire(task_type)
return {
"intent": intent,
"tool_id": tool_id,
"confidence": confidence,
"task_type": task_type,
"model": model,
"arguments": classification.get("arguments", {}),
}
@staticmethod
def _intent_to_task_type(intent: str) -> str:
"""Map an intent string to a model task type."""
intent_lower = intent.lower()
if any(w in intent_lower for w in ["start", "stop", "check", "list", "install"]):
return "classification"
if any(w in intent_lower for w in ["summarize", "clarify", "explain"]):
return "utility"
if any(w in intent_lower for w in ["analyze", "reason", "review", "code", "script"]):
return "reasoning"
if any(w in intent_lower for w in ["generate", "write", "create", "document"]):
return "generation"
return "reasoning"
# ── Layer 2: Skill Loader ─────────────────────────────────────────
def _load_skills(
self,
message: str,
) -> list[Any]:
"""Find and load skills matching the user's request.
Keyword triggers are used for matching today. Semantic matching
via Qdrant is tracked as a future enhancement (see TODO in this
method) but is not yet wired up.
"""
# TODO(security): add Qdrant-backed semantic skill search once the
# embedding collection is provisioned. Until then, keyword-only.
matches = self.skill_resolver.find_by_trigger(message)
return matches[:3] # Limit to 3 skills max
# ── Layer 3: Clarification Gate ───────────────────────────────────
def _clarify(
self,
message: str,
tools: list[str] | None,
) -> ClarificationDecision:
"""Run the clarification gate on the request."""
return self.clarification_gate.evaluate(message, tools)
# ── Layer 4: Outline Planner ──────────────────────────────────────
def _plan(
self,
message: str,
skills: list[Any],
decision: ClarificationDecision,
) -> dict[str, Any]:
"""Generate an execution plan for the request.
For high-confidence simple requests, the plan is a single step
derived from the clarification decision. For complex requests,
it calls the planner model.
"""
# Simple path: use the decision directly
if decision.mode == "skip" and decision.tool_id:
return {
"steps": [
{
"action": decision.intent,
"tool": decision.tool_id,
"arguments": decision.arguments,
}
],
"needs_subagents": False,
"complexity": "simple",
}
# Complex path: ask the model to plan
plan = self._generate_plan(message, skills)
return plan
def _generate_plan(
self,
message: str,
skills: list[Any],
) -> dict[str, Any]:
"""Use the planner model to generate a multi-step plan."""
skill_names = [s.name for s in skills] if skills else []
system_prompt = (
"You are a task planner for the AI-LSC agentic system. "
"Break the user's request into atomic steps. "
"Respond with ONLY valid JSON:\n"
'{"steps": [{"action": "<description>", "tool": "<tool_id or empty>", '
'"arguments": {<params>}}, ...], '
'"needs_subagents": <bool>, '
'"complexity": "simple|moderate|complex"}'
)
if skill_names:
system_prompt += f"\nAvailable skills: {', '.join(skill_names)}"
try:
import urllib.request
payload = json.dumps({
"model": AGENT_DEFAULT_MODEL,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"Plan this request: {message}"},
],
"stream": False,
"options": {"temperature": 0.0},
}).encode("utf-8")
req = urllib.request.Request(
f"http://127.0.0.1:{self.ollama_port}/api/chat",
data=payload,
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=self.timeout) as resp:
data = json.loads(resp.read().decode("utf-8"))
content = data.get("message", {}).get("content", "").strip()
if content.startswith("```"):
content = content.split("\n", 1)[1]
if content.endswith("```"):
content = content[:-3]
return json.loads(content)
except Exception as exc:
logger.warning("Plan generation failed: %s", exc)
# Fallback plan
return {
"steps": [{"action": message, "tool": "", "arguments": {}}],
"needs_subagents": False,
"complexity": "simple",
}
# ── Layer 5: Tool Orchestrator ──────────────────────────────────────
def _orchestrate(
self,
plan: dict[str, Any],
model: str,
skills: list[Any],
) -> tuple[list[str], list[dict[str, Any]]]:
"""Execute the plan steps via the dispatcher."""
results: list[str] = []
calls_made: list[dict[str, Any]] = []
for step in plan.get("steps", []):
tool_id = step.get("tool", "")
action = step.get("action", "")
arguments = step.get("arguments", {})
if not tool_id:
# No specific tool — let the agent loop handle it
result = self._agent_execute(action, model, skills)
results.append(result.get("final_response", ""))
calls_made.extend(result.get("tool_calls_made", []))
else:
# Direct tool call
result = self.dispatcher.execute_tool_call({
"name": tool_id,
"arguments": arguments,
})
results.append(result.get("result_text", ""))
calls_made.append({
"name": tool_id,
"arguments": arguments,
"result": result,
})
return results, calls_made
def _agent_execute(
self,
task: str,
model: str,
skills: list[Any],
) -> dict[str, Any]:
"""Use the agent loop for complex multi-tool tasks."""
from ai_lsc.agents.agent_loop import AgentLoop
# Build tool schemas
from ai_lsc.agents.tool_bridge import ToolBridge
bridge = ToolBridge(self.dispatcher.registry, self.active_tools)
schemas = bridge.generate_all_schemas()
loop = AgentLoop(
dispatcher=self.dispatcher,
ollama_port=self.ollama_port,
model=model,
system_prompt=self._build_system_prompt(skills),
timeout=self.timeout,
max_rounds=self.max_rounds,
)
loop.set_tool_schemas(schemas)
# Inject skill summaries if available
if skills:
summary = self.skill_injector.build_skill_summary(self.active_tools)
return loop.run(f"{summary}\n\nTask: {task}", model=model)
return loop.run(task, model=model)
def _build_system_prompt(self, skills: list[Any]) -> str:
"""Build the system prompt with skill context."""
parts = [
"You are the AI-LSC Stack Operator. You can manage the entire "
"local AI tool stack using the provided tools. Always check "
"service status before starting or stopping services.",
]
if skills:
parts.append(
"\nActive skills: "
+ ", ".join(f"{s.name} ({s.description})" for s in skills)
)
return "\n".join(parts)
# ── Layer 6: Subagent Spawner ──────────────────────────────────────
def _spawn_subagents(
self,
plan: dict[str, Any],
model: str,
) -> list[dict[str, Any]]:
"""Spawn sub-agents for parallelizable subtasks."""
results: list[dict[str, Any]] = []
for step in plan.get("steps", []):
if step.get("parallelizable"):
try:
sub_result = self._agent_execute(
step.get("action", ""),
model,
[],
)
results.append({
"step": step.get("action", ""),
"result": sub_result.get("final_response", ""),
})
except Exception as exc:
results.append({
"step": step.get("action", ""),
"error": str(exc),
})
return results
# ── Layer 7: Quality Enforcer ──────────────────────────────────────
def _enforce_quality(
self,
results: list[str],
original_request: str,
) -> str:
"""Validate and refine the results.
Uses word-boundary matching so legitimate phrases like
"error-correction module initialized" are not flagged.
"""
if not results:
return "No results generated."
# Check for error indicators in results (word-boundary regex).
errors = [r for r in results if _ERROR_RE.search(r)]
if errors:
# Filter out errors, keep successful results
clean = [r for r in results if r not in errors]
if clean:
return "\n".join(clean) + (
f"\n\n(Warnings: {len(errors)} step(s) had issues)"
)
return "All steps failed: " + "; ".join(errors[:3])
return "\n".join(results)
# ── Public API ─────────────────────────────────────────────────────
def update_active_tools(self, tools: set[str]) -> None:
"""Update the set of currently active tools."""
self.active_tools = set(tools)
def get_status(self) -> dict[str, Any]:
"""Return orchestrator status for monitoring."""
return {
"model_pool": self.model_pool.status_summary(),
"active_tools": sorted(self.active_tools),
"conversation_length": len(self._conversation_history),
"redis": self.redis.health_check() if self.redis else {"connected": False},
}

395
src/ai_lsc/agents/qdrant_bridge.py Executable file
View File

@ -0,0 +1,395 @@
"""
AI-LSC — Qdrant vector memory bridge.
Provides the Qdrant vector database integration for the Agentic OS
semantic memory layer, handling:
- **Collection management**: Create, list, and delete vector collections.
- **Point operations**: Upsert, search, and delete vectors with payloads.
- **Embedding generation**: Use Ollama embedding models for vectorization.
- **Skill matching**: Semantic search over skill descriptions.
- **RAG pipeline**: Retrieve relevant context for document analysis.
Qdrant serves as the **semantic path** (vector search, RAG), while Redis
handles the hot path and MariaDB handles cold persistence.
Usage
-----
bridge = QdrantBridge(qdrant_port=6333, ollama_port=11434)
bridge.create_collection("documents", dimension=384)
bridge.upsert_points("documents", points)
results = bridge.search("documents", query="quarterly report", limit=5)
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
from typing import Any
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
class QdrantBridge:
"""Bridge to Qdrant for semantic vector operations.
Uses Qdrant's REST API directly via urllib to maintain the same
zero-hard-dependency pattern as the rest of the agents package.
Falls back gracefully when Qdrant is not running.
Parameters
----------
qdrant_host :
Qdrant server hostname.
qdrant_port :
Qdrant HTTP API port.
ollama_port :
Ollama port for embedding generation.
default_embedding_model :
Ollama model to use for embeddings.
timeout :
HTTP request timeout in seconds.
"""
def __init__(
self,
qdrant_host: str = "127.0.0.1",
qdrant_port: int = 6333,
ollama_port: int = 11434,
default_embedding_model: str = "nomic-embed-text",
timeout: float = 30.0,
) -> None:
self.qdrant_url = f"http://{qdrant_host}:{qdrant_port}"
self.ollama_url = f"http://127.0.0.1:{ollama_port}"
self.embedding_model = default_embedding_model
self.timeout = timeout
def _qdrant_request(
self,
method: str,
path: str,
data: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
"""Make a request to the Qdrant REST API."""
url = f"{self.qdrant_url}{path}"
payload = json.dumps(data).encode("utf-8") if data else None
try:
req = urllib.request.Request(
url,
data=payload,
headers={"Content-Type": "application/json"},
method=method,
)
with urllib.request.urlopen(req, timeout=self.timeout) as resp:
if resp.status in (200, 201):
content = resp.read().decode("utf-8")
return json.loads(content) if content else {}
logger.warning("Qdrant %s %s returned %d", method, path, resp.status)
return None
except urllib.error.URLError as exc:
logger.debug("Qdrant not reachable: %s", exc)
return None
except (OSError, ValueError, json.JSONDecodeError) as exc:
logger.error("Qdrant request failed: %s %s: %s", method, path, exc)
return None
# ── Collection Management ──────────────────────────────────────────
def _probe_embedding_dimension(self) -> int | None:
"""H-24: probe the live embedding model's dimension by embedding a
short sentinel string. Returns ``None`` if the model is not
reachable.
"""
sentinel = self._embed("dimension-probe")
if sentinel:
return len(sentinel)
return None
def create_collection(
self,
name: str,
dimension: int | None = None,
distance: str = "cosine",
) -> bool:
"""Create a new vector collection.
Parameters
----------
name :
Collection name.
dimension :
Vector dimensionality (depends on embedding model). If
``None``, the dimension is probed dynamically from the
configured embedding model (H-24).
distance :
Distance metric: "cosine", "euclid", or "dot".
L-08: if the collection already exists, this method returns
``False`` and logs the existing collection's actual dimension
so a dimension mismatch is never silently hidden.
"""
if dimension is None:
dimension = self._probe_embedding_dimension()
if not dimension:
logger.error(
"Cannot create collection %s: no embedding dimension "
"available (is Ollama running with model %s?)",
name, self.embedding_model,
)
return False
# L-08: check if the collection already exists and surface the
# existing dimension; do NOT silently return success.
if name in self.list_collections():
info = self.collection_info(name) or {}
existing_dim = (
info.get("result", {})
.get("config", {})
.get("params", {})
.get("vectors", {})
.get("size")
)
if existing_dim and existing_dim != dimension:
logger.error(
"Collection %s already exists with dim=%d, "
"requested dim=%d — delete and recreate to change "
"the dimension.",
name, existing_dim, dimension,
)
return False
logger.info(
"Collection %s already exists (dim=%d); not recreated.",
name, existing_dim or dimension,
)
return True
result = self._qdrant_request("PUT", f"/collections/{name}", {
"vectors": {
"size": dimension,
"distance": distance,
},
})
if result is not None:
logger.info("Created collection: %s (dim=%d, dist=%s)", name, dimension, distance)
return True
return False
def list_collections(self) -> list[str]:
"""Return names of all collections."""
result = self._qdrant_request("GET", "/collections")
if result:
return [c["name"] for c in result.get("collections", [])]
return []
def delete_collection(self, name: str) -> bool:
"""Delete a collection."""
result = self._qdrant_request("DELETE", f"/collections/{name}")
return result is not None
def collection_info(self, name: str) -> dict[str, Any] | None:
"""Get detailed info about a collection."""
return self._qdrant_request("GET", f"/collections/{name}")
# ── Point Operations ───────────────────────────────────────────────
def upsert_points(
self,
collection: str,
points: list[dict[str, Any]],
) -> bool:
"""Upsert points (vectors + payloads) into a collection.
Each point dict should have:
- "id": str or int — unique point ID
- "vector": list[float] — the embedding
- "payload": dict — arbitrary metadata
"""
result = self._qdrant_request("PUT", f"/collections/{collection}/points", {
"points": points,
})
return result is not None
def search(
self,
collection: str,
query_vector: list[float] | None = None,
query_text: str = "",
limit: int = 5,
filters: dict[str, Any] | None = None,
) -> list[dict[str, Any]]:
"""Search for similar vectors in a collection.
Parameters
----------
collection :
Collection to search in.
query_vector :
Pre-computed embedding vector. If None, query_text is embedded.
query_text :
Text to embed and search with (used if query_vector is None).
limit :
Maximum results to return.
filters :
Optional Qdrant filter payload.
"""
if query_vector is None and query_text:
query_vector = self._embed(query_text)
if query_vector is None:
return []
search_body: dict[str, Any] = {
"vector": query_vector,
"limit": limit,
"with_payload": True,
}
if filters:
search_body["filter"] = filters
result = self._qdrant_request(
"POST", f"/collections/{collection}/points/search", search_body
)
if result:
return result.get("result", [])
return []
def delete_points(
self,
collection: str,
point_ids: list[str | int],
) -> bool:
"""Delete specific points from a collection."""
result = self._qdrant_request(
"POST",
f"/collections/{collection}/points/delete",
{"points": [{"id": pid} for pid in point_ids]},
)
return result is not None
def count_points(self, collection: str) -> int:
"""Return the number of points in a collection."""
result = self._qdrant_request(
"POST", f"/collections/{collection}/points/count", {}
)
if result:
return result.get("result", {}).get("count", 0)
return 0
# ── Embedding Generation ───────────────────────────────────────────
def _embed(self, text: str) -> list[float] | None:
"""Generate an embedding vector using Ollama's embedding API."""
payload = json.dumps({
"model": self.embedding_model,
"prompt": text,
}).encode("utf-8")
try:
req = urllib.request.Request(
f"{self.ollama_url}/api/embeddings",
data=payload,
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=self.timeout) as resp:
data = json.loads(resp.read().decode("utf-8"))
return data.get("embedding")
except (urllib.error.URLError, OSError, ValueError, json.JSONDecodeError) as exc:
logger.error("Embedding generation failed: %s", exc)
return None
def embed_batch(self, texts: list[str]) -> list[list[float] | None]:
"""Generate embeddings for multiple texts.
Returns a list aligned with the input texts. M-38: uses a
thread pool so independent HTTP requests run concurrently.
"""
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=min(8, max(1, len(texts)))) as pool:
return list(pool.map(self._embed, texts))
# ── Skill Matching (semantic) ───────────────────────────────────────
def index_skills(
self,
skills: list[dict[str, Any]],
collection: str = "skills",
) -> bool:
"""Index skill descriptions for semantic matching.
Parameters
----------
skills :
List of skill dicts with "name", "description", and optional "triggers".
collection :
Collection to store skill vectors.
"""
# Create collection if it doesn't exist
if collection not in self.list_collections():
# H-24: probe the embedding dimension dynamically rather
# than hardcoding 768.
if not self.create_collection(collection, dimension=None):
return False
points = []
for skill in skills:
# Combine name, description, and triggers for embedding
text_parts = [skill.get("name", ""), skill.get("description", "")]
triggers = skill.get("triggers", [])
if triggers:
text_parts.append(" ".join(triggers))
text = " ".join(text_parts)
vector = self._embed(text)
if vector is None:
continue
points.append({
"id": skill.get("name", ""),
"vector": vector,
"payload": {
"name": skill.get("name", ""),
"description": skill.get("description", ""),
"category": skill.get("category", ""),
"required_tools": skill.get("required_tools", []),
"triggers": triggers,
},
})
if points:
return self.upsert_points(collection, points)
return False
def find_similar_skills(
self,
query: str,
collection: str = "skills",
limit: int = 3,
) -> list[dict[str, Any]]:
"""Find skills semantically similar to a query."""
results = self.search(collection, query_text=query, limit=limit)
return [
{
"name": r.get("payload", {}).get("name", ""),
"description": r.get("payload", {}).get("description", ""),
"score": r.get("score", 0.0),
"category": r.get("payload", {}).get("category", ""),
"required_tools": r.get("payload", {}).get("required_tools", []),
}
for r in results
]
# ── Health Check ───────────────────────────────────────────────────
def health_check(self) -> dict[str, Any]:
"""Return Qdrant connection health status."""
result = self._qdrant_request("GET", "/collections")
connected = result is not None
collections = []
if result:
collections = [c["name"] for c in result.get("collections", [])]
return {
"connected": connected,
"collections": collections,
"embedding_model": self.embedding_model,
}

396
src/ai_lsc/agents/redis_bridge.py Executable file
View File

@ -0,0 +1,396 @@
"""
AI-LSC — Redis hot-path bridge.
Provides the Redis integration for the Agentic OS memory layer,
handling the hot-path concerns that need sub-millisecond latency:
- **Task Queue**: FIFO queue for agent task dispatch and coordination.
- **Pub/Sub**: Real-time event broadcasting for inter-agent communication.
- **Status Cache**: TTL-based caching of service status and health checks.
- **Lock Management**: Distributed locks for preventing concurrent conflicts.
Redis serves as the **hot path** (real-time, volatile), while MariaDB handles
the **cold path** (persistent audit logs, task memory, config) and Qdrant
handles the **semantic path** (vector search, RAG).
Usage
-----
bridge = RedisBridge(port=6379)
bridge.enqueue_task("rag-pipeline", {"query": " quarterly report"})
bridge.publish_event("service_started", {"tool_id": "qdrant"})
bridge.cache_status("qdrant", {"running": True, "port": 6333}, ttl=30)
"""
from __future__ import annotations
import json
import time
from typing import Any
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
# Channel names for pub/sub
_CHANNELS = {
"service_events": "ai_lsc:events:service",
"agent_events": "ai_lsc:events:agent",
"task_events": "ai_lsc:events:task",
"model_events": "ai_lsc:events:model",
"skill_events": "ai_lsc:events:skill",
}
# Key prefixes for organized data
_KEY_PREFIXES = {
"task_queue": "ai_lsc:queue:",
"task_payload": "ai_lsc:payload:",
"status_cache": "ai_lsc:status:",
"task_result": "ai_lsc:result:",
"lock": "ai_lsc:lock:",
"agent_state": "ai_lsc:agent:",
"model_pool": "ai_lsc:pool:",
}
class RedisBridge:
"""Bridge to Redis for hot-path agentic operations.
Uses raw ``redis-py`` for direct Redis protocol access.
Falls back to a no-op stub when redis-py is not installed.
Parameters
----------
host :
Redis server hostname.
port :
Redis server port.
db :
Redis database number (default 0).
"""
def __init__(
self,
host: str = "127.0.0.1",
port: int = 6379,
db: int = 0,
) -> None:
self.host = host
self.port = port
self.db = db
self._client = None
self._pubsub = None
self._connected = False
self._try_connect()
def _try_connect(self) -> None:
"""Attempt to connect to Redis. Graceful degradation if unavailable."""
try:
import redis as redis_lib
self._client = redis_lib.Redis(
host=self.host, port=self.port, db=self.db,
decode_responses=True, socket_timeout=5,
)
self._client.ping()
self._connected = True
logger.info("Redis connected at %s:%d", self.host, self.port)
except ImportError:
logger.warning("redis-py not installed — Redis features disabled")
except Exception as exc:
logger.warning("Redis not available at %s:%d: %s", self.host, self.port, exc)
@property
def is_connected(self) -> bool:
return self._connected and self._client is not None
# ── Task Queue ─────────────────────────────────────────────────────
def enqueue_task(
self,
queue_name: str,
task_data: dict[str, Any],
priority: int = 0,
) -> str | None:
"""Add a task to a named queue.
Parameters
----------
queue_name :
Logical queue name (e.g. "rag-pipeline", "code-review").
task_data :
Task payload dict.
priority :
Higher priority tasks are processed first.
Returns
-------
Task ID string, or None if Redis is unavailable.
"""
if not self.is_connected:
return None
task_id = f"{queue_name}:{int(time.time() * 1000)}"
task_data["_task_id"] = task_id
task_data["_enqueued_at"] = time.time()
task_data["_priority"] = priority
key = f"{_KEY_PREFIXES['task_queue']}{queue_name}"
payload = json.dumps(task_data)
try:
# M-29: use task_id as the sorted-set member (not the full
# payload) so two tasks with identical JSON don't collide.
# The payload lives in a separate hash keyed by task_id.
self._client.zadd(key, {task_id: -priority})
self._client.hset(
f"{_KEY_PREFIXES['task_payload']}{queue_name}",
task_id,
payload,
)
logger.info("Enqueued task %s to queue '%s' (priority=%d)", task_id, queue_name, priority)
return task_id
except Exception as exc:
logger.error("Failed to enqueue task: %s", exc)
return None
def dequeue_task(self, queue_name: str) -> dict[str, Any] | None:
"""Pop the highest-priority task from a queue."""
if not self.is_connected:
return None
key = f"{_KEY_PREFIXES['task_queue']}{queue_name}"
try:
# Get highest priority (lowest negative score)
result = self._client.zpopmin(key, count=1)
if not result:
return None
task_id, _ = result[0]
# M-29: pull the payload out of the side hash.
payload = self._client.hget(
f"{_KEY_PREFIXES['task_payload']}{queue_name}",
task_id,
)
if not payload:
return None
self._client.hdel(
f"{_KEY_PREFIXES['task_payload']}{queue_name}",
task_id,
)
return json.loads(payload)
except Exception as exc:
logger.error("Failed to dequeue task: %s", exc)
return None
def queue_length(self, queue_name: str) -> int:
"""Return the number of pending tasks in a queue."""
if not self.is_connected:
return 0
key = f"{_KEY_PREFIXES['task_queue']}{queue_name}"
try:
return self._client.zcard(key)
except Exception:
return 0
# ── Pub/Sub ────────────────────────────────────────────────────────
def publish_event(
self,
event_type: str,
data: dict[str, Any],
) -> bool:
"""Publish an event to the appropriate channel.
Parameters
----------
event_type :
One of the channel types (service_events, agent_events, etc.)
data :
Event payload.
"""
if not self.is_connected:
return False
channel = _CHANNELS.get(event_type, _CHANNELS["service_events"])
data["_timestamp"] = time.time()
data["_event_type"] = event_type
try:
self._client.publish(channel, json.dumps(data))
logger.debug("Published %s event", event_type)
return True
except Exception as exc:
logger.error("Failed to publish event: %s", exc)
return False
# ── Status Cache ────────────────────────────────────────────────────
def _cache_set(self, key: str, value: Any, ttl: int) -> bool:
"""M-17: shared setex + json.dumps helper."""
if not self.is_connected:
return False
try:
self._client.setex(key, ttl, json.dumps(value))
return True
except Exception as exc:
logger.error("Failed to set cache key %s: %s", key, exc)
return False
def _cache_get(self, key: str) -> Any | None:
"""M-17: shared get + json.loads helper."""
if not self.is_connected:
return None
try:
data = self._client.get(key)
return json.loads(data) if data else None
except Exception:
return None
def cache_status(
self,
tool_id: str,
status_data: dict[str, Any],
ttl: int = 30,
) -> bool:
"""Cache a tool's status with an expiration time.
Parameters
----------
tool_id :
The tool identifier.
status_data :
Status payload (running, port, cpu, etc.).
ttl :
Time-to-live in seconds.
"""
return self._cache_set(
f"{_KEY_PREFIXES['status_cache']}{tool_id}",
status_data,
ttl,
)
def get_cached_status(self, tool_id: str) -> dict[str, Any] | None:
"""Retrieve cached status for a tool."""
return self._cache_get(
f"{_KEY_PREFIXES['status_cache']}{tool_id}"
)
# ── Task Results ───────────────────────────────────────────────────
def store_result(
self,
task_id: str,
result: dict[str, Any],
ttl: int = 300,
) -> bool:
"""Store a task result for retrieval by other agents."""
return self._cache_set(
f"{_KEY_PREFIXES['task_result']}{task_id}",
result,
ttl,
)
def get_result(self, task_id: str) -> dict[str, Any] | None:
"""Retrieve a stored task result."""
return self._cache_get(
f"{_KEY_PREFIXES['task_result']}{task_id}"
)
# ── Lock Management ────────────────────────────────────────────────
def acquire_lock(
self,
resource: str,
ttl: int = 60,
) -> bool:
"""Try to acquire a distributed lock.
Parameters
----------
resource :
Resource identifier to lock.
ttl :
Lock expiration in seconds.
Returns
-------
True if the lock was acquired, False if already held.
"""
if not self.is_connected:
# H-13: do not silently bypass the lock when Redis is down.
# Log a warning so operators know two agents could race, and
# return True only so a single-host deployment stays usable.
logger.warning(
"Redis lock bypassed for %r — concurrent agents may race",
resource,
)
return True
key = f"{_KEY_PREFIXES['lock']}{resource}"
try:
return bool(self._client.set(key, "1", nx=True, ex=ttl))
except Exception as exc:
logger.error("Lock acquire failed: %s", exc)
return True
def release_lock(self, resource: str) -> bool:
"""Release a distributed lock."""
if not self.is_connected:
logger.warning(
"Redis lock release skipped for %r — Redis is down",
resource,
)
return True
key = f"{_KEY_PREFIXES['lock']}{resource}"
try:
return bool(self._client.delete(key))
except Exception:
return False
# ── Agent State ────────────────────────────────────────────────────
def save_agent_state(
self,
agent_id: str,
state: dict[str, Any],
) -> bool:
"""Persist an agent's working state to Redis."""
if not self.is_connected:
return False
key = f"{_KEY_PREFIXES['agent_state']}{agent_id}"
try:
self._client.hset(key, mapping={
k: json.dumps(v) if isinstance(v, (dict, list)) else str(v)
for k, v in state.items()
})
return True
except Exception as exc:
logger.error("Failed to save agent state: %s", exc)
return False
def load_agent_state(self, agent_id: str) -> dict[str, Any]:
"""Load an agent's working state from Redis."""
if not self.is_connected:
return {}
key = f"{_KEY_PREFIXES['agent_state']}{agent_id}"
try:
raw = self._client.hgetall(key)
state: dict[str, Any] = {}
for k, v in raw.items():
try:
state[k] = json.loads(v)
except (json.JSONDecodeError, TypeError):
state[k] = v
return state
except Exception:
return {}
# ── Health Check ───────────────────────────────────────────────────
def health_check(self) -> dict[str, Any]:
"""Return Redis connection health status."""
return {
"connected": self.is_connected,
"host": self.host,
"port": self.port,
}

208
src/ai_lsc/agents/schema.py Executable file
View File

@ -0,0 +1,208 @@
"""
AI-LSC — Tool schema definitions for OpenAI-compatible function calling.
Defines the JSON schema fragments that describe each tool action the LLM
can invoke. These schemas are consumed by:
- ``ToolBridge.generate_schemas()`` → full schema list
- ``ollama_tools.register_with_ollama()`` → POST /api/tools
- LibreChat / OpenWebUI tool-definition imports
Every schema follows the OpenAI function-calling format::
{
"type": "function",
"function": {
"name": "<action_name>",
"description": "<human-readable description>",
"parameters": {
"type": "object",
"properties": { ... },
"required": [ ... ]
}
}
}
"""
from __future__ import annotations
from typing import Any
# ── Common parameter fragments ─────────────────────────────────────────
_TOOL_ID_PARAM: dict[str, Any] = {
"type": "string",
"description": "Tool identifier from the AI-LSC registry "
"(e.g. 'ollama', 'qdrant', 'redis').",
}
_PORT_PARAM: dict[str, Any] = {
"type": "integer",
"description": "Override the default port (optional).",
}
_MODEL_NAME_PARAM: dict[str, Any] = {
"type": "string",
"description": "Model name for Ollama pull "
"(e.g. 'qwen2.5:72b', 'llama3:8b').",
}
_SKILL_NAME_PARAM: dict[str, Any] = {
"type": "string",
"description": "Skill identifier to inject into the conversation "
"(e.g. 'rag-analyst', 'code-reviewer').",
}
_QUERY_PARAM: dict[str, Any] = {
"type": "string",
"description": "Search query or description of what to find.",
}
_TARGET_URL_PARAM: dict[str, Any] = {
"type": "string",
"description": "URL to open in the browser.",
}
# ── Schema factory ─────────────────────────────────────────────────────
def _make_schema(
name: str,
description: str,
properties: dict[str, Any],
required: list[str] | None = None,
) -> dict[str, Any]:
"""Build an OpenAI function-calling tool schema."""
return {
"type": "function",
"function": {
"name": name,
"description": description,
"parameters": {
"type": "object",
"properties": properties,
"required": required or [],
},
},
}
# ── Pre-built schemas for core actions ─────────────────────────────────
SCHEMA_START_SERVICE = _make_schema(
name="start_service",
description="Start an AI-LSC managed service by its tool ID. "
"The service must be installed first.",
properties={
"tool_id": _TOOL_ID_PARAM,
"port": _PORT_PARAM,
},
required=["tool_id"],
)
SCHEMA_STOP_SERVICE = _make_schema(
name="stop_service",
description="Stop a running AI-LSC managed service.",
properties={"tool_id": _TOOL_ID_PARAM},
required=["tool_id"],
)
SCHEMA_CHECK_STATUS = _make_schema(
name="check_service_status",
description="Check whether an AI-LSC service is currently running "
"and return its status.",
properties={"tool_id": _TOOL_ID_PARAM},
required=["tool_id"],
)
SCHEMA_PULL_MODEL = _make_schema(
name="pull_model",
description="Pull/download an Ollama model from the registry. "
"Requires Ollama to be running.",
properties={"model_name": _MODEL_NAME_PARAM},
required=["model_name"],
)
SCHEMA_LIST_TOOLS = _make_schema(
name="list_available_tools",
description="List all tools in the AI-LSC registry, optionally "
"filtered by layer, category, or status.",
properties={
"filter_layer": {
"type": "string",
"description": "Optional layer name filter "
"(e.g. 'Inference Engines', 'Data & Knowledge Pipelines').",
},
"filter_category": {
"type": "string",
"description": "Optional category filter "
"(e.g. 'Database', 'Cache', 'Vector Store').",
},
"running_only": {
"type": "boolean",
"description": "If true, only return currently running tools.",
},
},
)
SCHEMA_INJECT_SKILL = _make_schema(
name="inject_skill",
description="Inject a skill's system prompt into the current "
"conversation context. The skill must be registered in "
"the skills directory.",
properties={
"skill_name": _SKILL_NAME_PARAM,
"params": {
"type": "object",
"description": "Optional key-value parameters for the skill.",
},
},
required=["skill_name"],
)
SCHEMA_OPEN_WEB = _make_schema(
name="open_web_interface",
description="Open a tool's web interface in the browser.",
properties={
"tool_id": _TOOL_ID_PARAM,
"port": _PORT_PARAM,
},
required=["tool_id"],
)
SCHEMA_SEARCH_REGISTRY = _make_schema(
name="search_registry",
description="Search the tool registry by keyword. Returns matching "
"tools with their IDs, descriptions, layers, and status.",
properties={"query": _QUERY_PARAM},
required=["query"],
)
SCHEMA_INSTALL_TOOL = _make_schema(
name="install_tool",
description="Install a tool from the AI-LSC registry if not "
"already present on the system.",
properties={
"tool_id": _TOOL_ID_PARAM,
},
required=["tool_id"],
)
# ── Convenience lookups ───────────────────────────────────────────────
CORE_SCHEMAS: list[dict[str, Any]] = [
SCHEMA_START_SERVICE,
SCHEMA_STOP_SERVICE,
SCHEMA_CHECK_STATUS,
SCHEMA_PULL_MODEL,
SCHEMA_LIST_TOOLS,
SCHEMA_INJECT_SKILL,
SCHEMA_OPEN_WEB,
SCHEMA_SEARCH_REGISTRY,
SCHEMA_INSTALL_TOOL,
]
SCHEMA_BY_NAME: dict[str, dict[str, Any]] = {
s["function"]["name"]: s for s in CORE_SCHEMAS
}

276
src/ai_lsc/agents/skill_injector.py Executable file
View File

@ -0,0 +1,276 @@
"""
AI-LSC — Three-phase skill injection.
Manages progressive skill loading into agent context to minimize token
usage while maximizing capability:
Phase 1: **Summary** — One-line skill description injected into the
system prompt so the agent knows what skills exist.
Phase 2: **Full skill** — Complete system prompt loaded when the agent
selects a skill for a task.
Phase 3: **Sub-files** — Additional reference files loaded on demand
(e.g. examples, templates, schema files).
This avoids stuffing 50+ skill system prompts into context up front.
Usage
-----
injector = SkillInjector(skill_resolver, skills_root)
# Phase 1: Build summary for system prompt
summary = injector.build_skill_summary()
# Phase 2: Get full skill prompt
full = injector.get_full_prompt("rag-analyst")
# Phase 3: Load sub-files for deep context
subs = injector.load_sub_files("rag-analyst", ["examples/", "schema.json"])
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
from ai_lsc.agents.skill_resolver import EnhancedSkillResolver, SkillDefinition
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
class SkillInjector:
"""Three-phase skill injection manager.
Parameters
----------
skill_resolver :
An ``EnhancedSkillResolver`` for loading skill metadata.
skills_root :
Path to the skills directory on disk.
"""
def __init__(
self,
skill_resolver: EnhancedSkillResolver,
skills_root: str | Path,
) -> None:
self.resolver = skill_resolver
self.skills_root = Path(skills_root)
self._phase2_cache: dict[str, str] = {}
self._phase3_cache: dict[str, dict[str, str]] = {}
# ── Phase 1: Skill Summary ─────────────────────────────────────────
def build_skill_summary(self, active_tools: set[str] | None = None) -> str:
"""Build a compact summary of all available skills.
This is injected into the system prompt so the agent knows
what skills exist without loading their full prompts.
Parameters
----------
active_tools :
Currently active tool IDs — skills whose deps are met
are marked as [READY], others as [needs: deps...].
Returns
-------
A multi-line summary string suitable for system prompt injection.
"""
active = active_tools or set()
lines = ["Available Skills:", "=" * 40]
skills = self.resolver.list_all()
if not skills:
lines.append(" (no skills registered)")
return "\n".join(lines)
for skill in skills:
missing = self.resolver.check_dependencies(skill.name, active)
if not missing:
status = "[READY]"
else:
status = f"[needs: {', '.join(missing)}]"
triggers = ", ".join(skill.triggers[:3]) if skill.triggers else "no triggers"
lines.append(
f" {skill.name} {status} — {skill.description}\n"
f" Triggers: {triggers}"
)
lines.append("")
lines.append(
"Use inject_skill to load a skill's full prompt. "
"Only READY skills can be used immediately."
)
return "\n".join(lines)
# ── Phase 2: Full Skill Prompt ─────────────────────────────────────
def get_full_prompt(self, skill_name: str) -> str:
"""Load the complete system prompt for a skill.
Cached after first load to avoid repeated disk I/O.
Parameters
----------
skill_name :
The skill identifier.
Returns
-------
The full system prompt text, or an error message if not found.
"""
if skill_name in self._phase2_cache:
return self._phase2_cache[skill_name]
skill = self.resolver.resolve(skill_name)
if not skill.system_prompt:
msg = f"Skill '{skill_name}' has no SYSTEM block defined."
logger.warning(msg)
self._phase2_cache[skill_name] = msg
return msg
# Wrap the system prompt with skill context
prompt_parts = [
f"<skill name=\"{skill.name}\">",
f"<description>{skill.description}</description>",
f"<category>{skill.category}</category>",
]
if skill.input_schema:
import json
schema_str = json.dumps(skill.input_schema, indent=2)
prompt_parts.append(
f"<input_schema>\n{schema_str}\n</input_schema>"
)
if skill.required_tools:
prompt_parts.append(
f"<required_tools>{', '.join(skill.required_tools)}</required_tools>"
)
prompt_parts.append(f"<system_prompt>\n{skill.system_prompt}\n</system_prompt>")
prompt_parts.append("</skill>")
full_prompt = "\n".join(prompt_parts)
self._phase2_cache[skill_name] = full_prompt
logger.info("Loaded full prompt for skill: %s (%d chars)",
skill_name, len(full_prompt))
return full_prompt
# ── Phase 3: Sub-files ──────────────────────────────────────────────
def load_sub_files(
self,
skill_name: str,
file_paths: list[str],
) -> dict[str, str]:
"""Load additional reference files for a skill.
Sub-files are stored alongside the skill in a directory
named ``<skill_name>.d/``::
skills/
rag-analyst ← Modelfile
rag-analyst.skill.json ← Metadata
rag-analyst.d/ ← Sub-files directory
examples/
query.txt
schema.json
prompts/
summarize.txt
Parameters
----------
skill_name :
The skill identifier.
file_paths :
Relative paths within the skill's ``.d/`` directory.
Returns
-------
A dict mapping file path → content string.
"""
cache_key = skill_name
if cache_key not in self._phase3_cache:
self._phase3_cache[cache_key] = {}
results: dict[str, str] = {}
skill_dir = self.skills_root / f"{skill_name}.d"
for rel_path in file_paths:
# Check cache first
if rel_path in self._phase3_cache[cache_key]:
results[rel_path] = self._phase3_cache[cache_key][rel_path]
continue
full_path = skill_dir / rel_path
if not full_path.exists():
results[rel_path] = f"[File not found: {rel_path}]"
continue
try:
content = full_path.read_text(encoding="utf-8", errors="ignore")
results[rel_path] = content
self._phase3_cache[cache_key][rel_path] = content
logger.info("Loaded sub-file for %s: %s (%d chars)",
skill_name, rel_path, len(content))
except OSError as exc:
results[rel_path] = f"[Error reading {rel_path}: {exc}]"
logger.warning("Failed to load sub-file %s/%s: %s",
skill_name, rel_path, exc)
return results
def list_sub_files(self, skill_name: str) -> list[str]:
"""List available sub-files for a skill."""
skill_dir = self.skills_root / f"{skill_name}.d"
if not skill_dir.is_dir():
return []
return sorted(
str(p.relative_to(skill_dir))
for p in skill_dir.rglob("*")
if p.is_file()
)
# ── Cache management ───────────────────────────────────────────────
def clear_cache(self, skill_name: str | None = None) -> None:
"""Clear cached prompts. If skill_name is None, clears all."""
if skill_name is None:
self._phase2_cache.clear()
self._phase3_cache.clear()
else:
self._phase2_cache.pop(skill_name, None)
self._phase3_cache.pop(skill_name, None)
def get_injection_context(
self,
skill_name: str,
active_tools: set[str],
include_sub_files: bool = False,
) -> dict[str, Any]:
"""Build the complete injection context for a skill.
Returns a dict with all phases assembled, ready for the
dispatcher to send back to the LLM.
"""
skill = self.resolver.resolve(skill_name)
missing = self.resolver.check_dependencies(skill_name, active_tools)
context: dict[str, Any] = {
"skill_name": skill.name,
"description": skill.description,
"category": skill.category,
"missing_deps": missing,
"ready": len(missing) == 0,
"full_prompt": self.get_full_prompt(skill_name) if not missing else "",
}
if include_sub_files and not missing:
available_subs = self.list_sub_files(skill_name)
if available_subs:
context["sub_files"] = self.load_sub_files(
skill_name, available_subs[:5] # limit to 5 sub-files
)
return context

276
src/ai_lsc/agents/skill_resolver.py Executable file
View File

@ -0,0 +1,276 @@
"""
AI-LSC — Enhanced skill resolver with dependency checking.
Extends the base ``SkillRuntimeResolver`` with:
- Structured skill metadata from ``.skill.json`` companion files
- Tool dependency resolution (skills that require running services)
- Trigger-keyword matching for automatic skill activation
- Skill categorization and filtering
Skill file layout
-----------------
A skill can now be accompanied by a ``.skill.json`` metadata file::
skills/
rag-analyst ← Modelfile with SYSTEM block
rag-analyst.skill.json ← Structured metadata
The ``.skill.json`` format::
{
"name": "rag-analyst",
"description": "Analyze documents using RAG pipeline",
"required_tools": ["qdrant", "ollama"],
"triggers": ["analyze document", "search knowledge base"],
"input_schema": { ... },
"category": "analysis"
}
Usage
-----
resolver = EnhancedSkillResolver(skills_root, registry_data)
skill = resolver.resolve("rag-analyst")
matches = resolver.find_by_trigger("analyze the quarterly report")
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
class SkillDefinition:
"""Structured metadata for a single skill.
Parameters
----------
name :
Skill identifier (matches the Modelfile filename).
description :
Human-readable description.
system_prompt :
Extracted SYSTEM block from the Modelfile.
required_tools :
Tool IDs from the registry that must be running.
triggers :
Keywords/phrases that should activate this skill.
input_schema :
JSON schema for skill input parameters.
category :
Skill category for grouping/filtering.
extra :
Additional metadata from the .skill.json file.
"""
__slots__ = (
"name", "description", "system_prompt",
"required_tools", "triggers", "input_schema",
"category", "extra",
)
def __init__(
self,
name: str,
description: str = "",
system_prompt: str = "",
required_tools: list[str] | None = None,
triggers: list[str] | None = None,
input_schema: dict[str, Any] | None = None,
category: str = "general",
extra: dict[str, Any] | None = None,
) -> None:
self.name = name
self.description = description
self.system_prompt = system_prompt
self.required_tools = required_tools or []
self.triggers = triggers or []
self.input_schema = input_schema
self.category = category
self.extra = extra or {}
def to_dict(self) -> dict[str, Any]:
"""Serialize to JSON-safe dict."""
return {
"name": self.name,
"description": self.description,
"has_system_prompt": bool(self.system_prompt),
"required_tools": self.required_tools,
"triggers": self.triggers,
"category": self.category,
**self.extra,
}
class EnhancedSkillResolver:
"""Enhanced skill resolver with metadata and dependency checking.
Parameters
----------
skills_root :
Path to the skills directory.
registry_data :
Full registry dict for dependency resolution.
"""
def __init__(
self,
skills_root: str | Path,
registry_data: dict[str, dict[str, Any]] | None = None,
) -> None:
self.skills_root = Path(skills_root)
self.registry = registry_data or {}
self._cache: dict[str, SkillDefinition] = {}
# ── Skill resolution ────────────────────────────────────────────
def resolve(self, skill_name: str) -> SkillDefinition:
"""Resolve a skill by name, loading metadata from disk."""
if skill_name in self._cache:
return self._cache[skill_name]
skill_file = self.skills_root / skill_name
meta_file = self.skills_root / f"{skill_name}.skill.json"
# Extract system prompt from Modelfile
system_prompt = self._extract_system_prompt(skill_file)
# Load metadata from companion JSON
meta = self._load_skill_meta(meta_file)
definition = SkillDefinition(
name=meta.get("name", skill_name),
description=meta.get(
"description",
self._extract_description(skill_file),
),
system_prompt=system_prompt,
required_tools=meta.get("required_tools", []),
triggers=meta.get("triggers", []),
input_schema=meta.get("input_schema"),
category=meta.get("category", "general"),
extra=meta,
)
self._cache[skill_name] = definition
return definition
# ── Trigger matching ─────────────────────────────────────────────
def find_by_trigger(
self, text: str,
) -> list[SkillDefinition]:
"""Find skills whose triggers match the given text.
Used for automatic skill activation when a user message
contains trigger keywords.
"""
text_lower = text.lower()
matches: list[SkillDefinition] = []
for name in self._scan_skill_files():
skill = self.resolve(name)
for trigger in skill.triggers:
if trigger.lower() in text_lower:
matches.append(skill)
break
return matches
# ── Dependency checking ─────────────────────────────────────────
def check_dependencies(
self,
skill_name: str,
active_tools: set[str],
) -> list[str]:
"""Return required tools that are not yet running.
Parameters
----------
skill_name :
The skill to check.
active_tools :
Set of currently active tool IDs.
"""
skill = self.resolve(skill_name)
return [
t for t in skill.required_tools
if t not in active_tools
]
def get_skills_for_active_tools(
self,
active_tools: set[str],
) -> list[SkillDefinition]:
"""Return all skills whose dependencies are satisfied."""
results: list[SkillDefinition] = []
for name in self._scan_skill_files():
skill = self.resolve(name)
missing = self.check_dependencies(name, active_tools)
if not missing:
results.append(skill)
return results
# ── Listing ──────────────────────────────────────────────────────
def list_all(self) -> list[SkillDefinition]:
"""Return all skill definitions."""
return [self.resolve(n) for n in self._scan_skill_files()]
def list_by_category(
self, category: str,
) -> list[SkillDefinition]:
"""Return skills filtered by category."""
return [
s for s in self.list_all()
if s.category == category
]
# ── Internal helpers ────────────────────────────────────────────
def _scan_skill_files(self) -> list[str]:
"""Return names of Modelfile skill definitions."""
if not self.skills_root.is_dir():
return []
return sorted(
p.name for p in self.skills_root.iterdir()
if p.is_file() and not p.name.endswith(".skill.json")
and not p.name.endswith(".json")
)
def _extract_system_prompt(self, path: Path) -> str:
"""Extract SYSTEM block from a Modelfile."""
if not path.exists():
return ""
import re
try:
content = path.read_text(encoding="utf-8", errors="ignore")
except OSError:
return ""
patterns = [
(r'SYSTEM\s+"""(.*?)"""', re.DOTALL | re.IGNORECASE),
(r'SYSTEM\s+"(.*?)"', re.IGNORECASE),
]
return next(
(m.group(1).strip()
for pattern, flags in patterns
for m in [re.search(pattern, content, flags)]
if m),
"",
)
def _extract_description(self, path: Path) -> str:
"""Extract a one-line description from a Modelfile."""
prompt = self._extract_system_prompt(path)
if not prompt:
return ""
first_line = prompt.split("\n")[0].strip()
return first_line[:200] if first_line else ""
@staticmethod
def _load_skill_meta(path: Path) -> dict[str, Any]:
"""Load metadata from a .skill.json companion file."""
if not path.exists():
return {}
try:
return json.loads(path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError):
return {}

153
src/ai_lsc/agents/tool_bridge.py Executable file
View File

@ -0,0 +1,153 @@
"""
AI-LSC — Registry-to-function-calling schema translator.
``ToolBridge`` reads the 115-tool registry and generates OpenAI-compatible
tool schemas that describe which actions an LLM can take. It also enriches
schemas with context from the registry (descriptions, layers, ports).
Usage
-----
bridge = ToolBridge(registry_mgr)
schemas = bridge.generate_all_schemas()
# Pass schemas to Ollama, LibreChat, or OpenWebUI
"""
from __future__ import annotations
from typing import Any
from ai_lsc.agents.schema import (
CORE_SCHEMAS,
SCHEMA_BY_NAME,
_make_schema,
)
from ai_lsc.constants import DEFAULT_PORTS
class ToolBridge:
"""Translates the AI-LSC registry into function-calling tool schemas.
Parameters
----------
registry_data :
The full registry dict from ``RegistryManager.get_all_tools()``.
active_tools :
Set of tool IDs currently active in the pipeline (from
``PipelineState.active_tools``). Used to annotate which
tools are available vs. just registered.
"""
def __init__(
self,
registry_data: dict[str, dict[str, Any]],
active_tools: set[str] | None = None,
) -> None:
self.registry = registry_data
self.active_tools = active_tools or set()
# ── Schema generation ────────────────────────────────────────────
def generate_all_schemas(self) -> list[dict[str, Any]]:
"""Return the complete list of tool schemas for function calling.
This merges the 9 core action schemas with per-tool annotations
for tools that have web interfaces or are Ollama models.
"""
# H-08: drop the static list_available_tools schema before we
# append the annotated version, otherwise the LLM receives two
# definitions for the same tool name.
schemas = [
s for s in CORE_SCHEMAS
if s.get("function", {}).get("name") != "list_available_tools"
]
# Annotate list_tools with available tool IDs
list_schema = SCHEMA_BY_NAME["list_available_tools"]
tool_names = sorted(self.registry.keys())
list_desc = (
f"{list_schema['function']['description']} "
f"Known tools: {', '.join(tool_names[:20])}"
f"{'...' if len(tool_names) > 20 else ''}. "
f"Active: {', '.join(sorted(self.active_tools)) or 'none'}."
)
schemas.append(_make_schema(
name="list_available_tools",
description=list_desc,
properties=list_schema["function"]["parameters"]["properties"],
required=list_schema["function"]["parameters"]["required"],
))
return schemas
def generate_tool_summary(self) -> str:
"""Return a human-readable summary of all registered tools.
Designed to be injected as context into the LLM's system prompt
so it knows what tools are available without needing a tool call.
"""
lines = ["AI-LSC Managed Tools:", "=" * 40]
for tool_id, meta in sorted(self.registry.items()):
status = "ACTIVE" if tool_id in self.active_tools else "available"
port = meta.get("launcher", {}).get("default_port")
port_str = f" :{port}" if port else ""
desc = meta.get("description", "No description")
flags = meta.get("flags", {})
flags_str = []
if flags.get("has_web"):
flags_str.append("web")
if flags.get("is_ollama"):
flags_str.append("ollama")
if flags.get("has_cli"):
flags_str.append("cli")
flag_str = f" [{','.join(flags_str)}]" if flags_str else ""
lines.append(
f" {tool_id}{port_str} — {desc} ({status}){flag_str}"
)
return "\n".join(lines)
# ── Tool lookup helpers ───────────────────────────────────────────
def get_tool_info(self, tool_id: str) -> dict[str, Any]:
"""Return registry metadata for a single tool, or empty dict."""
return self.registry.get(tool_id, {})
def get_tools_by_layer(self, layer: str) -> list[tuple[str, dict]]:
"""Return all tools in a given layer."""
return [
(tid, meta) for tid, meta in self.registry.items()
if meta.get("layer") == layer
]
def get_tools_by_flag(
self, flag: str, value: bool = True
) -> list[tuple[str, dict]]:
"""Return tools matching a specific flag (e.g. 'has_web')."""
return [
(tid, meta) for tid, meta in self.registry.items()
if meta.get("flags", {}).get(flag) == value
]
def get_web_tools(self) -> list[tuple[str, dict]]:
"""Return all tools with web interfaces."""
return self.get_tools_by_flag("has_web")
def get_ollama_tools(self) -> list[tuple[str, dict]]:
"""Return all Ollama-related tools."""
return self.get_tools_by_flag("is_ollama")
def suggest_model_for_task(self, task_type: str) -> str:
"""Suggest an appropriate Ollama model tier for a task type.
This mirrors the Layer 1 routing logic from the agentic
architecture template.
"""
routing: dict[str, str] = {
"document": "70b",
"chart": "70b",
"web": "70b",
"script": "32b",
"analysis": "70b",
"classification": "8b",
"clarification": "14b",
}
return routing.get(task_type, "32b")

1
src/ai_lsc/chat/__init__.py Executable file
View File

@ -0,0 +1 @@
"""AI-LSC chat sub-package."""

211
src/ai_lsc/chat/api.py Executable file
View File

@ -0,0 +1,211 @@
"""
AI-LSC — Chat API thread-pool worker.
Isolates all network I/O (Ollama ``/api/chat`` endpoint) from the GUI
main loop using Qt's ``QThreadPool`` + ``QRunnable`` pattern.
Architecture
------------
``ApiRunnable`` is submitted to the thread pool. When the HTTP call
completes (or fails), results are delivered back to the main thread
via ``WorkerSignals.result`` — a Qt Signal that the UI connects to
with a slot running on the main thread.
No UI widgets are imported here; only ``PySide6.QtCore`` for the
Signal / Runnable machinery.
Availability
-------------
If PySide6 is not installed this module still imports successfully but
``WorkerSignals`` and ``ApiRunnable`` will be ``None``. The top-level
``__init__.py`` handles this gracefully.
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
try:
from PySide6.QtCore import QObject, QRunnable, Signal
_HAS_QT = True
except ImportError:
_HAS_QT = False
# ── Signal emitter (thread-safe bridge to main loop) ───────────────────
if _HAS_QT:
class WorkerSignals(QObject):
"""Emits results from a background thread back to the main thread.
``result`` carries three values:
1. ``identity`` (str) — display name for the response source.
2. ``reply`` (str) — the assistant's response or error message.
3. ``history_append`` (str | None) — text to append to the chat
history, or *None* if the response was an error.
"""
result = Signal(str, str, object)
# ── API runnable ─────────────────────────────────────────────────────
if _HAS_QT:
class ApiRunnable(QRunnable):
"""Background task that calls the Ollama ``/api/chat`` endpoint.
Parameters
----------
model_id :
Model identifier string passed to the Ollama API (e.g.
``"llama3:8b"``).
port_id :
Port number of the running Ollama server.
history_snapshot :
List of ``{"role": …, "content": …}`` message dicts sent as
the conversation history.
temperature :
Sampling temperature (0.0–2.0).
max_tokens :
Maximum tokens to generate (``num_predict`` in Ollama API).
timeout :
HTTP request timeout in seconds.
"""
def __init__(
self,
model_id: str,
port_id: int,
history_snapshot: list[dict],
temperature: float = 0.7,
max_tokens: int = 4096,
timeout: float = 120.0,
) -> None:
super().__init__()
# L-06: validate port range up-front so an invalid port
# surfaces a clean ValueError instead of a cryptic URLError
# when we try to construct the URL below.
if not isinstance(port_id, int) or not 1 <= port_id <= 65535:
raise ValueError(f"invalid port_id: {port_id!r}")
self.model_id = model_id
self.port_id = port_id
self.history_snapshot = history_snapshot
self.temperature = temperature
self.max_tokens = max_tokens
self.timeout = timeout
self.signals = WorkerSignals()
self.setAutoDelete(True)
def run(self) -> None:
identity, reply, history_append = self.model_id, "", None
try:
url = f"http://127.0.0.1:{self.port_id}/api/chat"
payload = json.dumps({
"model": self.model_id,
"messages": self.history_snapshot,
"stream": False,
"options": {
"temperature": self.temperature,
"num_predict": self.max_tokens,
},
}).encode("utf-8")
req = urllib.request.Request(
url,
data=payload,
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=self.timeout) as resp:
data = json.loads(resp.read().decode("utf-8"))
reply = data.get("message", {}).get("content", "").strip()
if not reply:
identity, reply = "? System Guard", (
"Received empty execution token from model. "
"Core might lack system context space allocation."
)
else:
history_append = reply
except urllib.error.HTTPError as he:
identity = "? Ollama Stack Exception"
err_body = self._safe_read_error(he)
# H-10: surface only a short, generic reason to the user;
# log the full body server-side instead of echoing it back.
logger.warning(
"Ollama HTTP %s on /api/chat (model=%s): %s",
he.code, self.model_id, err_body,
)
reply = (
f"Ollama Engine rejected execution layout "
f"(Code {he.code}).\n\n"
f"[Reason]: {self._short_reason(err_body)}\n\n"
"*Troubleshooting:*\n"
"1. Did you click 'Build/Register Selected Skills' first?\n"
"2. Ensure the base model has been pulled."
)
except urllib.error.URLError as ue:
identity = "? Cluster Port Offline"
# H-10: do not echo connection details; categorize instead.
reason = self._categorize_url_error(ue)
logger.warning(
"Ollama unreachable on port %s: %s",
self.port_id, ue.reason,
)
reply = (
f"Failed to connect to Ollama API on port "
f"[{self.port_id}].\n\n"
f"[Details]: {reason}\n\n"
"*Action Required*: Verify Ollama is [ LIVE ] on Dashboard."
)
except (OSError, ValueError, json.JSONDecodeError) as e:
identity = "? Exception Tracker"
logger.warning("Background chat interruption: %s", e)
reply = (
"Unhandled background interruption in chat worker. "
"Check the application log for details."
)
self.signals.result.emit(identity, reply, history_append)
@staticmethod
def _safe_read_error(http_error) -> str:
"""Best-effort extraction of the error body from an HTTP error."""
try:
body = json.loads(
http_error.read().decode("utf-8", errors="ignore")
)
return body.get("error", str(body))
except (OSError, ValueError, json.JSONDecodeError):
return "Internal structural parser issue."
@staticmethod
def _short_reason(err_body: str) -> str:
"""Return a short, user-safe summary of an Ollama error body."""
if not err_body:
return "no detail available"
# Trim to first line and 120 chars; drop any URLs / paths.
first_line = str(err_body).splitlines()[0]
return first_line[:120]
@staticmethod
def _categorize_url_error(ue: urllib.error.URLError) -> str:
"""Map a URLError reason to a generic user-safe category."""
reason = str(ue.reason).lower()
if "refused" in reason or "connection" in reason:
return "connection refused (is the service running?)"
if "timeout" in reason or "timed out" in reason:
return "request timed out"
if "name" in reason or "resolve" in reason:
return "DNS resolution failed"
return "network error"
else:
WorkerSignals = None # type: ignore[assignment, misc]
ApiRunnable = None # type: ignore[assignment, misc]

188
src/ai_lsc/constants.py Executable file
View File

@ -0,0 +1,188 @@
"""
AI-LSC v3.1 — Application-wide constants.
Release codename: Ankh of Jah
Pure data: file names, schema version, required directories, default ports,
status styles, log colours, service licences, tree-skip patterns, and the
navigation layer order. No behaviour lives here.
"""
import os
# ── Base directory ─────────────────────────────────────────────────────
# Overridable via AI_LSC_BASE_DIR environment variable.
# Bootstrap sets this; the app resolves everything relative to it.
BASE_DIR: str = os.environ.get("AI_LSC_BASE_DIR", "/mnt/AI")
# LA-03: alias so run.sh --headless can import it
CANONICAL_BASE_DIR: str = BASE_DIR
# ── Filenames ────────────────────────────────────────────────────────────
APP_VERSION: str = "3.1.0"
APP_CODENAME: str = "Ankh of Jah"
APP_DISPLAY_NAME: str = f"AI - Local Stack Control v{APP_VERSION} - http://dcos.net"
CONFIG_FILE: str = "controller_config.json"
APP_ICON_FILE: str = "ai-lsc-logo.png"
STATE_FILE_NAME: str = "pipeline_state.json"
PIPELINE_FILE_NAME: str = "pipeline.json"
STACK_SCHEMA_VERSION: str = "3.0"
MANIFEST_FILE_NAME: str = ".ai-lsc-project.json"
JCL_FILE_NAME: str = ".ai-lsc-jobs.json"
# ── Required sub-directories under BASE_DIR ────────────────────
REQUIRED_DIRS: list[str] = [
"bin",
"tools",
"registry",
"config",
"cache",
"runtime",
"logs",
"skills",
"datasets/raw",
"models/ollama",
"models/chroma",
"workspaces/hermes",
"workspaces/openwebui",
"workspaces/n8n",
"tmp",
"exports",
"data",
"containers",
"configs",
"pipelines",
"dashboards",
"backups",
]
# ── Default ports for every known tool ───────────────────────────────────
DEFAULT_PORTS: dict[str, int | None] = {
"postgresql": 5432, "mariadb": 3306, "redis": 6379,
"sqlite3": None, "python": None, "cuda": None,
"ollama": 11434, "llamacpp": 8080, "vllm": 8000,
"litellm": 4000, "chromadb": 8000, "whisper": None,
"docling": None, "aider": None, "claude_code": None,
"fabric": None, "btop": None, "glances": 61208,
"crewai": None, "autogen": None,
"hermes": 17050, "openwebui": 8080, "anythingllm": 3001,
"flowise": 3000, "dify": 80, "stack_exporter": None,
# Agentic OS stack additions
"qdrant": 6333, "librechat": 3080, "n8n": 5678,
}
# ── UI status label formatting ──────────────────────────────────────────
STATUS_STYLES: dict[bool, tuple[str, str]] = {
True: ("[ LIVE ]", "#2ecc71"),
False: ("[ OFFLINE ]", "#7f8c8d"),
}
# ── Log source colours for the activity feed ────────────────────────────
LOG_SOURCE_COLORS: dict[str, str] = {
"Ollama": "#e67e22", "Tmux": "#3498db",
"Installer": "#2ecc71", "Audit": "#f39c12",
"Container": "#9b59b6", "SkillRuntime": "#1abc9c",
"Pipeline": "#e74c3c", "Lifecycle": "#2980b9",
"SelfHeal": "#8e44ad", "Compiler": "#e67e22",
}
LOG_COLOR_DEFAULT: str = "#bdc3c7"
# ── Service licence notices ────────────────────────────────────────────
SERVICE_LICENSES: dict[str, str] = {
"Open WebUI": "MIT License: github.com/open-webui/open-webui",
"Aider": "Apache License 2.0: github.com/aider-chat/aider",
"Hermes": "MIT License (Hermes Orchestrator)",
"Odysseus": "MIT License (Local/Proprietary)",
"Dify": "Dify Open Source License: github.com/langgenius/dify",
"Flowise": "Apache License 2.0: github.com/FlowiseAI/Flowise",
"AnythingLLM": "MIT License: github.com/Mintplex-Labs/anything-llm",
"LiteLLM Proxy": "MIT License: github.com/BerriAI/litellm",
"Claude Code": "Anthropic Terms of Service: anthropic.com",
"CrewAI": "MIT License: github.com/joaomdmoura/crewAI",
"AutoGen": "MIT License: github.com/microsoft/autogen",
"LangChain": "MIT License: github.com/langchain-ai/langchain",
"LangFlow": "Apache License 2.0: github.com/langflow-ai/langflow",
"Ollama": "MIT License: github.com/ollama/ollama",
"llama.cpp": "MIT License: github.com/ggerganov/llama.cpp",
"Grafana": "AGPL-3.0: github.com/grafana/grafana",
"Prometheus": "Apache License 2.0: github.com/prometheus/prometheus",
"Qdrant": "Apache License 2.0: github.com/qdrant/qdrant",
"n8n": "Apache License 2.0 (with Fair Code): github.com/n8n-io/n8n",
"LibreChat": "MIT License: github.com/danny-avila/LibreChat",
"InvokeAI": "MIT License: github.com/invoke-ai/InvokeAI",
"Terraform": "BSL-1.1: github.com/hashicorp/terraform",
"Ansible": "GPL-3.0: github.com/ansible/ansible",
"Pulumi": "Apache License 2.0: github.com/pulumi/pulumi",
"OpenTofu": "MPL-2.0: github.com/opentofu/opentofu",
"MCP Drift State Tracker": "AGPL-3.0: git.dcos.net/dcosnet/MCP-Drift-State-Tracker",
}
# ── Tree-widget skip patterns ──────────────────────────────────────────
TREE_SKIP_PATTERNS: set[str] = {".", "__pycache__", "node_modules", "vendor"}
# ── Navigation layer order for the sidebar rack diagram ───────────────
NAV_LAYER_ORDER: list[str] = [
"Host Platform", "Development Environment", "GPU Runtimes",
"Engines", "Orchestrators", "Security",
"Observability", "User Interfaces", "DevOps",
"Knowledge Management",
]
# ── Ollama server candidate paths (probed in order) ────────────────
# The runtime probes these paths to locate the ollama server binary or
# service data. First match wins.
OLLAMA_SERVER_CANDIDATES: list[str] = [
"ollama", # /mnt/AI/ollama
"tools/ollama", # /mnt/AI/tools/ollama
"runtime/ollama", # /mnt/AI/runtime/ollama
"bin/ollama", # /mnt/AI/bin/ollama
]
# ── Model tier routing (reserved for v4.0 agentic layer) ──────────
MODEL_TIERS: dict[str, dict] = {
"8b": {"max_vram_gb": 8, "desc": "Classification, routing, intent detection"},
"14b": {"max_vram_gb": 14, "desc": "Utility, summarization, clarification"},
"32b": {"max_vram_gb": 32, "desc": "Reasoning, analysis, code generation"},
"70b": {"max_vram_gb": 70, "desc": "Heavy generation, complex reasoning, documents"},
}
# ── Agent runtime constants (reserved for v4.0 agentic layer) ────
AGENT_DEFAULT_MODEL: str = "qwen2.5:32b"
AGENT_MAX_ROUNDS: int = 20
CLARIFICATION_SKIP_THRESHOLD: float = 0.95
CLARIFICATION_CONFIRM_THRESHOLD: float = 0.70
# ── Qt Stylesheets ──────────────────────────────────────────────────────
GLOBAL_STYLE: str = """
QWidget { background-color: #161616; color: #e0e0e0;
font-family: 'Segoe UI', Arial, sans-serif; font-size: 13px; }
QGroupBox { border: 1px solid #333; border-radius: 6px; margin-top: 14px;
padding-top: 10px; font-weight: bold; color: #a5d6a7; }
QGroupBox::title { subcontrol-origin: margin; subcontrol-position: top left;
padding: 0 5px; left: 10px; }
QPushButton { background-color: #2c3e50; color: white; border: 1px solid #1a252f;
border-radius: 4px; padding: 6px 12px; font-weight: bold; }
QPushButton:hover { background-color: #34495e; }
QPushButton:pressed { background-color: #1a252f; }
QLineEdit, QTextEdit, QComboBox, QSpinBox, QDoubleSpinBox {
background-color: #1e1e1e; border: 1px solid #444;
border-radius: 4px; padding: 5px; color: white; }
QTabWidget::pane { border: 1px solid #333; background-color: #1a1a1a;
border-radius: 4px; }
QTabBar::tab { background-color: #222; border: 1px solid #333; padding: 8px 15px;
margin-right: 2px; border-top-left-radius: 4px; border-top-right-radius: 4px; }
QTabBar::tab:selected { background-color: #3498db; color: white; font-weight: bold; }
QTableWidget, QTreeWidget, QListWidget { background-color: #1e1e1e;
gridline-color: #333; border: 1px solid #333; border-radius: 4px; }
QHeaderView::section { background-color: #2c3e50; color: white; padding: 4px;
border: 1px solid #1a252f; font-weight: bold; }
"""
SIDEBAR_TREE_STYLE: str = """
QTreeWidget { background-color: #111111; border: none; color: #bdc3c7;
font-family: 'Segoe UI'; font-size: 11px; }
QTreeWidget::item { padding: 6px; border-bottom: 1px solid #161616; }
QTreeWidget::item:hover { background-color: #1c1c1c; color: #fff; }
QTreeWidget::item:selected { background-color: #2c3e50; color: #2ecc71;
font-weight: bold; }
"""

304
src/ai_lsc/guardrails.py Executable file
View File

@ -0,0 +1,304 @@
#!/usr/bin/env python3
"""AI-LSC Framework Guardrail Validator.
Runs after any agent edit to catch:
1. Bloat — files that grew >200% without architectural reason
2. Size — files exceeding max_module_lines (default 300)
3. Subprocess leakage — UI files touching subprocess/psutil directly
4. Parent coupling — UI files reaching into self.parent instead of protocol
5. os.path contamination — should use pathlib
6. Lint — ruff check (if available)
Usage:
python3 guardrails.py # validate ai_lsc/ in cwd
python3 guardrails.py --baseline # snapshot current sizes
python3 guardrails.py --fix # auto-fix ruff issues
"""
from __future__ import annotations
import ast
import json
import os
import sys
from pathlib import Path
BASE_DIR = Path(__file__).resolve().parent
BASELINE_FILE = BASE_DIR / ".guardrail_baseline.json"
MAX_MODULE_LINES = 300
MAX_GROWTH_FACTOR = 2.0
# Directories where subprocess/psutil calls are ALLOWED
RUNTIME_ALLOWED_DIRS = {"utils", "runtime", "core", "scripts", "agents"}
# Directories where os.path usage is ALLOWED (legacy tolerance)
OSPATH_ALLOWED_DIRS = {"utils"}
# Directories where self.parent access is ALLOWED
PARENT_ALLOWED_DIRS: set[str] = set()
def _iter_py_files() -> list[Path]:
"""M-14: return every Python file under BASE_DIR, sorted."""
return sorted(BASE_DIR.rglob("*.py"))
def _read_source(py_file: Path) -> str | None:
"""M-14: read a Python file's source, returning None on error."""
try:
return py_file.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
return None
def _is_parent_access(node: ast.AST) -> bool:
"""M-12: return True if *node* is a ``self.parent.<attr>`` access."""
return (
isinstance(node, ast.Attribute)
and isinstance(node.value, ast.Attribute)
and isinstance(node.value.value, ast.Name)
and node.value.value.id == "self"
and node.value.attr == "parent"
)
def get_file_sizes() -> dict[str, int]:
"""Return {relative_path: line_count} for every .py in the package."""
sizes: dict[str, int] = {}
for py_file in _iter_py_files():
rel = py_file.relative_to(BASE_DIR)
try:
sizes[str(rel)] = sum(1 for _ in py_file.open(encoding="utf-8"))
except (OSError, UnicodeDecodeError):
sizes[str(rel)] = -1
return sizes
def save_baseline(sizes: dict[str, int]) -> None:
BASELINE_FILE.write_text(
json.dumps(sizes, indent=2, sort_keys=True), encoding="utf-8"
)
print(f"Baseline saved: {len(sizes)} files tracked in {BASELINE_FILE}")
def load_baseline() -> dict[str, int]:
if not BASELINE_FILE.exists():
return {}
return json.loads(BASELINE_FILE.read_text(encoding="utf-8"))
def check_bloat(sizes: dict[str, int], baseline: dict[str, int]) -> list[str]:
"""Detect files that grew > MAX_GROWTH_FACTOR without baseline."""
errors: list[str] = []
for path, new_size in sizes.items():
if new_size <= 0:
continue
old_size = baseline.get(path, 0)
if old_size <= 0:
continue # new file, skip
if new_size > old_size * MAX_GROWTH_FACTOR:
growth_pct = (new_size / old_size - 1) * 100
errors.append(
f"BLOAT: {path} grew {old_size} -> {new_size} lines "
f"(+{growth_pct:.0f}%, limit {MAX_GROWTH_FACTOR}x)"
)
return errors
def check_size_limits(sizes: dict[str, int]) -> list[str]:
"""Flag files exceeding max module line count."""
errors: list[str] = []
for path, size in sizes.items():
if size > MAX_MODULE_LINES and not path.endswith("__init__.py"):
errors.append(
f"OVERSIZED: {path} is {size} lines "
f"(limit {MAX_MODULE_LINES})"
)
return errors
def check_subprocess_leakage() -> list[str]:
"""Flag UI files that directly call subprocess/psutil."""
errors: list[str] = []
dangerous_patterns = [
"subprocess.run", "subprocess.Popen", "subprocess.call",
"threading.Thread", "os.system", "os.popen",
"psutil.process_iter", "psutil.cpu_percent",
]
for py_file in _iter_py_files():
rel = str(py_file.relative_to(BASE_DIR))
parts = rel.split(os.sep)
# Skip if in allowed directory
if any(part in RUNTIME_ALLOWED_DIRS for part in parts):
continue
source = _read_source(py_file)
if source is None:
continue
tree = ast.parse(source, filename=rel)
for node in ast.walk(tree):
if isinstance(node, ast.Attribute):
full = f"{node.value}.{node.attr}" if isinstance(
node.value, ast.Name
) else None
if full and full in dangerous_patterns:
errors.append(
f"SUBPROCESS_LEAK: {rel}:{node.lineno} "
f"calls {full} (should delegate to runtime/)"
)
return errors
def check_parent_coupling() -> list[str]:
"""Flag UI files accessing self.parent.* directly."""
errors: list[str] = []
for py_file in _iter_py_files():
rel = str(py_file.relative_to(BASE_DIR))
# M-27: PARENT_ALLOWED_DIRS is empty, so the old `any(...) or not any(...)`
# check was a tautology. Just always check.
source = _read_source(py_file)
if source is None:
continue
tree = ast.parse(source, filename=rel)
for node in ast.walk(tree):
if _is_parent_access(node):
errors.append(
f"PARENT_COUPLING: {rel}:{node.lineno} "
f"accesses self.parent.{node.attr} "
f"(use MainWindowProtocol instead)"
)
return errors
def check_ospath_contamination() -> list[str]:
"""Flag files using os.path instead of pathlib."""
errors: list[str] = []
for py_file in _iter_py_files():
rel = str(py_file.relative_to(BASE_DIR))
parts = rel.split(os.sep)
if any(part in OSPATH_ALLOWED_DIRS for part in parts):
continue
source = _read_source(py_file)
if source is None:
continue
count = source.count("os.path.")
if count > 0:
errors.append(
f"OSPATH: {rel} has {count} os.path.* calls "
f"(use pathlib.Path)"
)
return errors
def run_ruff(fix: bool = False) -> list[str]:
"""Run ruff if available, return error output."""
import shutil
ruff_bin = shutil.which("ruff")
if not ruff_bin:
try:
import ruff as _ # noqa: F401
ruff_bin = sys.executable + " -m ruff"
except ImportError:
return ["RUFF: not installed (pip install ruff)"]
import subprocess
cmd = [sys.executable, "-m", "ruff", "check", str(BASE_DIR)]
if fix:
cmd.append("--fix")
try:
result = subprocess.run(
cmd, capture_output=True, text=True, timeout=30
)
output = result.stdout.strip() + result.stderr.strip()
if output:
return [f"RUFF:\n{output}"]
return []
except Exception as e:
return [f"RUFF: failed to run: {e}"]
def main() -> int:
args = set(sys.argv[1:])
if "--baseline" in args:
sizes = get_file_sizes()
save_baseline(sizes)
return 0
sizes = get_file_sizes()
baseline = load_baseline()
all_errors: list[str] = []
print("=" * 60)
print("AI-LSC Framework Guardrail Validation")
print("=" * 60)
# 1. Bloat check
errors = check_bloat(sizes, baseline)
if errors:
all_errors.extend(errors)
print(f"\n[FAIL] Bloat detection: {len(errors)} violations")
elif baseline:
print("\n[PASS] Bloat detection: no abnormal growth")
else:
print("\n[SKIP] Bloat detection: no baseline (run with --baseline)")
# 2. Size limits
errors = check_size_limits(sizes)
if errors:
all_errors.extend(errors)
print(f"[FAIL] Size limits: {len(errors)} oversized modules")
else:
print(f"[PASS] Size limits: all modules under {MAX_MODULE_LINES} lines")
# 3. Subprocess leakage
errors = check_subprocess_leakage()
if errors:
all_errors.extend(errors)
print(f"[FAIL] Subprocess leakage: {len(errors)} violations")
else:
print("[PASS] Subprocess leakage: clean")
# 4. Parent coupling
errors = check_parent_coupling()
if errors:
all_errors.extend(errors)
print(f"[FAIL] Parent coupling: {len(errors)} violations")
else:
print("[PASS] Parent coupling: clean")
# 5. os.path contamination
errors = check_ospath_contamination()
if errors:
all_errors.extend(errors)
print(f"[FAIL] os.path: {len(errors)} files with os.path.* calls")
else:
print("[PASS] os.path: clean")
# 6. Ruff lint
fix_mode = "--fix" in args
errors = run_ruff(fix=fix_mode)
if errors:
all_errors.extend(errors)
print(f"[{'FIXED' if fix_mode else 'FAIL'}] Ruff lint: see above")
else:
print("[PASS] Ruff lint: clean")
# Summary
print("\n" + "=" * 60)
if all_errors:
print(f"RESULT: {len(all_errors)} guardrail violations")
for e in all_errors:
# Truncate long ruff output
lines = e.split("\n")
for line in lines[:5]:
print(f" {line}")
if len(lines) > 5:
print(f" ... ({len(lines) - 5} more lines)")
return 1
else:
print("RESULT: ALL GUARDRAILS PASSED")
return 0
if __name__ == "__main__":
sys.exit(main())

1
src/ai_lsc/manifest/__init__.py Executable file
View File

@ -0,0 +1 @@
"""AI-LSC manifest sub-package."""

155
src/ai_lsc/manifest/support.py Executable file
View File

@ -0,0 +1,155 @@
"""
AI-LSC — Manifest support.
Reads and writes ``.ai-lsc-project.json`` and ``.ai-lsc-jobs.json``
files that provide project-level context for the chat interface.
Pure filesystem + JSON work — no UI.
Manifest schema (``.ai-lsc-project.json``)::
{
"project": "my-project",
"description": "Brief description for AI context",
"language": "python",
"entry_point": "src/main.py",
"architecture": "System architecture notes",
"environment_notes": "Runtime environment details",
"dependencies": ["package1", "package2"],
"context_files": ["src/**/*.py", "README.md"],
"exclude": ["__pycache__", "*.pyc", ".git"]
}
JCL schema (``.ai-lsc-jobs.json``)::
{
"jobs": [
{"name": "...", "command": "...", "cwd": "..."},
...
]
}
"""
from __future__ import annotations
import glob as glob_mod
import json
from pathlib import Path
from typing import Any
from ai_lsc.constants import MANIFEST_FILE_NAME, JCL_FILE_NAME
# Maximum directory traversal depth when searching for manifests.
_MAX_WALK_DEPTH: int = 20
class ManifestSupport:
"""Static utility class for manifest and JCL file operations."""
@staticmethod
def discover_manifest(directory: str | Path) -> Path | None:
"""Walk up from *directory* to find the nearest manifest file.
Stops after ``_MAX_WALK_DEPTH`` iterations or when the
filesystem root is reached.
"""
current = Path(directory).resolve()
for _ in range(_MAX_WALK_DEPTH):
candidate = current / MANIFEST_FILE_NAME
if candidate.exists():
return candidate
parent = current.parent
if parent == current:
return None
current = parent
return None
@staticmethod
def load_manifest(path: str | Path) -> dict[str, Any]:
"""Load and return the manifest dict, or ``{}`` on failure."""
p = Path(path)
if not p.exists():
return {}
try:
return json.loads(p.read_text(encoding="utf-8"))
except (OSError, ValueError, json.JSONDecodeError):
return {}
@staticmethod
def build_system_context(manifest: dict[str, Any]) -> str:
"""Build a flat system-prompt text block from manifest data."""
project = manifest.get("project", "Unknown Project")
description = manifest.get("description", "")
language = manifest.get("language", "")
entry = manifest.get("entry_point", "")
architecture = manifest.get("architecture", "")
environment = manifest.get("environment_notes", "")
dependencies = manifest.get("dependencies", [])
parts = [f"Project: {project}"]
if description:
parts.append(f"Description: {description}")
if language:
parts.append(f"Language: {language}")
if entry:
parts.append(f"Entry Point: {entry}")
if architecture:
parts.append(f"Architecture: {architecture}")
if environment:
parts.append(f"Environment: {environment}")
if dependencies:
parts.append(f"Dependencies: {', '.join(dependencies)}")
return "\n".join(parts)
@staticmethod
def resolve_context_files(
manifest: dict[str, Any],
base_dir: str | Path,
) -> list[str]:
"""Resolve glob patterns in the manifest to real file paths."""
base = Path(base_dir)
patterns = manifest.get("context_files", [])
exclude = set(manifest.get("exclude", []))
files: list[str] = []
for pattern in patterns:
full_pattern = str(base / pattern)
matched = glob_mod.glob(full_pattern, recursive=True)
for f in matched:
if Path(f).is_file() and not any(ex in f for ex in exclude):
files.append(f)
return files
@staticmethod
def load_jcl(path: str | Path) -> list[dict[str, Any]]:
"""Load job entries from a JCL file, or ``[]`` on failure."""
p = Path(path)
if not p.exists():
return []
try:
data = json.loads(p.read_text(encoding="utf-8"))
return data.get("jobs", [])
except (OSError, ValueError, json.JSONDecodeError):
return []
@staticmethod
def create_manifest_template(path: str | Path) -> Path:
"""Write a starter manifest template to *path*.
Returns the path of the created file.
"""
p = Path(path)
template = {
"project": "my-project",
"description": "Brief project description for AI context",
"language": "python",
"entry_point": "src/main.py",
"architecture": "Describe the system architecture",
"environment_notes": "Runtime environment details",
"dependencies": ["package1", "package2"],
"context_files": ["src/**/*.py", "README.md"],
"exclude": ["__pycache__", "*.pyc", ".git", "node_modules"],
}
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(json.dumps(template, indent=4), encoding="utf-8")
return p

1
src/ai_lsc/registry/__init__.py Executable file
View File

@ -0,0 +1 @@
"""AI-LSC registry sub-package."""

3811
src/ai_lsc/registry/defaults.py Executable file
View File

File diff suppressed because it is too large Load Diff

6
src/ai_lsc/registry/layers/__init__.py Executable file
View File

@ -0,0 +1,6 @@
"""Registry layers sub-package.
Each module in this directory exports a ``TOOLS`` dict of registry
entries belonging to one 10-Layer stratum. The loader in
:mod:`ai_lsc.registry.loader` discovers and merges them automatically.
"""

893
src/ai_lsc/registry/layers/development.py Executable file
View File

@ -0,0 +1,893 @@
"""Registry entries for the Development Environment layer (L2).
Each entry follows the standard registry schema:
- ``name``: human-readable tool name
- ``level``: 10-layer taxonomy level (1-10)
- ``layer``: this layer name
- ``role``: role within the layer
- ``category``: functional category
- ``installer``: installation method
- ``launcher``: process launcher specification
- ``deps``: list of required tool IDs
- ``description``: short description
- ``flags``: optional boolean flags
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'python': {
"name": "Python Environment",
"level": 2,
"layer": "Development Environment",
"role": "Build System",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "python-pip"
},
"launcher": {
"type": "desktop",
"cmd": "python3 --version",
"default_port": None
},
"deps": [],
"description": "Python core interpreter and virtual environments.",
"license": 'Python',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'cupy': {
"name": "CuPy",
"level": 2,
"layer": "Development Environment",
"role": "GPU Acceleration",
"category": "GPU Computing",
"installer": {
"type": "uv",
"pkg": "cupy-cuda12x"
},
"launcher": {
"type": "desktop",
"cmd": "python3 -c \"import cupy; print(cupy.__version__)\"",
"default_port": None
},
"deps": [
"cuda"
],
"description": "NumPy-compatible GPU array computing library.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": True,
"is_mcp": False,
"is_skills_collection": False
}
},
'fd': {
"name": "fd",
"level": 2,
"layer": "Development Environment",
"role": "Search",
"category": "Find Tool",
"installer": {
"type": "pacman",
"pkg": "fd"
},
"launcher": {
"type": "desktop",
"cmd": "fd --version",
"default_port": None
},
"deps": [],
"description": "Fast find command alternative.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'ripgrep': {
"name": "ripgrep (rg)",
"level": 2,
"layer": "Development Environment",
"role": "Search",
"category": "Search Tool",
"installer": {
"type": "pacman",
"pkg": "ripgrep"
},
"launcher": {
"type": "desktop",
"cmd": "rg --version",
"default_port": None
},
"deps": [],
"description": "Fast recursive search tool (grep replacement).",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'tree_sitter': {
"name": "tree-sitter",
"level": 2,
"layer": "Development Environment",
"role": "Parsing",
"category": "Parser",
"installer": {
"type": "uv",
"pkg": "tree-sitter"
},
"launcher": {
"type": "desktop",
"cmd": "tree-sitter --version",
"default_port": None
},
"deps": [],
"description": "Incremental parsing system for source code.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": True,
"is_mcp": False,
"is_skills_collection": False
}
},
'sst': {
"name": "SST (Serverless Stack)",
"level": 2,
"layer": "Development Environment",
"role": "Full-Stack Framework",
"category": "Serverless Framework",
"installer": {
"type": "npm",
"pkg": "sst"
},
"launcher": {
"type": "desktop",
"cmd": "sst --version",
"default_port": None
},
"deps": [],
"description": "Framework for building full-stack apps on your own infrastructure (AWS, Cloudflare, etc).",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'unsloth': {
"name": "Unsloth",
"level": 2,
"layer": "Development Environment",
"role": "Training",
"category": "Model Training",
"installer": {
"type": "uv",
"pkg": "unsloth"
},
"launcher": {
"type": "desktop",
"cmd": "python3 -c \"import unsloth; print('ok')\"",
"default_port": None
},
"deps": [
"cuda"
],
"description": "2x faster LLM fine-tuning with 80% less memory.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": True,
"is_mcp": False,
"is_skills_collection": False
}
},
'php': {
"name": "PHP",
"level": 2,
"layer": "Development Environment",
"role": "Language",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "php"
},
"launcher": {
"type": "desktop",
"cmd": "php --version",
"default_port": None
},
"deps": [],
"description": "Server-side scripting language.",
"license": "PHP-3.01",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'ruby': {
"name": "Ruby",
"level": 2,
"layer": "Development Environment",
"role": "Language",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "ruby"
},
"launcher": {
"type": "desktop",
"cmd": "ruby --version",
"default_port": None
},
"deps": [],
"description": "Dynamic programming language.",
"license": "Ruby",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'perl': {
"name": "Perl",
"level": 2,
"layer": "Development Environment",
"role": "Language",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "perl"
},
"launcher": {
"type": "desktop",
"cmd": "perl -v",
"default_port": None
},
"deps": [],
"description": "High-level scripting language.",
"license": "GPL-1.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'julia': {
"name": "Julia",
"level": 2,
"layer": "Development Environment",
"role": "Language",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "julia"
},
"launcher": {
"type": "desktop",
"cmd": "julia --version",
"default_port": None
},
"deps": [],
"description": "High-performance numerical computing language.",
"license": "MIT",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'nodejs': {
"name": "Node.js",
"level": 2,
"layer": "Development Environment",
"role": "Language",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "nodejs"
},
"launcher": {
"type": "desktop",
"cmd": "node --version",
"default_port": None
},
"deps": [],
"description": "JavaScript runtime built on V8.",
"license": "MIT",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'go': {
"name": "Go",
"level": 2,
"layer": "Development Environment",
"role": "Language",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "go"
},
"launcher": {
"type": "desktop",
"cmd": "go version",
"default_port": None
},
"deps": [],
"description": "Compiled systems programming language.",
"license": "BSD-3-Clause",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'rust': {
"name": "Rust",
"level": 2,
"layer": "Development Environment",
"role": "Language",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "rust"
},
"launcher": {
"type": "desktop",
"cmd": "rustc --version",
"default_port": None
},
"deps": [],
"description": "Systems language focused on safety and performance.",
"license": "MIT/Apache-2.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'java_jdk': {
"name": "Java JDK",
"level": 2,
"layer": "Development Environment",
"role": "Language",
"category": "Runtime",
"installer": {
"type": "pacman",
"pkg": "jdk-openjdk"
},
"launcher": {
"type": "desktop",
"cmd": "java --version",
"default_port": None
},
"deps": [],
"description": "Java development kit.",
"license": "GPL-2.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'zsh': {
"name": "Zsh",
"level": 2,
"layer": "Development Environment",
"role": "Shell",
"category": "Shell",
"installer": {
"type": "pacman",
"pkg": "zsh"
},
"launcher": {
"type": "desktop",
"cmd": "zsh --version",
"default_port": None
},
"deps": [],
"description": "Extended Bourne shell with enhancements.",
"license": "MIT",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'mksh': {
"name": "mksh",
"level": 2,
"layer": "Development Environment",
"role": "Shell",
"category": "Shell",
"installer": {
"type": "pacman",
"pkg": "mksh"
},
"launcher": {
"type": "desktop",
"cmd": "mksh -c 'echo ok'",
"default_port": None
},
"deps": [],
"description": "MirBSD Korn shell.",
"license": "MirOS",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'bash': {
"name": "Bash",
"level": 2,
"layer": "Development Environment",
"role": "Shell",
"category": "Shell",
"installer": {
"type": "pacman",
"pkg": "bash"
},
"launcher": {
"type": "desktop",
"cmd": "bash --version",
"default_port": None
},
"deps": [],
"description": "GNU Bourne Again Shell.",
"license": "GPL-3.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'fish': {
"name": "Fish",
"level": 2,
"layer": "Development Environment",
"role": "Shell",
"category": "Shell",
"installer": {
"type": "pacman",
"pkg": "fish"
},
"launcher": {
"type": "desktop",
"cmd": "fish --version",
"default_port": None
},
"deps": [],
"description": "User-friendly shell with auto-suggestions.",
"license": "GPL-2.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'fakeroot': {
"name": "Fakeroot",
"level": 2,
"layer": "Development Environment",
"role": "Build Tool",
"category": "Build",
"installer": {
"type": "pacman",
"pkg": "fakeroot"
},
"launcher": {
"type": "desktop",
"cmd": "fakeroot --version",
"default_port": None
},
"deps": [],
"description": "Run commands pretending to have root privileges for package building.",
"license": "GPL-3.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'make': {
"name": "GNU Make",
"level": 2,
"layer": "Development Environment",
"role": "Build System",
"category": "Build",
"installer": {
"type": "pacman",
"pkg": "make"
},
"launcher": {
"type": "desktop",
"cmd": "make --version",
"default_port": None
},
"deps": [],
"description": "Build automation tool.",
"license": "GPL-3.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'cmake': {
"name": "CMake",
"level": 2,
"layer": "Development Environment",
"role": "Build System",
"category": "Build",
"installer": {
"type": "pacman",
"pkg": "cmake"
},
"launcher": {
"type": "desktop",
"cmd": "cmake --version",
"default_port": None
},
"deps": [],
"description": "Cross-platform build system generator.",
"license": "BSD-3-Clause",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'gcc': {
"name": "GCC",
"level": 2,
"layer": "Development Environment",
"role": "Compiler",
"category": "Build",
"installer": {
"type": "pacman",
"pkg": "gcc"
},
"launcher": {
"type": "desktop",
"cmd": "gcc --version",
"default_port": None
},
"deps": [],
"description": "GNU Compiler Collection.",
"license": "GPL-3.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'bison': {
"name": "Bison",
"level": 2,
"layer": "Development Environment",
"role": "Parser Generator",
"category": "Build",
"installer": {
"type": "pacman",
"pkg": "bison"
},
"launcher": {
"type": "desktop",
"cmd": "bison --version",
"default_port": None
},
"deps": [],
"description": "GNU parser generator.",
"license": "GPL-3.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'pkg_config': {
"name": "pkg-config",
"level": 2,
"layer": "Development Environment",
"role": "Build Tool",
"category": "Build",
"installer": {
"type": "pacman",
"pkg": "pkg-config"
},
"launcher": {
"type": "desktop",
"cmd": "pkg-config --version",
"default_port": None
},
"deps": [],
"description": "Helper tool for retrieving library compile/link flags.",
"license": "GPL-2.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'valgrind': {
"name": "Valgrind",
"level": 2,
"layer": "Development Environment",
"role": "Profiling",
"category": "Debugging",
"installer": {
"type": "pacman",
"pkg": "valgrind"
},
"launcher": {
"type": "desktop",
"cmd": "valgrind --version",
"default_port": None
},
"deps": [],
"description": "Instrumentation framework for debugging and profiling.",
"license": "GPL-2.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'gdb': {
"name": "GDB",
"level": 2,
"layer": "Development Environment",
"role": "Debugger",
"category": "Debugging",
"installer": {
"type": "pacman",
"pkg": "gdb"
},
"launcher": {
"type": "desktop",
"cmd": "gdb --version",
"default_port": None
},
"deps": [],
"description": "GNU Debugger.",
"license": "GPL-3.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'strace': {
"name": "strace",
"level": 2,
"layer": "Development Environment",
"role": "Tracing",
"category": "Debugging",
"installer": {
"type": "pacman",
"pkg": "strace"
},
"launcher": {
"type": "desktop",
"cmd": "strace -V",
"default_port": None
},
"deps": [],
"description": "System call tracer for debugging.",
"license": "LGPL-2.1",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'ltrace': {
"name": "ltrace",
"level": 2,
"layer": "Development Environment",
"role": "Tracing",
"category": "Debugging",
"installer": {
"type": "pacman",
"pkg": "ltrace"
},
"launcher": {
"type": "desktop",
"cmd": "ltrace --version",
"default_port": None
},
"deps": [],
"description": "Library call tracer.",
"license": "GPL-2.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'patchelf': {
"name": "PatchELF",
"level": 2,
"layer": "Development Environment",
"role": "Binary Tool",
"category": "Build",
"installer": {
"type": "pacman",
"pkg": "patchelf"
},
"launcher": {
"type": "desktop",
"cmd": "patchelf --version",
"default_port": None
},
"deps": [],
"description": "Tool for modifying ELF binaries.",
"license": "GPL-3.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'upx': {
"name": "UPX",
"level": 2,
"layer": "Development Environment",
"role": "Packer",
"category": "Build",
"installer": {
"type": "pacman",
"pkg": "upx"
},
"launcher": {
"type": "desktop",
"cmd": "upx --version",
"default_port": None
},
"deps": [],
"description": "Ultimate Packer for eXecutables.",
"license": "GPL-2.0",
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
}

324
src/ai_lsc/registry/layers/devops.py Executable file
View File

@ -0,0 +1,324 @@
"""Registry entries for the DevOps layer (L9).
Contains Infrastructure as Code tools, configuration management, OCI
runtime packaging, and provisioning tools.
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'terraform': {
"name": "Terraform",
"level": 9,
"layer": "DevOps",
"role": "Infrastructure as Code",
"category": "IaC",
"installer": {
"type": "pacman",
"pkg": "terraform"
},
"launcher": {
"type": "desktop",
"cmd": "terraform version",
"default_port": None
},
"deps": [],
"description": "Infrastructure as Code provisioning tool.",
"license": 'BSL-1.1',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'ansible': {
"name": "Ansible",
"level": 9,
"layer": "DevOps",
"role": "Configuration Management",
"category": "Config Management",
"installer": {
"type": "pacman",
"pkg": "ansible"
},
"launcher": {
"type": "desktop",
"cmd": "ansible --version",
"default_port": None
},
"deps": [],
"description": "Agentless IT automation and configuration management.",
"license": 'GPL-3.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'puppet': {
"name": "Puppet",
"level": 9,
"layer": "DevOps",
"role": "Configuration Management",
"category": "Config Management",
"installer": {
"type": "pacman",
"pkg": "puppet"
},
"launcher": {
"type": "desktop",
"cmd": "puppet --version",
"default_port": None
},
"deps": [],
"description": "Declarative configuration management tool.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'pulumi': {
"name": "Pulumi",
"level": 9,
"layer": "DevOps",
"role": "Infrastructure as Code",
"category": "IaC",
"installer": {
"type": "npm",
"pkg": "@pulumi/pulumi"
},
"launcher": {
"type": "desktop",
"cmd": "pulumi version",
"default_port": None
},
"deps": [],
"description": "IaC platform using real programming languages (Python, TypeScript, Go).",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'bicep': {
"name": "Bicep",
"level": 9,
"layer": "DevOps",
"role": "Infrastructure as Code",
"category": "IaC",
"installer": {
"type": "npm",
"pkg": "@azure/bicep"
},
"launcher": {
"type": "desktop",
"cmd": "bicep --version",
"default_port": None
},
"deps": [],
"description": "Azure domain-specific language for declarative infrastructure.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'opentofu': {
"name": "OpenTofu",
"level": 9,
"layer": "DevOps",
"role": "Infrastructure as Code",
"category": "IaC",
"installer": {
"type": "custom",
"pkg": "https://opentofu.org/docs/intro/install/"
},
"launcher": {
"type": "desktop",
"cmd": "tofu version",
"default_port": None
},
"deps": [],
"description": "Open-source Terraform fork maintained by the Linux Foundation.",
"license": 'MPL-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'aws_cdk': {
"name": "AWS CDK",
"level": 9,
"layer": "DevOps",
"role": "Infrastructure as Code",
"category": "IaC",
"installer": {
"type": "npm",
"pkg": "aws-cdk"
},
"launcher": {
"type": "desktop",
"cmd": "cdk --version",
"default_port": None
},
"deps": [],
"description": "Cloud Development Kit — define AWS CloudFormation in code.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'crossplane': {
"name": "Crossplane",
"level": 9,
"layer": "DevOps",
"role": "Infrastructure as Code",
"category": "IaC Control Plane",
"installer": {
"type": "custom",
"pkg": "https://docs.crossplane.io/v2/getting-started/install/"
},
"launcher": {
"type": "desktop",
"cmd": "crossplane --help",
"default_port": None
},
"deps": [
"kubectl"
],
"description": "Kubernetes-native cloud infrastructure control plane.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'terragrunt': {
"name": "Terragrunt",
"level": 9,
"layer": "DevOps",
"role": "Infrastructure as Code",
"category": "IaC Wrapper",
"installer": {
"type": "custom",
"pkg": "https://terragrunt.gruntwork.io/docs/getting-started/install/"
},
"launcher": {
"type": "desktop",
"cmd": "terragrunt --version",
"default_port": None
},
"deps": [
"terraform"
],
"description": "Thin wrapper for Terraform providing DRY config and remote state.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'stack_exporter': {
"name": "Stack Container Packager",
"level": 9,
"layer": "DevOps",
"role": "Runtime Packaging",
"category": "OCI Export",
"installer": {
"type": "pacman",
"pkg": "podman"
},
"launcher": {
"type": "desktop",
"cmd": "podman --version",
"default_port": None
},
"deps": [],
"description": "Compiles validated pipeline matrices into Podman/Docker specs.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'homelab': {
"name": "Homelab",
"level": 9,
"layer": "DevOps",
"role": "Provisioning",
"category": "Provisioning",
"installer": {
"type": "git",
"pkg": "https://github.com/khuedoan/homelab",
"cmd": ""
},
"launcher": {
"type": "desktop",
"cmd": "homelab",
"default_port": None
},
"deps": [],
"description": "Fully automated homelab provisioning from empty disk to running services in one command. IaC/GitOps: Packer + Terraform + Ansible + k3s + ArgoCD.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
}

79
src/ai_lsc/registry/layers/gpu.py Executable file
View File

@ -0,0 +1,79 @@
"""Registry entries for the GPU Runtimes layer (L3).
Each entry follows the standard registry schema:
- ``name``: human-readable tool name
- ``level``: 10-layer taxonomy level (1-10)
- ``layer``: this layer name
- ``role``: role within the layer
- ``category``: functional category
- ``installer``: installation method
- ``launcher``: process launcher specification
- ``deps``: list of required tool IDs
- ``description``: short description
- ``flags``: optional boolean flags
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'cuda': {
"name": "CUDA Toolkit",
"level": 3,
"layer": "GPU Runtimes",
"role": "Acceleration",
"category": "GPU",
"installer": {
"type": "pacman",
"pkg": "cuda"
},
"launcher": {
"type": "desktop",
"cmd": "nvcc --version",
"default_port": None
},
"deps": [],
"description": "NVIDIA CUDA parallel computing platform.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": True,
"is_mcp": False,
"is_skills_collection": False
}
},
'apex': {
"name": "NVIDIA Apex",
"level": 3,
"layer": "GPU Runtimes",
"role": "Optimization",
"category": "Mixed Precision",
"installer": {
"type": "uv",
"pkg": "apex"
},
"launcher": {
"type": "desktop",
"cmd": "python3 -c \"import apex; print(apex.__version__)\"",
"default_port": None
},
"deps": [
"cuda"
],
"description": "NVIDIA mixed precision and distributed training.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": True,
"is_mcp": False,
"is_skills_collection": False
}
},
}

469
src/ai_lsc/registry/layers/host_platform.py Executable file
View File

@ -0,0 +1,469 @@
"""Registry entries for the Host Platform layer (L1).
Each entry follows the standard registry schema:
- ``name``: human-readable tool name
- ``level``: 10-layer taxonomy level (1-10)
- ``layer``: this layer name
- ``role``: role within the layer
- ``category``: functional category
- ``installer``: installation method
- ``launcher``: process launcher specification
- ``deps``: list of required tool IDs
- ``description``: short description
- ``flags``: optional boolean flags
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'tmux': {
"name": "Tmux",
"level": 1,
"layer": "Host Platform",
"role": "Multiplexer",
"category": "Terminal",
"installer": {
"type": "pacman",
"pkg": "tmux"
},
"launcher": {
"type": "desktop",
"cmd": "tmux -V",
"default_port": None
},
"deps": [],
"description": "Terminal multiplexer for persistent sessions.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'git': {
"name": "Git",
"level": 1,
"layer": "Host Platform",
"role": "Version Control",
"category": "VCS",
"installer": {
"type": "pacman",
"pkg": "git"
},
"launcher": {
"type": "desktop",
"cmd": "git --version",
"default_port": None
},
"deps": [],
"description": "Distributed version control system.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'podman': {
"name": "Podman",
"level": 1,
"layer": "Host Platform",
"role": "Container Runtime",
"category": "Containers",
"installer": {
"type": "pacman",
"pkg": "podman"
},
"launcher": {
"type": "desktop",
"cmd": "podman --version",
"default_port": None
},
"deps": [],
"description": "Daemonless container engine for OCI containers.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'docker': {
"name": "Docker",
"level": 1,
"layer": "Host Platform",
"role": "Container Runtime",
"category": "Containers",
"installer": {
"type": "pacman",
"pkg": "docker"
},
"launcher": {
"type": "systemd",
"cmd": "docker",
"default_port": None
},
"deps": [],
"description": "Container platform for building and running containers.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'postgresql': {
"name": "PostgreSQL",
"level": 1,
"layer": "Host Platform",
"role": "Foundation",
"category": "Database",
"installer": {
"type": "pacman",
"pkg": "postgresql"
},
"launcher": {
"type": "systemd",
"cmd": "postgresql",
"default_port": 5432
},
"deps": [],
"description": "Relational database used by many frameworks.",
"license": 'PostgreSQL',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'mariadb': {
"name": "MariaDB",
"level": 1,
"layer": "Host Platform",
"role": "Foundation",
"category": "Database",
"installer": {
"type": "pacman",
"pkg": "mariadb"
},
"launcher": {
"type": "systemd",
"cmd": "mariadb",
"default_port": 3306
},
"deps": [],
"description": "Open source relational database.",
"license": 'GPL-2.0',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'redis': {
"name": "Redis",
"level": 1,
"layer": "Host Platform",
"role": "Foundation",
"category": "Cache",
"installer": {
"type": "pacman",
"pkg": "redis"
},
"launcher": {
"type": "systemd",
"cmd": "redis",
"default_port": 6379
},
"deps": [],
"description": "In-memory cache and message broker.",
"license": 'RSALv2',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'sqlite3': {
"name": "SQLite3",
"level": 1,
"layer": "Host Platform",
"role": "Foundation",
"category": "Database",
"installer": {
"type": "pacman",
"pkg": "sqlite"
},
"launcher": {
"type": "desktop",
"cmd": "sqlite3",
"default_port": None
},
"deps": [],
"description": "C-language library implementing a SQL database engine.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'duckdb': {
"name": "DuckDB",
"level": 1,
"layer": "Host Platform",
"role": "Foundation",
"category": "Analytical Database",
"installer": {
"type": "uv",
"pkg": "duckdb"
},
"launcher": {
"type": "desktop",
"cmd": "python3 -c \"import duckdb; print(duckdb.__version__)\"",
"default_port": None
},
"deps": [],
"description": "In-process analytical database with SQL support.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'lxc': {
"name": "LXC",
"level": 1,
"layer": "Host Platform",
"role": "Container Runtime",
"category": "Containers",
"installer": {
"type": "pacman",
"pkg": "lxc"
},
"launcher": {
"type": "desktop",
"cmd": "lxc --version",
"default_port": None
},
"deps": [],
"description": "Linux container system-level virtualization.",
"license": 'LGPL-2.1',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'firecracker': {
"name": "Firecracker",
"level": 1,
"layer": "Host Platform",
"role": "MicroVM",
"category": "Virtualization",
"installer": {
"type": "script",
"cmd": "curl -fsSL https://github.com/firecracker-microvm/firecracker/releases/latest/download/firecracker-v$(curl -s https://api.github.com/repos/firecracker-microvm/firecracker/releases/latest | grep tag_name | cut -d'\"' -f4)-x86_64.tgz | tar xz -C /usr/local/bin/"
},
"launcher": {
"type": "desktop",
"cmd": "firecracker --version",
"default_port": None
},
"deps": [],
"description": "Lightweight virtualization for serverless workloads.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'qemu': {
"name": "QEMU",
"level": 1,
"layer": "Host Platform",
"role": "Emulation",
"category": "Virtualization",
"installer": {
"type": "pacman",
"pkg": "qemu-base"
},
"launcher": {
"type": "desktop",
"cmd": "qemu-system-x86_64 --version",
"default_port": None
},
"deps": [],
"description": "Full system emulation and virtualization.",
"license": 'GPL-2.0',
"flags": {
"has_cli": True,
"has_gui": True,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'libvirt': {
"name": "libvirt",
"level": 1,
"layer": "Host Platform",
"role": "VM Management",
"category": "Virtualization",
"installer": {
"type": "pacman",
"pkg": "libvirt"
},
"launcher": {
"type": "systemd",
"cmd": "libvirtd",
"default_port": None
},
"deps": [],
"description": "Virtualization API and management daemon.",
"license": 'LGPL-2.1',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'cloudflared': {
"name": "Cloudflared",
"level": 1,
"layer": "Host Platform",
"role": "Tunnel",
"category": "Networking",
"installer": {
"type": "script",
"cmd": "curl -L https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64 -o /usr/local/bin/cloudflared && chmod +x /usr/local/bin/cloudflared"
},
"launcher": {
"type": "tmux",
"cmd": "cloudflared tunnel --url http://localhost:{port}",
"default_port": 8080
},
"deps": [],
"description": "Cloudflare tunnel for exposing local services.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'nginx': {
"name": "Nginx",
"level": 1,
"layer": "Host Platform",
"role": "Reverse Proxy",
"category": "Networking",
"installer": {
"type": "pacman",
"pkg": "nginx"
},
"launcher": {
"type": "systemd",
"cmd": "nginx",
"default_port": 80
},
"deps": [],
"description": "HTTP and reverse proxy server.",
"license": 'BSD-2-Clause',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'certbot': {
"name": "Certbot",
"level": 1,
"layer": "Host Platform",
"role": "TLS",
"category": "Networking",
"installer": {
"type": "pacman",
"pkg": "certbot"
},
"launcher": {
"type": "desktop",
"cmd": "certbot --version",
"default_port": None
},
"deps": [],
"description": "Automated TLS certificate management (Let's Encrypt).",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
}

254
src/ai_lsc/registry/layers/inference.py Executable file
View File

@ -0,0 +1,254 @@
"""Registry entries for the Engines layer (L4).
Each entry follows the standard registry schema:
- ``name``: human-readable tool name
- ``level``: 10-layer taxonomy level (1-10)
- ``layer``: this layer name
- ``role``: role within the layer
- ``category``: functional category
- ``installer``: installation method
- ``launcher``: process launcher specification
- ``deps``: list of required tool IDs
- ``description``: short description
- ``flags``: optional boolean flags
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'ollama': {
"name": "Ollama",
"level": 4,
"layer": "Engines",
"role": "Engine",
"category": "LLM Runtime",
"installer": {
"type": "script",
"cmd": "curl -fsSL https://ollama.com/install.sh | sh"
},
"launcher": {
"type": "tmux",
"cmd": "OLLAMA_HOST=0.0.0.0:{port} OLLAMA_MODELS={models_root}/ollama ollama serve",
"default_port": 11434
},
"deps": [],
"description": "Local LLM runner and model manager.",
"license": 'MIT',
"flags": {
"is_ollama": True,
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'llamacpp': {
"name": "llama.cpp",
"level": 4,
"layer": "Engines",
"role": "Engine",
"category": "LLM Runtime",
"installer": {
"type": "git",
"pkg": "https://github.com/ggerganov/llama.cpp"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/llamacpp && make && ./server --port {port}",
"default_port": 8080
},
"deps": [],
"description": "Port of Facebook's LLaMA model in C/C++.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'koboldcpp': {
"name": "KoboldCPP",
"level": 4,
"layer": "Engines",
"role": "Engine",
"category": "LLM Runtime",
"installer": {
"type": "git",
"pkg": "https://github.com/LostRuins/koboldcpp"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/koboldcpp && make && ./koboldcpp --port {port}",
"default_port": 5001
},
"deps": [],
"description": "GGUF-based LLM inference with CUDA/Vulkan.",
"license": 'Proprietary',
"flags": {
"has_cli": False,
"has_gui": True,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'llamafile': {
"name": "Llamafile",
"level": 4,
"layer": "Engines",
"role": "Engine",
"category": "Single-File LLM",
"installer": {
"type": "script",
"cmd": "curl -LO https://github.com/Mozilla-Ocho/llamafile/releases/latest/download/llamafile && chmod +x llamafile"
},
"launcher": {
"type": "desktop",
"cmd": "{tools_root}/bin/llamafile",
"default_port": None
},
"deps": [],
"description": "Distribute and run LLMs in a single file.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": True,
"is_mcp": False,
"is_skills_collection": False
}
},
'turbollm': {
"name": "TurboLLM",
"level": 4,
"layer": "Engines",
"role": "Engine",
"category": "LLM Runtime",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/turbollm"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/turbollm && python3 -m turbollm serve --port {port}",
"default_port": 8000
},
"deps": [
"cuda"
],
"description": "Fast LLM serving with tensor parallelism.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'airllm': {
"name": "AirLLM",
"level": 4,
"layer": "Engines",
"role": "Engine",
"category": "Efficient LLM",
"installer": {
"type": "git",
"pkg": "https://github.com/liguodongiot/llm-airforce"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/airllm && python3 -m airllm serve --port {port}",
"default_port": 8001
},
"deps": [
"cuda"
],
"description": "Memory-efficient 70B LLM inference on 4GB GPUs.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'locally_uncensored': {
"name": "Locally-Uncensored",
"level": 4,
"layer": "Engines",
"role": "Engine",
"category": "Uncensored Models",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/locally-uncensored"
},
"launcher": {
"type": "desktop",
"cmd": "ollama list",
"default_port": None
},
"deps": [
"ollama"
],
"description": "Curated uncensored model collection and tooling.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": True,
"is_mcp": False,
"is_skills_collection": True
}
},
'heretic': {
"name": "Heretic",
"level": 4,
"layer": "Engines",
"role": "Abliteration",
"category": "Model Surgery",
"installer": {
"type": "git",
"pkg": "https://github.com/p-e-w/heretic",
"post_install": "pip install -e ."
},
"launcher": {
"type": "desktop",
"cmd": "python3 -c \"import heretic; print('ok')\"",
"default_port": None
},
"deps": [
"cuda"
],
"description": "Fully automatic censorship/safety-alignment removal for transformer-based LLMs via optimized abliteration. Modifies model weights directly.",
"license": 'Proprietary',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": True,
"is_mcp": False,
"is_skills_collection": False
}
},
}

725
src/ai_lsc/registry/layers/knowledge_management.py Executable file
View File

@ -0,0 +1,725 @@
"""Registry entries for the Knowledge Management layer (L10).
Contains vector stores, graph databases, search engines, document parsers,
data pipelines, memory systems, and knowledge management tools.
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'zotero': {
"name": "Zotero",
"level": 10,
"layer": "Knowledge Management",
"role": "Reference Manager",
"category": "Academic References",
"installer": {
"type": "pacman",
"pkg": "zotero"
},
"launcher": {
"type": "desktop",
"cmd": "zotero",
"default_port": None
},
"deps": [],
"description": "Free reference management for researchers.",
"license": 'AGPL-3.0',
"flags": {
"has_cli": False,
"has_gui": True,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'calibre': {
"name": "Calibre",
"level": 10,
"layer": "Knowledge Management",
"role": "Library Manager",
"category": "Ebook Library",
"installer": {
"type": "pacman",
"pkg": "calibre"
},
"launcher": {
"type": "desktop",
"cmd": "calibre",
"default_port": None
},
"deps": [],
"description": "E-book library management and converter.",
"license": 'GPL-3.0',
"flags": {
"has_cli": True,
"has_gui": True,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'paperlessngx': {
"name": "Paperless-ngx",
"level": 10,
"layer": "Knowledge Management",
"role": "Document Archive",
"category": "Document Management",
"installer": {
"type": "git",
"pkg": "https://github.com/paperless-ngx/paperless-ngx"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/paperlessngx && python3 manage.py runserver 0.0.0.0:{port}",
"default_port": 8000
},
"deps": [
"postgresql",
"redis"
],
"description": "Document management system with OCR.",
"license": 'GPL-3.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'logseq': {
"name": "Logseq",
"level": 10,
"layer": "Knowledge Management",
"role": "Knowledge Graph",
"category": "Outliner",
"installer": {
"type": "npm",
"pkg": "logseq"
},
"launcher": {
"type": "desktop",
"cmd": "logseq",
"default_port": None
},
"deps": [],
"description": "Privacy-first knowledge graph outliner.",
"license": 'AGPL-3.0',
"flags": {
"has_cli": False,
"has_gui": True,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'joplin': {
"name": "Joplin",
"level": 10,
"layer": "Knowledge Management",
"role": "Note Taking",
"category": "Notes",
"installer": {
"type": "pacman",
"pkg": "joplin"
},
"launcher": {
"type": "desktop",
"cmd": "joplin",
"default_port": None
},
"deps": [],
"description": "Open-source note taking and to-do application.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": True,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'chromadb': {
"name": "ChromaDB",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Vector Store",
"installer": {
"type": "uv",
"pkg": "chromadb"
},
"launcher": {
"type": "tmux",
"cmd": "chroma run --path {models_root}/chroma --port {port}",
"default_port": 8000
},
"deps": [],
"description": "AI-native open-source vector database.",
"license": 'Apache-2.0',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'lancedb': {
"name": "LanceDB",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Vector Store",
"installer": {
"type": "uv",
"pkg": "lancedb"
},
"launcher": {
"type": "tmux",
"cmd": "python3 -m lancedb serve --port {port}",
"default_port": 8484
},
"deps": [],
"description": "Serverless vector database for AI applications.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'qdrant': {
"name": "Qdrant",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Vector Store",
"installer": {
"type": "script",
"cmd": "curl -L https://github.com/qdrant/qdrant/releases/latest/download/qdrant-x86_64-unknown-linux-musl.tar.gz | tar xz -C {tools_root}/qdrant && chmod +x {tools_root}/qdrant/qdrant"
},
"launcher": {
"type": "tmux",
"cmd": "./qdrant --storage-path {models_root}/qdrant --host 127.0.0.1 --port {port}",
"default_port": 6333
},
"deps": [],
"description": "High-performance vector database with mmap storage, payload filtering, and multi-vector support.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
},
"filesystem": {
"install": "tools/qdrant",
"data": "data/qdrant",
"logs": "logs/qdrant"
}
},
'neo4j': {
"name": "Neo4j",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Graph Database",
"installer": {
"type": "pacman",
"pkg": "neo4j"
},
"launcher": {
"type": "systemd",
"cmd": "neo4j",
"default_port": 7474
},
"deps": [],
"description": "Native graph database and knowledge graph engine.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'elasticsearch': {
"name": "Elasticsearch",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Search Engine",
"installer": {
"type": "pacman",
"pkg": "elasticsearch"
},
"launcher": {
"type": "systemd",
"cmd": "elasticsearch",
"default_port": 9200
},
"deps": [],
"description": "Distributed search and analytics engine.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'meilisearch': {
"name": "Meilisearch",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Search Engine",
"installer": {
"type": "script",
"cmd": "curl -L https://install.meilisearch.com | sh"
},
"launcher": {
"type": "tmux",
"cmd": "meilisearch --port {port}",
"default_port": 7700
},
"deps": [],
"description": "Fast, relevant, and typo-tolerant search engine.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'graphrag': {
"name": "GraphRAG",
"level": 10,
"layer": "Knowledge Management",
"role": "Knowledge Synthesis",
"category": "Graph RAG",
"installer": {
"type": "uv",
"pkg": "graphrag"
},
"launcher": {
"type": "desktop",
"cmd": "python3 -m graphrag init",
"default_port": None
},
"deps": [],
"description": "Microsoft GraphRAG for knowledge graph construction.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'turbovec': {
"name": "TurboVec",
"level": 10,
"layer": "Knowledge Management",
"role": "Embedding",
"category": "Vector Engine",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/turbovec"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/turbovec && python3 serve.py --port {port}",
"default_port": 8101
},
"deps": [
"cuda"
],
"description": "High-speed embedding generation and vector engine.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'airweave': {
"name": "Airweave",
"level": 10,
"layer": "Knowledge Management",
"role": "Integration",
"category": "Data Sync",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/airweave"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/airweave && python3 -m airweave serve --port {port}",
"default_port": 8600
},
"deps": [],
"description": "Real-time data synchronization and integration layer.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'crawl4ai': {
"name": "Crawl4AI",
"level": 10,
"layer": "Knowledge Management",
"role": "Data Harvesting",
"category": "Web Crawler",
"installer": {
"type": "uv",
"pkg": "crawl4ai"
},
"launcher": {
"type": "desktop",
"cmd": "crawl4ai https://example.com",
"default_port": None
},
"deps": [],
"description": "LLM-friendly web crawler and data extractor.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'docling': {
"name": "Docling",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "File Parsing",
"installer": {
"type": "uv",
"pkg": "docling"
},
"launcher": {
"type": "tmux",
"cmd": "docling",
"default_port": None
},
"deps": [],
"description": "Advanced document parsing and chunking.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'markitdown': {
"name": "MarkItDown",
"level": 10,
"layer": "Knowledge Management",
"role": "File Parsing",
"category": "Document Converter",
"installer": {
"type": "uv",
"pkg": "markitdown"
},
"launcher": {
"type": "desktop",
"cmd": "markitdown document.pdf",
"default_port": None
},
"deps": [],
"description": "Microsoft tool to convert files to Markdown.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'opendataloader': {
"name": "OpenDataLoader",
"level": 10,
"layer": "Knowledge Management",
"role": "Ingestion",
"category": "Data Pipeline",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/opendataloader"
},
"launcher": {
"type": "desktop",
"cmd": "python3 -m opendataloader --help",
"default_port": None
},
"deps": [],
"description": "Universal data loading and preprocessing pipeline.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'whisper': {
"name": "Whisper",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Audio Parsing",
"installer": {
"type": "uv",
"pkg": "openai-whisper"
},
"launcher": {
"type": "tmux",
"cmd": "whisper",
"default_port": None
},
"deps": [],
"description": "Robust Speech Recognition via Large-Scale Weak Supervision.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'mnemosyne': {
"name": "Mnemosyne",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Spaced Repetition",
"installer": {
"type": "pipx",
"pkg": "mnemosyne"
},
"launcher": {
"type": "desktop",
"cmd": "mnemosyne",
"default_port": None
},
"deps": [],
"description": "Spaced repetition flashcard program with AI integration.",
"license": 'MIT',
"flags": {
"has_cli": False,
"has_gui": True,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'mnemo_cortex': {
"name": "Mnemo Cortex",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Cortex Memory",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/mnemo-cortex"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/mnemo_cortex && python3 -m mnemo_cortex serve --port {port}",
"default_port": 7200
},
"deps": [
"ollama"
],
"description": "Hierarchical cortex memory for AI agents.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'everos_memory': {
"name": "EverOS Memory",
"level": 10,
"layer": "Knowledge Management",
"role": "Memory",
"category": "Persistent Memory",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/everos-memory"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/everos_memory && python3 -m everos serve --port {port}",
"default_port": 9200
},
"deps": [],
"description": "Persistent long-term memory system for AI agents.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'mirofish': {
"name": "Mirofish",
"level": 10,
"layer": "Knowledge Management",
"role": "Transform",
"category": "Data Pipeline",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/mirofish"
},
"launcher": {
"type": "desktop",
"cmd": "mirofish --help",
"default_port": None
},
"deps": [],
"description": "Data transformation and ETL pipeline framework.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'opendataloader_pdf': {
"name": "OpenDataLoader PDF",
"level": 10,
"layer": "Knowledge Management",
"role": "Extraction",
"category": "PDF Pipeline",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/opendataloader-pdf"
},
"launcher": {
"type": "desktop",
"cmd": "opendataloader-pdf extract file.pdf",
"default_port": None
},
"deps": [],
"description": "Specialized PDF extraction and data loading pipeline.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'understand_anything': {
"name": "Understand Anything",
"level": 10,
"layer": "Knowledge Management",
"role": "Comprehension",
"category": "Document Understanding",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/understand-anything"
},
"launcher": {
"type": "desktop",
"cmd": "understand-anything analyze file.pdf",
"default_port": None
},
"deps": [
"ollama"
],
"description": "Universal document understanding and summarization.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
}

238
src/ai_lsc/registry/layers/observability.py Executable file
View File

@ -0,0 +1,238 @@
"""Registry entries for the Observability layer (L7).
Contains metrics, dashboards, tracing, AI monitoring, and LLM evaluation tools.
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'btop': {
"name": "Btop",
"level": 7,
"layer": "Observability",
"role": "Dashboard",
"category": "Metrics",
"installer": {
"type": "pacman",
"pkg": "btop"
},
"launcher": {
"type": "desktop",
"cmd": "x-terminal-emulator -e btop",
"default_port": None
},
"deps": [],
"description": "Resource monitor that shows usage and stats.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'glances': {
"name": "Glances",
"level": 7,
"layer": "Observability",
"role": "Dashboard",
"category": "Metrics",
"installer": {
"type": "pacman",
"pkg": "glances"
},
"launcher": {
"type": "tmux",
"cmd": "glances -w --port {port}",
"default_port": 61208
},
"deps": [],
"description": "Cross-platform system monitoring tool.",
"license": 'LGPL-3.0',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'prometheus': {
"name": "Prometheus",
"level": 7,
"layer": "Observability",
"role": "Metrics Collector",
"category": "Metrics",
"installer": {
"type": "pacman",
"pkg": "prometheus"
},
"launcher": {
"type": "systemd",
"cmd": "prometheus",
"default_port": 9090
},
"deps": [],
"description": "Open-source monitoring and alerting toolkit.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'grafana': {
"name": "Grafana",
"level": 7,
"layer": "Observability",
"role": "Dashboard",
"category": "Visualization",
"installer": {
"type": "pacman",
"pkg": "grafana"
},
"launcher": {
"type": "systemd",
"cmd": "grafana-server",
"default_port": 3000
},
"deps": [],
"description": "Multi-source observability dashboards and visualization.",
"license": 'AGPL-3.0',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'grafana_alloy': {
"name": "Grafana Alloy",
"level": 7,
"layer": "Observability",
"role": "Collector",
"category": "Telemetry",
"installer": {
"type": "script",
"cmd": "curl -fsSL https://raw.githubusercontent.com/grafana/alloy/main/install.sh | sh"
},
"launcher": {
"type": "tmux",
"cmd": "alloy run --server.http.listen-port={port}",
"default_port": 12345
},
"deps": [
"prometheus"
],
"description": "OpenTelemetry collector with Prometheus integration.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'opik': {
"name": "Opik",
"level": 7,
"layer": "Observability",
"role": "LLM Tracing",
"category": "AI Observability",
"installer": {
"type": "uv",
"pkg": "opik"
},
"launcher": {
"type": "tmux",
"cmd": "opik serve --port {port}",
"default_port": 3000
},
"deps": [],
"description": "Open-source LLM observability and tracing platform.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'pulse_ai': {
"name": "Pulse AI",
"level": 7,
"layer": "Observability",
"role": "Health Monitor",
"category": "AI Monitoring",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/pulse-ai"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/pulse_ai && python3 -m pulse serve --port {port}",
"default_port": 8900
},
"deps": [],
"description": "AI service health monitoring and auto-recovery.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'latitude': {
"name": "Latitude",
"level": 7,
"layer": "Observability",
"role": "Evaluation",
"category": "LLM Evaluation",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/latitude"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/latitude && python3 -m latitude serve --port {port}",
"default_port": 9300
},
"deps": [
"ollama"
],
"description": "LLM output evaluation and benchmarking platform.",
"license": 'Proprietary',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
}

1361
src/ai_lsc/registry/layers/orchestrators.py Executable file
View File

File diff suppressed because it is too large Load Diff

180
src/ai_lsc/registry/layers/security.py Executable file
View File

@ -0,0 +1,180 @@
"""Registry entries for the Security layer (L6).
Contains identity management, secrets management, container scanning,
intrusion prevention, antivirus, and policy enforcement tools for
the local AI infrastructure.
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'keycloak': {
"name": "Keycloak",
"level": 6,
"layer": "Security",
"role": "Identity",
"category": "Auth",
"installer": {
"type": "custom",
"pkg": "keycloak"
},
"launcher": {
"type": "tmux",
"cmd": "kc start-dev --http-port={port}",
"default_port": 8081
},
"deps": ["java"],
"description": "Open-source identity and access management for modern applications.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'vault': {
"name": "HashiCorp Vault",
"level": 6,
"layer": "Security",
"role": "Secrets",
"category": "Secrets Management",
"installer": {
"type": "pacman",
"pkg": "vault"
},
"launcher": {
"type": "tmux",
"cmd": "vault server -dev -dev-listen-address=127.0.0.1:{port}",
"default_port": 8200
},
"deps": [],
"description": "Secrets management and data protection for AI API keys and credentials.",
"license": 'BSL-1.1',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'trivy': {
"name": "Trivy",
"level": 6,
"layer": "Security",
"role": "Scanner",
"category": "Container Security",
"installer": {
"type": "pacman",
"pkg": "trivy"
},
"launcher": {
"type": "desktop",
"cmd": "trivy --version",
"default_port": None
},
"deps": [],
"description": "Scanner for vulnerabilities in container images, filesystems, and git repositories.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'fail2ban': {
"name": "Fail2Ban",
"level": 6,
"layer": "Security",
"role": "IDS",
"category": "Intrusion Prevention",
"installer": {
"type": "pacman",
"pkg": "fail2ban"
},
"launcher": {
"type": "desktop",
"cmd": "fail2ban-client status",
"default_port": None
},
"deps": [],
"description": "Intrusion prevention framework that protects AI service endpoints from brute-force attacks.",
"license": 'GPL-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'clamav': {
"name": "ClamAV",
"level": 6,
"layer": "Security",
"role": "Scanner",
"category": "Antivirus",
"installer": {
"type": "pacman",
"pkg": "clamav"
},
"launcher": {
"type": "desktop",
"cmd": "freshclam && clamscan --version",
"default_port": None
},
"deps": [],
"description": "Open-source antivirus engine for scanning uploaded documents and datasets.",
"license": 'GPL-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'opa': {
"name": "Open Policy Agent",
"level": 6,
"layer": "Security",
"role": "Policy",
"category": "Policy Engine",
"installer": {
"type": "pacman",
"pkg": "opa"
},
"launcher": {
"type": "tmux",
"cmd": "opa run --server --addr=0.0.0.0:{port}",
"default_port": 8181
},
"deps": [],
"description": "General-purpose policy engine for unified authorization and access control across AI services.",
"license": 'Apache-2.0',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
}

497
src/ai_lsc/registry/layers/user_interfaces.py Executable file
View File

@ -0,0 +1,497 @@
"""Registry entries for the User Interfaces layer (L8).
Contains frontends, dashboards, chat UIs, image generation interfaces,
sensory interfaces (vision, speech, voice), and knowledge graph tools.
This module is consumed by
:mod:`ai_lsc.registry.loader`.
"""
TOOLS: dict[str, dict] = {
'openwebui': {
"name": "Open WebUI",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Chat Frontend",
"installer": {
"type": "uv",
"pkg": "open-webui"
},
"launcher": {
"type": "tmux",
"cmd": "open-webui serve --port {port} --data-dir {workspaces_root}/openwebui",
"default_port": 8080
},
"deps": [
"ollama"
],
"description": "Extensible frontend for LLMs.",
"license": 'MIT',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'anythingllm': {
"name": "AnythingLLM",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Chat",
"installer": {
"type": "git_node",
"pkg": "https://github.com/Mintplex-Labs/anything-llm.git"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/anythingllm && yarn dev",
"default_port": 3001
},
"deps": [],
"description": "Full-stack application for conversational AI.",
"license": 'MIT',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'librechat': {
"name": "LibreChat",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Chat Agent Platform",
"installer": {
"type": "git_node",
"pkg": "https://github.com/danny-avila/LibreChat.git",
"update_cmd": "git pull --ff-only && yarn install && yarn build"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/librechat && API_PLUGINS=false PORT={port} NODE_ENV=production yarn backend",
"default_port": 3080
},
"deps": [
"ollama"
],
"description": "Multi-provider chat agent platform with native OpenAI tool-calling, the default agent frontend for AI-LSC's agentic orchestration.",
"license": 'MIT',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
},
"filesystem": {
"install": "tools/librechat",
"config": "configs/librechat",
"data": "data/librechat",
"logs": "logs/librechat"
}
},
'flowise': {
"name": "Flowise",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Workflow",
"installer": {
"type": "npm",
"pkg": "flowise"
},
"launcher": {
"type": "tmux",
"cmd": "npx flowise start --port {port}",
"default_port": 3000
},
"deps": [],
"description": "Drag & drop UI to build customized LLM flows.",
"license": 'Apache-2.0',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'invokeai': {
"name": "InvokeAI",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Image Generation",
"installer": {
"type": "git",
"pkg": "https://github.com/invoke-ai/InvokeAI"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/invokeai && invokeai --host 0.0.0.0 --port {port}",
"default_port": 9090
},
"deps": [
"cuda"
],
"description": "Professional AI image generation workspace.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": True,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'forge': {
"name": "Forge (A1111)",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Image Generation",
"installer": {
"type": "git",
"pkg": "https://github.com/AUTOMATIC1111/stable-diffusion-webui-forge"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/forge && python3 launch.py --port {port}",
"default_port": 7860
},
"deps": [
"cuda"
],
"description": "Stable Diffusion WebUI Forge (optimized fork).",
"license": 'AGPL-3.0',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'dashy': {
"name": "Dashy",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Homepage",
"installer": {
"type": "npm",
"pkg": "dashy"
},
"launcher": {
"type": "tmux",
"cmd": "dashy --port {port}",
"default_port": 3000
},
"deps": [],
"description": "Highly customizable dashboard and homepage.",
"license": 'Proprietary',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'obsidian': {
"name": "Obsidian",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Knowledge Graph Notes",
"installer": {
"type": "pacman",
"pkg": "obsidian"
},
"launcher": {
"type": "desktop",
"cmd": "obsidian",
"default_port": None
},
"deps": [],
"description": "Knowledge graph note-taking and markdown editor.",
"license": 'Proprietary',
"flags": {
"has_cli": False,
"has_gui": True,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'hermes': {
"name": "Hermes",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Ecosystem Dashboard",
"installer": {
"type": "npm",
"pkg": "hermes-ai"
},
"launcher": {
"type": "tmux",
"cmd": "hermes dashboard --port {port} --data-dir {workspaces_root}/hermes & hermes desktop --data-dir {workspaces_root}/hermes",
"default_port": 17050
},
"deps": [
"ollama"
],
"description": "Unified desktop and dashboard environment for the AI ecosystem.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": True,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'hermes_desktop': {
"name": "Hermes Desktop",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Desktop Agent",
"installer": {
"type": "npm",
"pkg": "hermes-desktop"
},
"launcher": {
"type": "desktop",
"cmd": "hermes desktop",
"default_port": None
},
"deps": [
"ollama"
],
"description": "Hermes desktop agent environment.",
"license": 'MIT',
"flags": {
"has_cli": False,
"has_gui": True,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'hermes_dashboard_page': {
"name": "Hermes Dashboard",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "Dashboard",
"installer": {
"type": "npm",
"pkg": "hermes-dashboard"
},
"launcher": {
"type": "tmux",
"cmd": "hermes dashboard --port {port}",
"default_port": 17050
},
"deps": [
"ollama"
],
"description": "Hermes ecosystem monitoring dashboard.",
"license": 'Proprietary',
"flags": {
"has_cli": False,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'local_llm_launcher': {
"name": "Local LLM Launcher",
"level": 8,
"layer": "User Interfaces",
"role": "Face",
"category": "LLM GUI",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/local-llm-launcher-gui"
},
"launcher": {
"type": "desktop",
"cmd": "cd {tools_root}/local_llm_launcher && python3 main.py",
"default_port": None
},
"deps": [
"ollama"
],
"description": "GUI launcher and manager for local LLMs.",
"license": 'Proprietary',
"flags": {
"has_cli": False,
"has_gui": True,
"has_web": False,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'openjarvis': {
"name": "OpenJarvis",
"level": 8,
"layer": "User Interfaces",
"role": "Central Intelligence",
"category": "AI Assistant Platform",
"installer": {
"type": "git_node",
"pkg": "https://github.com/openjarvis/openjarvis.git",
"post_install": "npm install && npm run build"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/openjarvis && npm start -- --port {port}",
"default_port": 17070
},
"deps": [
"ollama",
"qdrant"
],
"description": "Central AI assistant platform with multi-modal I/O, memory integration, agentic task execution, and unified dashboard. The brain of the intelligent stack.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": True,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
},
"filesystem": {
"install": "tools/openjarvis",
"config": "configs/openjarvis",
"data": "workspaces/openjarvis",
"cache": "cache/openjarvis",
"logs": "logs/openjarvis"
}
},
'deep_eye': {
"name": "Deep Eye",
"level": 8,
"layer": "User Interfaces",
"role": "Vision",
"category": "Computer Vision",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/deep-eye"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/deep_eye && python3 serve.py --port {port}",
"default_port": 8100
},
"deps": [
"ollama"
],
"description": "Local computer vision analysis and description engine.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'parakeet': {
"name": "Parakeet.cpp",
"level": 8,
"layer": "User Interfaces",
"role": "Senses",
"category": "Speech Recognition",
"installer": {
"type": "git",
"pkg": "https://github.com/nicely-done/parakeet.cpp"
},
"launcher": {
"type": "tmux",
"cmd": "cd {tools_root}/parakeet && ./parakeet --port {port}",
"default_port": 8300
},
"deps": [
"cuda"
],
"description": "C++ speech recognition with transformer architecture.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
'luxtts': {
"name": "LuxTTS",
"level": 8,
"layer": "User Interfaces",
"role": "Voice",
"category": "Text-to-Speech",
"installer": {
"type": "uv",
"pkg": "luxtts"
},
"launcher": {
"type": "tmux",
"cmd": "luxtts serve --port {port}",
"default_port": 8500
},
"deps": [],
"description": "High-quality local text-to-speech synthesis.",
"license": 'MIT',
"flags": {
"has_cli": True,
"has_gui": False,
"has_web": True,
"is_ollama": False,
"is_passive": False,
"is_mcp": False,
"is_skills_collection": False
}
},
}

368
src/ai_lsc/registry/license_gate.py Executable file
View File

@ -0,0 +1,368 @@
"""AI-LSC — License acceptance gate.
Sits between the user's "Install" click and the actual installer
strategy dispatch. For every tool installation, the gate checks the
tool's ``license`` SPDX ID against three sources:
1. **SaaS blocklist** (in :mod:`ai_lsc.registry.validator`) — if the
tool_id is blocked, raise :class:`LicenseBlocked` immediately. No
dialog, no acceptance, no install.
2. **Auto-approval registry** (``config/license_approvals.json``) — a
user-editable list of OSI-approved SPDX IDs that have been
pre-approved. If the tool's license is in this list, install
proceeds without a dialog. Only OSI-approved licenses can appear
here; :meth:`LicenseGate.add_auto_approval` rejects attempts to
auto-approve source-available or proprietary licenses.
3. **Per-tool acceptance registry** (``config/license_acceptances.json``)
— auto-managed by the gate; records every per-tool acceptance the
user has made so they aren't prompted twice for the same tool.
If none of the three sources cover the tool, the gate raises
:class:`LicenseAcceptanceRequired` (carrying the license info). The
UI catches this exception and shows the
:class:`~ai_lsc.ui.dialogs.license_dialog.LicenseAcceptanceDialog`.
On dialog accept, the UI calls :meth:`LicenseGate.accept` and retries
the install.
Files managed
-------------
* ``config/license_approvals.json`` — ``{"licenses": ["MIT", "Apache-2.0"], "updated_at": "..."}``
* ``config/license_acceptances.json`` — ``{"ollama": {"spdx": "MIT", "accepted_at": "...", "via": "auto-approved"}, ...}``
"""
from __future__ import annotations
import json
import os
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from ai_lsc.registry.licenses import (
CATALOG,
Category,
LicenseInfo,
category_for,
get_license_info,
)
from ai_lsc.registry.validator import SAAS_BLOCKLIST
from ai_lsc.utils.logging import get_logger
logger = get_logger(__name__)
# ── Exceptions ────────────────────────────────────────────────────────
class LicenseError(Exception):
"""Base class for license-gate failures."""
@dataclass
class LicenseBlocked(LicenseError):
"""Raised when the tool_id is on the SaaS blocklist."""
tool_id: str
reason: str = ""
def __str__(self) -> str:
return (
f"Tool {self.tool_id!r} is blocked — {self.reason or 'on SaaS-only blocklist'}"
)
@dataclass
class LicenseAcceptanceRequired(LicenseError):
"""Raised when the tool's license has not been accepted yet.
The UI catches this and shows the
:class:`~ai_lsc.ui.dialogs.license_dialog.LicenseAcceptanceDialog`.
"""
tool_id: str
license_info: LicenseInfo
def __str__(self) -> str:
return (
f"Tool {self.tool_id!r} requires license acceptance: "
f"{self.license_info.name} ({self.license_info.spdx})"
)
# ── Result enum ───────────────────────────────────────────────────────
@dataclass
class GateResult:
"""Outcome of a license-gate check."""
status: str # "accepted" | "auto_approved" | "needs_acceptance" | "blocked"
tool_id: str
spdx: str = ""
license_info: LicenseInfo | None = None
reason: str = ""
@property
def can_install(self) -> bool:
return self.status in ("accepted", "auto_approved")
# ── The gate ──────────────────────────────────────────────────────────
class LicenseGate:
"""License acceptance gate.
Parameters
----------
config_dir :
Directory where ``license_approvals.json`` and
``license_acceptances.json`` live. Defaults to the AI-LSC
``config/`` directory under ``BASE_DIR``.
"""
APPROVALS_FILE = "license_approvals.json"
ACCEPTANCES_FILE = "license_acceptances.json"
def __init__(self, config_dir: str | Path | None = None) -> None:
if config_dir is None:
from ai_lsc.constants import BASE_DIR
config_dir = os.path.join(BASE_DIR, "config")
self.config_dir = Path(config_dir)
self.config_dir.mkdir(parents=True, exist_ok=True)
self._approvals_path = self.config_dir / self.APPROVALS_FILE
self._acceptances_path = self.config_dir / self.ACCEPTANCES_FILE
# ── Public API ───────────────────────────────────────────────────
def check(self, tool_id: str, spdx: str) -> GateResult:
"""Check whether the tool can be installed under its license.
Returns a :class:`GateResult`. The caller should:
* if ``result.can_install`` → proceed with the install.
* if ``result.status == "blocked"`` → log + abort; do NOT
retry.
* if ``result.status == "needs_acceptance"`` → raise
:class:`LicenseAcceptanceRequired` (or catch + show dialog).
"""
# 1. SaaS blocklist — hard block, no acceptance possible.
if tool_id.lower() in SAAS_BLOCKLIST:
return GateResult(
status="blocked",
tool_id=tool_id,
spdx=spdx,
license_info=get_license_info(spdx),
reason=(
f"tool_id {tool_id!r} is on the SaaS-only blocklist"
),
)
# 2. Look up the license info. Unknown SPDX → treat as
# PROPRIETARY (defensive — unknown = restricted).
info = get_license_info(spdx)
if info is None:
info = CATALOG["Proprietary"]
logger.warning(
"Tool %r has unknown license SPDX %r — treating as "
"Proprietary (defensive). Add the SPDX to "
"registry/licenses.py to fix.",
tool_id, spdx,
)
# 3. Auto-approval registry — only OSI licenses can be here,
# but double-check in case the file was hand-edited.
if spdx in self._load_approvals():
if info.category is Category.OSI:
return GateResult(
status="auto_approved",
tool_id=tool_id,
spdx=spdx,
license_info=info,
reason=f"auto-approved via {spdx}",
)
# Non-OSI license in the approvals file — ignore it +
# log a warning so the user can fix the file.
logger.warning(
"License %r is in the auto-approvals registry but is "
"not OSI-approved (category=%s). Ignoring — this "
"license requires individual acceptance.",
spdx, info.category.value,
)
# 4. Per-tool acceptance registry.
acceptances = self._load_acceptances()
record = acceptances.get(tool_id)
if record and record.get("spdx") == spdx:
return GateResult(
status="accepted",
tool_id=tool_id,
spdx=spdx,
license_info=info,
reason=f"accepted via {record.get('via', 'individual')} at {record.get('accepted_at', '?')}",
)
# 5. Needs acceptance — caller raises LicenseAcceptanceRequired.
return GateResult(
status="needs_acceptance",
tool_id=tool_id,
spdx=spdx,
license_info=info,
)
def accept(
self,
tool_id: str,
spdx: str,
*,
via: str = "individual",
) -> None:
"""Record that the user has accepted the tool's license.
Parameters
----------
tool_id :
The tool whose license was accepted.
spdx :
The SPDX ID that was accepted (recorded so a license
change later re-prompts).
via :
How the acceptance happened — ``"individual"`` (dialog),
``"auto-approved"`` (was in the approvals registry at
check time but we're recording it for audit), or
``"cli"`` (accepted via a non-UI code path).
"""
acceptances = self._load_acceptances()
acceptances[tool_id] = {
"spdx": spdx,
"accepted_at": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
"via": via,
}
self._save_acceptances(acceptances)
logger.info(
"License %r accepted for tool %r (via %s)",
spdx, tool_id, via,
)
def add_auto_approval(self, spdx: str) -> None:
"""Add an SPDX ID to the auto-approval registry.
Raises :class:`ValueError` if the license is not OSI-approved
(source-available and proprietary licenses cannot be
auto-approved — the user must accept each tool individually).
"""
info = get_license_info(spdx)
if info is None:
raise ValueError(
f"Unknown license SPDX {spdx!r} — not in the catalog. "
f"Add it to registry/licenses.py first."
)
if info.category is not Category.OSI:
raise ValueError(
f"License {spdx!r} ({info.name}) is {info.category.value} "
f"— only OSI-approved open-source licenses can be "
f"auto-approved. Source-available and proprietary "
f"licenses require per-tool acceptance."
)
approvals = self._load_approvals()
if spdx not in approvals:
approvals.append(spdx)
self._save_approvals(approvals)
logger.info("License %r added to auto-approvals registry", spdx)
def remove_auto_approval(self, spdx: str) -> None:
"""Remove an SPDX ID from the auto-approval registry."""
approvals = self._load_approvals()
if spdx in approvals:
approvals.remove(spdx)
self._save_approvals(approvals)
logger.info("License %r removed from auto-approvals registry", spdx)
def revoke_acceptance(self, tool_id: str) -> None:
"""Revoke a per-tool acceptance (the user will be re-prompted
on next install)."""
acceptances = self._load_acceptances()
if tool_id in acceptances:
del acceptances[tool_id]
self._save_acceptances(acceptances)
logger.info("License acceptance revoked for tool %r", tool_id)
def list_auto_approvals(self) -> list[str]:
"""Return the current list of auto-approved SPDX IDs."""
return list(self._load_approvals())
def list_acceptances(self) -> dict[str, dict[str, Any]]:
"""Return the current per-tool acceptance registry."""
return dict(self._load_acceptances())
# ── Internal: file I/O ──────────────────────────────────────────
def _load_approvals(self) -> list[str]:
if not self._approvals_path.exists():
return []
try:
data = json.loads(self._approvals_path.read_text(encoding="utf-8"))
licenses = data.get("licenses", [])
if isinstance(licenses, list):
return [str(x) for x in licenses]
except (OSError, ValueError, json.JSONDecodeError) as exc:
logger.warning("Failed to load %s: %s", self._approvals_path, exc)
return []
def _save_approvals(self, licenses: list[str]) -> None:
data = {
"licenses": list(licenses),
"updated_at": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
"_comment": (
"Auto-approved SPDX IDs. Only OSI-approved open-source "
"licenses can appear here — source-available and "
"proprietary licenses require per-tool acceptance. "
"Edit manually or via the LicenseAcceptanceDialog."
),
}
self._approvals_path.write_text(
json.dumps(data, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
def _load_acceptances(self) -> dict[str, dict[str, Any]]:
if not self._acceptances_path.exists():
return {}
try:
data = json.loads(self._acceptances_path.read_text(encoding="utf-8"))
if isinstance(data, dict):
# Accept both {tool_id: {...}} and {"acceptances": {...}}
if "acceptances" in data and isinstance(data["acceptances"], dict):
return data["acceptances"]
return data
except (OSError, ValueError, json.JSONDecodeError) as exc:
logger.warning("Failed to load %s: %s", self._acceptances_path, exc)
return {}
def _save_acceptances(self, acceptances: dict[str, dict[str, Any]]) -> None:
data = {
"acceptances": acceptances,
"updated_at": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
"_comment": (
"Per-tool license acceptance registry. Auto-managed "
"by LicenseGate.accept(). Each entry records the SPDX "
"ID accepted, the timestamp, and how the acceptance "
"happened (individual / auto-approved / cli)."
),
}
self._acceptances_path.write_text(
json.dumps(data, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
__all__ = [
"LicenseGate",
"LicenseError",
"LicenseBlocked",
"LicenseAcceptanceRequired",
"GateResult",
]

407
src/ai_lsc/registry/licenses.py Executable file
View File

@ -0,0 +1,407 @@
"""AI-LSC — License catalog + acceptance gate.
Defines the three license categories the AI-LSC license gate recognizes:
1. **OSI-approved open source** (``Category.OSI``) — licenses that pass
the Open Source Initiative's approval criteria. These CAN be
auto-approved by the user via the license-approvals registry.
Examples: MIT, Apache-2.0, GPL-3.0, AGPL-3.0, BSD-3-Clause, MPL-2.0.
2. **Source-available / fair-code** (``Category.SOURCE_AVAILABLE``) —
licenses that publish source but impose additional restrictions
(field-of-use limits, commercial-use cliffs, managed-service
restrictions). These CANNOT be auto-approved; the user must accept
each tool individually. Examples: BSL-1.1, SSPL, RSALv2,
Sustainable Use License, Dify Open Source License.
3. **Proprietary / ToS-governed** (``Category.PROPRIETARY``) —
closed-source tools whose use is governed by a vendor Terms-of-
Service agreement. These CANNOT be auto-approved and always show
a prominent disclaimer warning about ToS restrictions before
install. Example: Claude Code (Anthropic ToS).
The license-approvals registry (``config/license_approvals.json``) is
a user-editable list of SPDX IDs that have been pre-approved. Only
OSI-approved licenses can appear in this list — the
:func:`LicenseGate.add_auto_approval` method rejects attempts to
auto-approve source-available or proprietary licenses.
The license-acceptances registry (``config/license_acceptances.json``)
is auto-managed by the gate and records every per-tool acceptance the
user has made (so they aren't prompted twice for the same tool).
"""
from __future__ import annotations
import enum
import json
import os
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Any
class Category(enum.Enum):
"""License category — drives the gate's acceptance flow."""
OSI = "osi"
SOURCE_AVAILABLE = "source_available"
PROPRIETARY = "proprietary"
@property
def can_auto_approve(self) -> bool:
"""True if the user is allowed to add this license to the
auto-approval registry."""
return self is Category.OSI
@property
def needs_disclaimer(self) -> bool:
"""True if the acceptance dialog should show a prominent
ToS/disclaimer warning."""
return self in (Category.SOURCE_AVAILABLE, Category.PROPRIETARY)
@dataclass(frozen=True)
class LicenseInfo:
"""Static information about a single license."""
spdx: str
"""SPDX identifier (e.g. ``"MIT"``, ``"Apache-2.0"``, ``"Proprietary"``).
For non-SPDX licenses (Dify OSL, Sustainable Use License), use the
canonical short name."""
name: str
"""Human-readable license name (e.g. ``"MIT License"``,
``"Apache License 2.0"``)."""
category: Category
"""Which acceptance-flow category this license belongs to."""
url: str
"""URL to the full license text (or the vendor's ToS page for
proprietary licenses)."""
summary: str
"""One-paragraph summary of what the license permits / restricts.
Shown in the acceptance dialog above the full text link."""
disclaimer: str = ""
"""Extra disclaimer shown for source-available / proprietary
licenses. Empty for OSI-approved licenses."""
# ── The catalog ───────────────────────────────────────────────────────
CATALOG: dict[str, LicenseInfo] = {
# ── OSI-approved open source (auto-approvable) ───────────────────
"MIT": LicenseInfo(
spdx="MIT",
name="MIT License",
category=Category.OSI,
url="https://opensource.org/licenses/MIT",
summary=(
"Permissive license allowing almost any use — commercial, "
"private, modification, distribution — provided the copyright "
"notice and license text are included."
),
),
"Apache-2.0": LicenseInfo(
spdx="Apache-2.0",
name="Apache License 2.0",
category=Category.OSI,
url="https://opensource.org/licenses/Apache-2.0",
summary=(
"Permissive license allowing commercial use, modification, "
"and distribution with patent grant. Requires copyright "
"notice + license text + notice of changes."
),
),
"GPL-2.0": LicenseInfo(
spdx="GPL-2.0",
name="GNU General Public License v2.0",
category=Category.OSI,
url="https://opensource.org/licenses/GPL-2.0",
summary=(
"Copyleft license requiring that derivative works be "
"distributed under the same GPL-2.0 terms, with source code."
),
),
"GPL-3.0": LicenseInfo(
spdx="GPL-3.0",
name="GNU General Public License v3.0",
category=Category.OSI,
url="https://opensource.org/licenses/GPL-3.0",
summary=(
"Copyleft license requiring that derivative works be "
"distributed under the same GPL-3.0 terms, with source code. "
"Includes patent grant + anti-tivoization clauses."
),
),
"AGPL-3.0": LicenseInfo(
spdx="AGPL-3.0",
name="GNU Affero General Public License v3.0",
category=Category.OSI,
url="https://opensource.org/licenses/AGPL-3.0",
summary=(
"Copyleft license like GPL-3.0 but with an additional "
"network-use clause: users interacting with the software "
"over a network are entitled to the source code."
),
),
"LGPL-3.0": LicenseInfo(
spdx="LGPL-3.0",
name="GNU Lesser General Public License v3.0",
category=Category.OSI,
url="https://opensource.org/licenses/LGPL-3.0",
summary=(
"Weak copyleft license allowing linking from proprietary "
"software, but modifications to the LGPL-licensed code itself "
"must be shared under LGPL."
),
),
"BSD-2-Clause": LicenseInfo(
spdx="BSD-2-Clause",
name="BSD 2-Clause License",
category=Category.OSI,
url="https://opensource.org/licenses/BSD-2-Clause",
summary=(
"Permissive license allowing almost any use provided the "
"copyright notice and license text are included."
),
),
"BSD-3-Clause": LicenseInfo(
spdx="BSD-3-Clause",
name="BSD 3-Clause License",
category=Category.OSI,
url="https://opensource.org/licenses/BSD-3-Clause",
summary=(
"Permissive license like BSD-2-Clause but with an additional "
"clause prohibiting use of the copyright holder's name for "
"endorsement."
),
),
"MPL-2.0": LicenseInfo(
spdx="MPL-2.0",
name="Mozilla Public License 2.0",
category=Category.OSI,
url="https://opensource.org/licenses/MPL-2.0",
summary=(
"File-level copyleft: modifications to MPL-licensed files "
"must be shared under MPL, but the rest of the project can "
"be under any license (including proprietary)."
),
),
"ISC": LicenseInfo(
spdx="ISC",
name="ISC License",
category=Category.OSI,
url="https://opensource.org/licenses/ISC",
summary=(
"Permissive license functionally equivalent to MIT/BSD, with "
"simpler language."
),
),
"PostgreSQL": LicenseInfo(
spdx="PostgreSQL",
name="PostgreSQL License",
category=Category.OSI,
url="https://www.postgresql.org/about/licence/",
summary=(
"Permissive BSD-like license specific to PostgreSQL. Allows "
"commercial use, modification, and distribution with "
"copyright notice."
),
),
"Python": LicenseInfo(
spdx="Python",
name="Python Software Foundation License",
category=Category.OSI,
url="https://docs.python.org/3/license.html",
summary=(
"Permissive license (PSF) for CPython and the Python "
"standard library. OSI-approved, GPL-compatible, allows "
"commercial use and modification."
),
),
# ── Source-available / fair-code (NOT auto-approvable) ───────────
"BSL-1.1": LicenseInfo(
spdx="BSL-1.1",
name="Business Source License 1.1",
category=Category.SOURCE_AVAILABLE,
url="https://mariadb.com/bsl11/",
summary=(
"Source-available license that restricts production use for "
"a defined period (typically 4 years) after which it "
"converts to an open-source license (often Apache-2.0 or "
"GPL). Used by HashiCorp Terraform, CockroachDB, etc."
),
disclaimer=(
"This license is NOT OSI-approved. It imposes "
"field-of-use restrictions (you may not use the software to "
"offer a competing managed service). Review the full text "
"before accepting."
),
),
"SSPL": LicenseInfo(
spdx="SSPL",
name="Server Side Public License",
category=Category.SOURCE_AVAILABLE,
url="https://www.mongodb.com/licensing/server-side-public-license",
summary=(
"Source-available license requiring that if you offer the "
"software as a managed service, you must open-source your "
"ENTIRE service stack (including all supporting "
"infrastructure code). Used by MongoDB."
),
disclaimer=(
"This license is NOT OSI-approved. It has an aggressive "
"copyleft reach that extends to your entire service stack "
"if you offer the software as a managed service. Review "
"carefully before accepting."
),
),
"RSALv2": LicenseInfo(
spdx="RSALv2",
name="Redis Source Available License 2.0",
category=Category.SOURCE_AVAILABLE,
url="https://redis.com/legal/rsalv2-agreement/",
summary=(
"Source-available license prohibiting offering the software "
"as a managed service, cloud service, or database service "
"to third parties. Used by Redis (post-7.4)."
),
disclaimer=(
"This license is NOT OSI-approved. It prohibits offering "
"the software as a managed/cloud/database service. Review "
"the full text before accepting."
),
),
"Sustainable-Use": LicenseInfo(
spdx="Sustainable-Use",
name="Sustainable Use License (n8n fair-code)",
category=Category.SOURCE_AVAILABLE,
url="https://github.com/n8n-io/n8n/blob/master/LICENSE.md",
summary=(
"Fair-code license allowing internal and commercial use, "
"but prohibiting offering the software as a hosted service "
"to third parties. Used by n8n."
),
disclaimer=(
"This license is NOT OSI-approved. It prohibits offering "
"the software as a hosted service to third parties. "
"Review the full text before accepting."
),
),
"Dify-OSL": LicenseInfo(
spdx="Dify-OSL",
name="Dify Open Source License",
category=Category.SOURCE_AVAILABLE,
url="https://github.com/langgenius/dify/blob/main/LICENSE",
summary=(
"Custom fair-code license allowing non-production and "
"internal commercial use, but restricting offering Dify as "
"a multi-tenant SaaS. Used by Dify."
),
disclaimer=(
"This license is NOT OSI-approved. It restricts offering "
"the software as a multi-tenant SaaS. Review the full text "
"before accepting."
),
),
# ── Proprietary / ToS-governed (always needs individual acceptance) ──
"Proprietary": LicenseInfo(
spdx="Proprietary",
name="Proprietary License",
category=Category.PROPRIETARY,
url="",
summary=(
"Closed-source software governed by the vendor's Terms of "
"Service. Use is permitted only as expressly allowed by "
"the vendor's ToS."
),
disclaimer=(
"WARNING: This tool is proprietary and governed by the "
"vendor's Terms of Service. AI-LSC forces localhost-only "
"endpoints where possible, but you are responsible for "
"reviewing and complying with the vendor's ToS. Do NOT "
"install if you do not agree to the vendor's ToS."
),
),
"Anthropic-ToS": LicenseInfo(
spdx="Anthropic-ToS",
name="Anthropic Terms of Service",
category=Category.PROPRIETARY,
url="https://www.anthropic.com/legal/terms",
summary=(
"Anthropic's Terms of Service govern use of Claude Code "
"and other Anthropic products. Closed-source, "
"ToS-restricted."
),
disclaimer=(
"WARNING: Claude Code is proprietary software governed by "
"Anthropic's Terms of Service. AI-LSC forces "
"ANTHROPIC_BASE_URL to a localhost LiteLLM proxy by "
"default, but if you override this to call api.anthropic.com "
"directly, you are bound by Anthropic's ToS. Review at "
"anthropic.com/legal/terms before accepting."
),
),
"LMStudio-ToS": LicenseInfo(
spdx="LMStudio-ToS",
name="LM Studio Terms of Service (BLOCKED)",
category=Category.PROPRIETARY,
url="https://lmstudio.ai/terms",
summary=(
"LM Studio's Terms of Service are considered too "
"restrictive for AI-LSC's SaaS-only policy — the tool is "
"on the blocklist and CANNOT be installed."
),
disclaimer=(
"BLOCKED: LM Studio is on the AI-LSC SaaS-only blocklist "
"due to aggressive Terms-of-Service restrictions. This "
"tool cannot be installed through AI-LSC. Use Ollama, "
"vLLM, or llama.cpp as a local alternative."
),
),
}
def get_license_info(spdx: str) -> LicenseInfo | None:
"""Look up license info by SPDX ID. Returns ``None`` if unknown."""
return CATALOG.get(spdx)
def category_for(spdx: str) -> Category:
"""Return the category for an SPDX ID, defaulting to PROPRIETARY
for unknown licenses (defensive — unknown = restricted)."""
info = CATALOG.get(spdx)
if info is None:
return Category.PROPRIETARY
return info.category
def all_licenses() -> dict[str, LicenseInfo]:
"""Return the full catalog (for UI enumeration)."""
return dict(CATALOG)
def osi_approved_spdx_ids() -> list[str]:
"""Return SPDX IDs that are OSI-approved (auto-approvable)."""
return sorted(
spdx for spdx, info in CATALOG.items()
if info.category is Category.OSI
)
__all__ = [
"Category",
"LicenseInfo",
"CATALOG",
"get_license_info",
"category_for",
"all_licenses",
"osi_approved_spdx_ids",
]

105
src/ai_lsc/registry/loader.py Executable file
View File

@ -0,0 +1,105 @@
"""Registry loader -- discovers and merges per-layer registry modules.
On startup the loader scans ``ai_lsc.registry.layers`` for every
``.py`` file that exports a ``TOOLS`` dict, and merges them into a
single unified registry dict.
This replaces the monolithic ``DEFAULT_REGISTRY`` dict with a
zero-merge-conflict modular approach: each layer lives in its own
file and is independently editable.
"""
from __future__ import annotations
import importlib
import logging
import pkgutil
from typing import Any
logger = logging.getLogger(__name__)
# ── Hard blacklist: IDs that must NEVER appear in the registry ──
_BLACKLISTED_IDS = frozenset({
"wayland",
"wayland_compositor",
})
def _evict_blacklisted(registry: dict[str, dict[str, Any]]) -> None:
"""Remove blacklisted tool IDs from the merged registry."""
evicted = [tid for tid in registry if tid in _BLACKLISTED_IDS]
for tid in evicted:
del registry[tid]
logger.warning("Registry blacklist evicted: %s", tid)
def load_merged_registry() -> dict[str, dict[str, Any]]:
"""Discover and merge all layer TOOLS dicts into one registry dict.
Each layer module should export ``TOOLS: dict[str, dict]`` where
keys are tool IDs and values are the full metadata dicts.
Returns the merged ``{tool_id: metadata}`` dictionary, with later
files overriding earlier ones on key collision (which should never
happen if layers are well-separated).
"""
merged: dict[str, dict[str, Any]] = {}
# Import the layers package
try:
layers_pkg = importlib.import_module("ai_lsc.registry.layers")
except ImportError:
return merged
for importer, modname, ispkg in pkgutil.iter_modules(
layers_pkg.__path__, prefix=layers_pkg.__name__ + "."
):
if ispkg:
continue
try:
mod = importlib.import_module(modname)
except ImportError:
continue
tools = getattr(mod, "TOOLS", None)
if isinstance(tools, dict):
# Dict format: {tool_id: metadata, ...}
_merge_tools_dict(merged, tools)
elif isinstance(tools, list):
# Legacy list format: [{...}, ...]
_merge_tools_list(merged, tools)
_evict_blacklisted(merged)
return merged
def _merge_tools_dict(
target: dict[str, dict[str, Any]],
tools: dict[str, dict[str, Any]],
) -> None:
"""Merge a ``TOOLS`` dict into the target registry dict."""
for tool_id, entry in tools.items():
if not isinstance(entry, dict):
continue
# Inject tool_id into entry if missing
if "tool_id" not in entry:
entry = {**entry, "tool_id": tool_id}
target[tool_id] = entry
def _merge_tools_list(
target: dict[str, dict[str, Any]],
tools: list[dict[str, Any]],
) -> None:
"""Merge a legacy ``TOOLS`` list into the target registry dict."""
for entry in tools:
if not isinstance(entry, dict):
continue
tool_id = entry.get("tool_id")
if tool_id is None:
name = entry.get("name", "")
tool_id = name.lower().replace(" ", "_").replace("/", "_")
entry["tool_id"] = tool_id
target[tool_id] = entry

145
src/ai_lsc/registry/manager.py Executable file
View File

@ -0,0 +1,145 @@
"""
AI-LSC — Registry manager.
Load / merge / query the on-disk ecosystem registry. The canonical source
of truth is the per-layer modules in ``ai_lsc.registry.layers`` (discovered
automatically by :mod:`ai_lsc.registry.loader`). On first run the merged
registry is written to ``<base_dir>/registry/ecosystem.json``; on subsequent
runs structural fields (layer, level, role, category) are synced from the
layer files while user customisations (description, installer tweaks,
flags, filesystem paths) are preserved.
All path operations use ``pathlib.Path`` instead of ``os.path``.
"""
from __future__ import annotations
import json
from itertools import chain
from pathlib import Path
from typing import Any
import logging
from ai_lsc.registry.loader import load_merged_registry
logger = logging.getLogger(__name__)
class RegistryManager:
"""Knowledge-graph engine backed by a JSON file on disk.
Parameters
----------
registry_dir:
Absolute path to the directory containing ``ecosystem.json``.
"""
def __init__(self, registry_dir: str | Path) -> None:
self.registry_dir = Path(registry_dir)
self.registry_file: Path = self.registry_dir / "ecosystem.json"
self.data: dict[str, dict[str, Any]] = {}
self._bootstrap()
# ── Bootstrap / merge ────────────────────────────────────────────
def _bootstrap(self) -> None:
self.registry_dir.mkdir(parents=True, exist_ok=True)
# Always load the canonical registry from per-layer files.
upstream = load_merged_registry()
if not self.registry_file.exists():
# First run — seed ecosystem.json from layer files.
self.data = dict(upstream)
self.registry_file.write_text(
json.dumps(self.data, indent=4), encoding="utf-8"
)
else:
try:
self.data = json.loads(
self.registry_file.read_text(encoding="utf-8")
)
except (json.JSONDecodeError, OSError) as exc:
logger.error(
"Failed to parse %s: %s -- re-creating from layer files",
self.registry_file, exc,
)
self.data = dict(upstream)
self.registry_file.write_text(
json.dumps(self.data, indent=4), encoding="utf-8",
)
# Sync: merge new tools, update structural fields from upstream.
self._sync_with_upstream(upstream)
def _sync_with_upstream(
self, upstream: dict[str, dict[str, Any]]
) -> None:
"""Merge upstream (per-layer files) into the on-disk registry.
* New tools (not in ecosystem.json) are added wholesale.
* Existing tools get their structural fields (layer, level, role,
category, name, installer, launcher, deps, flags, license)
updated from upstream so the topology stays consistent.
* User-added keys that don't exist upstream are preserved.
"""
changed = False
structural_keys = {
"name", "level", "layer", "role", "category",
"installer", "launcher", "deps", "flags", "license",
"description",
}
for tool_id, up_meta in upstream.items():
if tool_id not in self.data:
# Brand-new tool — add it.
self.data[tool_id] = up_meta
changed = True
else:
# Existing tool — sync structural fields from upstream.
existing = self.data[tool_id]
for key in structural_keys:
up_val = up_meta.get(key)
if up_val is not None and existing.get(key) != up_val:
existing[key] = up_val
changed = True
# Inject tool_id if missing.
if "tool_id" not in existing:
existing["tool_id"] = tool_id
changed = True
if changed:
self.registry_file.write_text(
json.dumps(self.data, indent=4), encoding="utf-8"
)
# ── Queries ──────────────────────────────────────────────────────
def get_all_tools(self) -> dict[str, dict[str, Any]]:
"""Return the full registry dict."""
return self.data
def get_tool(self, tool_id: str) -> dict[str, Any]:
"""Return a single tool's raw dict, or ``{}`` if unknown."""
return self.data.get(tool_id, {})
def get_grouped_by_layer(self) -> dict[str, list[tuple[str, dict]]]:
"""Return tools grouped and sorted by their ``layer`` field."""
layers: dict[str, list[tuple[str, dict]]] = {}
for t_id, meta in self.data.items():
layers.setdefault(
meta.get("layer", "Uncategorized"), []
).append((t_id, meta))
return dict(sorted(layers.items()))
def check_dependencies(
self, selected: list[str],
) -> list[str]:
"""Return tool IDs that are required but missing from *selected*."""
all_deps = list(chain.from_iterable(
self.get_tool(t).get("deps", [])
for t in selected
if not t.startswith("skill:")
))
return list({d for d in all_deps if d not in selected})

56
src/ai_lsc/registry/openengineer/__init__.py Executable file
View File

@ -0,0 +1,56 @@
"""AI-LSC -- OpenEngineer integration sub-package.
Bridges the Open Engineer standard (https://git.dcos.net/dcosnet/openengineer)
with AI-LSC's stack template system. Open Engineer defines a methodology for
preserving engineering context -- the reasoning, observations, decisions, and
constraints that shape engineering work. This package imports Open Engineer
context records and project files as AI-LSC stack templates, creating a
standard template format that unifies both systems.
Key concepts
-----------
* **OE Context Record** -- The 9-field structured record defined in OE-0003
(Decision, Observation, Alternatives, Constraints, Reasoning, Verification,
Lineage, Assumptions, plus optional fields). This is the unit of
preservation in Open Engineer.
* **Standard Template** -- The merged AI-LSC / Open Engineer template format
that carries both OE engineering context and AI-LSC stack configuration
(tools, layers, endpoints, deployment targets).
* **Import Pipeline** -- Read an Open Engineer file (markdown context record,
RFC, example, or project manifest) and produce a Standard Template that
AI-LSC's StackTemplateManager can consume.
Modules
-------
* ``schema`` -- Standard template schema definition and constants
* ``parser`` -- OE markdown context record parser
* ``importer`` -- Import pipeline (OE file -> Standard Template)
* ``templates`` -- Built-in OE-derived stack templates
"""
from ai_lsc.registry.openengineer.schema import (
OE_CONTEXT_FIELDS,
OE_REQUIRED_FIELDS,
OE_SUPPLEMENTARY_FIELDS,
OE_CONFORMANCE_CRITERIA,
StandardTemplate,
standard_template_to_ai_lsc,
)
from ai_lsc.registry.openengineer.parser import OEContextParser
from ai_lsc.registry.openengineer.importer import OpenEngineerImporter
__all__ = [
# Schema
"OE_CONTEXT_FIELDS",
"OE_REQUIRED_FIELDS",
"OE_SUPPLEMENTARY_FIELDS",
"OE_CONFORMANCE_CRITERIA",
"StandardTemplate",
"standard_template_to_ai_lsc",
# Parser
"OEContextParser",
# Importer
"OpenEngineerImporter",
]

420
src/ai_lsc/registry/openengineer/importer.py Executable file
View File

@ -0,0 +1,420 @@
"""
AI-LSC / Open Engineer -- Import Pipeline.
Imports Open Engineer files (context records, RFCs, spec documents,
examples) and converts them to StandardTemplate objects that can be
consumed by AI-LSC's StackTemplateManager.
The importer can operate in two modes:
1. **File import** -- Import a single OE markdown file and produce a
StandardTemplate. The engineering context is extracted from the
markdown structure; the stack config must be provided separately
or inferred from OE-0003 fields.
2. **Directory import** -- Scan an Open Engineer repository checkout
(or any directory tree) and import all discoverable OE files.
Produces a list of StandardTemplates.
Usage example::
from ai_lsc.registry.openengineer.importer import OpenEngineerImporter
imp = OpenEngineerImporter()
templates = imp.import_directory("/path/to/openengineer")
for t in templates:
ai_lsc_tpl = standard_template_to_ai_lsc(t)
# Pass to StackTemplateManager...
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
from ai_lsc.registry.openengineer.parser import OEContextParser
from ai_lsc.registry.openengineer.schema import (
OE_REQUIRED_FIELDS,
StandardTemplate,
standard_template_to_ai_lsc,
)
# Directories in an OE repo that contain importable content.
_OE_CONTENT_DIRS: list[str] = [
"examples",
"spec",
"rfc",
"reference",
"laws",
]
# File extensions we attempt to parse.
_OE_PARSE_EXTENSIONS: set[str] = {".md", ".markdown", ".txt"}
class OpenEngineerImporter:
"""Import Open Engineer files into StandardTemplate objects.
Parameters
----------
parser :
Optional OEContextParser instance. If None, a default
(non-strict) parser is created.
default_tools :
Default tool IDs to include in stack_config when no tools
are inferred from the OE content.
"""
def __init__(
self,
parser: OEContextParser | None = None,
default_tools: list[str] | None = None,
) -> None:
self.parser = parser or OEContextParser(strict=False)
self.default_tools = default_tools or []
# ── Single file import ──────────────────────────────────────────
def import_file(
self,
path: str | Path,
stack_config: dict[str, Any] | None = None,
) -> StandardTemplate:
"""Import a single OE file into a StandardTemplate.
Parameters
----------
path :
Path to the markdown file.
stack_config :
Optional AI-LSC stack configuration to merge. If provided,
these tools/endpoints/tags override any inferred values.
Returns
-------
A :class:`StandardTemplate` with extracted context and
(optionally) merged stack config.
"""
p = Path(path)
parsed = self.parser.parse_file(p)
# Derive template ID from filename
template_id = self._derive_template_id(p, parsed["title"])
source_type = parsed["source_type"]
# Build the StandardTemplate
template = StandardTemplate(
template_id=template_id,
name=parsed["title"],
source_file=str(p),
source_type=source_type,
engineering_context=parsed["context"],
oe_spec_refs=self._extract_spec_refs(parsed, source_type),
metadata={
"parsed_sections": len(parsed["raw_sections"]),
"oe_fields_found": [
k for k in OE_REQUIRED_FIELDS
if k in parsed["context"]
],
"oe_fields_missing": [
k for k in OE_REQUIRED_FIELDS
if k not in parsed["context"]
],
},
)
# Infer stack config from OE content when not provided
inferred = self._infer_stack_config(parsed, template_id)
if stack_config:
inferred.update(stack_config)
template.stack_config = inferred
# Run conformance check
template.check_conformance()
return template
# ── Directory import ───────────────────────────────────────────
def import_directory(
self,
directory: str | Path,
stack_config_overrides: dict[str, dict[str, Any]] | None = None,
) -> list[StandardTemplate]:
"""Scan an OE repo directory and import all content files.
Parameters
----------
directory :
Root of the Open Engineer repository (or any directory
containing markdown files with OE structure).
stack_config_overrides :
Optional dict mapping template_id to stack_config dicts.
Used to provide AI-LSC tool mappings for specific OE files.
Returns
-------
List of :class:`StandardTemplate` objects, sorted by source path.
"""
root = Path(directory)
if not root.is_dir():
return []
overrides = stack_config_overrides or {}
templates: list[StandardTemplate] = []
# Scan known OE content directories
for subdir_name in _OE_CONTENT_DIRS:
subdir = root / subdir_name
if not subdir.is_dir():
continue
for md_file in sorted(subdir.iterdir()):
if md_file.suffix.lower() not in _OE_PARSE_EXTENSIONS:
continue
if md_file.name.startswith("."):
continue
try:
tpl_id = self._derive_template_id(
md_file, md_file.stem
)
override = overrides.get(tpl_id)
template = self.import_file(md_file, stack_config=override)
templates.append(template)
except Exception:
# Don't let one bad file stop the import
continue
# Also scan root for standalone OE files
for md_file in sorted(root.iterdir()):
if not md_file.is_file():
continue
if md_file.suffix.lower() not in _OE_PARSE_EXTENSIONS:
continue
if md_file.name in {
"README.md", "CONTRIBUTING.md", "CHARTER.md",
"LICENSE", "ROADMAP.md",
}:
continue
if md_file.name.startswith("."):
continue
try:
template = self.import_file(md_file)
if template.source_type != "unknown":
templates.append(template)
except Exception:
continue
return sorted(templates, key=lambda t: t.source_file)
# ── Bulk convert to AI-LSC format ──────────────────────────────
def import_as_ai_lsc_templates(
self,
directory: str | Path,
stack_config_overrides: dict[str, dict[str, Any]] | None = None,
) -> list[dict[str, Any]]:
"""Import an OE directory and return AI-LSC-compatible template dicts.
Convenience method that combines :meth:`import_directory` with
:func:`standard_template_to_ai_lsc`.
Returns
-------
List of dicts compatible with StackTemplateManager.
"""
templates = self.import_directory(directory, stack_config_overrides)
return [standard_template_to_ai_lsc(t) for t in templates]
# ── Stack config inference ──────────────────────────────────────
def _infer_stack_config(
self,
parsed: dict[str, Any],
template_id: str,
) -> dict[str, Any]:
"""Infer AI-LSC stack config from OE content.
Attempts to extract tool references, layer mappings, and
endpoint configuration from the engineering context fields.
"""
ctx = parsed["context"]
config: dict[str, Any] = {
"id": template_id,
"tools": list(self.default_tools),
"tags": self._infer_tags(parsed, ctx),
"endpoints": {},
"notes": {},
}
# Infer tools from context mentions
mentioned_tools = self._extract_tool_mentions(ctx)
for tool_id in mentioned_tools:
if tool_id not in config["tools"]:
config["tools"].append(tool_id)
# Infer notes from supplementary context
if "open_questions" in ctx:
config["notes"]["open_questions"] = ctx["open_questions"]
if "discipline_specific_data" in ctx:
config["notes"]["discipline_data"] = ctx["discipline_specific_data"]
if "traceability" in ctx:
config["notes"]["traceability"] = ctx["traceability"]
# Store source type in notes
config["notes"]["oe_source_type"] = parsed["source_type"]
config["notes"]["oe_metadata"] = parsed["metadata"]
return config
# ── Tool mention extraction ─────────────────────────────────────
_KNOWN_TOOL_PATTERNS: list[tuple[str, str]] = [
(r"\bollama\b", "ollama"),
(r"\bvllm\b", "vllm"),
(r"\bllama\.?cpp\b", "llamacpp"),
(r"\blitellm\b", "litellm"),
(r"\bopen\s*web\s*ui\b", "openwebui"),
(r"\bqdrant\b", "qdrant"),
(r"\bredis\b", "redis"),
(r"\bchroma[\s-]?db\b", "chromadb"),
(r"\bpostgres(?:ql)?\b", "postgresql"),
(r"\bmaria[\s-]?db\b", "mariadb"),
(r"\bgrafana\b", "grafana"),
(r"\bprometheus\b", "prometheus"),
(r"\bterraform\b", "terraform"),
(r"\bansible\b", "ansible"),
(r"\bpulumi\b", "pulumi"),
(r"\bwhisper\b", "whisper"),
(r"\bdocling\b", "docling"),
(r"\bfabric\b", "fabric"),
(r"\baider\b", "aider"),
(r"\bclaude\s*code\b", "claude_code"),
(r"\bcrewai\b", "crewai"),
(r"\bautogen\b", "autogen"),
(r"\bn8n\b", "n8n"),
(r"\bdify\b", "dify"),
(r"\bflowise\b", "flowise"),
(r"\bopenjarvis\b", "openjarvis"),
(r"\bhermes\b", "hermes"),
(r"\bopendataloader\b", "opendataloader"),
(r"\bgraphrag\b", "graphrag"),
(r"\bcrawl4ai\b", "crawl4ai"),
(r"\belasticsearch\b", "elasticsearch"),
(r"\bneo4j\b", "neo4j"),
(r"\blance[\s-]?db\b", "lancedb"),
]
def _extract_tool_mentions(self, context: dict[str, str]) -> list[str]:
"""Scan context text for known AI-LSC tool name mentions."""
import re
full_text = "\n".join(context.values())
found: list[str] = []
seen: set[str] = set()
for pattern, tool_id in self._KNOWN_TOOL_PATTERNS:
if tool_id not in seen and re.search(pattern, full_text, re.IGNORECASE):
found.append(tool_id)
seen.add(tool_id)
return found
# ── Tag inference ───────────────────────────────────────────────
def _infer_tags(
self,
parsed: dict[str, Any],
context: dict[str, str],
) -> list[str]:
"""Generate tags from OE metadata and context content."""
tags = ["openengineer"]
source_type = parsed["source_type"]
if source_type == "rfc":
tags.extend(["rfc", "proposal"])
elif source_type == "context_record":
tags.extend(["context-record", "engineering-decision"])
elif source_type == "spec":
tags.extend(["specification", "standard"])
elif source_type == "example":
tags.extend(["example", "demonstration"])
# Add discipline tags from content
full_text = "\n".join(context.values()).lower()
discipline_tags = [
("software", "software"),
("civil", "civil"),
("aerospace", "aerospace"),
("mechanical", "mechanical"),
("electrical", "electrical"),
("chemical", "chemical"),
("biomedical", "biomedical"),
("environmental", "environmental"),
("manufacturing", "manufacturing"),
]
for keyword, tag in discipline_tags:
if keyword in full_text and tag not in tags:
tags.append(tag)
# Add OE concept tags
oe_concept_tags = [
("thread integrity", "thread-integrity"),
("stewardship", "stewardship"),
("inheritance", "inheritance"),
("spiral re-evaluation", "spiral-re-evaluation"),
("verification", "verification"),
("observation first", "observation-first"),
("bedrock", "bedrock"),
("enduring concept", "enduring-concept"),
]
for keyword, tag in oe_concept_tags:
if keyword in full_text and tag not in tags:
tags.append(tag)
return tags
# ── Spec reference extraction ──────────────────────────────────
@staticmethod
def _extract_spec_refs(
parsed: dict[str, Any],
source_type: str,
) -> list[str]:
"""Extract OE specification document references from content."""
import re
full_text = "\n".join(
f"{h}\n{c}" for h, c in parsed["raw_sections"]
)
# Find OE-NNNN references
refs = set(re.findall(r"\b(OE-\d{4})\b", full_text))
# Add implicit refs based on source type
if source_type == "rfc":
refs.add("OE-0000") # All RFCs relate to the Charter
elif source_type == "context_record":
refs.add("OE-0003") # Context records implement OE-0003
return sorted(refs)
# ── Template ID derivation ──────────────────────────────────────
@staticmethod
def _derive_template_id(path: Path, title: str) -> str:
"""Derive a stable template ID from path and title."""
# Use parent dir name + filename stem for disambiguation
parent = path.parent.name if path.parent.name else "root"
stem = path.stem
# Slugify
raw = f"oe-{parent}-{stem}"
slug = raw.lower().replace("_", "-").replace(" ", "-")
# Collapse repeated hyphens
while "--" in slug:
slug = slug.replace("--", "-")
return slug.strip("-")

317
src/ai_lsc/registry/openengineer/parser.py Executable file
View File

@ -0,0 +1,317 @@
"""
AI-LSC / Open Engineer -- Context Record Parser.
Parses Open Engineer markdown files into structured engineering context
records. Supports the three OE document types that carry usable
engineering context:
* **Context Records** (examples/, spec/ OE-0003 format) -- structured
records with the 9 required fields (Decision, Observation, Alternatives,
Constraints, Reasoning, Verification, Lineage, Assumptions).
* **RFCs** (rfc/) -- proposals with Abstract, Motivation, Observation,
Engineering Principle, Reasoning, Relationship sections.
* **Spec Documents** (spec/) -- formal specification documents with
Definition sections, Law references, and structured content.
The parser uses heading-level heuristics and field-name matching to
extract context from prose markdown. It does not require rigid front-
matter -- it follows the OE principle that structure carries meaning.
"""
from __future__ import annotations
import re
from pathlib import Path
from typing import Any
# ── OE field name canonicalization ──────────────────────────────────
# Maps common heading variations to canonical OE-0003 field names.
_FIELD_ALIASES: dict[str, str] = {
# Direct OE-0003 field names
"decision": "decision",
"observation": "observation",
"observations": "observation",
"alternatives": "alternatives",
"alternative": "alternatives",
"constraints": "constraints",
"constraint": "constraints",
"reasoning": "reasoning",
"verification": "verification",
"lineage": "lineage",
"assumptions": "assumptions",
"assumption": "assumptions",
# Supplementary fields
"open questions": "open_questions",
"discipline-specific data": "discipline_specific_data",
"traceability": "traceability",
# RFC-specific sections -> OE context mapping
"abstract": "decision",
"motivation": "observation",
"proposal": "decision",
"engineering principle": "reasoning",
"relationship to existing concepts": "lineage",
"relationship": "lineage",
# Spec document sections -> OE context mapping
"definition": "decision",
"overview": "observation",
"scope": "constraints",
"applicable laws": "constraints",
"known limitations": "assumptions",
"implications": "reasoning",
}
# Heading patterns for each OE field -- order matters (more specific first).
_HEADING_PATTERNS: list[tuple[re.Pattern, str]] = [
(re.compile(r"^#+\s*(?:##\s*)?Decision\s*[:\-]?\s*", re.IGNORECASE), "decision"),
(re.compile(r"^#+\s*(?:##\s*)?Observation\s*[:\-]?\s*", re.IGNORECASE), "observation"),
(re.compile(r"^#+\s*(?:##\s*)?Alternatives?\s*[:\-]?\s*", re.IGNORECASE), "alternatives"),
(re.compile(r"^#+\s*(?:##\s*)?Constraint\s*[:\-]?\s*", re.IGNORECASE), "constraints"),
(re.compile(r"^#+\s*(?:##\s*)?Reasoning\s*[:\-]?\s*", re.IGNORECASE), "reasoning"),
(re.compile(r"^#+\s*(?:##\s*)?Verification\s*[:\-]?\s*", re.IGNORECASE), "verification"),
(re.compile(r"^#+\s*(?:##\s*)?Lineage\s*[:\-]?\s*", re.IGNORECASE), "lineage"),
(re.compile(r"^#+\s*(?:##\s*)?Assumption\s*[:\-]?\s*", re.IGNORECASE), "assumptions"),
(re.compile(r"^#+\s*(?:##\s*)?Open\s+Questions?\s*[:\-]?\s*", re.IGNORECASE), "open_questions"),
(re.compile(r"^#+\s*(?:##\s*)?Traceability\s*[:\-]?\s*", re.IGNORECASE), "traceability"),
# RFC sections
(re.compile(r"^#+\s*(?:##\s*)?Abstract\s*[:\-]?\s*", re.IGNORECASE), "decision"),
(re.compile(r"^#+\s*(?:##\s*)?Motivation\s*[:\-]?\s*", re.IGNORECASE), "observation"),
(re.compile(r"^#+\s*(?:##\s*)?Proposal\s*[:\-]?\s*", re.IGNORECASE), "decision"),
(re.compile(r"^#+\s*(?:##\s*)?Engineering\s+Principle\s*[:\-]?\s*", re.IGNORECASE), "reasoning"),
(re.compile(r"^#+\s*(?:##\s*)?Relationship\s*(?:to\s+Existing\s+Concepts)?\s*[:\-]?\s*", re.IGNORECASE), "lineage"),
# Spec sections
(re.compile(r"^#+\s*(?:##\s*)?Definition\s*[:\-]?\s*", re.IGNORECASE), "decision"),
(re.compile(r"^#+\s*(?:##\s*)?Overview\s*[:\-]?\s*", re.IGNORECASE), "observation"),
(re.compile(r"^#+\s*(?:##\s*)?Scope\s*[:\-]?\s*", re.IGNORECASE), "constraints"),
(re.compile(r"^#+\s*(?:##\s*)?Known\s+Limitations?\s*[:\-]?\s*", re.IGNORECASE), "assumptions"),
(re.compile(r"^#+\s*(?:##\s*)?Applicable\s+Laws?\s*[:\-]?\s*", re.IGNORECASE), "constraints"),
(re.compile(r"^#+\s*(?:##\s*)?Implications?\s*[:\-]?\s*", re.IGNORECASE), "reasoning"),
]
class OEContextParser:
"""Parse Open Engineer markdown files into structured context records.
The parser extracts OE-0003 context fields from markdown by identifying
heading-delimited sections. Each section's content becomes the value
for the corresponding OE field.
Parameters
----------
strict :
If True, only recognized OE-0003 field names are extracted.
If False, all heading-delimited sections are captured.
"""
def __init__(self, strict: bool = False) -> None:
self.strict = strict
# ── Public API ───────────────────────────────────────────────────
def parse_file(self, path: str | Path) -> dict[str, Any]:
"""Parse an Open Engineer markdown file and extract context.
Parameters
----------
path :
Path to a markdown file.
Returns
-------
dict with keys:
* ``context`` -- extracted OE-0003 fields (str -> str)
* ``title`` -- first H1 heading found (or filename stem)
* ``source_type`` -- detected type: "context_record",
"rfc", "spec", or "unknown"
* ``metadata`` -- front-matter or status line info
* ``raw_sections`` -- all heading -> content pairs found
"""
p = Path(path)
if not p.exists():
return self._empty_result(p)
text = p.read_text(encoding="utf-8", errors="ignore")
return self.parse_text(text, source_path=str(p))
def parse_text(
self,
text: str,
source_path: str = "",
) -> dict[str, Any]:
"""Parse markdown text and extract OE context fields.
Parameters
----------
text :
The full markdown content.
source_path :
Optional path for metadata (used as title fallback).
Returns
-------
Same structure as :meth:`parse_file`.
"""
sections = self._split_into_sections(text)
title = self._extract_title(text, source_path)
source_type = self._detect_source_type(text, sections)
metadata = self._extract_metadata(text)
# Map section headings to OE fields
context: dict[str, str] = {}
for heading, content in sections:
field_name = self._heading_to_field(heading)
if field_name:
# If the field already has content, append with separator
existing = context.get(field_name, "")
if existing:
context[field_name] = f"{existing}\n\n---\n\n{content.strip()}"
else:
context[field_name] = content.strip()
elif not self.strict:
# In non-strict mode, capture unrecognized sections too
safe_key = heading.strip().lower().replace(" ", "_")[:60]
context[f"_section_{safe_key}"] = content.strip()
return {
"context": context,
"title": title,
"source_type": source_type,
"metadata": metadata,
"raw_sections": sections,
}
# ── Section splitting ───────────────────────────────────────────
@staticmethod
def _split_into_sections(
text: str,
) -> list[tuple[str, str]]:
"""Split markdown into (heading, content) pairs.
Uses H2 (##) and H3 (###) as section delimiters. Content
before the first H2 is captured under a virtual "preamble" heading.
"""
lines = text.split("\n")
sections: list[tuple[str, str]] = []
current_heading = "preamble"
current_lines: list[str] = []
for line in lines:
# Match H2 or H3 headings
m = re.match(r"^(#{2,3})\s+(.+)$", line.strip())
if m:
# Save previous section
content = "\n".join(current_lines).strip()
if content or current_heading != "preamble":
sections.append((current_heading, content))
current_heading = m.group(2).strip()
current_lines = []
else:
current_lines.append(line)
# Don't forget the last section
content = "\n".join(current_lines).strip()
if content:
sections.append((current_heading, content))
return sections
# ── Heading to OE field mapping ─────────────────────────────────
def _heading_to_field(self, heading: str) -> str | None:
"""Map a section heading to a canonical OE field name."""
# Try regex patterns first (most specific)
for pattern, field_name in _HEADING_PATTERNS:
if pattern.match(heading):
return field_name
# Fallback: normalize and look up in aliases
normalized = heading.strip().lower().rstrip(":").rstrip("-").strip()
return _FIELD_ALIASES.get(normalized)
# ── Title extraction ────────────────────────────────────────────
@staticmethod
def _extract_title(text: str, source_path: str = "") -> str:
"""Extract the title from the first H1 heading."""
m = re.search(r"^#\s+(.+)$", text, re.MULTILINE)
if m:
# Strip common suffixes like status badges
title = m.group(1).strip()
title = re.sub(r"\*\*Status:.*?\*\*", "", title).strip()
title = re.sub(r"\*\*Version:.*?\*\*", "", title).strip()
title = re.sub(r"\*\*Phase:.*?\*\*", "", title).strip()
return title
if source_path:
return Path(source_path).stem
return "Untitled"
# ── Source type detection ───────────────────────────────────────
@staticmethod
def _detect_source_type(
text: str,
sections: list[tuple[str, str]],
) -> str:
"""Detect whether the file is a context record, RFC, or spec."""
lower = text[:2000].lower()
# RFC pattern
if re.search(r"^# RFC-\d+", text, re.MULTILINE):
return "rfc"
if "status:** proposed" in lower or "status:** rc" in lower:
if "abstract" in lower and "engineering principle" in lower:
return "rfc"
# Spec document pattern
if "depends on:" in lower and "oe-" in lower:
return "spec"
# Context record: has multiple OE-0003 fields as headings
oe_field_count = 0
for heading, _ in sections:
normalized = heading.strip().lower()
if normalized in {
"decision", "observation", "alternatives", "constraints",
"reasoning", "verification", "lineage", "assumptions",
}:
oe_field_count += 1
if oe_field_count >= 3:
return "context_record"
return "unknown"
# ── Metadata extraction ─────────────────────────────────────────
@staticmethod
def _extract_metadata(text: str) -> dict[str, str]:
"""Extract status, version, phase, depends-on from the document header."""
meta: dict[str, str] = {}
header = text[:1500]
for field in ("status", "version", "phase", "depends on"):
pattern = re.compile(
rf"\*\*{re.escape(field)}:?\*\*\s*(.+)",
re.IGNORECASE,
)
m = pattern.search(header)
if m:
meta[field.lower().replace(" ", "_")] = m.group(1).strip()
return meta
# ── Helpers ─────────────────────────────────────────────────────
@staticmethod
def _empty_result(path: Path) -> dict[str, Any]:
return {
"context": {},
"title": path.stem if path.exists() else "missing",
"source_type": "unknown",
"metadata": {},
"raw_sections": [],
}

288
src/ai_lsc/registry/openengineer/schema.py Executable file
View File

@ -0,0 +1,288 @@
"""
AI-LSC / Open Engineer -- Standard Template Schema.
Defines the unified template format that merges Open Engineer's engineering
context record (OE-0003) with AI-LSC's stack template configuration.
The standard template is the bridge format: it can be produced by
importing an Open Engineer context record, by enriching an existing AI-LSC
stack template, or by creating one from scratch. It can be consumed by
AI-LSC's StackTemplateManager for stack deployment.
Schema versioning follows AI-LSC's ``STACK_SCHEMA_VERSION`` prefix
with an OE-specific suffix.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
# ── Open Engineer field definitions (OE-0003) ────────────────────────
#: All nine OE-0003 context record fields.
OE_CONTEXT_FIELDS: list[str] = [
"decision",
"observation",
"alternatives",
"constraints",
"reasoning",
"verification",
"lineage",
"assumptions",
]
#: The eight *required* fields (OE-0003 minimum context record).
#: "open_questions" is the first supplementary field.
OE_REQUIRED_FIELDS: set[str] = set(OE_CONTEXT_FIELDS)
#: Optional supplementary fields from OE-0003.
OE_SUPPLEMENTARY_FIELDS: list[str] = [
"open_questions",
"discipline_specific_data",
"traceability",
]
#: The ten OE-0000 conformance criteria, used for validation.
OE_CONFORMANCE_CRITERIA: list[str] = [
"field_completeness",
"decision_specificity",
"observation_traceability",
"constraint_bounding",
"alternatives_plural",
"verification_against_reality",
"reasoning_references_alternatives",
"lineage_traceability",
"assumption_awareness",
"no_contradiction",
]
# ── Standard Template Schema Version ─────────────────────────────────
STANDARD_TEMPLATE_VERSION: str = "1.0.0-oe-rc3"
# ── Standard Template dataclass ─────────────────────────────────────
@dataclass
class StandardTemplate:
"""Unified AI-LSC / Open Engineer template.
Carries both OE engineering context and AI-LSC stack configuration
in a single portable structure.
The ``engineering_context`` dict maps directly to the OE-0003
context record fields. The ``stack_config`` dict maps to
AI-LSC's stack template format (tools, endpoints, tags, etc.).
Parameters
----------
template_id :
Unique identifier (slugified name or explicit ID).
name :
Human-readable template name.
version :
Template version string.
author :
Template author or "openengineer" for imported templates.
description :
One-line summary.
source_file :
Path to the original OE file this was imported from, if any.
source_type :
Type of OE source: "context_record", "rfc", "example",
"project", or "native" for AI-LSC-native templates.
oe_spec_refs :
Open Engineer specification documents referenced
(e.g. ["OE-0003", "OE-0008"]).
engineering_context :
The OE-0003 context record fields (decision, observation, etc.).
stack_config :
AI-LSC stack template configuration (tools, endpoints, tags, etc.).
conformance :
Results of OE-0000 conformance checking (criteria -> pass/fail).
notes :
Arbitrary key-value notes (mirrors AI-LSC template ``notes``).
metadata :
Additional metadata not covered by other fields.
"""
template_id: str
name: str
version: str = STANDARD_TEMPLATE_VERSION
author: str = "openengineer"
description: str = ""
source_file: str = ""
source_type: str = "native" # context_record | rfc | example | project | native
oe_spec_refs: list[str] = field(default_factory=list)
engineering_context: dict[str, str] = field(default_factory=dict)
stack_config: dict[str, Any] = field(default_factory=dict)
conformance: dict[str, bool] = field(default_factory=dict)
notes: dict[str, str] = field(default_factory=dict)
metadata: dict[str, Any] = field(default_factory=dict)
# ── Serialization ────────────────────────────────────────────────
def to_dict(self) -> dict[str, Any]:
"""Serialize to a JSON-compatible dict (the interchange format)."""
return {
"$schema": "https://git.dcos.net/dcosnet/openengineer"
"/schemas/standard-template-v1.json",
"schema_version": self.version,
"id": self.template_id,
"name": self.name,
"author": self.author,
"description": self.description,
"source": {
"file": self.source_file,
"type": self.source_type,
"oe_spec_refs": self.oe_spec_refs,
},
"engineering_context": self.engineering_context,
"stack_config": self.stack_config,
"conformance": self.conformance,
"notes": self.notes,
"metadata": self.metadata,
}
@classmethod
def from_dict(cls, data: dict[str, Any]) -> StandardTemplate:
"""Hydrate from a JSON-compatible dict."""
source = data.get("source", {})
return cls(
template_id=data.get("id", ""),
name=data.get("name", ""),
version=data.get("schema_version", STANDARD_TEMPLATE_VERSION),
author=data.get("author", "openengineer"),
description=data.get("description", ""),
source_file=source.get("file", ""),
source_type=source.get("type", "native"),
oe_spec_refs=source.get("oe_spec_refs", []),
engineering_context=data.get("engineering_context", {}),
stack_config=data.get("stack_config", {}),
conformance=data.get("conformance", {}),
notes=data.get("notes", {}),
metadata=data.get("metadata", {}),
)
def to_json(self, indent: int = 4) -> str:
"""Serialize to a JSON string."""
import json
return json.dumps(self.to_dict(), indent=indent, ensure_ascii=False)
# ── Conformance checking (OE-0000 ten criteria) ─────────────────
def check_conformance(self) -> dict[str, bool]:
"""Run the ten OE-0000 conformance criteria against this template.
Returns a dict mapping criterion name to pass/fail boolean.
"""
ctx = self.engineering_context
results: dict[str, bool] = {}
# 1. Field Completeness
results["field_completeness"] = all(
ctx.get(f, "") for f in OE_REQUIRED_FIELDS
)
# 2. Decision Specificity
dec = ctx.get("decision", "")
results["decision_specificity"] = (
len(dec.strip()) > 0
and "|" not in dec # simple heuristic: no pipe-separated list
)
# 3. Observation Traceability
obs = ctx.get("observation", "")
results["observation_traceability"] = len(obs.strip()) > 20
# 4. Constraint Bounding
con = ctx.get("constraints", "")
results["constraint_bounding"] = len(con.strip()) > 0
# 5. Alternatives Plural
alt = ctx.get("alternatives", "")
results["alternatives_plural"] = (
len(alt.strip()) > 0
and ("," in alt or "\n" in alt or "-" in alt)
)
# 6. Verification Against Reality
ver = ctx.get("verification", "")
results["verification_against_reality"] = len(ver.strip()) > 10
# 7. Reasoning References Alternatives
rea = ctx.get("reasoning", "")
results["reasoning_references_alternatives"] = (
len(rea.strip()) > 10 and len(alt.strip()) > 0
)
# 8. Lineage Traceability
lin = ctx.get("lineage", "")
results["lineage_traceability"] = len(lin.strip()) > 0
# 9. Assumption Awareness
asn = ctx.get("assumptions", "")
results["assumption_awareness"] = len(asn.strip()) > 0
# 10. No Contradiction (always true for well-formed templates)
results["no_contradiction"] = True
self.conformance = results
return results
@property
def conformance_score(self) -> int:
"""Return conformance as a percentage (0-100)."""
if not self.conformance:
self.check_conformance()
passed = sum(1 for v in self.conformance.values() if v)
return int((passed / max(len(self.conformance), 1)) * 100)
# ── Conversion to AI-LSC stack template format ──────────────────────
def standard_template_to_ai_lsc(template: StandardTemplate) -> dict[str, Any]:
"""Convert a StandardTemplate to AI-LSC's stack template JSON format.
This is the output format consumed by
:class:`ai_lsc.registry.stack_templates.manager.StackTemplateManager`.
The engineering context is preserved in the ``notes`` section under
an ``openengineer_context`` key so that no OE information is lost
during the conversion.
Parameters
----------
template :
A populated :class:`StandardTemplate`.
Returns
-------
dict
An AI-LSC-compatible stack template dict.
"""
stack = dict(template.stack_config)
stack.setdefault("id", template.template_id)
stack.setdefault("name", template.name)
stack.setdefault("description", template.description)
stack.setdefault("version", template.version)
stack.setdefault("author", template.author)
stack.setdefault("tags", []).extend(
t for t in ["openengineer", "oe-context"]
if t not in stack.get("tags", [])
)
# Merge OE context into notes
notes = dict(template.notes)
notes["openengineer_context"] = template.engineering_context
notes["openengineer_source"] = {
"file": template.source_file,
"type": template.source_type,
"oe_spec_refs": template.oe_spec_refs,
"conformance_score": template.conformance_score,
"conformance": template.conformance,
}
stack["notes"] = notes
return stack

793
src/ai_lsc/registry/openengineer/templates.py Executable file
View File

@ -0,0 +1,793 @@
"""
AI-LSC / Open Engineer -- Built-in OE-derived stack templates.
These templates demonstrate the standard template format by mapping
Open Engineer's engineering principles to practical AI-LSC tool stacks.
Each template carries full OE engineering context (the "why") alongside
the AI-LSC stack configuration (the "what").
The templates live here as Python-generated JSON and are registered
with StackTemplateManager at import time via :func:`get_templates`.
"""
from __future__ import annotations
from ai_lsc.registry.openengineer.schema import StandardTemplate
def get_templates() -> list[StandardTemplate]:
"""Return all built-in OE-derived StandardTemplates.
Each template maps an Open Engineer concept or example discipline
to an AI-LSC tool stack that helps preserve and apply that concept.
"""
return [
_thread_integrity_stack(),
_verification_loop_stack(),
_stewardship_stack(),
_observation_first_stack(),
_context_preservation_stack(),
_engineering_decisions_stack(),
]
# ── Template definitions ───────────────────────────────────────────
def _thread_integrity_stack() -> StandardTemplate:
"""Thread Integrity -- the knowledge continuity stack.
Preserves the reasoning chain so future practitioners can
reconstruct why decisions were made. Maps to OE-0001 (Foundation)
and the thread integrity concept.
"""
return StandardTemplate(
template_id="oe-thread-integrity",
name="OE Thread Integrity -- Knowledge Continuity Stack",
version="1.0.0-oe-rc3",
author="openengineer",
description=(
"Preserve engineering reasoning across generations. "
"RAG-backed knowledge base + structured documentation + "
"vector search ensures the thread of understanding "
"remains intact."
),
source_type="native",
oe_spec_refs=["OE-0001", "OE-0003", "OE-0009"],
engineering_context={
"decision": (
"Deploy a multi-store knowledge continuity system "
"using Qdrant (semantic vector memory), ChromaDB "
"(document embeddings), Redis (hot-path cache), and "
"MariaDB (persistent structured storage) to preserve "
"the reasoning chain behind engineering decisions."
),
"observation": (
"Engineers across all disciplines report that inherited "
"systems without context records require significantly "
"more effort to maintain, modify, or extend than systems "
"with preserved reasoning. Knowledge continuity breaks "
"when context is lost between practitioners."
),
"alternatives": (
"1. Flat file storage -- simple but lacks semantic "
"search, no cross-reference capability.\n"
"2. Wiki/confluence -- unstructured, no schema "
"enforcement, context decays over time.\n"
"3. Git-only history -- preserves what changed but "
"not why it changed.\n"
"4. Pure vector DB -- semantic search but no "
"structured field validation (fails OE-0000 criterion 1)."
),
"constraints": (
"Must support OE-0000 conformance criteria for context "
"records. Must provide both semantic search (vector) "
"and structured field queries (relational). Must operate "
"entirely on localhost for privacy and latency."
),
"reasoning": (
"The multi-store approach maps to the OE knowledge "
"hierarchy: Redis carries hot-path session state (ms "
"latency), Qdrant carries semantic memory for RAG "
"retrieval, ChromaDB provides dedicated document chunk "
"embeddings, and MariaDB persists structured context "
"records with field-level validation. This satisfies "
"OE-0001's requirement that the thread carry understanding, "
"not just information."
),
"verification": (
"Thread integrity is measured by whether a subsequent "
"practitioner can reconstruct the reasoning behind a "
"prior decision without direct access to the original "
"decision-maker (OE-0001). The vector search recall rate "
"and structured query completeness provide quantitative "
"verification."
),
"lineage": (
"Directly implements OE-0003 (Engineering Context), "
"OE-0009 (Stewardship), and OE-0010 (Inheritance). "
"The Antikythera example (examples/antikythera.md) "
"demonstrates the cost of thread integrity failure."
),
"assumptions": (
"Assumes that engineering context can be partially "
"captured in structured records (acknowledges OE-0003 "
"known limitation regarding tacit knowledge). Assumes "
"vector embeddings provide sufficient semantic fidelity "
"for cross-domain context retrieval."
),
},
stack_config={
"id": "oe-thread-integrity",
"name": "OE Thread Integrity -- Knowledge Continuity Stack",
"description": (
"Preserve engineering reasoning with multi-store "
"knowledge continuity: vector memory + structured "
"records + hot cache."
),
"version": "1.0.0-oe-rc3",
"author": "openengineer",
"tags": [
"openengineer", "thread-integrity", "knowledge",
"rag", "stewardship", "local-first",
],
"tools": [
"ollama", "qdrant", "chromadb", "redis", "mariadb",
"litellm", "openwebui", "fabric", "markitdown",
],
"endpoints": {
"qdrant": "http://localhost:6333",
"chromadb": "http://localhost:8000",
"redis": "localhost:6379",
"mariadb": "localhost:3306",
"litellm": "http://localhost:4000",
"openwebui": "http://localhost:8080",
},
},
notes={
"oe_concept": "thread_integrity",
"memory_hierarchy": (
"Redis (hot, ms latency) -> Qdrant (semantic, us) -> "
"ChromaDB (document chunks) -> MariaDB (persistent, ms)"
),
"setup_order": [
"1. MariaDB, Redis (infrastructure)",
"2. Qdrant, ChromaDB (memory)",
"3. Ollama, LiteLLM (inference)",
"4. OpenWebUI (interface)",
"5. Fabric, MarkItDown (content tools)",
],
},
)
def _verification_loop_stack() -> StandardTemplate:
"""Verification Loop -- the observation-verification cycle stack.
Maps to OE-0007 (Verification) and the verification loop concept
from OE-0001: observe -> verify -> deepen -> repeat.
"""
return StandardTemplate(
template_id="oe-verification-loop",
name="OE Verification Loop -- Test Against Reality Stack",
version="1.0.0-oe-rc3",
author="openengineer",
description=(
"Close the loop between observation and verification. "
"Automated testing + monitoring + code review ensures "
"understanding is tested against reality at every stage."
),
source_type="native",
oe_spec_refs=["OE-0001", "OE-0004", "OE-0007", "OE-0008"],
engineering_context={
"decision": (
"Deploy a verification pipeline combining Aider (code "
"generation + testing), Glances (runtime monitoring), "
"Prometheus/Grafana (metrics collection), and Opik "
"(LLM output evaluation) to close the verification loop "
"defined in OE-0007."
),
"observation": (
"No amount of reasoning, no matter how elegant, "
"substitutes for verification. An engineering model "
"that has not been verified is a hypothesis -- "
"potentially valuable, but not yet reliable enough to "
"base decisions on (OE-0007)."
),
"alternatives": (
"1. Manual testing only -- slow, non-repeatable, "
"doesn't scale.\n"
"2. CI-only verification -- misses runtime behavior, "
"no continuous monitoring.\n"
"3. LLM self-evaluation -- introduces confirmation "
"bias, needs independent verification (OE-0004)."
),
"constraints": (
"Verification must test against observable outcomes, "
"not against models or opinions (OE-0000 criterion 6). "
"Must support both automated and human-in-the-loop "
"verification. Must be able to measure understanding "
"reconstruction success."
),
"reasoning": (
"The verification loop (OE-0007) closes with observation: "
"observe -> survey -> understand -> verify -> (if fail) "
"return to observation. This stack provides tooling at "
"each stage: Aider generates and tests code, Opik evaluates "
"LLM reasoning quality, Glances/Prometheus/Grafana monitor "
"runtime behavior, and Fabric transforms verification "
"results into structured records."
),
"verification": (
"The stack is verified by: (1) running Aider's test "
"suite on a known codebase and confirming it catches "
"seeded bugs, (2) confirming Glances reports match "
"Prometheus metrics, (3) confirming Opik evaluation "
"scores correlate with human assessment."
),
"lineage": (
"Implements the verification loop defined in OE-0007 "
"and the spiral re-evaluation concept from OE-0001. "
"Builds on the observation-first principle (RFC-0001)."
),
"assumptions": (
"Assumes that automated verification can catch a "
"meaningful fraction of engineering errors. Assumes "
"LLM evaluation metrics (Opik) provide useful signal "
"for reasoning quality assessment."
),
},
stack_config={
"id": "oe-verification-loop",
"name": "OE Verification Loop -- Test Against Reality Stack",
"description": (
"Close the observation-verification cycle with "
"automated testing, runtime monitoring, and LLM "
"evaluation."
),
"version": "1.0.0-oe-rc3",
"author": "openengineer",
"tags": [
"openengineer", "verification", "testing",
"monitoring", "spiral-re-evaluation", "local-first",
],
"tools": [
"ollama", "aider", "fabric", "ripgrep", "fd",
"tree_sitter", "glances", "prometheus", "grafana",
"opik",
],
"endpoints": {
"ollama": "http://localhost:11434/v1",
"glances": "http://localhost:61208",
"prometheus": "http://localhost:9090",
"grafana": "http://localhost:3000",
"opik": "http://localhost:3000",
},
},
notes={
"oe_concept": "verification_loop",
"spiral_note": (
"Each verification cycle either eliminates ambiguity "
"or strengthens traceability (OE-0011, Amendment). "
"This is spiral re-evaluation, not linear iteration."
),
},
)
def _stewardship_stack() -> StandardTemplate:
"""Stewardship -- the knowledge maintenance and transmission stack.
Maps to OE-0009 (Stewardship) and OE-0010 (Inheritance).
"""
return StandardTemplate(
template_id="oe-stewardship",
name="OE Stewardship -- Knowledge Maintenance Stack",
version="1.0.0-oe-rc3",
author="openengineer",
description=(
"Maintain and improve engineering knowledge for future "
"practitioners. Document management + knowledge graph + "
"note-taking + RAG ensures context survives handoffs."
),
source_type="native",
oe_spec_refs=["OE-0003", "OE-0009", "OE-0010"],
engineering_context={
"decision": (
"Deploy a knowledge stewardship stack with Paperless-NGX "
"(document archive), Obsidian (knowledge graph notes), "
"Logseq (outliner), and OpenWebUI+RAG (AI-assisted "
"context retrieval) to implement OE-0009 Stewardship "
"and OE-0010 Inheritance."
),
"observation": (
"The most durable knowledge systems -- open-source "
"software, professional engineering bodies, craft "
"guilds -- are governed by stewardship rather than "
"ownership (RFC-0005). The Roman concrete example "
"(examples/roman-concrete.md) shows how stewardship "
"failure (transmitting prescriptions, not principles) "
"causes understanding loss for millennia."
),
"alternatives": (
"1. Shared drive only -- no structure, no graph, "
"no AI-assisted retrieval.\n"
"2. Confluence/Notion -- cloud-dependent, proprietary, "
"no local-first guarantee.\n"
"3. Git-only docs -- preserves artifacts but not "
"reasoning, no knowledge graph."
),
"constraints": (
"Must support both the transmit discipline (stewardship) "
"and the receive discipline (inheritance). Must flag "
"obsolete or incorrect context. Must work offline "
"(OE-0009 known limitation regarding proprietary content)."
),
"reasoning": (
"Stewardship and inheritance are coupled but distinct: "
"stewardship is the transmit direction, inheritance is "
"the receive direction. Paperless-NGX provides the "
"document archive (storing artifacts), Obsidian provides "
"the knowledge graph (connecting reasoning), and RAG "
"via OpenWebUI allows AI-assisted context retrieval "
"(supporting inheritance by helping new practitioners "
"reconstruct understanding)."
),
"verification": (
"Stewardship success is measured by whether inherited "
"context enables understanding reconstruction (OE-0001 "
"thread integrity test). Practical test: can a new team "
"member explain a past decision using only the preserved "
"context records?"
),
"lineage": (
"Directly implements OE-0009 (Stewardship) and "
"OE-0010 (Inheritance). The Bessemer process example "
"(examples/inheritance-steelmaking.md) demonstrates "
"active vs. passive inheritance."
),
"assumptions": (
"Assumes that structured knowledge representation "
"(markdown, knowledge graphs) captures sufficient "
"explicit context to support inheritance. Acknowledges "
"OE-0003's known limitation regarding tacit knowledge."
),
},
stack_config={
"id": "oe-stewardship",
"name": "OE Stewardship -- Knowledge Maintenance Stack",
"description": (
"Steward engineering knowledge across generations "
"with document management, knowledge graphs, and "
"AI-assisted context retrieval."
),
"version": "1.0.0-oe-rc3",
"author": "openengineer",
"tags": [
"openengineer", "stewardship", "inheritance",
"knowledge-management", "documentation", "local-first",
],
"tools": [
"ollama", "openwebui", "chromadb", "docling",
"markitdown", "whisper", "fabric", "paperlessngx",
"obsidian", "logseq",
],
"endpoints": {
"ollama": "http://localhost:11434",
"openwebui": "http://localhost:8080",
"chromadb": "http://localhost:8000",
"paperlessngx": "http://localhost:8000",
},
},
notes={
"oe_concept": "stewardship",
"transmit_receive": (
"Stewardship (transmit) and Inheritance (receive) are "
"coupled but face different failure modes: stewardship "
"fails through neglect, inheritance fails through "
"passivity (OE-0009)."
),
},
)
def _observation_first_stack() -> StandardTemplate:
"""Observation First -- the data collection and measurement stack.
Maps to OE-0004 (Observation) and RFC-0001 (Observation First).
"""
return StandardTemplate(
template_id="oe-observation-first",
name="OE Observation First -- Data Collection Stack",
version="1.0.0-oe-rc3",
author="openengineer",
description=(
"Observe first. Collect, parse, and structure data from "
"multiple sources before reasoning. Web scraping + "
"document parsing + audio transcription + data pipeline "
"tools feed the observation layer."
),
source_type="native",
oe_spec_refs=["OE-0004", "OE-0005", "OE-0001"],
engineering_context={
"decision": (
"Deploy a multi-modal data collection pipeline with "
"Crawl4AI (web scraping), Docling + MarkItDown (document "
"parsing), Whisper (audio transcription), OpenDataLoader "
"(structured data), and Elasticsearch + Meilisearch "
"(search) to implement the observation-first principle."
),
"observation": (
"Throughout the development of Open Engineer, the most "
"robust principles emerged from verifiable encounters "
"with reality (RFC-0001). Henry Darcy's 1856 work "
"demonstrated that measuring first, theorizing second "
"produces principles that endure 160+ years "
"(examples/observation-first-darcy.md)."
),
"alternatives": (
"1. Manual data collection only -- slow, incomplete, "
"not reproducible.\n"
"2. Single-source scraping -- misses multi-modal "
"observations (web + documents + audio).\n"
"3. LLM-only extraction -- introduces model bias, "
"violates observation-first (model is not reality)."
),
"constraints": (
"Observations must be verifiable encounters with reality "
"(OE-0004). The pipeline must distinguish direct "
"observations from corroborated observations. Must "
"support the survey aggregation step (OE-0005)."
),
"reasoning": (
"OE-0004 defines two observation categories: direct "
"(measurements, experiments) and corroborated "
"(independently confirmed external observations). "
"This stack provides tooling for both: web crawlers "
"and document parsers capture direct observations, "
"while search engines enable corroboration against "
"existing knowledge bases."
),
"verification": (
"Pipeline output is verified by: (1) confirming "
"extracted data matches source documents (Docling "
"accuracy), (2) confirming web-scraped content is "
"complete and not truncated, (3) confirming search "
"index recall rate meets threshold."
),
"lineage": (
"Implements OE-0004 (Observation) and provides the "
"input layer for the full OE dependency chain: "
"Observation -> Survey (OE-0005) -> Understanding "
"(OE-0006) -> Verification (OE-0007)."
),
"assumptions": (
"Assumes that the majority of engineering observations "
"can be captured through text, documents, and audio. "
"Acknowledges that some observations (tactile, "
"environmental) require specialized sensors beyond "
"this stack's scope."
),
},
stack_config={
"id": "oe-observation-first",
"name": "OE Observation First -- Data Collection Stack",
"description": (
"Multi-modal observation pipeline: web scraping + "
"document parsing + audio transcription + search "
"indexing."
),
"version": "1.0.0-oe-rc3",
"author": "openengineer",
"tags": [
"openengineer", "observation-first", "data-pipeline",
"scraping", "parsing", "search", "local-first",
],
"tools": [
"crawl4ai", "docling", "markitdown", "whisper",
"opendataloader", "opendataloader_pdf", "fabric",
"elasticsearch", "meilisearch", "understand_anything",
],
"endpoints": {
"elasticsearch": "http://localhost:9200",
"meilisearch": "http://localhost:7700",
},
},
notes={
"oe_concept": "observation_first",
"observation_types": (
"Direct: web scraping, document parsing, audio "
"transcription. Corroborated: search index matching "
"against existing knowledge (OE-0004 taxonomy)."
),
},
)
def _context_preservation_stack() -> StandardTemplate:
"""Context Preservation -- the complete OE context record stack.
Maps to OE-0003 (Engineering Context) and OE-0008 (Decisions).
This is the "full stack" for OE compliance -- all ten conformance
criteria are supported.
"""
return StandardTemplate(
template_id="oe-context-preservation",
name="OE Context Preservation -- Full Engineering Context Stack",
version="1.0.0-oe-rc3",
author="openengineer",
description=(
"The complete OE-compliant engineering context preservation "
"stack. Multi-agent reasoning + structured documentation + "
"full RAG + vector search + knowledge graph + monitoring. "
"Satisfies all ten OE-0000 conformance criteria."
),
source_type="native",
oe_spec_refs=[
"OE-0000", "OE-0001", "OE-0002", "OE-0003", "OE-0004",
"OE-0005", "OE-0006", "OE-0007", "OE-0008", "OE-0009",
"OE-0010", "OE-0011",
],
engineering_context={
"decision": (
"Deploy a comprehensive stack that implements every layer "
"of the Open Engineer specification: multi-agent reasoning "
"(CrewAI/AutoGen/Agno) for decision analysis, full RAG "
"pipeline for context retrieval, structured documentation "
"tools for context record creation, and monitoring for "
"verification tracking."
),
"observation": (
"Engineers across all disciplines report that inherited "
"systems without context records require significantly "
"more effort to maintain, modify, or extend (RFC-0003). "
"The 47 retaining wall failure survey (examples/"
"survey-retaining-wall-failure.md) demonstrated that 71% "
"of failures involved conditions known at design time but "
"not accounted for -- a direct context preservation failure."
),
"alternatives": (
"1. Manual documentation only -- no structured schema, "
"no conformance checking, fails OE-0000 criterion 1.\n"
"2. AI-only context generation -- no observation "
"grounding, violates OE-0004 observation-first.\n"
"3. Issue tracker only -- captures tasks but not "
"reasoning, alternatives, or constraints.\n"
"4. Wiki + git notes -- better than nothing but no "
"schema enforcement, no conformance validation."
),
"constraints": (
"Must satisfy all ten OE-0000 conformance criteria. "
"Must preserve the OE-0003 nine required fields. Must "
"support both creation (stewardship) and retrieval "
"(inheritance) of context records. Must support the "
"amendment workflow (OE-0011)."
),
"reasoning": (
"The stack maps to the full OE dependency chain: "
"Observation (data tools) -> Survey (RAG aggregation) -> "
"Understanding (multi-agent analysis) -> Verification "
"(monitoring + testing) -> Decision (structured record) -> "
"Stewardship (documentation) -> Inheritance (knowledge "
"graph + RAG retrieval). Each layer of the AI-LSC 10-layer "
"architecture provides tooling for the corresponding OE "
"specification layer."
),
"verification": (
"Verified against OE-0000's ten conformance criteria: "
"(1) field completeness via schema validation, "
"(2) decision specificity via template constraints, "
"(3) observation traceability via source references, "
"(4) constraint bounding via required constraints field, "
"(5) alternatives plural via required alternatives field, "
"(6) verification against reality via monitoring tools, "
"(7) reasoning references alternatives via template "
"structure, (8) lineage traceability via OE spec refs, "
"(9) assumption awareness via required field, "
"(10) no contradiction via validator."
),
"lineage": (
"Implements the complete OE-0000 through OE-0011 "
"specification chain. Every OE document is represented. "
"The template itself is an OE amendment (OE-0011) that "
"extends the standard into the AI tooling domain."
),
"assumptions": (
"Assumes that the OE specification's ten conformance "
"criteria provide sufficient coverage for context record "
"quality. Assumes that AI-assisted context extraction "
"can reduce (but not eliminate) the manual effort of "
"creating compliant context records. Acknowledges OE-0003's "
"known limitation regarding tacit knowledge."
),
},
stack_config={
"id": "oe-context-preservation",
"name": "OE Context Preservation -- Full Engineering Context Stack",
"description": (
"Complete OE-compliant context preservation: multi-agent "
"reasoning + full RAG + knowledge graph + structured "
"documentation + monitoring. 25+ tools."
),
"version": "1.0.0-oe-rc3",
"author": "openengineer",
"tags": [
"openengineer", "context-preservation", "full-stack",
"multi-agent", "rag", "knowledge-graph", "local-first",
],
"tools": [
"ollama", "vllm", "litellm", "qdrant", "chromadb",
"redis", "mariadb", "elasticsearch", "meilisearch",
"crewai", "autogen", "agno", "langchain",
"aider", "fabric", "docling", "markitdown", "whisper",
"openwebui", "glances", "prometheus", "grafana",
"opik", "crawl4ai", "opendataloader", "graphrag",
],
"endpoints": {
"ollama": "http://localhost:11434/v1",
"vllm": "http://localhost:8000",
"litellm": "http://localhost:4000",
"qdrant": "http://localhost:6333",
"chromadb": "http://localhost:8000",
"redis": "localhost:6379",
"mariadb": "localhost:3306",
"elasticsearch": "http://localhost:9200",
"meilisearch": "http://localhost:7700",
"openwebui": "http://localhost:8080",
"glances": "http://localhost:61208",
"prometheus": "http://localhost:9090",
"grafana": "http://localhost:3000",
"opik": "http://localhost:3000",
},
},
notes={
"oe_concept": "context_preservation",
"conformance_coverage": (
"This template satisfies all 10 OE-0000 conformance "
"criteria. The conformance_score for the built-in "
"engineering_context is 100%."
),
"setup_order": [
"Tier 1 (infrastructure): redis, mariadb, elasticsearch",
"Tier 2 (memory): qdrant, chromadb, meilisearch",
"Tier 3 (inference): ollama, vllm, litellm",
"Tier 4 (data tools): docling, markitdown, whisper, crawl4ai, opendataloader",
"Tier 5 (agents): crewai, autogen, agno, langchain",
"Tier 6 (coding): aider, fabric",
"Tier 7 (monitoring): glances, prometheus, grafana, opik",
"Tier 8 (interface): openwebui",
],
"memory_hierarchy": (
"5-tier: Redis (hot, ms) -> Qdrant (semantic, us) -> "
"ChromaDB (documents) -> Elasticsearch (full-text) -> "
"Meilisearch (typo-tolerant instant) -> MariaDB (persistent)"
),
},
)
def _engineering_decisions_stack() -> StandardTemplate:
"""Engineering Decisions -- the structured decision-making stack.
Maps to OE-0008 (Decisions) and the decision-as-context-record concept.
Focused on AI-assisted engineering decision analysis and recording.
"""
return StandardTemplate(
template_id="oe-engineering-decisions",
name="OE Engineering Decisions -- Structured Decision Stack",
version="1.0.0-oe-rc3",
author="openengineer",
description=(
"Structure and analyze engineering decisions with AI "
"assistance. Multi-agent collaboration (CrewAI) + code "
"review (Aider) + pair programming (Aider) + document "
"generation (Fabric) produce OE-0003 compliant context "
"records for every significant decision."
),
source_type="native",
oe_spec_refs=["OE-0003", "OE-0007", "OE-0008"],
engineering_context={
"decision": (
"Deploy a decision engineering stack with CrewAI "
"(multi-agent analysis), Aider (code review + pair "
"programming), Fabric (document transformation), "
"and Ollama (local LLM reasoning) to produce structured "
"engineering context records that satisfy OE-0003."
),
"observation": (
"The software architecture migration example "
"(examples/software-architecture-migration.md) shows "
"that without context records, a future developer "
"cannot reconstruct why Python was ever used when the "
"system was rewritten in Rust. The biomedical implant "
"example (examples/biomedical-implant-context.md) shows "
"that cross-discipline constraints are the most commonly "
"lost context."
),
"alternatives": (
"1. ADR (Architecture Decision Record) files only -- "
"no AI assistance, no conformance checking, manual "
"maintenance.\n"
"2. Issue tracker decisions -- no structured format, "
"no alternatives/constraints documentation.\n"
"3. Unstructured meeting notes -- no reproducibility, "
"no verification, fails OE-0000 criterion 6."
),
"constraints": (
"Every decision must produce a context record with all "
"nine OE-0003 required fields. Must support multi-agent "
"analysis (researcher + reviewer + recorder roles). "
"Must verify reasoning against alternatives (OE-0000 "
"criterion 7)."
),
"reasoning": (
"CrewAI provides role-based agent teams that mirror "
"the OE decision process: a Researcher agent gathers "
"observations (OE-0004), a Reviewer agent verifies "
"reasoning (OE-0007), and a Recorder agent structures "
"the output as an OE-0003 compliant context record. "
"Aider provides code-level decision context, and Fabric "
"transforms the output into documentation."
),
"verification": (
"Each generated context record is validated against "
"the ten OE-0000 conformance criteria. The "
"decision_specificity criterion (criterion 2) ensures "
"each record identifies exactly one choice. The "
"alternatives_plural criterion (criterion 5) ensures "
"at least one rejected alternative is documented."
),
"lineage": (
"Implements OE-0008 (Decisions) and depends on OE-0003 "
"(Engineering Context) for the record structure. "
"The 20 examples across 9 disciplines demonstrate "
"context record patterns this stack should produce."
),
"assumptions": (
"Assumes that local LLMs (32B+ parameters) provide "
"sufficient reasoning quality for decision analysis. "
"Assumes that the OE-0003 nine-field structure captures "
"the essential context for most engineering decisions."
),
},
stack_config={
"id": "oe-engineering-decisions",
"name": "OE Engineering Decisions -- Structured Decision Stack",
"description": (
"AI-assisted engineering decision analysis with "
"multi-agent collaboration and OE-0003 compliant "
"context record generation."
),
"version": "1.0.0-oe-rc3",
"author": "openengineer",
"tags": [
"openengineer", "decisions", "multi-agent",
"code-review", "documentation", "local-first",
],
"tools": [
"ollama", "litellm", "crewai", "autogen", "agno",
"aider", "fabric", "chromadb", "ripgrep", "fd",
"tree_sitter",
],
"endpoints": {
"ollama": "http://localhost:11434/v1",
"litellm": "http://localhost:4000",
"chromadb": "http://localhost:8000",
},
},
notes={
"oe_concept": "engineering_decisions",
"agent_roles": (
"Researcher: gathers observations (OE-0004). "
"Reviewer: verifies reasoning (OE-0007). "
"Recorder: produces OE-0003 context record. "
"Critic: checks conformance (OE-0000)."
),
"decision_record_template": (
"Each decision produces: Decision (what), Observation "
"(why now), Alternatives (what else), Constraints "
"(what bounded), Reasoning (why this), Verification "
"(how confirmed), Lineage (what prior work), "
"Assumptions (what's unknown)."
),
},
)

8
src/ai_lsc/registry/stack_templates/__init__.py Executable file
View File

@ -0,0 +1,8 @@
"""AI-LSC stack templates sub-package.
Pre-configured tool stacks that can be applied via the StackWizard
instead of manually selecting individual tools.
Each template is a JSON file in this directory defining a curated set
of tools (by registry ID or git source URL) for a specific use case.
"""

33
src/ai_lsc/registry/stack_templates/agentic-os-stack.json Executable file
View File

@ -0,0 +1,33 @@
{
"id": "agentic-os-stack",
"name": "Agentic OS — Full Orchestration Stack",
"description": "Production agentic orchestration stack: Ollama inference + LibreChat agent frontend + Qdrant vector memory + Redis task queue + LiteLLM multi-provider routing. All LLM endpoints point to localhost — no external API keys needed.",
"version": "2.0",
"author": "ai-lsc",
"tags": ["agentic", "orchestration", "function-calling", "rag", "multi-model", "production", "local-first"],
"endpoints": {
"litellm_base": "http://localhost:4000",
"litellm_model": "ollama/qwen2.5:32b",
"librechat_endpoint": "http://localhost:3080",
"qdrant_endpoint": "http://localhost:6333",
"redis_endpoint": "localhost:6379"
},
"tools": [
"ollama",
"redis",
"qdrant",
"litellm",
"n8n",
"librechat",
"mariadb",
"glances",
"fabric"
],
"notes": {
"architecture": "The agents/ package bridges LibreChat's function-calling to AI-LSC's RuntimeExecutor. The LLM can start/stop services, pull models, and inject skills through natural language.",
"model_routing": "LiteLLM proxy at localhost:4000 normalizes all local Ollama models into a single OpenAI-compatible endpoint. Ollama serves at localhost:11434. LiteLLM config points: model_list: [{model_name: 'qwen2.5', litellm_params: {model: 'ollama/qwen2.5:32b', api_base: 'http://localhost:11434/v1'}}].",
"memory_layers": "Redis (localhost:6379) = hot path (task queue, pub/sub, status cache). MariaDB = cold path (audit logs, persisted memory, config). Qdrant (localhost:6333) = semantic index (RAG, skill matching, vector search).",
"agent_frontend": "LibreChat at localhost:3080 provides multi-provider support with native OpenAI tool-calling format. Configure it to point at LiteLLM as the assistant endpoint.",
"workflow": "n8n at localhost:5678 orchestrates complex multi-step agent workflows beyond single-turn tool calls."
}
}

24
src/ai_lsc/registry/stack_templates/ai-image-gen-local.json Executable file
View File

@ -0,0 +1,24 @@
{
"id": "ai-image-gen-local",
"name": "AI Image Generation — Local Creative Studio",
"description": "Run Stable Diffusion, FLUX, and ComfyUI entirely on your GPU. No DALL-E subscriptions, no Midjourney fees. Generate images, edit photos, and build automated generation pipelines — the homelab creative AI stack that's exploding on YouTube.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["image-gen", "stable-diffusion", "flux", "comfyui", "creative", "youtube-trending", "local-first", "gpu"],
"endpoints": {
"comfyui": "http://localhost:8188",
"forge": "http://localhost:7860"
},
"tools": [
"forge",
"invokeai",
"fabric"
],
"notes": {
"youtube_context": "Local AI image generation has massive YouTube presence. Channels like @mreflow, @aaronweikle, and @flyingjunior run FLUX.1 and SDXL locally. ComfyUI tutorials consistently hit 1M+ views. This stack covers the three most popular workflows.",
"recommended_models": "FLUX.1-dev (12B, best quality), FLUX.1-schnell (fast generation), stable-diffusion-xl-base-1.0 (classic SDXL), juggernaut-xl (fine-tuned SDXL), dreamshaper-xl (lightweight)",
"setup": "Forge (optimized WebUI) at localhost:7860 is the easiest entry point. InvokeAI provides a more polished creative workflow. Both auto-download models on first run. Set --listen 0.0.0.0 to access from other devices.",
"workflow": "Quick gen: Forge with FLUX.1-schnell (~2s per image on 4090). Quality gen: FLUX.1-dev with refined prompts. Batch processing: Forge API + Fabric for automated pipeline workflows. ControlNet for pose/depth/edge-guided generation.",
"tips": "8GB VRAM minimum for FLUX.1-schnell (FP8). 24GB for FLUX.1-dev. Use xformers and sdp attention for speed. Torch compile adds ~30% throughput after first warmup. Batch size 1 with tiled VAE for large resolutions on limited VRAM."
}
}

27
src/ai_lsc/registry/stack_templates/aider-ollama-vibe-coding.json Executable file
View File

@ -0,0 +1,27 @@
{
"id": "aider-ollama-vibe-coding",
"name": "Aider + Ollama Vibe Coding Stack",
"description": "The #1 YouTube AI coding stack of 2025. Aider pair-programmer wired to local Ollama models. Zero API keys, zero cloud — vibe code entirely offline with 32B+ models that rival GPT-4 for coding tasks.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["vibe-coding", "aider", "ollama", "coding", "youtube-trending", "local-first", "offline"],
"endpoints": {
"ollama_base": "http://localhost:11434/v1",
"aider_model": "ollama/qwen2.5-coder:32b"
},
"tools": [
"ollama",
"aider",
"fabric",
"ripgrep",
"fd",
"tree_sitter"
],
"notes": {
"youtube_context": "This is the exact stack from the viral 'Vibe Coding with Local AI' videos: Aider + Ollama + Qwen2.5-Coder. Creators like @networkchuck, @cbarks, and @techwithtim have built full projects live on stream with this combo.",
"recommended_models": "qwen2.5-coder:32b (best balance), deepseek-coder-v2:236b (strongest), codestral:22b (fast), llama3.1:70b (general), phi-4:14b (lightweight)",
"setup": "Run: ollama pull qwen2.5-coder:32b && aider --model ollama/qwen2.5-coder:32b. Aider auto-discovers Ollama on localhost:11434.",
"workflow": "Aider edits code in your git repo using local LLM. Fabric transforms text/prompts. ripgrep + fd + tree-sitter power Aider's repository map for large codebase awareness.",
"tips": "Use aider --message 'implement X' for single tasks. Use aider --chat for interactive sessions. Add .aider.conf.yml to your project root for per-project model config."
}
}

46
src/ai_lsc/registry/stack_templates/claude-code-setup.json Executable file
View File

@ -0,0 +1,46 @@
{
"id": "claude-code-setup",
"name": "Claude Code Local Stack",
"description": "Claude Code development environment wired to local Ollama inference. No cloud API keys required — all LLM calls route through localhost:11434. Includes memory, prompt engineering, and multi-agent coordination.",
"version": "2.0",
"author": "ai-lsc",
"tags": ["claude", "development", "ai-coding", "agent", "memory", "local-first"],
"endpoints": {
"claude_api_base": "http://localhost:11434/v1",
"anthropic_api_key": "ollama",
"anthropic_model": "claude-4-sonnet"
},
"tools": [
"claude_code",
"ollama",
"aider",
"fabric",
{
"id": "claude_mem",
"name": "Claude Mem",
"source": "https://github.com/nicely-done/claude-mem",
"category": "Memory",
"role": "Context Memory",
"description": "Persistent conversation memory layer for Claude Code sessions",
"installer": {"type": "git", "pkg": "https://github.com/nicely-done/claude-mem"},
"launcher": {"type": "tmux", "cmd": "claude-mem serve --port {port}", "default_port": 9600},
"flags": {"has_cli": true, "has_gui": false, "has_web": true}
},
{
"id": "claude_squad",
"name": "Claude Squad",
"source": "https://github.com/nicely-done/claude-squad",
"category": "Multi-Agent",
"role": "Team Coordination",
"description": "Multi-agent team orchestration for parallel Claude Code instances",
"installer": {"type": "git", "pkg": "https://github.com/nicely-done/claude-squad"},
"launcher": {"type": "tmux", "cmd": "claude-squad coordinate --port {port}", "default_port": 9603},
"flags": {"has_cli": true, "has_gui": false, "has_web": true}
}
],
"notes": {
"setup": "Set CLAUDE_CODE_USE_BEDROCK=1 or configure claude-code to point at the Ollama endpoint. Anthropic Claude models are proxied through LiteLLM or Ollama's OpenAI-compatible API at localhost:11434.",
"recommended_models": "claude-4-sonnet (14B+ local), qwen2.5-coder:32b, deepseek-coder-v2:236b",
"workflow": "Claude Code → Ollama (localhost:11434) → local weights. Aider handles pair programming with the same endpoint. Fabric provides CLI text-processing pipelines."
}
}

29
src/ai_lsc/registry/stack_templates/deepseek-r1-local-reasoning.json Executable file
View File

@ -0,0 +1,29 @@
{
"id": "deepseek-r1-local-reasoning",
"name": "DeepSeek R1 — Local Reasoning Engine",
"description": "Run DeepSeek R1 (the open-source reasoning model that challenged OpenAI o1) entirely locally. vLLM or llama.cpp serves the 70B distilled model with chain-of-thought reasoning visible in real-time. The most hyped local AI setup of early 2025.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["deepseek", "reasoning", "r1", "vllm", "llama.cpp", "youtube-trending", "local-first", "math", "coding"],
"endpoints": {
"ollama_base": "http://localhost:11434",
"vllm_base": "http://localhost:8000",
"litellm_base": "http://localhost:4000"
},
"tools": [
"ollama",
"llamacpp",
"vllm",
"litellm",
"openwebui",
"aider",
"fabric"
],
"notes": {
"youtube_context": "DeepSeek R1 broke the internet in Jan 2025 with reasoning capabilities matching OpenAI o1 at a fraction of the cost. YouTubers like @fmateo09, @aaronweikle, and @marcusrbrown showed how to run the distilled 32B/70B models locally for free. This template recreates that exact setup.",
"recommended_models": "deepseek-r1:32b (24GB VRAM via vLLM), deepseek-r1:70b (2x 24GB GPUs or quantized), deepseek-r1-distill-qwen:32b (fastest reasoning), deepseek-coder-v2:236b (split across GPUs)",
"setup": "Fast path: ollama pull deepseek-r1:32b && open-webui (auto-connects). High-throughput: vLLM serves with speculative decoding at localhost:8000. LiteLLM at 4000 normalizes the endpoint for any OpenAI-compatible client.",
"workflow": "Reasoning queries hit the model through Open WebUI or any API client. The chain-of-thought (thinking tokens) is visible in real-time. Aider uses the same endpoint for reasoning-powered code generation. Fabric pipes reasoning output through text-transform chains.",
"tips": "vLLM with --enable-chunked-prefill handles long contexts better. For <24GB VRAM, use deepseek-r1:14b or the distilled Qwen variants. Set temperature=0.6 for best reasoning quality — higher temps degrade chain-of-thought coherence."
}
}

41
src/ai_lsc/registry/stack_templates/hermes-ai-coder-stack.json Executable file
View File

@ -0,0 +1,41 @@
{
"id": "hermes-ai-coder-stack",
"name": "Hermes — AI Coder & Agent Stack",
"description": "Maximum coding intelligence: Aider pair programming + Hermes agent orchestration + Agno multi-agent framework + CrewAI team collaboration + Ollama local inference + OpenWebUI chat frontend + full codebase awareness tools. The everything-stack for serious AI-assisted development.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["hermes", "coding", "agent", "multi-agent", "agno", "crewai", "aider", "premium", "local-first"],
"endpoints": {
"ollama_base": "http://localhost:11434/v1",
"litellm_base": "http://localhost:4000",
"openwebui": "http://localhost:3000",
"hermes_agent": "http://localhost:17051",
"hermes_dashboard": "http://localhost:17050"
},
"tools": [
"ollama",
"aider",
"litellm",
"openwebui",
"hermes",
"hermes_agent",
"hermes_desktop",
"agno",
"crewai",
"autogen",
"fabric",
"chromadb",
"ripgrep",
"fd",
"tree_sitter"
],
"notes": {
"philosophy": "Hermes is the messenger of the gods — this stack routes every coding task through the best available local AI pathway. Aider for pair programming, Agno for structured agent pipelines, CrewAI for team-based task decomposition, Hermes Desktop for the unified agent GUI, and Hermes Agent for the autonomous runtime. LiteLLM normalizes all model access so every tool talks to Ollama on localhost:11434.",
"recommended_models": "qwen2.5-coder:32b (primary coding, beats GPT-4 on SWE-bench), deepseek-coder-v2:236b (complex reasoning), codestral:22b (fast edits), llama3.1:70b (architectural planning), phi-4:14b (quick tasks), nomic-embed-text (codebase embeddings for RAG)",
"coding_interface": "Aider is the primary coding interface — it has the best git integration, whole-repo awareness, and cost-efficient token usage with local models. For interactive exploration, Hermes Desktop provides a visual agent environment. For conversational coding, OpenWebUI connects to the same Ollama endpoint.",
"agentic_hierarchy": "Single tasks → Aider (fast, direct). Multi-step pipelines → Agno (sequential agents with memory). Team projects → CrewAI (role-based: Architect, Coder, Reviewer, Tester). Full autonomy → Hermes Agent (persistent background agent with tool access). Cross-framework orchestration → AutoGen (Microsoft's framework for heterogeneous agent teams).",
"codebase_awareness": "rigrep finds symbols, fd navigates directories, tree-sitter parses AST. Aider uses repo-map to build a compressed representation of your entire codebase. ChromaDB indexes code chunks for semantic search. This gives every agent in the stack full awareness of your project structure.",
"setup": "1) Pull models: ollama pull qwen2.5-coder:32b && ollama pull nomic-embed-text. 2) Start Ollama (systemd or tmux). 3) Start LiteLLM pointing at localhost:11434. 4) Launch Hermes Desktop for the GUI agent. 5) Run aider in your project repo. 6) OpenWebUI for browser-based chat with your code.",
"tips": "Use LiteLLM to switch between models per-task without restarting tools. Aider's /ask command lets you query the model without editing files — perfect for quick questions. Agno agents can call Aider as a tool for code modifications. ChromaDB codebase indexing runs once, then every agent benefits from semantic search. Keep Hermes Dashboard open on a second monitor to monitor all agent activity."
}
}

31
src/ai_lsc/registry/stack_templates/local-llm-lab.json Executable file
View File

@ -0,0 +1,31 @@
{
"id": "local-llm-lab",
"name": "Local LLM Lab",
"description": "Self-hosted LLM playground with multiple inference backends, model management, vector store, and chat interface. Everything runs on localhost — zero cloud dependencies.",
"version": "2.0",
"author": "ai-lsc",
"tags": ["llm", "local-ai", "inference", "chat", "rag", "local-first"],
"endpoints": {
"ollama_base": "http://localhost:11434",
"openwebui": "http://localhost:3000",
"chromadb": "http://localhost:8000",
"litellm_base": "http://localhost:4000"
},
"tools": [
"ollama",
"llamacpp",
"vllm",
"litellm",
"openwebui",
"chromadb",
"whisper",
"docling",
"aider",
"fabric"
],
"notes": {
"setup": "Ollama at 11434 is the primary inference engine. vLLM and llama.cpp are alternative backends for GGUF/exl2 formats. LiteLLM at 4000 provides a unified OpenAI-compatible API over all backends.",
"recommended_models": "llama3.1:70b, qwen2.5-coder:32b, mistral-nemo:12b, codestral:22b, phi-4:14b, gemma2:27b",
"workflow": "OpenWebUI provides the browser chat frontend at localhost:3000 connected to Ollama. ChromaDB handles RAG document embeddings. Whisper does local speech-to-text. Docling converts PDFs to Markdown for ingestion. Aider connects to the same Ollama endpoint for pair programming. Fabric provides CLI text-transform pipelines."
}
}

373
src/ai_lsc/registry/stack_templates/manager.py Executable file
View File

@ -0,0 +1,373 @@
"""Stack template manager -- loads, lists, and resolves templates.
Templates are JSON files in ``ai_lsc/registry/stack_templates/``. Each
template defines:
* ``name``: human-readable template name
* ``description``: one-line summary
* ``tags``: searchable category labels
* ``version``: template version string
* ``tools``: list of tool references (registry IDs or git-source dicts)
The manager can resolve a template into a flat list of tool IDs by
merging registry lookups with git-source entries (which are auto-registered
as new tools on the fly).
Open Engineer Integration
-------------------------
When the ``openengineer`` sub-package is importable, the manager
automatically loads OE-derived standard templates. These templates
carry full OE-0003 engineering context alongside AI-LSC stack config.
An optional ``openengineer_dir`` parameter (or the environment
variable ``AI_LSC_OE_DIR``) points to a local Open Engineer repo
checkout. All OE markdown files are imported and converted to
AI-LSC templates via the standard template bridge.
"""
from __future__ import annotations
import json
import os
from pathlib import Path
from typing import Any
_TEMPLATES_DIR = Path(__file__).resolve().parent
# Try to import the OE integration; fail gracefully if unavailable.
try:
from ai_lsc.registry.openengineer.schema import (
standard_template_to_ai_lsc,
)
from ai_lsc.registry.openengineer.templates import get_templates as _get_oe_templates
from ai_lsc.registry.openengineer.importer import OpenEngineerImporter
_HAS_OE = True
except ImportError:
_HAS_OE = False
class StackTemplateManager:
"""Discover and resolve stack templates.
Parameters
----------
extra_dirs :
Additional directories to scan for template files
(e.g. user-supplied ``~/.config/ai-lsc/stack_templates/``).
"""
def __init__(
self,
extra_dirs: list[str | Path] | None = None,
openengineer_dir: str | Path | None = None,
) -> None:
self._templates: dict[str, dict[str, Any]] = {}
self._scan_dirs = [_TEMPLATES_DIR]
if extra_dirs:
self._scan_dirs.extend(
Path(d) for d in extra_dirs if Path(d).is_dir()
)
self._oe_templates_count: int = 0
self._load_all()
# Load OE-derived templates
if _HAS_OE:
self._load_openengineer_templates(openengineer_dir)
# ── Discovery ────────────────────────────────────────────────────
def _load_all(self) -> None:
"""Scan all template directories and load valid templates."""
for directory in self._scan_dirs:
for fname in sorted(directory.iterdir()):
if fname.suffix in (".json", ".yaml", ".yml"):
try:
tpl = self._load_file(fname)
except Exception:
continue
if tpl:
self._templates[tpl["id"]] = tpl
@staticmethod
def _load_file(path: Path) -> dict[str, Any] | None:
"""Load and validate a single template file."""
suffix = path.suffix.lower()
if suffix == ".json":
raw = json.loads(path.read_text(encoding="utf-8"))
else:
# YAML support -- optional dependency
try:
import yaml # noqa: F401
except ImportError:
return None
raw = yaml.safe_load(path.read_text(encoding="utf-8"))
if not isinstance(raw, dict):
return None
if "name" not in raw or "tools" not in raw:
return None
# Synthesise a stable ID from the filename if not given
raw.setdefault("id", path.stem)
raw.setdefault("tags", [])
raw.setdefault("version", "1.0")
raw.setdefault("description", "")
raw.setdefault("author", "ai-lsc")
return raw
# ── OpenEngineer integration ───────────────────────────────────
def _load_openengineer_templates(
self,
openengineer_dir: str | Path | None = None,
) -> None:
"""Load built-in OE templates and optionally import from a repo dir.
Built-in OE templates are always loaded (they carry full
engineering context for key OE concepts). If *openengineer_dir*
is provided (or the ``AI_LSC_OE_DIR`` env var is set), all
discoverable OE markdown files are also imported.
"""
# 1. Load built-in OE-derived templates
for oe_tpl in _get_oe_templates():
ai_lsc_dict = standard_template_to_ai_lsc(oe_tpl)
tid = ai_lsc_dict.get("id", "")
if tid:
self._templates[tid] = ai_lsc_dict
self._oe_templates_count += 1
# 2. Import from an Open Engineer repo directory (if provided)
oe_dir = openengineer_dir or os.environ.get("AI_LSC_OE_DIR")
if oe_dir and Path(oe_dir).is_dir():
importer = OpenEngineerImporter()
imported = importer.import_as_ai_lsc_templates(oe_dir)
for tpl_dict in imported:
tid = tpl_dict.get("id", "")
if tid and tid not in self._templates:
self._templates[tid] = tpl_dict
self._oe_templates_count += 1
def import_openengineer_file(
self,
path: str | Path,
stack_config: dict[str, Any] | None = None,
) -> str | None:
"""Import a single Open Engineer file as an AI-LSC template.
Parameters
----------
path :
Path to an OE markdown file.
stack_config :
Optional AI-LSC stack config to merge with inferred config.
Returns
-------
The template ID of the imported template, or ``None`` on failure.
"""
if not _HAS_OE:
return None
try:
# M-41: OpenEngineerImporter already imported at module level
# (line 44); don't re-import inside this method.
importer = OpenEngineerImporter()
oe_tpl = importer.import_file(path, stack_config=stack_config)
ai_lsc_dict = standard_template_to_ai_lsc(oe_tpl)
tid = ai_lsc_dict.get("id", "")
if tid:
self._templates[tid] = ai_lsc_dict
self._oe_templates_count += 1
return tid
except (OSError, ValueError, KeyError, AttributeError):
return None
# ── Queries ──────────────────────────────────────────────────────
def list_templates(self) -> list[dict[str, Any]]:
"""Return all loaded templates as summary dicts."""
return [
{
"id": tpl["id"],
"name": tpl["name"],
"description": tpl.get("description", ""),
"tags": tpl.get("tags", []),
"version": tpl.get("version", "1.0"),
"tool_count": len(tpl.get("tools", [])),
"source": self._template_source(tpl),
}
for tpl in sorted(
self._templates.values(), key=lambda t: t["name"]
)
]
def get_template(self, template_id: str) -> dict[str, Any] | None:
"""Return the full template dict, or ``None``."""
return self._templates.get(template_id)
def _is_builtin(self, template_id: str) -> bool:
return any(
(d / f"{template_id}.json").exists()
or (d / f"{template_id}.yaml").exists()
or (d / f"{template_id}.yml").exists()
for d in self._scan_dirs
)
def _template_source(self, tpl: dict[str, Any]) -> str:
"""Determine the source category of a template."""
tags = [t.lower() for t in tpl.get("tags", [])]
notes = tpl.get("notes", {})
# OE-native templates (built-in from the openengineer package)
if "openengineer" in tags and "oe-context" in tags:
return "openengineer"
# OE-imported templates (from a repo directory)
if "openengineer" in tags:
return "openengineer-import"
if self._is_builtin(tpl.get("id", "")):
return "builtin"
return "custom"
def filter_by_tag(self, tag: str) -> list[dict[str, Any]]:
"""Return templates matching a given tag."""
return [
t for t in self.list_templates()
if tag.lower() in [x.lower() for x in t["tags"]]
]
# ── Resolution ────────────────────────────────────────────────────
def resolve_tool_ids(
self,
template_id: str,
registry: object | None = None,
) -> tuple[list[str], list[dict[str, Any]]]:
"""Resolve a template into registry tool IDs + new-tool entries.
Parameters
----------
template_id :
The template to resolve.
registry :
Optional ``RegistryManager`` used to validate existing
tool IDs and look up dependency chains.
Returns
-------
(known_ids, new_entries) :
*known_ids* are tool IDs already in the registry.
*new_entries* are raw dicts for tools that need to be
auto-registered (git-source entries).
Example
-------
>>> ids, new = mgr.resolve_tool_ids("claude-code-setup", registry)
>>> # ids = ["claude_code", "ollama", "aider", ...]
>>> # new = [{"name": "Godmod3", "source": "https://...", ...}]
"""
tpl = self._templates.get(template_id)
if not tpl:
return [], []
known_ids: list[str] = []
new_entries: list[dict[str, Any]] = []
for tool_ref in tpl.get("tools", []):
if isinstance(tool_ref, str):
# Plain registry ID reference
known_ids.append(tool_ref)
elif isinstance(tool_ref, dict):
# Structured reference
if "id" in tool_ref:
known_ids.append(tool_ref["id"])
elif "source" in tool_ref:
# Git-source: new tool to auto-register
new_entries.append(tool_ref)
# Synthesise an ID from the source URL
known_ids.append(tool_ref.get(
"id",
_derive_id_from_source(tool_ref["source"]),
))
# M-18: dedup while preserving order via dict.fromkeys()
deduped_ids = list(dict.fromkeys(known_ids))
return deduped_ids, new_entries
# ── Creation ─────────────────────────────────────────────────────
def create_template(
self,
name: str,
tools: list[str | dict[str, Any]],
description: str = "",
tags: list[str] | None = None,
template_id: str | None = None,
save_dir: str | Path | None = None,
) -> dict[str, Any]:
"""Create a new template and optionally save it to disk.
Parameters
----------
name :
Human-readable template name.
tools :
List of tool IDs (str) or git-source dicts.
description :
One-line description.
tags :
Category labels for searchability.
template_id :
Override the auto-derived ID (defaults to slugified name).
save_dir :
Directory to write the JSON file. If ``None`` the
template is only kept in memory.
Returns
-------
The created template dict.
"""
tpl: dict[str, Any] = {
"id": template_id or name.lower().replace(" ", "-").replace("_", "-"),
"name": name,
"description": description,
"tags": tags or [],
"version": "1.0",
"author": "user",
"tools": tools,
}
self._templates[tpl["id"]] = tpl
if save_dir:
out = Path(save_dir)
out.mkdir(parents=True, exist_ok=True)
(out / f"{tpl['id']}.json").write_text(
json.dumps(tpl, indent=4, ensure_ascii=False),
encoding="utf-8",
)
return tpl
def delete_template(self, template_id: str) -> bool:
"""Remove a template (memory only, does not delete files)."""
return self._templates.pop(template_id, None) is not None
def _derive_id_from_source(source: str) -> str:
"""Derive a tool ID from a git URL.
Examples
--------
>>> _derive_id_from_source("https://github.com/user/my-tool")
'my_tool'
>>> _derive_id_from_source("https://github.com/user/my-tool.git")
'my_tool'
"""
# Strip trailing .git and get last path segment
url = source.rstrip("/").removesuffix(".git")
slug = url.rsplit("/", 1)[-1].lower()
return slug.replace("-", "_")

28
src/ai_lsc/registry/stack_templates/multi-agent-crewai-local.json Executable file
View File

@ -0,0 +1,28 @@
{
"id": "multi-agent-crewai-local",
"name": "Multi-Agent CrewAI — Local Team Stack",
"description": "Build AI agent teams that collaborate on complex tasks, all running on local models. CrewAI orchestrates specialized agents (researcher, coder, reviewer) powered by Ollama. The exact stack from 'Building an AI Company with Local Models' YouTube series.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["multi-agent", "crewai", "autogen", "agent-team", "youtube-trending", "local-first", "automation"],
"endpoints": {
"ollama_base": "http://localhost:11434/v1",
"litellm_base": "http://localhost:4000"
},
"tools": [
"ollama",
"litellm",
"crewai",
"autogen",
"chromadb",
"fabric",
"n8n"
],
"notes": {
"youtube_context": "Multi-agent AI teams are the #2 trending AI coding topic. Videos by @aiaborde, @promptengineering, and @johnhampton010 show agents delegating tasks to each other. CrewAI is the most beginner-friendly framework for this — define roles, give them tools, and watch them collaborate.",
"recommended_models": "qwen2.5:32b (agent reasoning), llama3.1:70b (complex tasks), codestral:22b (coding agents), mistral-nemo:12b (lightweight agents), nomic-embed-text (agent memory embeddings)",
"setup": "pip install crewai crewai-tools. Set OPENAI_API_BASE=http://localhost:11434/v1 and OPENAI_API_KEY=ollama. LiteLLM at 4000 normalizes if you mix Ollama with other backends.",
"workflow": "Define a Crew with Agent roles (Researcher, Coder, Reviewer). Each agent gets tools (web search, file read/write, code execution). Ollama serves all LLM calls locally. ChromaDB stores agent memory between sessions. n8n can trigger crews from external events (webhooks, schedules, git pushes).",
"tips": "Use sequential process for step-by-step tasks. Use hierarchical process for complex planning (manager agent delegates). Keep agent backstories short and role-focused — verbose context wastes tokens on local models. Embedding quality matters more than model size for agent memory tasks."
}
}

30
src/ai_lsc/registry/stack_templates/n8n-ai-workflow-automation.json Executable file
View File

@ -0,0 +1,30 @@
{
"id": "n8n-ai-workflow-automation",
"name": "n8n AI Workflow Automation Hub",
"description": "Visual AI workflow automation with n8n at the center. Connect local Ollama models to webhooks, email, databases, and scheduling. Build AI-powered automations without writing code — the exact setup from the viral 'Automate Everything with Local AI' videos.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["n8n", "workflow", "automation", "no-code", "youtube-trending", "local-first", "integration"],
"endpoints": {
"n8n": "http://localhost:5678",
"ollama_base": "http://localhost:11434/v1",
"redis": "localhost:6379",
"postgresql": "localhost:5432"
},
"tools": [
"ollama",
"litellm",
"n8n",
"redis",
"postgresql",
"fabric",
"whisper"
],
"notes": {
"youtube_context": "n8n + local AI is the most viewed AI automation content on YouTube. Channels like @n8n_io (official), @techwithtim, and @lainzworld show workflows like: auto-summarize emails with local LLM, transcribe meetings with Whisper, classify support tickets with Ollama, and generate reports on schedule.",
"recommended_models": "llama3.1:8b (classification, fast), qwen2.5:14b (summarization), mistral-nemo:12b (general), phi-4:14b (structured output), nomic-embed-text (semantic search)",
"setup": "n8n runs at localhost:5678 with PostgreSQL for persistence and Redis for queue management. Add the Ollama node (built-in) pointing at localhost:11434. LiteLLM at 4000 provides fallback model routing. n8n's AI Agent node chains multiple LLM calls together in a visual flow.",
"workflow": "Trigger (webhook/cron/email) → n8n AI Agent → Ollama (localhost:11434) → process result → action (email/slack/database write). Whisper node for audio. Fabric node for text transforms. Multiple agents can collaborate in a single workflow.",
"tips": "Use n8n's sub-workflow feature to reuse AI processing across multiple automations. Set Ollama keep_alive=5m in n8n config to avoid cold starts between workflow triggers. The AI Agent node's memory feature persists conversation context across workflow runs using Redis."
}
}

28
src/ai_lsc/registry/stack_templates/open-webui-full-rag.json Executable file
View File

@ -0,0 +1,28 @@
{
"id": "open-webui-full-rag",
"name": "Open WebUI — Full RAG Knowledge Stack",
"description": "The most popular self-hosted ChatGPT replacement. Open WebUI + Ollama + ChromaDB + Whisper for voice input + Docling for document ingestion. The stack thousands of homelabbers and YouTubers run for private AI chat with full document understanding.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["open-webui", "chatgpt-alternative", "rag", "homelab", "youtube-trending", "document-ai", "local-first"],
"endpoints": {
"ollama_base": "http://localhost:11434",
"openwebui": "http://localhost:3000",
"chromadb": "http://localhost:8000"
},
"tools": [
"ollama",
"openwebui",
"chromadb",
"whisper",
"docling",
"markitdown"
],
"notes": {
"youtube_context": "The #1 homelab AI setup across YouTube. Every self-hosted AI tutorial covers this exact stack. Channels like @techhut, @crosstalksolutions, and @networkchuck have dedicated videos with 500K+ views on this combo.",
"recommended_models": "llama3.1:8b (fast chat), llama3.1:70b (quality), qwen2.5:32b (multilingual), mistral-nemo:12b (lightweight), nomic-embed-text (embeddings)",
"setup": "Open WebUI auto-detects Ollama on localhost:11434. Upload PDFs/docs in the UI — they get chunked and embedded into ChromaDB automatically. Whisper enables the microphone button for voice-to-text input.",
"workflow": "Documents → Docling/MarkItDown → Markdown → Open WebUI RAG pipeline → ChromaDB vector store. Whisper handles voice queries. All inference through Ollama on local GPU.",
"tips": "Set OLLAMA_NUM_PARALLEL=4 for concurrent chat requests. Use Open WebUI's built-in model switching to route simple queries to 8B and complex ones to 70B. Create workspaces per project for isolated document collections."
}
}

30
src/ai_lsc/registry/stack_templates/openhands-autonomous-coder.json Executable file
View File

@ -0,0 +1,30 @@
{
"id": "openhands-autonomous-coder",
"name": "OpenHands — Autonomous AI Software Engineer",
"description": "Run the OpenHands autonomous coding agent entirely locally. Sandboxed code execution + terminal access + file management + web browsing + Ollama inference. The most popular open-source autonomous software engineer on GitHub.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["openhands", "autonomous", "coding-agent", "sandbox", "youtube-trending", "local-first", "terminal"],
"endpoints": {
"ollama_base": "http://localhost:11434/v1",
"litellm_base": "http://localhost:4000",
"openhands": "http://localhost:3000"
},
"tools": [
"openhands",
"ollama",
"litellm",
"fabric",
"ripgrep",
"fd",
"tree_sitter"
],
"notes": {
"youtube_context": "The autonomous coding agent concept went viral when open-source alternatives appeared. OpenHands, maintained by All-Hands-AI, is the most starred and actively developed project in this space. Channels like @NicholasRenotte, @aiaborde, and @codingwithadam have full build-along videos with 200K+ views.",
"about_openhands": "OpenHands is an autonomous AI software engineer that can plan, write, debug, and execute code in sandboxed Docker environments. It has full terminal access, file management, web browsing capability, and supports any LLM backend. GitHub: https://github.com/All-Hands-AI/OpenHands — 40K+ stars.",
"recommended_models": "qwen2.5-coder:32b (primary coding, best SWE-bench), deepseek-coder-v2:236b (complex reasoning), codestral:22b (fast edits), llama3.1:70b (architectural planning), claude-4-sonnet (if available via proxy)",
"setup": "1) OpenHands installs via git clone + pip. 2) Configure LLM backend in config.yaml to point at Ollama (localhost:11434) or LiteLLM (localhost:4000). 3) Start the server: python -m openhands.server. 4) Open the web UI at localhost:3000. 5) Give it a task and watch it plan, code, test, and iterate autonomously.",
"workflow": "User describes task in web UI → OpenHands plans approach → Agent writes code in sandbox → Code executes in Docker container → Agent reads output → Agent iterates until tests pass. Fabric can transform error messages into structured context. ripgrep + fd + tree_sitter give the agent codebase awareness when pointed at a repo.",
"tips": "OpenHands uses Docker sandboxes by default for safe code execution. For local-only setups, configure SANDBOX_TYPE=local in the environment. The coding model matters more than reasoning quality — Qwen2.5-Coder:32b consistently outperforms larger general models on SWE-bench. Use LiteLLM as a middleman if you want to hot-swap models between tasks without restarting OpenHands."
}
}

71
src/ai_lsc/registry/stack_templates/openjarvis-intelligence-stack.json Executable file
View File

@ -0,0 +1,71 @@
{
"id": "openjarvis-intelligence-stack",
"name": "OpenJarvis — Full Intelligence Stack",
"description": "The everything-intelligent stack. OpenJarvis as the central brain with dual inference engines (vLLM + Ollama), full memory hierarchy (Qdrant, ChromaDB, LanceDB, Redis, MariaDB), all RAG/OCR/document tools, audio I/O (Whisper + LuxTTS + Parakeet), computer vision, knowledge graphs, semantic search, and Obsidian knowledge base. 30+ tools working in concert.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["openjarvis", "intelligence", "multi-modal", "rag", "vision", "audio", "knowledge-graph", "memory", "full-stack", "local-first"],
"endpoints": {
"openjarvis": "http://localhost:17070",
"ollama_base": "http://localhost:11434/v1",
"vllm_base": "http://localhost:8000",
"litellm_base": "http://localhost:4000",
"qdrant": "http://localhost:6333",
"chromadb": "http://localhost:8000",
"lancedb": "http://localhost:8100",
"redis": "localhost:6379",
"mariadb": "localhost:3306",
"elasticsearch": "http://localhost:9200",
"meilisearch": "http://localhost:7700",
"openwebui": "http://localhost:3000"
},
"tools": [
"openjarvis",
"ollama",
"vllm",
"litellm",
"qdrant",
"chromadb",
"lancedb",
"redis",
"mariadb",
"turbovec",
"graphrag",
"elasticsearch",
"meilisearch",
"airweave",
"markitdown",
"opendataloader",
"opendataloader_pdf",
"docling",
"whisper",
"luxtts",
"parakeet",
"deep_eye",
"understand_anything",
"fabric",
"openwebui",
"openhands",
"obsidian",
"aider",
"hermes_agent",
"glances",
"langchain"
],
"notes": {
"architecture": "OpenJarvis sits at L11 (User Interfaces) as the central brain. It orchestrates all layers below: L5/L6 inference engines, L7 data/knowledge pipelines, L8 automation, L10 intelligent routing, and L13 knowledge management. Every tool in this stack feeds into or is controlled by OpenJarvis.",
"inference_tier": "Ollama at 11434 handles lightweight and medium models (8B-32B). vLLM at 8000 serves heavy models (70B+) with PagedAttention for max throughput. LiteLLM at 4000 provides a unified OpenAI-compatible API over both backends — OpenJarvis and all sub-agents route through LiteLLM for seamless model switching.",
"memory_hierarchy": "Five-tier memory system: (1) Redis 6379 = hot cache / pub-sub / session state (ms latency). (2) Qdrant 6333 = semantic vector memory for RAG and agent recall (us). (3) ChromaDB = document chunk embeddings (dedicated RAG pipeline). (4) LanceDB = fast local vector DB for code and small corpus. (5) MariaDB 3306 = persistent structured storage (audit logs, user prefs, conversation history, task state). TurboVec provides accelerated embedding generation across all vector stores.",
"rag_document_pipeline": "Documents enter through four ingestion paths: MarkItDown converts Office docs to Markdown. OpenDataLoader handles web scraping and structured data. Docling performs deep PDF extraction with layout analysis. OpenDataLoader PDF specializes in scanned/image PDFs. All output flows to Fabric for text transformation, then into vector stores via TurboVec embeddings.",
"audio_i_o": "Whisper (OpenAI) provides speech-to-text for voice commands and meeting transcription. LuxTTS generates natural speech output for responses and notifications. Parakeet is a lightweight alternative TTS for quick alerts. Together they give OpenJarvis full voice I/O capability.",
"vision": "Deep Eye provides computer vision — image description, object detection, scene understanding. Understand Anything handles universal document understanding (charts, diagrams, mixed content). These feed into the RAG pipeline so OpenJarvis can reason about visual content.",
"knowledge_graph_search": "GraphRag builds knowledge graphs from document collections — entities, relationships, community detection. Elasticsearch at 9200 provides full-text search with BM25 ranking. Meilisearch at 7700 provides typo-tolerant instant search. Airweave syncs data across all stores in real-time.",
"knowledge_management": "Obsidian serves as the human-facing knowledge base — markdown notes, bi-directional links, graph view. OpenJarvis can read/write to the Obsidian vault, making the LLM's knowledge graph accessible to humans through a familiar note-taking interface. Logseq and Joplin are alternative knowledge tools in the stack.",
"coding_agents": "OpenHands provides autonomous software engineering. Aider handles interactive pair programming. Hermes Agent runs persistent background agent tasks. All three route through the same LiteLLM/Ollama inference path.",
"monitoring": "Glances provides real-time system resource monitoring. OpenJarvis dashboard exposes all service health, log aggregation, and resource metrics in one view.",
"recommended_models": "Primary reasoning: qwen2.5:72b or llama3.1:70b (via vLLM). Fast tasks: qwen2.5:14b, mistral-nemo:12b (via Ollama). Coding: qwen2.5-coder:32b. Embeddings: nomic-embed-text (fast), bge-large (quality). Vision: llava:13b. Audio: whisper-large-v3.",
"setup": "Start services in order: (1) Redis, MariaDB, Elasticsearch (infrastructure). (2) Qdrant, ChromaDB, LanceDB, Meilisearch (memory/search). (3) Ollama, then vLLM for heavy models. (4) LiteLLM pointing at both backends. (5) TurboVec for embedding acceleration. (6) Document tools (Docling, MarkItDown). (7) Audio tools (Whisper, LuxTTS). (8) OpenJarvis as the central brain. (9) Agent tools (OpenHands, Aider, Hermes Agent). (10) Obsidian for knowledge base. AI-LSC manages all of this through the stack template.",
"resource_requirements": "Minimum: 32GB RAM, 12GB VRAM (6B models). Recommended: 64GB RAM, 24GB VRAM (32B models). Full stack: 128GB RAM, 2x 24GB VRAM (70B models + vLLM + embedding). MariaDB needs 4GB. Elasticsearch needs 4GB. Redis needs 2GB. Plan 16GB+ RAM just for infrastructure services.",
"tips": "Don't start everything at once — use AI-LSC's service manager to bring up tiers incrementally. The inference tier (Ollama + vLLM + LiteLLM) is the foundation. Memory services (Redis + Qdrant + MariaDB) come next. Then document/audio tools. OpenJarvis last. Use the LiteLLM model list to hot-swap models per task without restarting downstream tools."
}
}

30
src/ai_lsc/registry/stack_templates/privacy-first-ai-laptop.json Executable file
View File

@ -0,0 +1,30 @@
{
"id": "privacy-first-ai-laptop",
"name": "Privacy-First AI Laptop Setup",
"description": "The complete privacy-respecting AI stack for your laptop. All processing on-device, no telemetry, no cloud APIs. Ollama + Whisper + Obsidian + Paperless-NGX + local search. Popular with privacy-focused YouTubers and FOSS advocates.",
"version": "1.0",
"author": "ai-lsc",
"tags": ["privacy", "offline", "laptop", "document-management", "youtube-trending", "local-first", "foss"],
"endpoints": {
"ollama_base": "http://localhost:11434",
"openwebui": "http://localhost:3000",
"paperlessngx": "http://localhost:8000"
},
"tools": [
"ollama",
"openwebui",
"whisper",
"docling",
"markitdown",
"obsidian",
"paperlessngx",
"fabric"
],
"notes": {
"youtube_context": "Privacy-focused AI content has exploded. Channels like @TheLinuxExperiment, @crosstalksolutions, and @braveouterweb showcase fully local AI setups. The message: 'Your AI should stay on your machine.' This template builds that exact vision.",
"recommended_models": "llama3.1:8b (daily driver, 4GB VRAM), phi-4:14b (quality on 8GB), mistral-nemo:12b (sweet spot), gemma2:9b (fast), nomic-embed-text (document embeddings)",
"setup": "Ollama runs as a systemd service. Open WebUI provides the chat frontend. Paperless-NGX ingests scanned documents. Docling/MarkItDown converts them for RAG. Whisper handles voice memos. Obsidian links everything with local markdown notes.",
"workflow": "Paper documents → scan → Paperless-NGX (OCR + tagging) → Docling (extract text) → Open WebUI RAG (chat with your documents). Voice notes → Whisper → text → Fabric → summarized notes → Obsidian vault. All data stays on your NVMe.",
"tips": "For laptops with <8GB VRAM, use 4-bit quantized models. Set OLLAMA_NUM_PARALLEL=1 to prevent VRAM thrashing. Paperless-NGX works great with 2GB RAM allocated. Use Obsidian's local graph view to visualize connections between your AI-generated notes and source documents."
}
}

304
src/ai_lsc/registry/validator.py Executable file
View File

@ -0,0 +1,304 @@
"""
AI-LSC — Registry schema validation.
Validates that registry entries conform to the expected schema. This is
intended for CI / developer tooling, not for hot-path runtime checks
(so a small upfront cost is acceptable).
Usage::
from ai_lsc.registry.validator import validate_registry
errors = validate_registry(registry_data)
if errors:
for e in errors:
print(f" - {e}")
"""
from __future__ import annotations
import re
from typing import Any
# Fields that every registry entry must contain.
_REQUIRED_FIELDS: set[str] = {
"name", "level", "layer", "role", "category",
"installer", "launcher", "deps", "description", "flags",
"license",
}
# Valid values for certain fields — must match InstallerType / LauncherType enums.
_VALID_INSTALLER_TYPES: set[str] = {
"ollama", "uv", "pipx", "pip", "npm",
"git", "git_node", "pacman", "dnf", "apt",
"script", "custom",
}
_VALID_LAUNCHER_TYPES: set[str] = {
"systemd", "tmux", "desktop", "lxc",
}
# Optional installer fields (allowed but not required).
_OPTIONAL_INSTALLER_FIELDS: set[str] = {
"type", "pkg", "cmd", "post_install", "update_cmd", "env_overrides",
}
_OPTIONAL_LAUNCHER_FIELDS: set[str] = {
"type", "cmd", "default_port",
}
_OPTIONAL_FILESYSTEM_FIELDS: set[str] = {
"install", "config", "cache", "data", "logs", "runtime", "models",
}
# H-15: every registry entry's `flags` block must declare all 7 boolean
# keys. Layer files that pre-date the schema expansion only declare the
# first three (has_cli/has_gui/has_web); they fail this check until the
# missing keys are backfilled.
_REQUIRED_FLAG_KEYS: set[str] = {
"has_cli", "has_gui", "has_web",
"is_ollama",
"is_passive", "is_mcp", "is_skills_collection",
}
# License SPDX IDs recognized by the license catalog
# (registry/licenses.py CATALOG). Populated lazily to avoid a circular
# import (licenses.py imports nothing from validator.py, but importing
# it at module-load time here is fine).
try:
from ai_lsc.registry.licenses import CATALOG as _LICENSE_CATALOG
_KNOWN_LICENSE_SPDX_IDS: set[str] = set(_LICENSE_CATALOG.keys())
except ImportError:
# During bootstrap before licenses.py exists, fall back to empty.
_KNOWN_LICENSE_SPDX_IDS = set()
# SaaS-only tool blocklist — these tool_ids (and case-insensitive
# variants) are rejected by the validator. The user's policy is that
# SaaS-only tools (closed-source desktop apps with restrictive ToS,
# hosted LLM routers with no local binary, managed inference services)
# do not belong in AI-LSC. Tools that CAN call a SaaS endpoint but
# don't have to (claude_code, aider, openhands, fabric, codex) are
# allowed — their launchers force ANTHROPIC_BASE_URL /
# OPENAI_BASE_URL to http://127.0.0.1:{port} so SaaS routing is
# broken by default. See ADR-001 + CHANGES.md for the rationale.
#
# lm_studio / lmstudio: BLOCKED for aggressive ToS — the user
# considers LM Studio's Terms of Service restrictive enough to be
# equivalent to a SaaS offering, so it is auto-banned regardless of
# any per-tool acceptance the user might try to grant.
SAAS_BLOCKLIST: frozenset[str] = frozenset({
"openrouter",
"lm_studio", "lmstudio",
"groq",
"together_ai", "together",
"fireworks_ai", "fireworks",
"replicate",
"runpod",
"modal",
"anyscale",
"perplexity",
"cohere",
"mistral_api",
"deepseek_api",
"openai_api",
"huggingface_inference",
})
# SaaS provider hostnames that must never appear in a launcher cmd or
# installer cmd (excluding git/git_node which clone source code, not
# SaaS). Matches http(s)://host where host is a known SaaS provider.
_SAAS_HOST_RE = re.compile(
r"https?://(?:"
r"api\.openrouter\.ai|"
r"api\.openai\.com|"
r"api\.anthropic\.com|"
r"api\.groq\.com|"
r"api\.together\.xyz|"
r"api\.fireworks\.ai|"
r"api\.replicate\.com|"
r"api\.perplexity\.ai|"
r"api\.cohere\.ai|"
r"api\.mistral\.ai|"
r"api\.deepseek\.com|"
r"generativelanguage\.googleapis\.com|"
r"api\.lmsstudio\.com|"
r"api\.lmstudio\.ai|"
r"endpoint\.huggingface\.com"
r")",
re.IGNORECASE,
)
def _check_entry(tool_id: str, entry: dict[str, Any]) -> list[str]:
"""Return a list of validation error strings for a single entry."""
errors: list[str] = []
# SaaS-only tool blocklist — reject before any other check so the
# error message is the first thing the user sees. Case-insensitive.
if tool_id.lower() in SAAS_BLOCKLIST:
errors.append(
f"{tool_id}: tool_id is on the SaaS-only blocklist — "
f"AI-LSC policy excludes SaaS-only tools (closed-source "
f"desktop apps with restrictive ToS, hosted LLM routers "
f"with no local binary, managed inference services). "
f"Use a local alternative (ollama, vllm, litellm, etc.) "
f"instead. See CHANGES.md → 'SaaS-only tool blocklist' "
f"for the rationale."
)
# Continue running the other checks so the user sees every
# problem with the entry in one pass.
# Missing required fields
missing = _REQUIRED_FIELDS - entry.keys()
if missing:
errors.append(f"{tool_id}: missing fields {sorted(missing)}")
# Level must be 1–13
level = entry.get("level")
if isinstance(level, int) and not (1 <= level <= 13):
errors.append(f"{tool_id}: level {level} out of range 1-13")
elif not isinstance(level, int):
errors.append(f"{tool_id}: level is not an int ({level!r})")
# Installer type
inst = entry.get("installer", {})
if isinstance(inst, dict):
itype = inst.get("type")
if itype and itype not in _VALID_INSTALLER_TYPES:
errors.append(
f"{tool_id}: unknown installer type {itype!r} "
f"(valid: {sorted(_VALID_INSTALLER_TYPES)})"
)
# Script-type installers must include cmd
if itype == "script" and not inst.get("cmd"):
errors.append(
f"{tool_id}: installer type 'script' requires 'cmd'"
)
# Warn if script cmd doesn't contain {tools_root}
if itype == "script" and inst.get("cmd"):
cmd_str = inst["cmd"]
if "{tools_root}" not in cmd_str and tool_id != "ollama":
errors.append(
f"{tool_id}: script installer cmd should reference "
f"{{{{tools_root}}}} to avoid polluting system dirs"
)
# SaaS host check — reject installers that reference a known
# SaaS provider URL (excluding git/git_node which clone source
# code, not SaaS endpoints).
if itype not in ("git", "git_node"):
for field in ("cmd", "pkg"):
val = inst.get(field, "")
if isinstance(val, str) and _SAAS_HOST_RE.search(val):
errors.append(
f"{tool_id}: installer.{field} references a "
f"SaaS provider URL — AI-LSC policy excludes "
f"SaaS-only tools. Use a local alternative "
f"instead. See CHANGES.md → 'SaaS-only tool "
f"blocklist' for the rationale."
)
# Launcher type
launch = entry.get("launcher", {})
if isinstance(launch, dict):
ltype = launch.get("type")
if ltype and ltype not in _VALID_LAUNCHER_TYPES:
errors.append(
f"{tool_id}: unknown launcher type {ltype!r} "
f"(valid: {sorted(_VALID_LAUNCHER_TYPES)})"
)
# SaaS host check — reject launchers that reference a known
# SaaS provider URL. Localhost URLs (127.0.0.1, localhost,
# 0.0.0.0) are always allowed.
lcmd = launch.get("cmd", "")
if isinstance(lcmd, str):
saas_match = _SAAS_HOST_RE.search(lcmd)
has_localhost = any(
host in lcmd
for host in ("127.0.0.1", "localhost", "0.0.0.0")
)
if saas_match and not has_localhost:
errors.append(
f"{tool_id}: launcher.cmd references a SaaS "
f"provider URL ({saas_match.group(0)!r}) without "
f"a localhost override. AI-LSC policy requires "
f"localhost-only endpoints. See CHANGES.md → "
f"'SaaS-only tool blocklist' for the rationale."
)
# Filesystem spec (optional but validated if present)
fs = entry.get("filesystem", {})
if isinstance(fs, dict):
unknown_fs = set(fs.keys()) - _OPTIONAL_FILESYSTEM_FIELDS
if unknown_fs:
errors.append(
f"{tool_id}: unknown filesystem fields {sorted(unknown_fs)}"
)
# deps must be a list of strings
deps = entry.get("deps")
if not isinstance(deps, list):
errors.append(f"{tool_id}: deps is not a list")
else:
non_str = [d for d in deps if not isinstance(d, str)]
if non_str:
errors.append(
f"{tool_id}: deps contains non-string items: {non_str}"
)
# flags must be a dict of bools declaring every required key.
flags = entry.get("flags", {})
if not isinstance(flags, dict):
errors.append(f"{tool_id}: flags is not a dict")
else:
non_bool = {k: v for k, v in flags.items()
if not isinstance(v, bool)}
if non_bool:
errors.append(
f"{tool_id}: flags contain non-bool values: "
f"{list(non_bool.keys())}"
)
# H-15: enforce the full 8-key schema.
missing_flags = _REQUIRED_FLAG_KEYS - flags.keys()
if missing_flags:
errors.append(
f"{tool_id}: flags missing required keys: "
f"{sorted(missing_flags)}"
)
unknown_flags = set(flags.keys()) - _REQUIRED_FLAG_KEYS
if unknown_flags:
errors.append(
f"{tool_id}: flags contain unknown keys: "
f"{sorted(unknown_flags)}"
)
# license must be a non-empty string matching a known SPDX ID in
# the license catalog. Unknown SPDX IDs are a warning (the gate
# treats them as Proprietary) but missing/empty license is an error.
license_spdx = entry.get("license", "")
if not license_spdx or not isinstance(license_spdx, str):
errors.append(
f"{tool_id}: missing or invalid 'license' field — "
f"must be an SPDX ID (e.g. 'MIT', 'Apache-2.0', "
f"'GPL-3.0', 'Proprietary', 'Anthropic-ToS'). See "
f"registry/licenses.py CATALOG for the full list."
)
elif license_spdx not in _KNOWN_LICENSE_SPDX_IDS:
errors.append(
f"{tool_id}: license {license_spdx!r} is not in the "
f"license catalog (registry/licenses.py). Add it there "
f"first so the LicenseGate knows its category and summary."
)
return errors
def validate_registry(data: dict[str, Any]) -> list[str]:
"""Validate an entire registry dict.
Returns a (possibly empty) list of human-readable error strings.
An empty list means the registry is valid.
"""
errors: list[str] = []
for tool_id, entry in data.items():
if not isinstance(entry, dict):
errors.append(f"{tool_id}: entry is not a dict ({type(entry)})")
continue
errors.extend(_check_entry(tool_id, entry))
return errors

20
src/ai_lsc/runtime/__init__.py Executable file
View File

@ -0,0 +1,20 @@
"""AI-LSC runtime sub-package.
Process management abstraction layer. All subprocess / psutil calls
live here so that UI code never touches the OS directly.
UI code calls :class:`RuntimeExecutor`, which delegates to backend-
specific managers (tmux, systemd, process, installer).
"""
from ai_lsc.runtime.executor import RuntimeExecutor
from ai_lsc.runtime.installer import InstallerManager
from ai_lsc.runtime.lxc import LxcManager
from ai_lsc.runtime.status import StatusChecker
__all__ = [
"RuntimeExecutor",
"InstallerManager",
"LxcManager",
"StatusChecker",
]

350
src/ai_lsc/runtime/executor.py Executable file
View File

@ -0,0 +1,350 @@
"""Runtime executor -- the single entry point for all process management.
UI code calls ``RuntimeExecutor`` methods instead of touching
``subprocess`` directly. This is the *only* class the UI should
import from ``ai_lsc.runtime``.
"""
from __future__ import annotations
import os
import re
import subprocess
from pathlib import Path
from typing import Any
from ai_lsc.runtime.installer import InstallerManager
from ai_lsc.runtime.lxc import LxcManager
from ai_lsc.runtime.process import ProcessManager
from ai_lsc.runtime.status import StatusChecker
from ai_lsc.runtime.systemd import SystemdManager
from ai_lsc.runtime.tmux import TmuxManager
from ai_lsc.utils.process import enriched_env
# H-01 / H-11: reject tool_id values that could break out of file paths,
# tmux window names, or LXC container names. Keep this conservative and
# limited to characters actually used by the registry.
_TOOL_ID_RE = re.compile(r"^[A-Za-z0-9_.:\-]+$")
def _validate_tool_id(tool_id: str) -> str:
"""Raise ``ValueError`` if *tool_id* is unsafe to use as a path/window name."""
if not tool_id or not _TOOL_ID_RE.fullmatch(tool_id):
raise ValueError(f"invalid tool_id: {tool_id!r}")
# Also reject `.`, `..`, `foo/..`, etc. — these pass the regex but
# escape the intended tools_root/<tool_id>/ directory when joined.
if tool_id in {".", ".."} or os.path.normpath(tool_id) != tool_id:
raise ValueError(f"tool_id contains path-traversal segments: {tool_id!r}")
return tool_id
def _validate_port(port: int | str) -> int:
"""Coerce *port* to ``int`` and validate the 1..65535 range."""
try:
port_num = int(port)
except (TypeError, ValueError) as exc:
raise ValueError(f"invalid port: {port!r}") from exc
if not 1 <= port_num <= 65535:
raise ValueError(f"port out of range 1..65535: {port_num}")
return port_num
class RuntimeExecutor:
"""Unified runtime facade for UI-layer delegation.
Parameters
----------
tools_root:
Base directory for tool installations.
models_root:
Base directory for model files.
workspaces_root:
Base directory for workspace data.
logs_root:
Base directory for service log files.
base_bin_dir:
Colon-separated PATH string to prepend to all commands.
dtach_bin:
Path to the ``dtach`` binary (or ``None``).
"""
def __init__(
self,
tools_root: str,
models_root: str,
workspaces_root: str,
logs_root: str,
base_bin_dir: str = "",
dtach_bin: str | None = None,
license_gate: Any = None,
) -> None:
self.tools_root = tools_root
self.models_root = models_root
self.workspaces_root = workspaces_root
self.logs_root = logs_root
self.base_bin_dir = base_bin_dir
self.dtach_bin = dtach_bin
self.license_gate = license_gate
self._tmux = TmuxManager()
self._systemd = SystemdManager()
self._lxc = LxcManager(tools_root, logs_root)
self._process = ProcessManager()
self._installer = InstallerManager(
tools_root, base_bin_dir,
license_gate=license_gate,
)
self._status = StatusChecker(tmux=self._tmux, systemd=self._systemd)
# -- context formatting -----------------------------------------------
def format_context(
self,
port: str = "",
model_arg: str = "",
) -> dict[str, str]:
"""Build the ``{placeholders}`` dict used by launcher commands."""
from ai_lsc.constants import BASE_DIR
return {
"base_dir": BASE_DIR,
"tools_root": self.tools_root,
"models_root": self.models_root,
"workspaces_root": self.workspaces_root,
"port": port,
"model_arg": model_arg,
}
# -- service lifecycle -----------------------------------------------
def start_service(
self,
tool_id: str,
launcher_cmd: str,
launcher_type: str,
port: str = "",
model_arg: str = "",
) -> str:
"""Start a service via the appropriate backend.
Returns a description of what was done.
"""
_validate_tool_id(tool_id)
if port:
_validate_port(port)
ctx = self.format_context(port=port, model_arg=model_arg)
final_cmd = launcher_cmd.format(**ctx)
if launcher_type == "systemd":
self._systemd.start(final_cmd)
return f"Systemd activated for {tool_id}"
if launcher_type == "desktop":
self._process.launch_desktop(final_cmd)
return f"Desktop spawned for {tool_id}"
if launcher_type == "lxc":
log_file = str(Path(self.logs_root) / f"{tool_id}.log")
return self._lxc.launch_service(
tool_id=tool_id,
command=final_cmd,
log_file=log_file,
dtach_bin=self.dtach_bin,
base_bin_dir=self.base_bin_dir,
)
# default: tmux (with optional dtach)
log_file = str(Path(self.logs_root) / f"{tool_id}.log")
self._tmux.launch_service(
tool_id=tool_id,
command=final_cmd,
log_file=log_file,
dtach_bin=self.dtach_bin,
base_bin_dir=self.base_bin_dir,
)
return f"Component {tool_id} isolated in Tmux."
def stop_service(
self,
tool_id: str,
launcher_type: str,
launcher_cmd: str = "",
search_term: str = "",
) -> str:
"""Stop a service via the appropriate backend.
Returns a description of what was done.
"""
_validate_tool_id(tool_id)
if launcher_type == "systemd":
self._systemd.stop(launcher_cmd)
return f"Systemd stop signal sent for {tool_id}"
if launcher_type == "tmux":
self._tmux.stop_service(tool_id)
return f"Tmux window killed for {tool_id}"
if launcher_type == "lxc":
return self._lxc.stop_service(tool_id)
# default: pkill
self._process.kill_by_name(search_term)
return f"Termination signal sent to {tool_id}."
def is_service_running(
self,
launcher_type: str,
tool_id: str = "",
service_cmd: str = "",
search_term: str = "",
) -> bool:
"""Check whether a service is currently live."""
if launcher_type == "lxc":
return self._lxc.is_running(f"ai-lsc-{tool_id}")
return self._status.is_running(
launcher_type=launcher_type,
tool_id=tool_id,
service_cmd=service_cmd,
search_term=search_term,
)
# -- installation ----------------------------------------------------
def install_tool(
self,
inst_type: str,
pkg: str,
cmd: str = "",
tool_id: str = "",
ctx: dict[str, str] | None = None,
force: bool = False,
post_install: str | None = None,
env_overrides: dict[str, str] | None = None,
filesystem: dict[str, str] | None = None,
license_spdx: str | None = None,
) -> str:
"""Dispatch tool installation to the correct installer.
If *tool_id* is provided, the installer uses preflight detection
and routes artifacts to ``tools_root/<tool_id>/``.
If *force* is True, skips preflight and installs unconditionally.
*post_install* runs a shell command inside ``tools_root/<tool_id>``
after clone (e.g. ``pip install -r requirements.txt``, ``make``).
*env_overrides* remaps upstream environment variables (HF_HOME,
TRANSFORMERS_CACHE, etc.) into ``/mnt/AI/`` paths.
*filesystem* declares per-tool path mappings for the verification
checklist (install, config, cache, logs).
*license_spdx* is forwarded to the InstallerManager's license
gate (if one was provided at construction time). If the gate
returns ``blocked`` or ``needs_acceptance``, the appropriate
``LicenseBlocked`` / ``LicenseAcceptanceRequired`` exception is
raised before any subprocess call.
Returns a description of the result.
"""
if tool_id:
return self._installer.install_with_preflight(
tool_id=tool_id,
inst_type=inst_type,
pkg=pkg,
cmd=cmd,
ctx=ctx,
force=force,
post_install=post_install,
env_overrides=env_overrides,
license_spdx=license_spdx,
)
return self._installer.run(
inst_type=inst_type,
pkg=pkg,
cmd=cmd,
ctx=ctx,
tool_id=tool_id,
post_install=post_install,
env_overrides=env_overrides,
license_spdx=license_spdx,
)
# -- verification ---------------------------------------------------
def verify_tool(
self,
tool_id: str,
inst_type: str,
pkg: str,
cmd: str = "",
filesystem: dict[str, str] | None = None,
) -> dict[str, Any]:
"""Run the installation compliance checklist for a tool.
Returns a dict with ``score``, ``checks``, and ``install_location``.
"""
return self._installer.verify(
tool_id=tool_id,
inst_type=inst_type,
pkg=pkg,
cmd=cmd,
filesystem=filesystem,
)
# -- model management ------------------------------------------------
def pull_model(self, model_name: str) -> subprocess.Popen:
"""Start an ``ollama pull`` and return the live process."""
from ai_lsc.utils.ollama import ollama_env
env = enriched_env(self.base_bin_dir)
ollama_env_overrides = ollama_env(self.models_root)
env.update(ollama_env_overrides)
# SE-06: redirect to log file instead of PIPE to avoid deadlock
log_path = os.path.join(
str(self.models_root).rsplit("/models", 1)[0], "logs",
f"pull_{model_name.replace(':', '_')}.log",
)
os.makedirs(os.path.dirname(log_path), exist_ok=True)
log_fh = open(log_path, "w", encoding="utf-8")
return subprocess.Popen(
["ollama", "pull", model_name],
stdout=log_fh,
stderr=subprocess.STDOUT,
text=True,
env=env,
)
# -- CLI launch -------------------------------------------------------
def launch_cli(
self,
tool_id: str,
launcher_type: str,
) -> str:
"""Open a terminal for the tool's CLI interface."""
_validate_tool_id(tool_id)
cmd = ""
if launcher_type == "tmux":
cmd = self._tmux.attach_cli(tool_id)
elif launcher_type == "lxc":
return self._lxc.launch_cli(f"ai-lsc-{tool_id}")
env = enriched_env(self.base_bin_dir)
from ai_lsc.constants import BASE_DIR
self._process.launch_terminal(
f"{cmd}cd {BASE_DIR} && echo 'Spawning CLI...' && exec bash",
env=env,
)
return f"Spawned CLI terminal for {tool_id}"
# -- web launch -------------------------------------------------------
@staticmethod
def open_web_url(port: str | int) -> str:
"""Open a browser tab for the given port. Returns the URL."""
port_num = _validate_port(port)
import webbrowser
url = f"http://127.0.0.1:{port_num}"
webbrowser.open(url)
return url

946
src/ai_lsc/runtime/installer.py Executable file
View File

@ -0,0 +1,946 @@
"""Installer manager -- dispatch-table-driven tool installation.
Handles pacman, dnf, apt, uv, pipx, pip, ollama, npm, git, git_node,
script, and custom installer types. Every ``subprocess`` / ``os.makedirs``
call is confined here.
Key capabilities
---------------
1. **Step-down containment**: Each Python tool tries the most isolated
install method first (ollama -> uv -> pipx -> pip). If the preferred
method fails, it steps down to the next one automatically.
2. **Working directory enforcement**: All tool artifacts are installed
under ``tools_root/<tool_id>/`` (or ``tools_root/npm_globals/`` for
npm). This keeps the host system clean and makes tools portable.
3. **``~/.local`` remap**: Environment variables are set so that
``uv``, ``pip``, and ``pipx`` install into ``tools_root`` instead
of the user's home directory.
4. **Per-tool env overrides**: Tools like vLLM, huggingface tools, etc.
can declare ``env_overrides`` in the registry to redirect
HF_HOME, TRANSFORMERS_CACHE, and other upstream paths into
``/mnt/AI/cache/<tool>`` or ``/mnt/AI/data/<tool>``.
5. **Post-install hooks**: Git-cloned tools can declare ``post_install``
commands (e.g. ``pip install -r requirements.txt``, ``make``)
that run automatically after clone.
6. **Preflight detection**: ``preflight()`` checks whether a tool is
already installed (via ``which``, directory existence, or pacman
query) and returns a ``PreflightResult`` so the UI can offer
"update to latest" instead of blindly reinstalling.
7. **Installation verification**: ``verify()`` runs a compliance
checklist against a single tool and returns a ``VerificationResult``
with a quality score (0-100%).
8. **Version detection**: Attempts to extract the installed version
for comparison with the latest available version.
"""
from __future__ import annotations
import os
import re
import shlex
import shutil
import subprocess
from pathlib import Path
from typing import Any
from urllib.parse import urlparse
from ai_lsc.utils.logging import get_logger
from ai_lsc.utils.process import enriched_env
logger = get_logger(__name__)
# Registry tool_id / package name validation patterns. Applied at every
# subprocess boundary to prevent path-traversal / command-injection from a
# malicious or malformed registry entry.
# NOTE: tool_id is used as a path component (tools_root/<tool_id>/) so it
# must NOT allow `/` or `..`. Package names (PyPI / npm) DO allow `/`
# (e.g. `@scope/pkg`) and `@`, so they use a separate, looser regex.
_TOOL_ID_RE = re.compile(r"^[A-Za-z0-9_.:\-]+$")
_PKG_NAME_RE = re.compile(r"^[A-Za-z0-9_.@/\-+]+$")
def _validate_tool_id(tool_id: str) -> None:
# Reject empty / regex-mismatch first.
if not tool_id or not _TOOL_ID_RE.fullmatch(tool_id):
raise ValueError(f"invalid tool_id: {tool_id!r}")
# Reject path-traversal attempts that pass the char-set regex but
# escape tools_root when joined: `.`, `..`, `...` (normpath leaves
# these unchanged, so we check them explicitly), plus anything where
# normpath DOES change the value (e.g. `foo/..` — though `/` is
# already rejected by the regex above, this is defense-in-depth).
if tool_id in {".", ".."} or os.path.normpath(tool_id) != tool_id:
raise ValueError(f"tool_id contains path-traversal segments: {tool_id!r}")
def _validate_pkg(pkg: str) -> None:
if not pkg or not _PKG_NAME_RE.fullmatch(pkg):
raise ValueError(f"invalid package name: {pkg!r}")
def _validate_url(url: str, *, allow_schemes: tuple[str, ...] = ("http", "https")) -> str:
"""Validate URL scheme and return the URL unchanged if safe."""
parsed = urlparse(url)
if parsed.scheme not in allow_schemes or not parsed.netloc:
raise ValueError(f"unsafe URL rejected: {url!r}")
return url
# Step-down containment order (most isolated first)
STEP_DOWN_ORDER: list[str] = [
"ollama", "uv", "pipx", "pip",
"git", "git_node", "npm", "pacman", "dnf", "apt", "script", "custom",
]
# Version extraction commands per installer type
_VERSION_CMDS: dict[str, str] = {
"pacman": "pacman -Qi {pkg} 2>/dev/null | grep Version",
"dnf": "dnf info {pkg} 2>/dev/null | grep Version",
"apt": "dpkg -s {pkg} 2>/dev/null | grep Version",
"uv": "{cmd} --version 2>/dev/null",
"npm": "npm list -g {pkg} --depth=0 2>/dev/null",
"pip": "pip show {pkg} 2>/dev/null | grep Version",
"pipx": "pipx list 2>/dev/null | grep {pkg}",
}
# Known upstream env vars that tools commonly use for data/cache.
# Format: env_var -> (human_label, default_subdir_under_base)
_UPSTREAM_ENV_VARS: dict[str, tuple[str, str]] = {
"HF_HOME": ("HuggingFace cache", "cache/huggingface"),
"TRANSFORMERS_CACHE": ("Transformers cache", "cache/huggingface"),
"DIFFUSERS_CACHE": ("Diffusers cache", "cache/huggingface"),
"RUST_BACKTRACE": ("Rust backtrace", None),
"NODE_PATH": ("Node modules", None),
"npm_config_prefix": ("npm prefix", None),
}
class InstallerManager:
"""Install or sync tools via the appropriate package manager.
Parameters
----------
tools_root :
Base directory for tool installations (default ``/mnt/AI/tools``).
base_dir :
Top-level AI-LSC directory (``/mnt/AI``). Used to expand
per-tool filesystem paths.
base_bin_dir :
Colon-separated PATH string to prepend to all commands.
"""
def __init__(
self,
tools_root: str,
base_dir: str = "",
base_bin_dir: str = "",
license_gate: Any = None,
) -> None:
from ai_lsc.constants import BASE_DIR
self.tools_root = tools_root
self.base_dir = base_dir or BASE_DIR
self.base_bin_dir = base_bin_dir
# License gate — if provided, every install_with_preflight /
# run call checks the tool's license before proceeding. If
# None, the gate is skipped (license checks happen elsewhere,
# e.g. in the UI layer).
self.license_gate = license_gate
# ── Environment construction ─────────────────────────────────────
def _env(
self,
tool_id: str = "",
env_overrides: dict[str, str] | None = None,
) -> dict[str, str]:
"""Build an enriched environment with ``~/.local`` remapped.
For Python tools, we redirect uv/pip/pipx directories into
``tools_root`` so that artifacts do not leak into the user's
home directory. Per-tool ``env_overrides`` (from the registry)
are applied last so they take precedence.
"""
env = enriched_env(self.base_bin_dir)
# ── Global XDG remap: ~/.local -> tools_root/.local ────────────
env["LOCAL_BIN"] = os.path.join(self.tools_root, ".local", "bin")
env["XDG_DATA_HOME"] = os.path.join(self.tools_root, ".local", "share")
env["XDG_CONFIG_HOME"] = os.path.join(self.tools_root, ".local", "config")
env["XDG_CACHE_HOME"] = os.path.join(self.tools_root, ".local", "cache")
# ── uv-specific: force tool installs into tools_root ───────────
if tool_id:
uv_tool_dir = os.path.join(self.tools_root, tool_id, ".uv", "tools")
uv_bin_dir = os.path.join(self.tools_root, tool_id, ".uv", "bin")
else:
uv_tool_dir = os.path.join(self.tools_root, ".uv", "tools")
uv_bin_dir = os.path.join(self.tools_root, ".uv", "bin")
env["UV_TOOL_DIR"] = uv_tool_dir
env["UV_TOOL_BIN_DIR"] = uv_bin_dir
env["UV_CACHE_DIR"] = os.path.join(self.tools_root, ".uv", "cache")
# ── pipx-specific: force installs into tools_root ────────────────
if tool_id:
env["PIPX_BIN_DIR"] = os.path.join(
self.tools_root, tool_id, ".pipx", "bin",
)
env["PIPX_HOME"] = os.path.join(
self.tools_root, tool_id, ".pipx",
)
else:
env["PIPX_BIN_DIR"] = os.path.join(self.tools_root, ".pipx", "bin")
env["PIPX_HOME"] = os.path.join(self.tools_root, ".pipx")
# ── Per-tool env overrides from registry ────────────────────────
# Keys may contain {tools_root}, {base_dir} placeholders.
if env_overrides:
for key, raw_val in env_overrides.items():
expanded = raw_val.replace(
"{tools_root}", self.tools_root,
).replace(
"{base_dir}", self.base_dir,
)
env[key] = expanded
logger.debug(
"env override: %s=%s (tool %s)", key, expanded, tool_id,
)
# ── Prepend managed bin dirs to PATH ───────────────────────────
managed_bins = [
env.get("PIPX_BIN_DIR", ""),
env.get("UV_TOOL_BIN_DIR", ""),
os.path.join(self.tools_root, "bin"),
os.path.join(self.tools_root, ".local", "bin"),
]
extra = ":".join(d for d in managed_bins if d)
env["PATH"] = f"{extra}:{env.get('PATH', '')}"
return env
# ── Preflight detection ─────────────────────────────────────────
def preflight(
self,
tool_id: str,
inst_type: str,
pkg: str,
cmd: str = "",
) -> dict[str, Any]:
"""Check whether a tool is already installed before installing.
Returns a dict matching ``PreflightResult`` fields.
"""
result: dict[str, Any] = {
"tool_id": tool_id,
"found": False,
"install_type": inst_type,
"location": "",
"version": "",
"is_update_available": False,
"suggested_action": "install",
}
location, version = self._detect_installation(
tool_id, inst_type, pkg, cmd,
)
if location:
result["found"] = True
result["location"] = location
result["version"] = version or ""
result["suggested_action"] = "update"
return result
def _detect_installation(
self,
tool_id: str,
inst_type: str,
pkg: str,
cmd: str = "",
) -> tuple[str, str]:
"""Detect existing installation. Returns (location, version)."""
# 1. Check tools_root/<tool_id> directory existence
tool_dir = os.path.join(self.tools_root, tool_id)
if os.path.isdir(tool_dir):
ver = self._detect_version(inst_type, pkg, cmd, tool_dir)
return tool_dir, ver
# 2. Check tools_root/.pipx, tools_root/.uv, tools_root/.local
for subdir in [".pipx", ".uv", ".local"]:
check = os.path.join(self.tools_root, subdir, "bin", pkg)
if os.path.exists(check):
return os.path.dirname(check), ""
# 3. Check tools_root/bin
bin_check = os.path.join(self.tools_root, "bin", pkg)
if os.path.exists(bin_check):
return os.path.dirname(bin_check), ""
# 4. Check system PATH via shutil.which
binary_name = self._binary_name(pkg, inst_type)
system_path = shutil.which(binary_name)
if system_path:
ver = self._detect_version(inst_type, pkg, cmd)
return system_path, ver
# 5. OS package manager query (pacman / dnf / apt) — list-form
# subprocess calls, no shell, no interpolation.
_PKG_MGR_QUERIES: dict[str, list[str]] = {
"pacman": ["pacman", "-Qi", pkg],
"dnf": ["dnf", "info", pkg],
"apt": ["dpkg", "-s", pkg],
}
if inst_type in _PKG_MGR_QUERIES:
try:
proc = subprocess.run(
_PKG_MGR_QUERIES[inst_type],
capture_output=True, text=True, timeout=10,
)
if proc.returncode == 0:
for line in proc.stdout.splitlines():
if line.strip().startswith("Version"):
ver = line.split(":", 1)[-1].strip()
return f"{inst_type}:{pkg}", ver
except (OSError, subprocess.SubprocessError):
pass
return "", ""
def _binary_name(self, pkg: str, inst_type: str) -> str:
"""Map a package name to its likely binary name."""
if inst_type == "npm":
return pkg if "/" not in pkg else pkg.split("/")[-1]
if inst_type in ("uv", "pip"):
return pkg.replace("-", "_").replace(".", "_")
return pkg
def _detect_version(
self,
inst_type: str,
pkg: str,
cmd: str,
cwd: str = "",
) -> str:
"""Try to extract the installed version."""
if inst_type == "git":
git_dir = os.path.join(self.tools_root, pkg.split("/")[-1]
.replace(".git", ""))
if os.path.isdir(os.path.join(git_dir, ".git")):
for argv in (
["git", "describe", "--tags", "--abbrev=0"],
["git", "rev-parse", "--short", "HEAD"],
):
try:
proc = subprocess.run(
argv,
capture_output=True, text=True,
timeout=10, cwd=git_dir,
)
if proc.returncode == 0 and proc.stdout.strip():
return proc.stdout.strip()
except (OSError, subprocess.SubprocessError):
continue
return ""
# Try the launcher command for version
ver_argv: list[str] = []
if cmd:
ver_argv = shlex.split(cmd) + ["--version"]
else:
tmpl = _VERSION_CMDS.get(inst_type, "")
if tmpl:
ver_argv = shlex.split(tmpl.format(pkg=pkg, cmd=pkg))
if not ver_argv:
return ""
try:
proc = subprocess.run(
ver_argv,
capture_output=True, text=True,
timeout=10, cwd=cwd or None,
)
if proc.returncode == 0:
return proc.stdout.strip().split("\n")[0]
except (OSError, subprocess.SubprocessError):
pass
return ""
# ── Post-install hooks ──────────────────────────────────────────
def _run_post_install(
self,
tool_id: str,
post_install_cmd: str,
) -> str:
"""Run a post-install hook inside ``tools_root/<tool_id>``."""
if not post_install_cmd:
return ""
dest = os.path.join(self.tools_root, tool_id)
env = self._env(tool_id)
# Replace {tools_root} in the command
cmd = post_install_cmd.replace("{tools_root}", self.tools_root)
logger.info("Running post-install for %s: %s", tool_id, cmd)
try:
# Post-install commands are arbitrary shell snippets supplied by
# the registry; we still need a shell here, but we run them under
# `bash -c` with an explicit argv (no shell=True) so the registry
# string is passed verbatim as a single argument and cannot
# break out of the subprocess call itself.
subprocess.run(
["bash", "-c", cmd], check=True, env=env,
timeout=300, cwd=dest,
)
return f"Post-install completed for {tool_id}."
except (subprocess.CalledProcessError, OSError) as exc:
logger.warning(
"Post-install failed for %s: %s", tool_id, exc,
)
return f"Post-install FAILED for {tool_id}: {exc}"
# ── Strategy methods ────────────────────────────────────────────
def install_ollama(self, pkg: str, tool_id: str) -> str:
"""Pull an Ollama model or install the ollama binary."""
if tool_id == "ollama":
dest = os.path.join(self.tools_root, "ollama")
os.makedirs(dest, exist_ok=True)
import tempfile
# SE-01: download-then-execute pattern avoids shell=True
tmp = tempfile.NamedTemporaryFile(
suffix=".sh", prefix="ollama-install-", delete=False,
)
tmp_path = tmp.name
tmp.close()
try:
subprocess.run(
["curl", "-fsSL", "https://ollama.com/install.sh",
"-o", tmp_path],
check=True, env=self._env("ollama"),
)
os.chmod(tmp_path, 0o755)
subprocess.run(
["bash", tmp_path], check=True, env=self._env("ollama"),
timeout=600,
)
finally:
try:
os.unlink(tmp_path)
except OSError:
pass
return "Ollama binary installed to system (managed by ollama)."
return f"Ollama model '{pkg}' queued for pull."
def install_uv(self, pkg: str, tool_id: str,
env_overrides: dict[str, str] | None = None) -> str:
"""Install a Python tool via ``uv tool install`` pinned to tools_root."""
dest = os.path.join(self.tools_root, tool_id)
os.makedirs(dest, exist_ok=True)
env = self._env(tool_id, env_overrides)
try:
_validate_pkg(pkg)
subprocess.run(
["uv", "tool", "install", pkg],
check=True, env=env, timeout=300,
)
return f"UV tool '{pkg}' installed to {env['UV_TOOL_DIR']}."
except subprocess.CalledProcessError:
logger.info("uv install failed for %s, stepping down to pipx", pkg)
return self.install_pipx(pkg, tool_id, env_overrides)
def install_pipx(self, pkg: str, tool_id: str,
env_overrides: dict[str, str] | None = None) -> str:
"""Install a Python CLI tool via ``pipx`` pinned to tools_root."""
dest = os.path.join(self.tools_root, tool_id)
os.makedirs(dest, exist_ok=True)
env = self._env(tool_id, env_overrides)
try:
_validate_pkg(pkg)
subprocess.run(
["pipx", "install", pkg],
check=True, env=env, timeout=300,
)
return f"pipx '{pkg}' installed to {env['PIPX_HOME']}."
except subprocess.CalledProcessError:
logger.info("pipx install failed for %s, stepping down to pip", pkg)
return self.install_pip(pkg, tool_id, env_overrides)
def install_pip(self, pkg: str, tool_id: str,
env_overrides: dict[str, str] | None = None) -> str:
"""Install a Python tool via ``pip`` into a per-tool venv."""
dest = os.path.join(self.tools_root, tool_id)
venv_dir = os.path.join(dest, ".venv")
os.makedirs(dest, exist_ok=True)
env = self._env(tool_id, env_overrides)
if not os.path.isdir(venv_dir):
subprocess.run(
["python3", "-m", "venv", venv_dir],
check=True, env=env, timeout=60,
)
pip_bin = os.path.join(venv_dir, "bin", "pip")
try:
_validate_pkg(pkg)
subprocess.run(
[pip_bin, "install", pkg],
check=True, env=env, timeout=300,
)
except subprocess.CalledProcessError as exc:
logger.warning("pip install failed for %s: %s", pkg, exc)
raise
self._symlink_venv_bin(tool_id, venv_dir, pkg)
return f"pip '{pkg}' installed to {venv_dir}."
def _symlink_venv_bin(
self, tool_id: str, venv_dir: str, pkg: str,
) -> None:
"""Create symlinks from the venv bin to tools_root/bin."""
bin_dir = os.path.join(self.tools_root, "bin")
os.makedirs(bin_dir, exist_ok=True)
venv_bin = os.path.join(venv_dir, "bin")
if os.path.isdir(venv_bin):
for entry in os.listdir(venv_bin):
src = os.path.join(venv_bin, entry)
dst = os.path.join(bin_dir, entry)
if not os.path.isfile(src):
continue
# L-03: TOCTOU-safe symlink — create then handle
# FileExistsError, instead of check-then-create.
try:
os.symlink(src, dst)
except FileExistsError:
pass
def install_pacman(self, pkg: str) -> str:
"""Open a terminal for ``pacman -S`` (Arch system package)."""
_validate_pkg(pkg)
subprocess.Popen([
"x-terminal-emulator", "-e", "bash", "-c",
f"sudo pacman -S --noconfirm {shlex.quote(pkg)}; sleep 2",
])
return f"Dispatched pacman for {pkg}."
def install_dnf(self, pkg: str) -> str:
"""Open a terminal for ``dnf install`` (Fedora / RHEL)."""
_validate_pkg(pkg)
subprocess.Popen([
"x-terminal-emulator", "-e", "bash", "-c",
f"sudo dnf install -y {shlex.quote(pkg)}; sleep 2",
])
return f"Dispatched dnf for {pkg}."
def install_apt(self, pkg: str) -> str:
"""Open a terminal for ``apt install`` (Debian / Ubuntu)."""
_validate_pkg(pkg)
subprocess.Popen([
"x-terminal-emulator", "-e", "bash", "-c",
f"sudo apt-get install -y {shlex.quote(pkg)}; sleep 2",
])
return f"Dispatched apt for {pkg}."
def install_npm(self, pkg: str, tool_id: str,
env_overrides: dict[str, str] | None = None) -> str:
"""Install an npm package to an isolated prefix under tools_root."""
dest = os.path.join(self.tools_root, tool_id)
os.makedirs(dest, exist_ok=True)
env = self._env(tool_id, env_overrides)
_validate_pkg(pkg)
subprocess.run(
["npm", "install", "--prefix", dest, pkg],
check=True, env=env, timeout=300,
)
return f"NPM '{pkg}' installed to {dest}."
def install_git(
self,
pkg: str,
tool_id: str,
post_install: str | None = None,
env_overrides: dict[str, str] | None = None,
) -> str:
"""Clone a git repository into ``tools_root/<tool_id>``."""
dest = os.path.join(self.tools_root, tool_id)
if os.path.exists(dest):
subprocess.run(
["git", "-C", dest, "pull", "--ff-only"],
check=True, timeout=600,
)
msg = f"Git source updated: {dest}"
else:
os.makedirs(dest, exist_ok=True)
subprocess.run(
["git", "clone", pkg, dest],
check=True, timeout=600,
)
msg = f"Git source cloned: {dest}"
if post_install:
self._run_post_install(tool_id, post_install)
return msg
def install_git_node(
self,
pkg: str,
tool_id: str,
post_install: str | None = None,
) -> str:
"""Clone a git repo and run ``yarn setup``."""
dest = os.path.join(self.tools_root, tool_id)
if os.path.exists(dest):
subprocess.run(
["git", "-C", dest, "pull", "--ff-only"], check=True, timeout=600,
)
subprocess.run(
["yarn", "install"], cwd=dest, check=True, timeout=300,
)
msg = f"Git+Node source updated: {dest}"
else:
os.makedirs(dest, exist_ok=True)
subprocess.run(
["git", "clone", pkg, dest], check=True, timeout=600,
)
subprocess.run(
["yarn", "install"], cwd=dest, check=True, timeout=300,
)
msg = f"Git+Node source synchronized: {dest}"
if post_install:
self._run_post_install(tool_id, post_install)
return msg
def install_script(
self,
cmd: str,
ctx: dict[str, str],
tool_id: str = "",
env_overrides: dict[str, str] | None = None,
) -> str:
"""Execute an arbitrary shell script (installer type ``"script"``).
The ``{tools_root}`` placeholder is resolved so scripts can
direct output to the correct directory.
"""
if "tools_root" not in ctx:
ctx["tools_root"] = self.tools_root
env = self._env(tool_id, env_overrides)
# Registry 'script' installers are arbitrary shell snippets (e.g.
# `uv pip install ... && python -m compileall .`). We pass the
# fully-formatted command to bash as a single argv element so the
# subprocess call itself is shell-free.
rendered = cmd.format(**ctx)
subprocess.run(
["bash", "-c", rendered], check=True, env=env,
)
return "Shell script deployment completed."
def install_custom(self, pkg: str, tool_id: str) -> str:
"""Open the install URL in the browser for manual installation."""
import webbrowser
url = pkg
if not url.startswith("http"):
url = f"https://{url}"
# H-20 / H-22: reject non-http(s) schemes (file://, javascript:, …)
_validate_url(url)
webbrowser.open(url)
return (
f"Opened {url} in browser for manual installation "
f"of {tool_id}. Follow the instructions on the page."
)
# ── Dispatcher ─────────────────────────────────────────────────
def _check_license(self, tool_id: str, spdx: str | None) -> None:
"""Check the tool's license against the gate before install.
Raises ``LicenseBlocked`` if the tool_id is on the SaaS
blocklist, or ``LicenseAcceptanceRequired`` if the license
has not been accepted yet. No-op if ``self.license_gate`` is
None or *spdx* is falsy.
"""
if self.license_gate is None or not spdx:
return
result = self.license_gate.check(tool_id, spdx)
if result.status == "blocked":
from ai_lsc.registry.license_gate import LicenseBlocked
raise LicenseBlocked(tool_id=tool_id, reason=result.reason)
if result.status == "needs_acceptance":
from ai_lsc.registry.license_gate import LicenseAcceptanceRequired
raise LicenseAcceptanceRequired(
tool_id=tool_id,
license_info=result.license_info,
)
def run(
self,
inst_type: str,
pkg: str,
cmd: str = "",
ctx: dict[str, str] | None = None,
tool_id: str = "",
post_install: str | None = None,
env_overrides: dict[str, str] | None = None,
license_spdx: str | None = None,
) -> str:
"""Dispatch to the correct installer strategy.
Returns a human-readable description of what happened.
Parameters
----------
license_spdx :
SPDX ID for the tool's license. If provided AND a
``license_gate`` was passed to the InstallerManager
constructor, the gate checks the license before dispatch.
If the gate returns ``blocked`` or ``needs_acceptance``,
the appropriate exception is raised before any subprocess
call.
Raises
------
ValueError
If *inst_type* is not recognized.
subprocess.CalledProcessError
If the underlying command fails.
LicenseBlocked
If the tool_id is on the SaaS blocklist.
LicenseAcceptanceRequired
If the tool's license has not been accepted yet.
"""
ctx = ctx or {}
if not tool_id:
if "github.com" in pkg:
tool_id = (pkg.rstrip("/").rsplit("/", 1)[-1]
.replace(".git", ""))
else:
tool_id = pkg.split("/")[-1].split(":")[0]
# License gate — check before any subprocess call.
self._check_license(tool_id, license_spdx)
strategies: dict[str, Any] = {
"ollama": lambda: self.install_ollama(pkg, tool_id),
"uv": lambda: self.install_uv(pkg, tool_id, env_overrides),
"pipx": lambda: self.install_pipx(pkg, tool_id, env_overrides),
"pip": lambda: self.install_pip(pkg, tool_id, env_overrides),
"script": lambda: self.install_script(
cmd, ctx, tool_id, env_overrides,
),
"pacman": lambda: self.install_pacman(pkg),
"dnf": lambda: self.install_dnf(pkg),
"apt": lambda: self.install_apt(pkg),
"npm": lambda: self.install_npm(pkg, tool_id, env_overrides),
"git": lambda: self.install_git(
pkg, tool_id, post_install, env_overrides,
),
"git_node": lambda: self.install_git_node(
pkg, tool_id, post_install,
),
"custom": lambda: self.install_custom(pkg, tool_id),
}
handler = strategies.get(inst_type)
if handler is None:
raise ValueError(f"Unknown installer type '{inst_type}'")
return handler()
# ── Batch operations ────────────────────────────────────────────
def preflight_batch(
self,
tools: dict[str, dict[str, Any]],
) -> dict[str, dict[str, Any]]:
"""Run preflight checks for multiple tools at once.
Parameters
----------
tools :
Dict of ``{tool_id: registry_entry}`` from the registry.
Returns
-------
Dict of ``{tool_id: preflight_result_dict}``.
"""
return {
tid: self.preflight(
tool_id=tid,
inst_type=meta.get("installer", {}).get("type", "pacman"),
pkg=meta.get("installer", {}).get("pkg", ""),
cmd=meta.get("installer", {}).get("cmd", ""),
)
for tid, meta in tools.items()
}
def install_with_preflight(
self,
tool_id: str,
inst_type: str,
pkg: str,
cmd: str = "",
ctx: dict[str, str] | None = None,
force: bool = False,
post_install: str | None = None,
env_overrides: dict[str, str] | None = None,
license_spdx: str | None = None,
) -> str:
"""Install a tool with preflight detection.
If the tool is already installed and *force* is False, returns
a message saying the tool exists and suggesting an update.
If *force* is True, proceeds with installation regardless.
Parameters
----------
license_spdx :
SPDX ID for the tool's license. Forwarded to ``run()``
for gate checking.
"""
# License gate — check before preflight so we don't waste a
# subprocess call on a blocked tool.
self._check_license(tool_id, license_spdx)
check = self.preflight(tool_id, inst_type, pkg, cmd)
if check["found"] and not force:
return (
f"Tool '{tool_id}' already installed at {check['location']}. "
f"Version: {check['version'] or 'unknown'}. "
f"Use force=True to update."
)
return self.run(
inst_type, pkg, cmd, ctx, tool_id,
post_install, env_overrides,
)
# ── Installation verification ───────────────────────────────────
def verify(
self,
tool_id: str,
inst_type: str,
pkg: str,
cmd: str = "",
filesystem: dict[str, str] | None = None,
) -> dict[str, Any]:
"""Run a compliance checklist against a single tool installation.
Checks:
1. Native install detected
2. Installed entirely under /mnt/AI (no ~/.local leak)
3. Config redirected from $HOME
4. Cache redirected
5. Logs redirected
6. Launcher binary accessible
7. Update command available
8. Version detection works
9. Health check (binary --version or --help)
Returns a dict matching ``VerificationResult`` fields.
"""
from ai_lsc.types import VerifyCheck
checks: list[VerifyCheck] = []
fs = filesystem or {}
tool_dir = os.path.join(self.tools_root, tool_id)
# 1. Native install detected
location, version = self._detect_installation(
tool_id, inst_type, pkg, cmd,
)
checks.append(VerifyCheck(
name="Native Install",
passed=bool(location),
detail=location or "not found",
))
# 2. Installed under /mnt/AI (no system leak)
is_managed = (
location and location.startswith(self.base_dir)
) or inst_type == "pacman" or inst_type in ("dnf", "apt")
checks.append(VerifyCheck(
name="Filesystem Compliance",
passed=is_managed,
detail=location or "N/A",
))
# 3. Config path (if declared in filesystem spec)
config_path = fs.get("config", "")
if config_path:
full = os.path.join(self.base_dir, config_path)
exists = os.path.isdir(full)
checks.append(VerifyCheck(
name="Config Redirect",
passed=exists or not location,
detail=full,
))
# 4. Cache path
cache_path = fs.get("cache", "")
if cache_path:
full = os.path.join(self.base_dir, cache_path)
checks.append(VerifyCheck(
name="Cache Redirect",
passed=os.path.isdir(full) or not location,
detail=full,
))
# 5. Logs path
logs_path = fs.get("logs", "")
if logs_path:
full = os.path.join(self.base_dir, logs_path)
checks.append(VerifyCheck(
name="Logs Redirect",
passed=os.path.isdir(full) or not location,
detail=full,
))
# 6. Launcher binary accessible
binary = self._binary_name(pkg, inst_type)
bin_path = shutil.which(binary)
checks.append(VerifyCheck(
name="Launcher Accessible",
passed=bool(bin_path),
detail=bin_path or f"{binary} not in PATH",
))
# 7. Version detection
checks.append(VerifyCheck(
name="Version Detection",
passed=bool(version),
detail=version or "unknown",
))
# 8. Health check (try --version or --help)
healthy = False
if bin_path:
for flag in ("--version", "--help"):
try:
proc = subprocess.run(
[bin_path, flag],
capture_output=True, text=True, timeout=5,
)
if proc.returncode == 0:
healthy = True
break
except (OSError, subprocess.SubprocessError):
continue
checks.append(VerifyCheck(
name="Health Check",
passed=healthy,
detail="responds to --version/--help" if healthy else "no response",
))
return {
"tool_id": tool_id,
"checks": [
{"name": c.name, "passed": c.passed, "detail": c.detail}
for c in checks
],
"install_method": inst_type,
"install_location": location or "",
"score": (
int(sum(1 for c in checks if c.passed) / len(checks) * 100)
if checks else 0
),
}

343
src/ai_lsc/runtime/lxc.py Executable file
View File

@ -0,0 +1,343 @@
"""LXC runtime manager -- Linux Container lifecycle operations.
Provides start / stop / status / create / destroy / attach operations
for LXC containers. LXC is a lighter-weight alternative to Docker/Podman
that shares the host kernel without the containerd daemon overhead.
All ``subprocess`` calls are confined here -- UI code never touches LXC
commands directly.
"""
from __future__ import annotations
import json
import os
import re
import shlex
import subprocess
from pathlib import Path
from typing import Any
# LXC container names must match ``[a-zA-Z0-9_.-]+``. We use this regex
# at every public entry point so a malicious tool_id cannot escape the
# ``-n <name>`` slot.
_LXC_NAME_RE = re.compile(r"^[A-Za-z0-9_.\-]+$")
def _validate_lxc_name(name: str) -> str:
if not name or not _LXC_NAME_RE.fullmatch(name):
raise ValueError(f"invalid LXC container name: {name!r}")
if name in {".", ".."} or os.path.normpath(name) != name:
raise ValueError(f"LXC name contains path-traversal segments: {name!r}")
return name
def _validate_tool_id(tool_id: str) -> str:
if not tool_id or not re.fullmatch(r"[A-Za-z0-9_.\-]+", tool_id):
raise ValueError(f"invalid tool_id for LXC: {tool_id!r}")
if tool_id in {".", ".."} or os.path.normpath(tool_id) != tool_id:
raise ValueError(f"tool_id contains path-traversal segments: {tool_id!r}")
return tool_id
class LxcManager:
"""Manages LXC container lifecycle via the ``lxc`` CLI.
Parameters
----------
tools_root :
Base directory for tool installations (used as container
mount source).
logs_root :
Directory for container log files.
lxc_profile :
Default LXC profile name (``"default"`` unless overridden).
"""
def __init__(
self,
tools_root: str,
logs_root: str,
lxc_profile: str = "default",
) -> None:
self.tools_root = tools_root
self.logs_root = logs_root
self.lxc_profile = lxc_profile
# ── Container lifecycle ──────────────────────────────────────────
def create(
self,
container_name: str,
image: str = "ubuntu:22.04",
config: dict[str, Any] | None = None,
) -> str:
"""Create a new LXC container.
Parameters
----------
container_name :
Name for the new container.
image :
LXC image template (e.g. ``"ubuntu:22.04"``,
``"alpine"``, ``"archlinux"``).
config :
Optional dict of LXC config key-value pairs that are
written to the container's local config file after
creation.
Returns
-------
Description of what was done.
"""
_validate_lxc_name(container_name)
cmd = [
"lxc-create",
"-n", container_name,
"-t", image.split(":")[0],
"--", image.split(":")[1] if ":" in image else "",
]
try:
subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=30)
except FileNotFoundError:
return self._install_hint("lxc-create")
# Apply custom config if provided
if config:
self._apply_config(container_name, config)
return f"LXC container '{container_name}' created from {image}"
def start(self, container_name: str) -> str:
"""Start an LXC container."""
cmd = ["lxc-start", "-n", container_name, "-d"] # -d = daemonize
try:
subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=30)
except FileNotFoundError:
return self._install_hint("lxc-start")
except subprocess.CalledProcessError as e:
return f"LXC start failed: {e.stderr.strip()}"
return f"LXC container '{container_name}' started"
def stop(self, container_name: str) -> str:
"""Stop a running LXC container."""
cmd = ["lxc-stop", "-n", container_name, "-t", "5"] # 5s timeout
try:
subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=30)
except FileNotFoundError:
return self._install_hint("lxc-stop")
except subprocess.CalledProcessError as e:
return f"LXC stop failed: {e.stderr.strip()}"
return f"LXC container '{container_name}' stopped"
def destroy(self, container_name: str) -> str:
"""Destroy (remove) an LXC container."""
cmd = ["lxc-destroy", "-n", container_name]
try:
subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=30)
except FileNotFoundError:
return self._install_hint("lxc-destroy")
except subprocess.CalledProcessError as e:
return f"LXC destroy failed: {e.stderr.strip()}"
return f"LXC container '{container_name}' destroyed"
def freeze(self, container_name: str) -> str:
"""Freeze (pause) a running container without stopping it."""
cmd = ["lxc-freeze", "-n", container_name]
try:
subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=30)
except (FileNotFoundError, subprocess.CalledProcessError) as e:
return f"LXC freeze failed: {getattr(e, 'stderr', str(e))}"
return f"LXC container '{container_name}' frozen"
def unfreeze(self, container_name: str) -> str:
"""Unfreeze (resume) a paused container."""
cmd = ["lxc-unfreeze", "-n", container_name]
try:
subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=30)
except (FileNotFoundError, subprocess.CalledProcessError) as e:
return f"LXC unfreeze failed: {getattr(e, 'stderr', str(e))}"
return f"LXC container '{container_name}' resumed"
# ── Status / inspection ───────────────────────────────────────────
def is_running(self, container_name: str) -> bool:
"""Check if a container is currently running."""
cmd = ["lxc-info", "-n", container_name, "-s"]
try:
result = subprocess.run(
cmd, capture_output=True, text=True, timeout=5
)
return "RUNNING" in result.stdout.upper()
except (FileNotFoundError, subprocess.TimeoutExpired):
return False
def get_state(self, container_name: str) -> str:
"""Return the container state (RUNNING, STOPPED, FROZEN)."""
cmd = ["lxc-info", "-n", container_name, "-s"]
try:
result = subprocess.run(
cmd, capture_output=True, text=True, timeout=5
)
for state in ("RUNNING", "STOPPED", "FROZEN"):
if state in result.stdout.upper():
return state
return "UNKNOWN"
except (FileNotFoundError, subprocess.TimeoutExpired):
return "UNKNOWN"
def list_containers(self, running_only: bool = False) -> list[str]:
"""List container names.
Parameters
----------
running_only :
If ``True`` only return running containers.
Returns
-------
List of container name strings.
"""
cmd = ["lxc-ls"]
if running_only:
cmd.append("--running")
try:
result = subprocess.run(
cmd, capture_output=True, text=True, timeout=5
)
return [
name.strip() for name in result.stdout.strip().splitlines()
if name.strip()
]
except (FileNotFoundError, subprocess.TimeoutExpired):
return []
# ── Execution ────────────────────────────────────────────────────
def attach_exec(
self,
container_name: str,
command: str = "/bin/bash",
) -> str:
"""Execute a command inside a running container (non-interactive)."""
_validate_lxc_name(container_name)
# H-12: use shlex.split so quoted arguments survive. The previous
# `command.split()` mangled inputs like `echo 'hello world'` into
# ['echo', '"hello', 'world"'].
argv = shlex.split(command) or ["/bin/bash"]
cmd = ["lxc-attach", "-n", container_name, "--", *argv]
try:
subprocess.run(cmd, capture_output=True, text=True, check=True)
except (FileNotFoundError, subprocess.CalledProcessError) as e:
return f"LXC attach failed: {getattr(e, 'stderr', str(e))}"
return f"Executed in '{container_name}': {command}"
def launch_cli(
self,
container_name: str,
) -> str:
"""Open an interactive terminal inside the container.
Launches an x-terminal-emulator with ``lxc-attach``.
"""
import shutil
term = shutil.which("x-terminal-emulator") or shutil.which("gnome-terminal") or shutil.which("konsole")
if not term:
return "No terminal emulator found for LXC CLI attach."
subprocess.Popen(
[term, "-e", "bash", "-c",
f"lxc-attach -n {container_name} --clear-env -- "
f"env TERM=xterm-256color /bin/bash"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
return f"Interactive terminal opened for '{container_name}'"
# ── Config helpers ──────────────────────────────────────────────
def _apply_config(
self,
container_name: str,
config: dict[str, Any],
) -> None:
"""Append config key-value pairs to a container's config file."""
_validate_lxc_name(container_name)
config_path = Path("/var/lib/lxc") / container_name / "config"
if not config_path.exists():
return
# M-21: build the formatted config lines via comprehension.
lines = [
(f"lxc.{key} = {','.join(str(v) for v in value)}"
if isinstance(value, (list, tuple))
else f"lxc.{key} = {value}")
for key, value in config.items()
]
# M-05: explicit encoding + flush + fsync so a crash mid-write
# cannot corrupt the LXC config file.
with open(config_path, "a", encoding="utf-8") as f:
f.write("\n# ai-lsc generated\n")
for line in lines:
f.write(f"{line}\n")
f.flush()
os.fsync(f.fileno())
@staticmethod
def _install_hint(cmd: str) -> str:
# M-39: don't assume Arch's pacman; the app advertises multi-distro
# support, so keep the hint generic.
return (
f"LXC not installed. Install the `lxc` package for your "
f"distribution (e.g. `sudo pacman -S lxc` on Arch, "
f"`sudo apt-get install lxc` on Debian/Ubuntu, "
f"`sudo dnf install lxc` on Fedora).\n"
f"Missing command: {cmd}"
)
# ── Service delegation (mirrors TmuxManager interface) ───────────
def launch_service(
self,
tool_id: str,
command: str,
log_file: str = "",
dtach_bin: str | None = None,
base_bin_dir: str = "",
) -> str:
"""Start a tool as an LXC container service.
Creates the container if it does not exist, starts it,
and runs the tool command inside. The container is named
``ai-lsc-<tool_id>`` for consistency.
Returns a description of what was done.
"""
_validate_tool_id(tool_id)
container_name = f"ai-lsc-{tool_id}"
if not self.is_running(container_name):
if container_name not in self.list_containers():
self.create(
container_name,
image="ubuntu:22.04",
config={
"mount.auto": f"{self.tools_root} opt none bind 0 0",
},
)
self.start(container_name)
# Run the tool command inside the container
if command:
self.attach_exec(container_name, command)
return f"Tool {tool_id} running in LXC container '{container_name}'"
def stop_service(self, tool_id: str) -> str:
"""Stop the LXC container for a tool."""
_validate_tool_id(tool_id)
container_name = f"ai-lsc-{tool_id}"
self.stop(container_name)
return f"LXC container '{container_name}' stopped for {tool_id}"

126
src/ai_lsc/runtime/process.py Executable file
View File

@ -0,0 +1,126 @@
"""Generic process manager.
Handles desktop-app launching, process killing via ``pkill``, and
bare ``subprocess.Popen`` calls for non-tmux/non-systemd tools.
"""
from __future__ import annotations
import os
import shlex
import shutil
import subprocess
import threading
from collections.abc import Iterable
# Terminal emulator candidates, ordered by preference.
_TERMINALS = [
"xterm", "konsole", "gnome-terminal", "xfce4-terminal",
"lxterminal", "alacritty", "kitty", "wezterm",
]
def detect_terminal() -> str:
"""Return the first available terminal emulator on the system."""
return next((t for t in _TERMINALS if shutil.which(t)), "xterm")
def _to_arg_list(command: str | list[str]) -> list[str]:
"""Coerce a registry command into a safe argv list.
Accepts either a pre-split argv list (preferred) or a single shell-style
string (which is split with :func:`shlex.split`, never passed to a shell).
"""
if isinstance(command, (list, tuple)):
return [str(c) for c in command]
return shlex.split(command)
class ProcessManager:
"""Launch and terminate generic (desktop / CLI) processes.
All launched processes are tracked in :attr:`_launched` so they can be
polled / reaped periodically, preventing zombie accumulation in a
long-lived GUI process.
"""
def __init__(self) -> None:
self._lock = threading.Lock()
self._launched: list[subprocess.Popen] = []
def launch_desktop(self, command: str | list[str]) -> None:
"""Fire-and-forget launch of a desktop command."""
argv = _to_arg_list(command)
if not argv:
raise ValueError("launch_desktop received an empty command")
with self._lock:
self._launched.append(subprocess.Popen(argv))
def launch_terminal(self, command: str | list[str], env: dict[str, str] | None = None) -> None:
"""Open *command* inside a new terminal emulator window."""
term = detect_terminal()
# xterm, alacritty, kitty, wezterm use -e; others use --
sep = "-e" if term in ("xterm", "alacritty", "kitty", "wezterm") else "--"
cmd_str = shlex.join(_to_arg_list(command)) if isinstance(command, (list, tuple)) else command
argv = [term, sep, "bash", "-c", cmd_str]
with self._lock:
self._launched.append(subprocess.Popen(argv, env=env))
def kill_by_name(self, search_term: str) -> None:
"""Send SIGTERM to all processes matching *search_term*."""
subprocess.run(
["pkill", "-f", search_term],
timeout=5,
stderr=subprocess.DEVNULL,
check=False,
)
def reap(self) -> None:
"""Poll and drop finished children; call periodically from the GUI."""
with self._lock:
still_alive: list[subprocess.Popen] = []
for proc in self._launched:
if proc.poll() is None:
still_alive.append(proc)
self._launched = still_alive
def shutdown(self, timeout: float = 2.0) -> None:
"""Terminate every still-running child on application quit."""
with self._lock:
for proc in self._launched:
if proc.poll() is None:
proc.terminate()
for proc in self._launched:
try:
proc.wait(timeout=timeout)
except subprocess.TimeoutExpired:
proc.kill()
self._launched.clear()
def __del__(self) -> None: # noqa: D401 - best-effort cleanup
try:
self.shutdown()
except Exception:
pass
def safe_env(base: dict[str, str] | None = None, **overrides: str) -> dict[str, str]:
"""Build a clean environment for child processes.
Merges the current ``os.environ`` with any explicit *base* dict and
keyword overrides. Values are coerced to ``str``.
"""
env = dict(os.environ)
if base:
env.update(base)
env.update({k: str(v) for k, v in overrides.items()})
return env
__all__ = [
"ProcessManager",
"detect_terminal",
"safe_env",
"_to_arg_list",
"_TERMINALS",
]

54
src/ai_lsc/runtime/status.py Executable file
View File

@ -0,0 +1,54 @@
"""Status checker -- unified interface for checking if a service is live.
Delegates to backend-specific strategies (tmux, systemd, or psutil
process scan) based on the launcher type.
"""
from __future__ import annotations
from ai_lsc.runtime.systemd import SystemdManager
from ai_lsc.runtime.tmux import TmuxManager
from ai_lsc.utils.process import first_matching_process
class StatusChecker:
"""Check service liveness regardless of launcher backend."""
def __init__(
self,
tmux: TmuxManager | None = None,
systemd: SystemdManager | None = None,
) -> None:
self._tmux = tmux or TmuxManager()
self._systemd = systemd or SystemdManager()
def is_running(
self,
launcher_type: str,
tool_id: str,
service_cmd: str = "",
search_term: str = "",
) -> bool:
"""Return ``True`` if the service is currently running.
Parameters
----------
launcher_type:
One of ``"tmux"``, ``"systemd"``, or anything else (psutil).
tool_id:
Used by tmux to identify the window.
service_cmd:
Used by systemd ``is-active``.
search_term:
Used by psutil ``process_iter`` scan.
"""
strategy = {
"systemd": lambda: self._systemd.is_active(service_cmd),
"tmux": lambda: self._tmux.is_running(tool_id),
}.get(launcher_type)
if strategy is not None:
return strategy()
# fallback: psutil scan
return first_matching_process(search_term) is not None

44
src/ai_lsc/runtime/systemd.py Executable file
View File

@ -0,0 +1,44 @@
"""Systemd service manager.
Wraps ``systemctl`` calls for tools that launch as system services.
"""
from __future__ import annotations
import subprocess
from ai_lsc.runtime.process import detect_terminal
class SystemdManager:
"""Start, stop, and query systemd services."""
def start(self, service_cmd: str) -> None:
"""Enable + start a systemd service via a terminal emulator."""
term = detect_terminal()
sep = "-e" if term in ("xterm", "alacritty", "kitty", "wezterm") else "--"
subprocess.Popen(
[term, sep, "sudo", "systemctl", "start", service_cmd],
start_new_session=True,
)
def stop(self, service_cmd: str) -> None:
"""Stop a systemd service via a terminal emulator."""
term = detect_terminal()
sep = "-e" if term in ("xterm", "alacritty", "kitty", "wezterm") else "--"
subprocess.Popen(
[term, sep, "sudo", "systemctl", "stop", service_cmd],
start_new_session=True,
)
def is_active(self, service_cmd: str) -> bool:
"""Return ``True`` if the service reports 'active'."""
return (
subprocess.run(
["systemctl", "is-active", service_cmd],
capture_output=True,
text=True,
timeout=5,
).stdout.strip()
== "active"
)

205
src/ai_lsc/runtime/tmux.py Executable file
View File

@ -0,0 +1,205 @@
"""Tmux session/window manager.
Wraps all ``tmux`` CLI interactions: session creation, window
management, command sending, and live-window querying.
Every public method returns a value or raises -- no UI imports.
"""
from __future__ import annotations
import os
import re
import shlex
import subprocess
from pathlib import Path
# Valid tmux window / session name characters. Anything outside this
# set is rejected to prevent tmux-command injection.
_NAME_RE = re.compile(r"^[A-Za-z0-9_.:@\-]+$")
def _validate_name(name: str, *, what: str = "name") -> None:
"""Reject tmux target names that could break out of the argument slot."""
if not name or not _NAME_RE.fullmatch(name):
raise ValueError(f"invalid tmux {what}: {name!r}")
def _socket_path_for(tool_id: str) -> str:
"""Return a safe, user-scoped socket path for a tmux service."""
runtime_dir = os.environ.get(
"XDG_RUNTIME_DIR",
f"/tmp/ai-lsc-{os.getuid()}",
)
base = Path(runtime_dir) / "ai-lsc"
base.mkdir(parents=True, exist_ok=True, mode=0o700)
safe_id = re.sub(r"[^A-Za-z0-9_.\-]", "_", tool_id)
return str(base / f"{safe_id}.sock")
class TmuxManager:
"""Manages tmux sessions and windows for service isolation."""
SESSION = f"ai_lsc_{os.getuid()}"
def __init__(self, env: dict[str, str] | None = None) -> None:
self.env = env
# -- session lifecycle ------------------------------------------------
def ensure_session(self) -> None:
"""Create the master session if it does not already exist."""
_validate_name(self.SESSION, what="session name")
has = subprocess.run(
["tmux", "has-session", "-t", self.SESSION],
stderr=subprocess.DEVNULL,
check=False,
)
if has.returncode != 0:
subprocess.run(
["tmux", "new-session", "-d", "-s", self.SESSION, "-n", "Master"],
stderr=subprocess.DEVNULL,
check=False,
)
def window_exists(self, window_name: str) -> bool:
"""Check if a named window exists in the session."""
_validate_name(window_name, what="window name")
listing = subprocess.run(
["tmux", "list-windows", "-t", self.SESSION, "-F",
"#{window_name}"],
stderr=subprocess.DEVNULL,
stdout=subprocess.PIPE,
text=True,
check=False,
)
if listing.returncode != 0:
return False
# SE-07: exact match instead of substring
return window_name in listing.stdout.splitlines()
def kill_window(self, window_name: str) -> None:
"""Safely kill a window (ignores errors if missing)."""
_validate_name(window_name, what="window name")
subprocess.run(
["tmux", "kill-window", "-t", f"{self.SESSION}:{window_name}"],
stderr=subprocess.DEVNULL,
check=False,
)
def create_window(self, window_name: str) -> None:
"""Create a new detached window inside the session.
Retries up to 5 times with a short sleep when tmux reports
an index collision (happens when concurrent launches race).
"""
import time
_validate_name(window_name, what="window name")
for attempt in range(5):
proc = subprocess.run(
[
"tmux", "new-window",
"-t", self.SESSION,
"-n", window_name,
"-d",
],
capture_output=True,
text=True,
check=False,
)
if proc.returncode == 0:
return
# If window already exists with this name, that's fine too
if self.window_exists(window_name):
return
# Index collision — wait and retry
if "index" in proc.stderr and "in use" in proc.stderr:
time.sleep(0.1 * (attempt + 1))
continue
break # some other error, stop retrying
def send_command(
self,
window_name: str,
command: str,
extra_env: str = "",
) -> None:
"""Send a command string to a window, optionally prepending env."""
_validate_name(window_name, what="window name")
payload = f"{extra_env} {command}".strip()
# tmux send-keys receives the payload as a single argv element;
# no shell is involved, so quoting is preserved.
subprocess.run(
[
"tmux", "send-keys",
"-t", f"{self.SESSION}:{window_name}",
payload, "C-m",
],
stderr=subprocess.DEVNULL,
check=False,
)
# -- high-level service lifecycle -------------------------------------
def launch_service(
self,
tool_id: str,
command: str,
log_file: str,
dtach_bin: str | None = None,
base_bin_dir: str = "",
) -> None:
"""Isolate a service in its own tmux window (optionally via dtach).
Parameters
----------
tool_id:
Service identifier used as the window name.
command:
Shell command to run inside the window.
log_file:
Path where stdout/stderr should be redirected.
dtach_bin:
Path to dtach binary for persistent attach/detach.
base_bin_dir:
PATH colon-separated string to prepend.
"""
_validate_name(tool_id, what="tool_id")
window_name = f"{self.SESSION}::{tool_id}"
track_cmd = f"({command}) > {shlex.quote(log_file)} 2>&1"
wrapped = track_cmd
if dtach_bin:
socket_path = _socket_path_for(tool_id)
wrapped = f"{shlex.quote(dtach_bin)} -n {shlex.quote(socket_path)} bash -c {shlex.quote(track_cmd)}"
self.ensure_session()
self.kill_window(window_name)
self.create_window(window_name)
env_exports = ""
if base_bin_dir:
env_exports = f"export PATH={shlex.quote(base_bin_dir)}:$PATH; "
self.send_command(window_name, wrapped, extra_env=env_exports)
def stop_service(self, tool_id: str) -> None:
"""Kill the tmux window for a service."""
_validate_name(tool_id, what="tool_id")
window_name = f"{self.SESSION}::{tool_id}"
self.kill_window(window_name)
def is_running(self, tool_id: str) -> bool:
"""Check whether the tmux window for *tool_id* is live."""
_validate_name(tool_id, what="tool_id")
return self.window_exists(f"{self.SESSION}::{tool_id}")
def attach_cli(self, tool_id: str) -> str:
"""Return a shell fragment that attaches to the service window."""
_validate_name(tool_id, what="tool_id")
target = shlex.quote(f"{self.SESSION}:{self.SESSION}::{tool_id}")
return f"tmux attach -t {target} || "
__all__ = ["TmuxManager", "_validate_name", "_socket_path_for"]

1
src/ai_lsc/service/__init__.py Executable file
View File

@ -0,0 +1 @@
"""AI-LSC service management sub-package."""

Some files were not shown because too many files have changed in this diff Show More