SysDeck/klanker-gate/docs/concepts/functionality-and-capabilities.md

Identified Functionality, Capabilities & Purpose

This document is the code-traceable inventory of what Frosty Deno actually does in the current repository state.

Application purpose statement

Frosty Deno is a Deno-native gateway for LLM traffic and operations. It provides one HTTP surface for many model providers, adds governance and metering, optionally caches responses, aggregates MCP tools, exposes a same-origin operator control plane, and stores durable configuration and counters in PostgreSQL.

Primary observed use cases:

• Send chat, completion, embedding, image, audio, file, or batch requests through one gateway surface instead of integrating each provider separately.
• Govern inference traffic with virtual keys, rate limits, request budgets, cost budgets, team hierarchy, and pricing metadata.
• Operate providers, settings, cache, logs, runtime diagnostics, and MCP clients through the same-origin control UI and /api/* endpoints.
• Run tool-aware and agentic workloads through MCP and the /v1/responses loop.

Capability map

#	Feature	Capability	Purpose	Primary Users	Implementation Location
1	Chat completions	OpenAI-compatible chat completions with stream and non-stream modes	Core inference routing	API clients	apps/gateway/routes/inference.ts, packages/providers/src/*
2	Legacy completions	OpenAI legacy text completions	Backward compatibility	API clients	apps/gateway/routes/inference.ts
3	Anthropic Messages compatibility	Anthropic-style message ingress and token counting	Protocol compatibility	API clients	apps/gateway/routes/compat.ts, apps/gateway/routes/compat_families.ts
4	Responses loop	Tool-aware /v1/responses execution path	Agentic workflows	API clients, agents	apps/gateway/routes/inference.ts, packages/core/src/*
5	Model catalog	List configured models and provider-backed catalog data	Discover available models	API clients, operators	apps/gateway/routes/inference.ts, apps/gateway/routes/catalog.ts
6	Embeddings	Embedding generation across supporting providers	Search, ranking, semantic cache	API clients	apps/gateway/routes/advanced.ts
7	Image generation	Image generation surface	Multimodal generation	API clients	apps/gateway/routes/advanced.ts, packages/providers/src/imagen.ts
8	Speech synthesis	Text-to-speech output	Audio generation	API clients	apps/gateway/routes/advanced.ts, packages/providers/src/audio.ts
9	Transcription	Speech-to-text ingestion	Audio understanding	API clients	apps/gateway/routes/advanced.ts, packages/providers/src/audio.ts
10	Files API	Provider-backed file create, list, fetch, and delete	File-oriented provider workflows	API clients, operators	apps/gateway/routes/advanced.ts, provider advanced adapters
11	Batches API	Provider-backed batch job create, list, inspect, cancel, and fetch results	Long-running inference jobs	API clients, operators	apps/gateway/routes/advanced.ts, provider advanced adapters
12	Provider management	CRUD provider accounts, refresh models, inspect health	Operate upstream accounts	Operators	apps/gateway/routes/admin.ts, packages/contracts/src/config.ts, control UI provider views
13	Gateway config import/export	Export, import, reload, and view config	Backup and restore operator state	Operators	apps/gateway/routes/admin.ts, packages/config/src/service.ts
14	Virtual keys	Create governed machine credentials with budgets and scopes	Traffic control and attribution	Operators	apps/gateway/routes/governance.ts, packages/governance/src/virtual_keys.ts
15	Team and customer hierarchy	Roll budgets and usage up the governance chain	Multi-tenant spend and usage control	Operators	apps/gateway/routes/governance.ts, packages/governance/src/hierarchy.ts
16	Pricing catalog and sync	Persist pricing overrides and optional upstream sync	Cost accounting and budget evaluation	Operators	apps/gateway/routes/governance.ts, packages/governance/src/pricing.ts, pricing_sync.ts
17	Settings API	Persist and expose effective operator settings with provenance	Runtime tuning and operator visibility	Operators	apps/gateway/routes/settings.ts, packages/contracts/src/settings.ts
18	Response cache	Exact and semantic cache with clear operations	Lower cost and repeated-response latency	Operators, API clients	packages/cache/src/*, apps/gateway/routes/extensions.ts
19	Logs and analytics	Live logs, stored logs, analytics rollups, metrics, runtime gauges	Observability and auditing	Operators	apps/gateway/routes/logs.ts, analytics.ts, runtime.ts, packages/telemetry/src/*
20	MCP client aggregation	Manage MCP clients, sync tools, expose health	Centralized tool integration	Operators, agents	apps/gateway/routes/extensions.ts, packages/mcp/src/*
21	MCP server surface	Serve the gateway as an MCP endpoint	Tool-serving interoperability	Agents, external MCP clients	apps/gateway/routes/mcpserver.ts
22	Code Mode	VFS metadata surface and gated run surface	Tool-oriented code execution workflows	Operators, agents	apps/gateway/routes/codemode.ts, packages/mcp/src/codemode/*
23	Same-origin control plane	React operator UI for the gateway surface	Browser-based operations	Operators	apps/control-ui/src/*, served by apps/gateway/main.ts

Functional domains

Inference and compatibility

What it does:

• Accepts canonical and compatibility-route inference traffic and dispatches it to configured providers.

Why it exists:

• Lets clients talk to one gateway rather than many provider SDKs and HTTP contracts.

How it is implemented:

• Route registration in inference.ts, compat.ts, compat_families.ts, azure_ingress.ts, and openrouter_ingress.ts
• Provider adapters in packages/providers/src/*
• Canonical translation and stream handling in packages/core/src/*

Entry points:

• /v1/*, /genai/*, /cohere/*, /openrouter/*, /openai/deployments/*, and alias prefixes such as /openai/* and /anthropic/*

Provider operations

What it does:

• Stores, redacts, reloads, and health-checks provider accounts.

Why it exists:

• Makes upstream-account management an operator concern rather than a code-edit concern.

How it is implemented:

• packages/contracts/src/config.ts
• packages/config/src/service.ts
• apps/gateway/routes/admin.ts
• provider-management screens in apps/control-ui/src/views/ProvidersView.tsx

Entry points:

• /api/providers*, /api/config*, /api/proxy-config

Governance and pricing

What it does:

• Admits or denies requests, meters usage, and rolls spend up through a hierarchy.

Why it exists:

• Provides quota, attribution, and cost control for multi-tenant inference traffic.

How it is implemented:

• VirtualKeyManager, GovernanceHierarchy, ProviderBudgetTracker, PricingCatalog, SharedRateLimiter

Entry points:

• /api/virtual-keys*, /api/teams*, /api/customers*, /api/pricing*, governance middleware on inference routes

Cache and invalidation

What it does:

• Returns cached responses when possible and keeps cache state coherent across processes.

Why it exists:

• Lowers repeated-request latency and upstream cost.

How it is implemented:

• SemanticCache, PgCacheStore, PgVectorStore, InvalidationBus

Entry points:

• cache use is implicit on eligible inference routes; cache clearing is exposed through /api/cache*

MCP and Code Mode

What it does:

• Aggregates external MCP tools and exposes Frosty as an MCP server, with a separate Code Mode surface.

Why it exists:

• Supports tool-oriented agent workflows and structured execution paths.

How it is implemented:

• packages/mcp/src/registry.ts, monitor.ts, codemode/*

Entry points:

• /api/mcp*, /mcp, /api/mcp/codemode/*

Observability and runtime diagnostics

What it does:

• Tracks logs, metrics, traces, analytics, runtime concurrency, and stored-log analytics.

Why it exists:

• Gives operators visibility into behavior, failures, and cost.

How it is implemented:

• packages/telemetry/src/*, apps/gateway/routes/logs.ts, analytics.ts, runtime.ts

Entry points:

• /api/logs*, /api/analytics, /api/runtime, /metrics

Operator UI

What it does:

• Presents the provider, governance, settings, logs, dashboard, and runtime surfaces in a browser.

Why it exists:

• Keeps operator workflows on the same origin and the same data contract as the gateway.

How it is implemented:

• React views under apps/control-ui/src/views/* and reusable UI/domain components under apps/control-ui/src/components/*

Entry points:

• hash-routed UI views such as #/providers, #/dashboard, #/logs, #/settings, #/virtual-keys, #/extensions

User roles and permissions matrix

The shipped code exposes operational identities rather than human user accounts.

Role	Capability	Access Level
Anonymous client	GET /healthz, GET /api/version	Read
Anonymous client	Public inference routes without governance credentials	Write
Virtual-key client	Governed inference routes with valid bearer key	Write
Virtual-key client	/api/* operator routes	None
Virtual-key client	Stored logs and pricing administration	None
Admin operator	Provider CRUD and config import/export	Admin
Admin operator	Virtual-key, team, and customer administration	Admin
Admin operator	Pricing, settings, and cache operations	Admin
Admin operator	Runtime, logs, and analytics inspection	Read
Admin operator	MCP client management and Code Mode inspection	Admin
MCP peer	/mcp server surface	Write

Assignment and enforcement:

• Admin capability is enforced by adminAuthMiddleware when FROSTY_ADMIN_TOKEN is configured.
• Virtual-key capability is enforced by governance middleware and VirtualKeyManager on inference routes.
• Team and customer chain enforcement is performed by GovernanceHierarchy.

Feature traceability

Feature	API endpoints	UI routes	Persistence	Test coverage
Chat completions	/v1/chat/completions	none direct	optional logs, counters, cache	tests/contract/golden_chat_test.ts, tests/integration/*
Legacy completions	/v1/completions	none direct	optional logs and counters	covered by route and integration tests in gateway and compat suites
Anthropic Messages compatibility	/v1/messages, /v1/messages/count_tokens	none direct	optional logs and counters	tests/integration/anthropic_ingress_test.ts, streaming and contract suites
Responses loop	/v1/responses	none direct	optional logs and counters	tests/contract/responses_agent_test.ts
Embeddings	/v1/embeddings	none direct	optional logs and counters	advanced API tests and integration suites
Images	/v1/images/generations	none direct	optional logs and counters	advanced API tests
Speech	/v1/audio/speech	none direct	optional logs and counters	advanced API tests
Transcriptions	/v1/audio/transcriptions	none direct	optional logs and counters	advanced API tests
Files	/v1/files*	none direct	provider-side file storage plus gateway logs/counters	advanced API tests
Batches	/v1/batches*	none direct	provider-side batch state plus gateway logs/counters	advanced API tests
Provider management	/api/providers*, /api/config*	#/providers, #/settings/config	frosty.state config keys	apps/gateway/routes/admin_test.ts, ProvidersView.test.tsx, integration suites
Virtual keys	/api/virtual-keys*	#/virtual-keys	frosty.state plus frosty.counters	packages/governance/src/virtual_keys_test.ts, VirtualKeysView.test.tsx, governance integration suites
Teams and customers	/api/teams*, /api/customers*	#/teams, #/customers	frosty.state plus frosty.counters	hierarchy and governance integration suites
Pricing	/api/pricing*	#/pricing	pricing records in durable state	pricing tests and integration suites
Settings	/api/settings	#/settings/*	['settings', <group>] overrides in frosty.state	tests/contract/settings_contract_test.ts, SettingsView.test.tsx, settings integration suites
Cache	implicit on inference routes, /api/cache*	#/settings/caching	frosty.response_cache, pgvector table, invalidation events	tests/integration/cache_test.ts, cache package tests
Logs and analytics	/api/logs*, /api/analytics, /metrics	#/logs, #/dashboard	in-memory LogBus, optional stored logs, counters	log, analytics, observability, and dashboard tests
Runtime diagnostics	/api/runtime	#/status	in-memory gauges and state projections	tests/integration/runtime_api_test.ts, StatusView.test.tsx
MCP management	/api/mcp*	#/extensions	MCP client config in durable state	MCP integration suites and extensions UI paths
MCP server surface	/mcp	none direct	request handling only	MCP transport and plugin integration suites
Code Mode	/api/mcp/codemode/*	settings-side Code Mode preview	no independent durable store beyond config state	Code Mode route and executor tests

No major capability reviewed here was found to be completely untested, but coverage density varies. Provider-specific advanced capabilities rely heavily on integration and route-level tests rather than UI tests.

External integrations

Integration	Purpose	Implementation location	Data exchanged
OpenAI API	Chat, embeddings, images, audio, files	packages/providers/src/openai.ts and related routes	prompts, messages, usage, media payloads
Anthropic API	Messages, tool use, files, batches	packages/providers/src/anthropic*.ts	messages, tool calls, file and batch payloads
Azure OpenAI	Azure deployment-scoped inference and advanced APIs	packages/providers/src/azure.ts, apps/gateway/routes/azure_ingress.ts	deployment-scoped requests and responses
Gemini / Vertex	Google model access and Imagen support	packages/providers/src/gemini*.ts, vertex.ts	prompts, embeddings, image requests, auth data
OpenRouter	OpenRouter-shaped ingress and provider access	packages/providers/src/openai_compat.ts, openrouter_ingress.ts	chat and embeddings payloads
Bedrock and S3	Bedrock inference and file/batch support	packages/providers/src/bedrock*.ts, s3.ts, sigv4.ts	signed requests, file objects, batch payloads
Cohere	Cohere-compatible chat ingress	packages/providers/src/cohere.ts, compat_families.ts	chat payloads
ElevenLabs	Text-to-speech	packages/providers/src/elevenlabs.ts	text and audio data
Hugging Face	model inference, images, embeddings, audio	packages/providers/src/huggingface.ts	model and media payloads
Ollama	local provider support	packages/providers/src/openai_compat.ts via LM-compatible setup	local HTTP model payloads
Groq, Mistral, xAI, Perplexity, Cerebras, Nebius, SGLang, Parasail	additional provider-account types	provider manager plus OpenAI-wire or dedicated adapters	standard inference payloads
Generic OpenAI-compatible endpoint	arbitrary compatible upstream	packages/providers/src/openai_compat.ts	compatible OpenAI payloads
Generic Anthropic-compatible endpoint	arbitrary compatible upstream	Anthropic compatibility path	compatible Anthropic payloads
MCP servers	tool aggregation and remote tool calls	packages/mcp/src/client.ts, registry.ts	JSON-RPC requests, tool metadata, tool outputs
PostgreSQL / pgvector	durable state, counters, cache, embeddings, invalidation coordination	packages/config/src/*, packages/cache/src/*	config documents, counters, cached responses, vectors
LiteLLM price catalog	optional pricing metadata sync	packages/governance/src/pricing_sync.ts	pricing JSON
OTLP collector	optional tracing export	packages/telemetry/src/otel.ts	spans and span-derived metrics

Non-functional capabilities

Implemented cross-cutting capabilities:

• Exact and semantic caching
• Rate limiting and budget enforcement
• Multi-process clustering on supported platforms
• Durable and in-memory request logging
• Prometheus metrics and OTLP tracing
• Same-origin SPA serving
• Optional encryption at rest for persisted secrets
• Provider fallback and load-balancing policy
• Background health monitoring and configuration reconciliation

Gaps and partial implementations

Area	Status	Evidence
OpenRouter native GET /generation and GET /key	Explicit 501	apps/gateway/routes/openrouter_ingress.ts
Aggregator-path Bedrock native ingress	Explicit 501	apps/gateway/routes/compat_families.ts
Some provider-panel config fields	Persisted and surfaced but not fully enforced	field comments in packages/contracts/src/config.ts
Cache-hit streaming	Not implemented by design	cache package behavior and route flow
Code Mode executor	Gated and intentionally constrained	packages/mcp/src/codemode/* and env defaults
Kubernetes and Helm packaging	Not present in the repo	deployment assets are Docker and Compose only