projects · llm-gateway
One front door for every model.
Local models for private questions, cloud frontier models for hard ones — and no reason anyone in the household should care which is which. The gateway puts one API and one chat surface over all of them, with per-client budgets so an experiment can't quietly become a bill.
The product
A model roster, not a model bet
Models come and go monthly; the surface stays. A router named
local-smart picks the best local model for the job, cloud
models sit one dropdown away, and every client — human or agent — spends
from its own budgeted key.
- StackLiteLLM gateway · Open WebUI chat · Postgres · local models on Apple silicon + cloud providers behind one API shape
- GovernancePer-client virtual keys with monthly budgets; agents get their own keys, so a runaway loop hits a ceiling, not a credit card
- AccessPublic behind SSO (the app handles the OIDC handshake itself); the platform's edge gate does the rest
Lessons
What it taught the platform
Theming a third-party app is a token test
Open WebUI ships its own design; making it wear the dusk theme through a thin CSS adapter proved the tokens-and-adapters pattern works even when you don't own the DOM.
Healthy-but-wasteful is a real failure class
An observability sidecar in this stack once burned two CPU cores for five days while every liveness check stayed green. That incident created the fourth axis of the monitoring pattern.
Budgets are architecture
Cost control designed in from the start — per-key, enforced at the gateway — is what makes saying "yes" to new clients and agents cheap.