How is OmnisRouter different from a black-box router like Weave Router?

OmnisRouter's routing is fully open and reproducible: the intent-cluster model and policy table ship in the repo, are rebuildable from public data, and every decision is stamped with the model version that produced it. Each response carries a routing receipt — chosen model, confidence, alternatives, and estimated savings — so 'we save you money' is inspectable per request, not an unverifiable claim. Routing itself is never paywalled; it is Apache-2.0 open core.

Does it preserve streaming, tools, vision and prompt caching?

Yes. Streaming, tool/function calling, vision inputs, prompt-caching directives and extended thinking are preserved across formats and providers. Where a capability cannot be faithfully carried to the chosen provider, OmnisRouter refuses with an explicit error rather than silently dropping it.

Open-core LLM routing proxy · v1

Route to the cheapest capable model. Prove why.

OmnisRouter is a drop-in proxy for the Anthropic, OpenAI and Gemini formats. Swap one base URL and every request goes to the cheapest model that can actually handle it — returned in your client's own format, with a receipt that shows exactly what was chosen, how confident, and what it saved. The routing model is open and reproducible; nothing about it is a black box.

Anthropic · OpenAI · Gemini ≤50ms routing overhead BYOK · self-host Apache-2.0 open core

wire formats in, any candidate model out — cross-provider

100%

of responses carry a routing receipt

process, embedded SQLite — no external dependency

paywalled routing — the model is open + reproducible

Cheapest capable, not cheapest

The same client, routed by intent

Each prompt is embedded in-process, matched to an intent cluster, and sent to the cheapest model whose measured quality clears the bar for that cluster — escalating only when the router isn't confident. Easy work gets cheap models; hard work gets strong ones. This is a live trace from the shipped model.

Request	Intent	Routed to	Tier	Decision

The math row is the tell: the benchmark shows the tiny gpt-5-nano is essentially as accurate as the frontier model on grade-school math — so OmnisRouter sends it there and pockets the ~25× price difference. That's a measured call, not a guess.

The wedge

A receipt on every request

Black-box routers ask you to trust an unpublished savings number. OmnisRouter attaches the decision to every response as headers, exposes a cost-free POST /v1/route that returns the full decision without calling any provider, and writes a content-free decision log you can export. Every figure is explainable and every decision cites the exact, downloadable routing model that made it.

# ask "what would you do?" — no upstream call, no cost
$ curl -s localhost:8080/v1/route -H "authorization: bearer $TOKEN" \
       -d '{"messages":[{"role":"user","content":"Summarize this thread."}]}'
{
  "policy_version": "v3-omnisbench-2026-08-20",
  "cluster_id": 4,        "confidence": 0.26,
  "decision": "ROUTED",   "reason": "cheapest_capable",
  "chosen": { "provider": "gemini", "model_id": "gemini-2.5-flash" },
  "est_cost_usd": 0.0004,  "est_cost_delta_vs_big_usd": -0.0121,
  "alternatives": [ ... ranked, with predicted quality + $ delta ... ]
}
# and on every real response:
X-Omnis-Model: gemini/gemini-2.5-flash   X-Omnis-Decision: Routed
X-Omnis-Confidence: 0.26                  X-Omnis-Policy: v3-omnisbench-2026-08-20
X-Omnis-Cost-Delta-Vs-Big: -0.0121

Under the hood

Open routing, in one process

No network hop to decide where to go — the whole routing decision happens in-process in well under the 50ms budget.

In-process embedding

A pinned open ONNX model (bge-small-en-v1.5) embeds each prompt on the request path — no external routing service, no extra hop.

Intent cluster → policy

The embedding maps to the nearest intent cluster; the cluster's published policy table picks the cheapest candidate that clears the quality bar, escalating when confidence is low.

Faithful translation

Streaming, tools, vision, prompt caching and extended thinking are preserved across formats and providers — or refused explicitly, never silently dropped.

Reproducible model

The centroids + policy table are built by a documented, deterministic job from public data. Same inputs → byte-identical model. policy_version is stamped into every decision.

Measured by OmnisBench

The coding + math policy is driven by real per-model quality from the companion OmnisBench benchmark. Every release must pass its gate.

BYOK, encrypted

Your provider keys are AES-256-GCM encrypted at rest; prompts leave your infra only to the chosen upstream. Nothing sensitive is logged.

Drop in

Change one base URL

Self-host in one process, add a provider key, and point your existing client at it — Claude Code, Codex, Cursor, or your own app. No client code changes.

# run it (single-file binary or docker)
$ docker compose -f deploy/docker-compose.yml up -d --build

# add a BYOK key, then wire a client with the installer
$ npx omnisrouter-cli --url http://localhost:8080 --token $TOKEN --client cursor --write

# or just swap the base URL by hand — your client's format is preserved
OPENAI_BASE_URL   = http://localhost:8080/v1
ANTHROPIC_BASE_URL= http://localhost:8080
# POST /v1/messages · /v1/chat/completions · /v1beta/models/{model}:generateContent

Questions

Frequently asked

Straight answers to what people actually ask about LLM routing proxies.

What is an LLM routing proxy?

It sits between your app and the model providers and sends each request to the cheapest model capable of handling it, instead of always using one expensive frontier model. OmnisRouter is drop-in — change only the base URL — speaks the Anthropic, OpenAI and Gemini formats, and returns the response in the format your client sent.

How is it different from a black-box router?

The routing model ships in the repo, rebuilds from public data, and every decision is stamped with the model version that made it. Each response carries a receipt — chosen model, confidence, alternatives, estimated savings — so the savings claim is inspectable per request. Routing is never paywalled; it's Apache-2.0 open core.

Does it preserve streaming, tools, vision and caching?

Yes — across formats and providers. Where a capability can't be faithfully carried to the chosen provider (e.g. a thinking-signature can't move to a different model), it refuses with an explicit error rather than silently dropping it.

Is it self-hosted and BYOK?

One self-hosted process with embedded SQLite (Postgres optional). You bring your own provider keys; they're encrypted at rest and prompt content leaves your infra only to the chosen upstream. No key or prompt content is written to logs, receipts or exports.

How does the routing stay honest?

The model is built by a documented, deterministic offline job (same inputs → byte-identical model), its coding + math policy is driven by real OmnisBench measurements, and each tagged release must pass an OmnisBench gate and publishes its results.

Read it honestly

Where v1 stands

Coding + math are measured

Those clusters are driven by a real OmnisBench run. The other domains use sensible estimates today — broader benchmark coverage is the next step, and the pipeline is already wired.

Self-host, open core

v1 is the self-hosted, Apache-2.0 router. Hosted team dashboards and SSO are a later, paid layer — the routing itself stays open, always.

Refuse, don't drop

When a capability can't cross to the chosen provider, the router returns an explicit error instead of silently degrading — surprising the first time, correct every time.

Pinned, dated prices

Cost figures in receipts come from a committed, dated pricing snapshot — reproducible even as list prices move — shared with OmnisBench's cost math.