Run every model as one system.
Route each request. Enforce every policy. Inspect every decision. One OpenAI-compatible gateway in front of every model.
- OpenAI compatible
- Bring your own keys
- No router to host
Prompt“Summarize this update in one sentence: "The migratio…”
Your request
Inferbase routes it
Model pool
Picked
Deepseek V4 Flash (fp4)
$0.09 in, $0.18 out per 1M tokens, as billed
- Task
- summarisation
- Qualified
- 20 models
- Objective
- balanced
Four capabilities, one gateway.
One OpenAI-compatible endpoint, with failover, rate limits and budgets behind it.
Catalog
Know the field.
An AI model catalog of curated creators and verified variants, with live price and latency per route.
Read the docs →Routing
Route with intent.
LLM routing: each prompt classified in about a tenth of a second and decided through a recorded pipeline.
Read the docs →Security
Policies on every key.
Which models may serve, which doors the guardrails check, which tools an agent may call, per key.
Read the docs →Observability
Keep the evidence.
The decision, verdicts, tokens, cost and latency of every request, with agent runs and an audit log.
Read the docs →How Inferbase works.
The gateway classifies the request, applies your policies, picks the model on your objective, and serves it. Each cycle below is one request.
Inferbase Gateway
summary → Mistral Small 3.2
- Catalog
- Routing
- Security
- Observability
More than another AI gateway.
Gateways proxy and routers recommend. Inferbase decides, applies your policies, serves and keeps the record.
Optimize
Lower cost by default. Each request goes to the cheapest model that clears its quality bar.
Observe
Understand every request. The decision, its candidates and the receipt are kept by request id.
Adapt
New models, new prices and new providers land in the catalog. Your code does not change.
Scale
One endpoint from prototype to production, with failover, rate limits and budgets behind it.
Bring your own keys
Your credentials, encrypted, serving under your accounts.
Zero-retention mode
Retention off per user or organization.
Per-request receipts
Model, tokens, cache reads and cost, itemized.
Session affinity
Multi-turn work stays on one model.
Guardrails at three doors
Prompts, responses and tool results, checked per policy.
Tool governance
What an agent may declare and call, per key.
Models change. Your API doesn't.
Change one line and keep your code: set the base URL, set the model to auto. Streaming, tools and structured output pass through.
- Works with the OpenAI, Anthropic and LangChain SDKs
- Pin a model any time
- Bring your own keys, or serve on ours
client = OpenAI(
base_url="https://api.inferbase.ai/v1",
api_key=os.environ["INFERBASE_API_KEY"],
)
r = client.chat.completions.create(
model="auto", # the router picks
messages=[{"role": "user", "content": prompt}],
)
print(r.id) # fetch the routing decision by this id{
"object": "routing",
"task": "summarisation",
"complexity": "simple",
"estimated": false,
"decisiveness": 0.81,
"forced_single": false,
"model": "deepseek-v4-flash",
"routing_time_ms": 104
}Audit every decision.
Before the first token: what was seen, which models qualified, which won and how clearly.
- Decisiveness says how clear-cut the pick was
- The full record is yours by request id
- Objective, task, complexity, verdicts and cost, per request
From the blog.
Benchmarks, cost analysis, and the thinking behind how we build.
LLM Sampling Parameters Explained: Temperature, Top-p, and Friends
What temperature, top_p, top_k, penalties, and seeds actually do to LLM output, which defaults to trust, and which knobs deserve adjustment for each task.
Fine-Tuning vs RAG vs Prompt Engineering: Which One Fixes Your Problem
The three ways to adapt an LLM solve different failure modes. A decision framework for choosing between prompting, retrieval, and fine-tuning, with costs.
What Is an LLM API? How Applications Talk to Language Models
How LLM APIs work: the request and response anatomy, tokens as the billing unit, streaming, API keys, and why one wire format became the standard.
How Inferbase compares.
Routers recommend, aggregators resell routing, frameworks make you host it. Inferbase decides, serves and keeps the record, in one API.
| Inferbase | OpenRouter | LiteLLM | Portkey | NotDiamond | RouteLLM | |
|---|---|---|---|---|---|---|
| Picks the best model per request | First-party | Via NotDiamond add-on | Beta tiers you map by hand | No, rules you define | Yes | Strong vs weak only |
| Routes and serves in one API | Yes | Yes | Proxies via your providers | Proxies via your providers | No, you run it | No, self-hosted |
| Per-request decision audit | Yes | No | Logs and cost tracking | Deep logs and traces | Recommend-side | Build your own |
| Nothing to self-host or calibrate | Yes | Yes | No | Hosted, rules are yours | Yes | No |
| Model breadth | Curated catalog of open models | Hundreds of models | 100+ providers, your keys | 1,600+ models, your keys | Your chosen pool | Two models |
A unified API where routing is a NotDiamond-powered add-on.
Read the comparisonRecommends a model per prompt; you run the inference.
Read the comparisonA self-hosted, strong-vs-weak router you operate.
Read the comparisonPut intelligence in the middle.
One gateway in front of every model, with your policies applied and every decision on record. Start with $5 of credit and 5,000 routing decisions a month, no card required.