⌘ AnyInt Intelligent Routing
Intelligent routing: make every model call earn its cost
Plain routing sends every request to one fixed model. Intelligent routing first understands the request, then picks the best-fit model and endpoint under quality, cost, latency, privacy, and safety constraints.
“Explain this invoice line item and tell me whether a human agent needs to review it.”
complexity = simple AND cache_similarity > 0.9Check the semantic cache first; on a miss, answer from the lightweight pool and skip flagship models entirely.
Routing Arena
Same goal, lower cost with intelligent routing
The numbers below are local demo data used to illustrate the product advantage: without lowering the quality threshold, simple requests go to cheaper models while high-risk or hard tasks stay on dedicated lanes.
Billing support question
Versus routing everything to a fixed strong model, it keeps the quality threshold while cutting unnecessary large-model calls.
Rule matched: complexity = simple AND cache_similarity > 0.9
Signal panel
- Domain
- Support / Billing
- Complexity
- Simple
- Context size
- ~2K tokens
- Sensitive spans
- None detected
- Cache similarity
- 0.91
Normal Route vs Smart Route
Fixed routing answers “can it run”; smart routing answers “is it worth it”
How plain routing runs
Every request goes to the same model or a set of static weights, so even simple tasks burn an expensive model.
Support, code, privacy, and long-document requests are all treated the same, with no differentiation.
All traffic goes through one lane — no signal analysis and no policy branching.
Quality is acceptable, but even the simplest tasks burn the most expensive compute.
How intelligent routing runs
Keep one complete routing chain: pick a task type on the left, match its rule and policy in the middle, and land on different model pools and endpoints on the right.
Classify task type, complexity, context, privacy, and cache similarity.
Merge signals into a decision vector, then hit concrete rules by priority.
complexity = simple AND cache_similarity > 0.9Run semantic cache, cost gates, prompt injection, verification, and redaction.
Select the candidate pool by quality, cost, latency, and deployment boundaries.
Signals in, policy out
Routing is a configurable orchestration layer, not random dispatch. Multi-dimensional signals profile every request, rules compose them into policies, and each policy binds the plugin chain that runs before, during, and after the model call.
Cost policies
When complexity and cache signals allow, check the semantic cache first, then climb from lightweight pools only as far as quality demands.
Quality policies
When a fact-check or schema signal fires, add verification and repair stages instead of blindly paying for a bigger model.
Compliance policies
When sensitive spans are detected, restrict the candidate pool to attested deployments, redact before logging, and write an audit record.