Smart Routing (Coming Soon)
Smart Routing (Coming Soon) is the coming AnyInt product direction for task-aware model selection, cost control, and fallback policy.
Smart Routing is an upcoming AnyInt product direction for task-aware model selection, cost control, and fallback policy. This section explains the planning model, but it is not a published automatic-routing API contract.
Availability
Smart Routing is not generally available yet. Use this section to understand the product model and integration planning assumptions, and confirm production availability before depending on an automated routing API.
What you can do today
Until an automated Smart Routing API is generally available, production applications should make routing decisions explicitly:
| Need | Current public pattern |
|---|---|
| Confirm account-level model access | Call Models API and use returned data[].id values |
| Pick a route family | Choose OpenAI-compatible, Anthropic-compatible, Gemini-compatible, media, or music APIs based on request shape |
| Control reliability | Document primary and fallback model IDs in application configuration |
| Control cost | Separate high-value workloads from lower-risk batch or extraction workloads |
| Detect route failures | Handle 401, 403, 429, and temporary upstream errors as described in Errors and Limits |
Planning checklist
If you are designing an application that will later benefit from Smart Routing, keep these decisions outside hardcoded prompt logic:
- Store primary and fallback model IDs in configuration.
- Keep workload labels such as
chat,extract,classify,code,image-understanding, ormedia-generationnear the call site. - Record whether the workload optimizes for quality, latency, cost, or reliability.
- Make retries and fallback behavior observable in logs or metrics.
- Keep provider-native request bodies isolated so they can be changed without rewriting product logic.
What not to assume
| Do not assume | Use instead |
|---|---|
A hidden /smart-routing endpoint is available | Only call published routes in API Reference |
| AnyInt will automatically choose a model for every request today | Send explicit model IDs returned by the Models API |
| A fallback is safe without testing | Verify the fallback model's request shape, latency, and output quality |
| Cost optimization is only a provider choice | Also consider prompt size, streaming behavior, caching, retries, and task design |
Read next
Related live patterns
Until Smart Routing (Coming Soon) is generally available, production applications should still choose explicit model IDs, document fallback behavior, and use Models API to confirm account-level model access.