atobotz
Book a Call
Back to blog
2026-06-02

Model Routing: The 10x AI Cost Savings Nobody Talks About

95% of enterprise AI work runs on models that cost 87x more than necessary. Not because those models are needed. Because nobody built the plumbing to use cheaper ones.

The Problem

Here's how most companies deploy AI today: pick the most powerful model available, send everything to it, hope the bill doesn't hurt too much.

It does hurt. We're in the middle of what Derek Thompson calls "The Great AI Cost Panic of 2026." One Anthropic enterprise client hit $500M/month in API spend. Uber burned through its 2026 AI budget by April. Microsoft cancelled its own Claude Code licenses. GitHub Copilot just switched to token billing — some developers saw costs spike 60x.

The dirty secret? Most of those API calls are doing trivial work. Text classification. Data formatting. Simple extraction. Summarization. Tasks that a model costing pennies handles just as well as one costing dollars.

It's like chartering a private jet for every trip to the corner store. Sometimes you need the jet. Most trips are under two miles.

Network routing diagram representing intelligent AI model selection
Network routing diagram representing intelligent AI model selection

The Solution

Model routing is exactly what it sounds like: a system that sends each AI task to the cheapest model that can handle it reliably.

The architecture is simple:

  1. Classify the task — Is it simple (extraction, classification, formatting), moderate (summarization, basic reasoning), or complex (multi-step reasoning, code generation, agent loops)?
  2. Route to the right tier — Simple → small open-weight model (LFM2.5-8B, Mellum2). Moderate → mid-tier commercial (Sonnet, GPT-4.5-mini). Complex → frontier only (Opus 4.8, GPT-5).
  3. Monitor and adjust — Track accuracy per task type. Promote models that overperform. Demote ones that underperform.

The key tools emerging right now:

  • AnythingLLM v1.13.0 — First consumer hybrid AI with a built-in Model Router. Routes between local and cloud models automatically.
  • DeepSeek V4 Pro — Made its 75% price cut permanent. 87x cheaper cache-reads than Western alternatives. The economics of "route to cheap" just got dramatically cheaper.
  • Liquid AI LFM2.5-8B — 253 tokens/second on a laptop with 128K context. For simple tasks, this runs locally at zero per-query cost.
  • Step-3.7-Flash — #1 on ClawEval for tool-use reliability at open-weight pricing. Apache 2.0 license. Run it yourself.

Benchmarks

The cost difference between routing and not routing is extreme:

  • DeepSeek V4 Pro vs. Western frontier: 7x cheaper inputs, 17x cheaper outputs, 87x cheaper cache-reads. Permanent pricing, not a promotion.
  • LFM2.5-8B on laptop: 253 tok/s with 128K context. Free after hardware. Four times faster than previous-gen local models.
  • Enterprise model routing case studies: Early adopters of AnythingLLM's Model Router report 8-12x cost reductions with negligible quality impact on routine tasks.
  • Step-3.7-Flash: 98%+ tool-use reliability at open-weight pricing. That's production-grade agent reliability without frontier-model costs.
  • Caveat: Routing adds latency (one extra classification step) and complexity (multiple models to manage). Start with two tiers, not five. Measure relentlessly.

AI cost comparison chart showing model tier pricing differences
AI cost comparison chart showing model tier pricing differences

Impact

Let's do the math for a hypothetical mid-size enterprise:

  • Current spend: $2M/month on a single frontier model for everything
  • With routing (estimated): 60% of tasks → small models (10x cheaper), 25% → mid-tier (3x cheaper), 15% → frontier (same cost)
  • New monthly spend: ~$200-400K/month
  • Annual savings: $19-22 million

That's a 10x improvement without reducing capability. You still use frontier models — just for the 15% of tasks that actually need them.

The strategic impact is bigger than the financials. Companies with cost-efficient AI can afford to deploy AI more broadly. More workflows, more experiments, more agents — all within the same budget. Cost efficiency isn't about doing less with AI. It's about doing more.

Why This Matters Now

Three things changed this week that make model routing urgent:

  1. DeepSeek made its pricing permanent. This isn't a loss leader anymore. Open-weight models at 87x cheaper cache-reads are the new baseline.
  2. GitHub Copilot switched to token billing. The era of "just use it, don't worry about cost" is officially over. Consumption pricing means every token matters.
  3. Frontier model costs are going up, not down. Anthropic's $965B valuation means investors expect premium pricing to hold. The cheapest GPT-5 call you'll ever make is the one you don't make.

The infrastructure for model routing exists today. The economics are undeniable. The only question is whether your team builds the plumbing this quarter or waits until the next budget review when someone asks why the AI bill doubled again.

My take: If you're not implementing model routing by Q3 2026, you're volunteering to overpay. This isn't speculative — the tools exist, the models exist, the savings are proven. The companies that figure this out first will have a structural cost advantage that compounds every month.