ai industry15 min read

Four frontier models in three days, and nobody will tell you which one you needed

Between September 1 and 3, 2026, Anthropic, Meta, Google and OpenAI all shipped frontier models, and CNBC gave the resulting exhaustion a name: model fatigue. The number of releases is the boring part. The interesting part is that every release creates an evaluation somebody downstream has to run, and the companies shipping them have no reason to help you skip it. A look at what a model swap actually costs, why cheaper per token is not cheaper per task, and why fully-managed software built for one narrow job is the only arrangement where somebody else can make the decision for you.

K
Ken Jo
#model-fatigue#llm-evaluation#ai-costs#managed-software#vertical-ai#model-routing#enterprise-ai

Four frontier model releases across three days: Anthropic on Tuesday September 1, Meta and Google on Wednesday September 2, OpenAI on Thursday September 3, 2026

Brand marks from Simple Icons (CC0), shown unmodified to identify the companies named. Release dates and the Altman quote from CNBC, September 6, 2026.

Between Tuesday and Thursday of the first week of September 2026, Anthropic shipped Claude Fable 5.1 and Claude Mythos 5.1, Meta shipped Muse Spark 1.3, Google shipped Gemini 3.8 Flash, and OpenAI shipped GPT-6 Astra. On that same Thursday, the Mohamed bin Zayed University of Artificial Intelligence in Abu Dhabi opened its K2 Horizon family to the world, and NVIDIA agreed to buy Hugging Face for $12,930,300,000.

Six announcements. Three days. CNBC ran the story on Sunday, September 6, under a phrase that had been circulating among practitioners for weeks: model fatigue.

The obvious reading is that the industry is moving faster than anyone can follow. That reading is true, and mostly uninteresting. Here is the part worth your attention: each release transfers a cost, and the transfer runs in exactly one direction. Every model a lab ships becomes an evaluation somebody downstream has to run — and nobody upstream has any reason at all to help you skip it.

TL;DR

  • Four frontier releases landed in three days (September 1–3, 2026); CNBC named the resulting exhaustion "model fatigue" on September 6.
  • The real cost is not attention, it is evaluation labour. One enterprise CEO told CNBC that when his team wants to assess 10 models for a task, it tests about 5.
  • Per-token price is a bad proxy for per-task cost — a stronger, pricier model can finish harder work for less by taking fewer steps.
  • Model vendors are structurally uninterested in telling you when a cheaper model would do. That gap is the entire reason managed, purpose-built software exists.

Model fatigue got a name on September 6

"I feel like model fatigue is a real thing," Zhen Lu, CEO of the AI infrastructure startup Runpod, told CNBC. "Don't get me wrong, I am extremely excited about all of the innovation that's happening, but I really do think that we are in an environment where there's just so much frothiness that you have to make noise."

Sam Altman offered a gentler framing to CNBC the same week: "we're all moving to faster cadences," he said, attributing some of the acceleration to everyone getting "back after summer vacation."

Both things can be true. The cadence is genuinely accelerating, and the acceleration is genuinely partly theatre. Ahmed Abbasi, a professor at Notre Dame's Mendoza School of Business with 25 years in AI, put the commercial logic plainly: the labs are "all playing the share-of-wallet game." Anthropic and OpenAI are pushing hardest as they head toward public markets, each already valued at close to $1 trillion by private investors.

The wallet in question is enormous. Gartner forecast in May 2026 that worldwide AI spending would reach $2.59 trillion this year, a 47% increase over 2025. Infrastructure takes the largest share, but well over $1 trillion of that lands on services, software, security, models and tools — which is to say, on decisions that somebody has to make.

Here's the point: a market that large, growing that fast, rewards being loud. It does not reward telling customers they could get by with less.

The cost isn't attention. It's the evaluation you can't finish

Ask a practitioner what model fatigue actually feels like and you don't get "I read too many launch posts." You get arithmetic.

Suresh Vasudevan, CEO of the enterprise AI startup Clockwork Systems, gave CNBC the most useful number in the whole story. If his company wants to evaluate 10 AI models for a particular task, he said, it may just pick five. "It's really challenging to go evaluate every one of the ones that are coming out right now."

Sit with that ratio for a second, because it is the load-bearing fact of this entire subject. A company whose business is enterprise AI, staffed by people who do this professionally, with compute budget allocated for exactly this purpose, evaluates half the candidates it believes are worth evaluating.

Call the other half the untested five. They are not rejected. Nobody found them wanting. They were simply never run, and so the question of whether one of them would have been faster, cheaper or more accurate on your specific workload stays permanently open. Every release week adds to that pile.

And the pile is expensive to work through, because a model swap is not a version bump. It is a re-qualification. Here is what actually has to happen before a new model reaches production:

What has to be re-checkedWhy a new model breaks it
Prompts and system instructionsTuned against the old model's quirks; phrasing that steered one model reliably can steer another somewhere else
Output format stabilityDownstream parsers depend on shape — JSON, headings, delimiters — and formatting habits shift between models
Accuracy on your real dataBenchmark deltas are measured on public sets, not on your meetings, your tickets, your codebase
Tool and API behaviourAgentic models differ in when they call, how often they retry, and what they do when a call fails
Latency and throughput under loadA model that is faster on a single request can be slower behind your rate limits
Total cost at your volumePer-token price times tokens consumed times attempts — see below
Safety and refusal behaviourBoundaries move between versions in both directions
Regression casesThe specific failures you already fixed, which nobody else's benchmark knows about

Eight categories, each needing real examples, a comparison run and someone's judgement. That is a project, not a config change. Multiply it by the number of frontier releases in any given month and the reason for the 10-to-5 ratio becomes obvious.

We wrote a whole piece on how to measure this on your own workload rather than trusting a leaderboard, and the honest summary is that doing it properly takes days, not hours.

Cheaper per token is not cheaper per task

The most counter-intuitive finding in this whole story came not from a lab but from an engineer trying to automate the choice.

Moe Khalil, a product engineer at the open-source AI gateway LiteLLM, published a post on September 4 titled "Model Fatigue is Real." He works on LiteLLM's auto router — software whose entire job is picking a model for you — and he disclosed that affiliation up front, which is more than most people writing about this do.

His team's starting assumption was the intuitive one. Take two models that can both solve a task; the one that costs half as much per token according to the model card should solve it for roughly half the price.

That is not what the tests showed. The stronger, more expensive model was solving the harder tasks at a fraction of the cheaper model's cost. Khalil's explanation: "Because it was smarter, it would take a smarter approach, and would get to the finished product way faster."

Cost per task equals price per token times tokens spent times attempts — which is why a model that is cheaper per token can be more expensive per finished task

Per-token price is the only term of the three that appears on a pricing page. The other two are properties of the model's competence, and they only show up when you run the work.

This reframes the pricing page entirely. The number a vendor advertises is one factor in a product of three: price per token, tokens spent, and attempts until the task actually succeeds. Vendors set the first. The model itself sets the second and third, and neither is knowable until you run your own work through it.

Which means the comparison you were told to make — scan the price column, pick the cheap one — is not merely incomplete. On real tasks it can point the wrong way.

Nobody upstream is incentivised to tell you

Khalil then says the quiet part, and it is the sharpest sentence written about this subject all year:

"For the model companies, this laziness is a feature — not a bug. They prefer that I use [the top-tier model] to summarize my email, and they have no incentive to tell me when a cheaper model would work just as well. Their job is to keep pushing the frontier. Figuring out when not to use the frontier is left to me."

Read that once more with the Gartner number in mind. A $2.59 trillion market, growing 47% a year, in which the suppliers benefit from every customer defaulting upward and bear no cost when customers overspend.

This is not a conspiracy and it does not require anyone to behave badly. It is just what the incentive gradient looks like. A frontier lab's job is to move the frontier. Telling you that last year's mid-tier model handles 80% of your workload is somebody else's job, and until recently it was nobody's job at all.

Abbasi's second observation belongs here too. He suggested it is "not a coincidence" that every major developer announced in the same week, and Noah Faro, technology chief at the AI finance startup Farsight, agreed — noting that rivals read each other's plans partly from cloud compute availability, since they are all competing for capacity from the same handful of vendors. Faro also made the deflationary point: unlike GPT-6 Astra, the Anthropic, Meta and Google launches that week were point releases, upgrades to existing models rather than new ones. In his assessment the last two releases that genuinely moved the needle were Anthropic's Fable 5 in June and Moonshot AI's Kimi K3 in July.

So: six announcements, one genuinely new frontier model, and a full evaluation cycle's worth of work generated for everyone downstream.

Two ways the industry is absorbing this, and they are not the same

If the labs won't make the decision, somebody has to. Two answers have emerged, and they are structurally different.

The horizontal answer is routing. Gateways like LiteLLM put every model behind one API key and try to select per request. Khalil describes the technical problem precisely: for any closed-ended task there exists a minimum viable model — the cheapest one that still succeeds — and the challenge is predicting which one that is before the task runs. His team's best current approach infers it from public benchmark data plus heuristics in the prompt; they have also tried LLM classification and hybrids. His own verdict: "All still imperfect, but a step up from nothing."

Routing is real engineering and it helps. But notice what it does and does not move. It absorbs the plumbing — one key, one interface, automatic access to new models. It does not absorb accountability. When the router picks wrong, the bad output is still yours, in your product, in front of your user. You still own the evaluation of whether the answer was good.

The vertical answer is fully-managed software built for one job. Here the vendor does not route between models on your behalf; it takes the entire task off your hands and delivers a finished result. You do not see a model name. You see a transcript, a summary, a translation, a filed expense report.

Three arrangements for absorbing the model decision: build it yourself, a gateway or router, or managed software built for one job — compared by who picks the model, who re-tests on every release, what you are accountable for, and what you see

The further right you go, the less of the decision remains yours — and the more you depend on the vendor having done the evaluation honestly.

The local-AI community arrived at a similar conclusion from the opposite direction. An analysis published this year described the small-model scene as a treadmill: download, configure, run a few prompts, half-judge it, move on to the next release. The recommendation was to stop — to standardise on one proven daily-driver setup and adopt a new model only after testing it long enough on real, representative tasks to confirm it is actually better for your specific workload. Same conclusion, reached by a hobbyist with two GPUs and by a CEO with a compute budget: pick once, properly, and stop re-litigating it every Thursday.

Why a narrow job is what makes the decision absorbable

Here's our take, and it is the reason this category exists at all.

The model decision can only be made by someone who knows the task. That sounds trivial. It is not, because it rules out almost everyone.

A general-purpose chat interface cannot make the decision for you, because on any given morning it might be asked to write a sonnet, debug a race condition, or summarise a lease. There is no minimum viable model for "whatever the user types next." The only safe default is the strongest model available — which is exactly the default that serves the vendor's interests, and exactly the default Khalil says nobody has an incentive to talk you out of.

A gateway can make a better guess, because it sees the prompt. But it sees the prompt with no knowledge of what a good answer looks like in your domain, and no consequence if it is wrong.

Purpose-built managed software sits in a different position entirely. When the job is fixed — transcribe a 60-minute meeting with four speakers switching between Korean and English, then produce action items — the task is closed-ended in exactly Khalil's sense. A minimum viable model exists for it. And critically, it is the same minimum viable model for every customer with that job, which means the evaluation has to be run once and its cost is amortised across everyone.

That is the whole economic argument for the category, and it is worth stating as a rule: the narrower the job, the more of the model decision somebody else can absorb on your behalf. A vendor serving one workload can afford to test all 10 candidates because it is testing them once, for thousands of customers, against a fixed definition of a good result. You testing five, for yourself, against a definition you have to invent, is the expensive way to get a worse answer.

The trade is real and worth naming honestly. You give up direct control of which model runs your work. In exchange you stop running the evaluation, you stop tracking release calendars, and the thing you judge is the output — which was always the only thing you actually cared about.

How to tell which arrangement you are buying

Before your next AI purchase, four questions in order:

  1. Is the job closed-ended? Can you state what a correct output looks like in one sentence? If yes, a managed vertical tool can own the decision. If the answer is "it depends what we ask it," you are buying a general model and the evaluation stays yours.
  2. Who re-tests when a new model ships? Ask the vendor directly. A managed vendor should be able to describe its evaluation set and its swap process. If the answer is that you will "get access to the latest models," you have bought plumbing, not judgement.
  3. What are you shown — a model or a result? If the product surfaces model names, context windows and token prices, it is handing the decision back to you. That is a legitimate product, but price it accordingly.
  4. What does the total cost look like at your volume? Price per token, times tokens actually consumed on your work, times attempts to get it right. Run your own tasks. The pricing page cannot answer this.

The bottom line: the frontier is a default, not a decision

Model fatigue is usually described as an attention problem — too much news, too fast. It isn't. It is a cost that the fastest-growing market in technology quietly moved onto its customers, where it sits unpriced and largely unmeasured.

The labs will keep shipping, because shipping is their job and the market pays for it. The evaluation burden will keep growing, because it grows mechanically with every release. And most teams will keep resolving it the only way a busy team can: by reaching for the newest, largest, most expensive model, which is the one option nobody has to defend in a meeting.

Every piece of software that takes a job off your hands completely — and never shows you a model name — is a bet that this is a bad way to run a business. The bet looks better every release week.


Where Telli.sh fits: this is the category we build in. Telli.sh records meetings, separates speakers, translates across 15 languages in real time, and produces summaries and action items — and it never asks you which model to use, because that is our job, not yours. Our browser extension does offer nine translation engines including a fully on-device option, for the people who genuinely want that control; the meeting product deliberately does not, because "transcribe this meeting well" is a closed-ended task and we would rather be judged on the transcript than on the model behind it.

Record your next meeting and let someone else pick the model

Sources


Back to Blog