Skip to main content
MindStudio
Pricing
BlogAbout
My Workspace
AI model nerfedClaude got worseGPT-5 dumber

Why Do AI Models Get Worse After Launch? The Dials Explained

Anthropic and OpenAI admit models feel worse post-launch. Here's how effort levels, routing, and system prompts explain it, not secret weight swaps.

Edited by Luis Chavez-Mattos, Director of Product RSS
Why Do AI Models Get Worse After Launch? The Dials Explained

Why do AI models seem to get worse after launch?

AI models rarely get worse because the underlying weights change. Instead, the companies running them adjust the settings around the model: how much “effort” it’s allowed to spend thinking, what hidden instructions it follows, which model actually answers a given request, and bugs in the app wrapped around it. Anthropic and OpenAI have both acknowledged versions of this in public postmortems. The model’s brain usually stays the same. The dials around it get turned down, often without warning.

TL;DR

  • Anthropic has published postmortems admitting Claude Code felt less intelligent for weeks, and in one case wrote that their own evaluations didn’t catch the degradation users were reporting.
  • OpenAI has had similar issues, including Sam Altman acknowledging GPT-5 seemed “way dumber” the day after launch due to a backend problem, and later shipping a usage reset to compensate users.
  • Effort level is one of the biggest levers: Anthropic once quietly dropped Claude Code’s default reasoning effort from high to medium to speed things up, then admitted it was the wrong tradeoff.
  • Benchmark scores often come from max effort settings, while most users run on medium by default, which explains why a model’s advertised score and real-world performance can diverge sharply.
  • Routing can silently swap which model answers you: one tracked case showed a model’s debugging score collapse because safety filters rerouted most requests to a different, separate model without telling users.
  • System prompts and app bugs cause real damage too, including one case where Anthropic added a hidden instruction limiting Claude’s text output, and later found it made responses worse.
  • No hard proof exists that labs quietly ship a smaller, quantized version of a flagship model after launch, though independent tracking projects exist specifically to catch this if it happens.
The AI software platform for companies that need regulatory compliance and security.
Patient record #20417Viewing as: Billing clerk
Name
Maria Alvarez
Date of birth
SSN
Diagnosis
Balance due
$412.80
✓ HIPAA✓ Role-based access✓ Audit log

What are AI model weights, and do they actually change after release?

A model is, at its core, a massive file of numbers called weights, tuned during training so the model produces useful outputs. Once training ends, those weights are fixed for that model version. When a new model name ships (say, an update to Claude Opus or GPT-5), the weights for that specific release generally don’t change afterward.

That’s part of why the “nerfing” phenomenon is confusing. Independent trackers built specifically to catch silent weight changes, running identical prompts against the same model every day, mostly haven’t found evidence of the core model being swapped out underneath a stable version number. One such tracker ran the same set of questions daily against a Claude Opus model for its first 50 days and detected no drop. That doesn’t prove nothing ever changes at the weight level. It means nobody has caught hard proof of it yet, and the people doing the checking admit smaller changes could slip past their tests.

Is quantization the hidden culprit?

Quantization is a real technique: rounding off a model’s numbers to make it smaller and cheaper to run, similar to streaming a movie in 720p instead of 4K. The content is the same, but detail gets lost. It’s a plausible way a company could quietly cut compute costs on a popular model.

The catch is that nobody has published proof this is happening to major flagship models post-launch. No leak, no admission, no confirmed detection from the daily-tracking projects built to catch exactly this. It remains a widely suspected explanation among users, but it stays unconfirmed. If it does happen, the degradation someone eventually proved would look exactly like what a lot of people already describe: same model name, noticeably fuzzier output.

What is “effort,” and why does it matter more than people realize?

Effort refers to how much computation a model is allowed to spend reasoning before producing an answer. It’s a dial, and labs adjust it for cost and speed reasons separate from the model’s underlying capability.

Anthropic has acknowledged turning down Claude Code’s default effort level from high to medium specifically to make responses faster, without disclosing the change at the time. They later admitted this was the wrong tradeoff for users. This matters because of how companies report benchmark scores. Published scores for a model often reflect runs at maximum effort, while the version most people interact with defaults to a lower setting to save on speed and cost. That gap between the headline benchmark number and the real-world default is one of the most concrete, acknowledged reasons a model can feel weaker than its marketing suggested, without any change to the weights at all.

How do system prompts and hidden instructions quietly degrade output?

Every AI application wraps its underlying model in a hidden system prompt, instructions the user never sees that shape tone, format, and behavior. These live somewhere in the app’s “harness,” on top of whatever custom instructions a user adds themselves.

› Build a loan-approval app for 2,000 bankers.
already included:✓ SOC 2✓ role-based access✓ audit log✓ self-hosted
The enterprise platform for building software with AI.

Anthropic has acknowledged adding an instruction telling Claude to keep its text between steps under a strict word limit, then found through their own testing that the constraint made output worse. OpenAI has dealt with a related issue, where an assistant product was triggering old, irrelevant skills too frequently, getting in the model’s way rather than helping it. These aren’t changes to the model’s intelligence. They’re changes to the scaffolding around it, and that scaffolding can be tuned (and mistuned) constantly, often without any announcement.

Can a model secretly reroute your request to a different, weaker model?

Yes, and this is one of the more concrete, documented mechanisms behind the “nerf” feeling. Routing determines which model actually generates the response to a given prompt, and it can happen invisibly.

One tracked example involved a model’s debugging benchmark score collapsing dramatically after a period of downtime, which users initially blamed on the model itself being gutted. Investigation found that most of the test tasks never actually reached that model. A safety filter was intercepting the requests and routing them to a different, separate model instead, without informing users that the swap had occurred. Independent testing of this behavior found that a meaningful share of a broad question set was being silently rerouted rather than answered by the model the user thought they were using. If you think you’re talking to one model but a safety layer has quietly handed your question to another, the output can feel like a downgrade because, in a sense, it is: you didn’t get the model you asked for.

Are app bugs part of the problem too?

Yes, and these are the most mundane but very real cases. Anthropic has shipped a bug that wiped out a model’s working memory of its own reasoning between conversational turns, causing it to repeatedly lose track of what it was doing. OpenAI has found misconfigured servers degrading a product for a portion of its traffic. Neither involves the model’s weights or even its effort or routing settings. It’s plain software bugs in the surrounding application, and they produce the same user complaint: “this got worse.”

Is there a financial incentive behind post-launch slowdowns?

There’s a plausible compute-cost incentive, though it remains an inference rather than a confirmed admission. Launch periods bring enormous spikes in demand. When GPT-5 launched, Sam Altman noted that API traffic roughly doubled within 24 hours. Every token a model processes costs real compute, and effort level directly affects how many tokens get burned per response. Running every user at maximum effort during a traffic spike is expensive. Turning the default down, even slightly, reduces cost at scale.

That creates a natural incentive structure: ship the most impressive version possible at launch to drive hype and adoption, then quietly dial things back once the sign-ups and demos have already happened. No company has stated this as a strategy. But the mechanisms that would make it possible (effort defaults, routing, system prompt tuning) are the same ones already documented in postmortems as things labs adjust after launch, often without telling users in advance.

Is any of this illegal or regulated?

Automation tools are the old way.Enterprises build with AI on Remy.
Zapier
n8n
Make
Airtable
RemyEnterprise-ready
  • Self-hosted
  • SOC 2
  • HIPAA
  • Audit log
  • Role-based access
Built into every app you make

Not meaningfully, at least not yet. No broad law requires AI companies to disclose when they adjust effort defaults, system prompts, or routing behavior on a model you’re paying for. Connecticut has passed a law requiring companies to disclose usage limits before a signup or renewal, but it doesn’t address whether model behavior or quality has changed. For now, the only real check on this behavior is public scrutiny: independent benchmark trackers, users comparing outputs over time, and public pressure after someone notices a drop.

Frequently Asked Questions

Do AI companies actually admit to nerfing their models?

Not in those words, but both Anthropic and OpenAI have published or stated acknowledgments of real degradation. Anthropic has released postmortems describing weeks-long drops in Claude Code’s perceived intelligence. OpenAI’s Sam Altman publicly acknowledged GPT-5 seemed noticeably worse the day after its launch due to a backend issue.

Is it proven that labs swap in a smaller, quantized model after launch?

No. It’s a widely suspected explanation among users and a technically plausible way to cut costs, but no leak, confirmed benchmark detection, or company admission has proven it happens to major models after release.

Why do benchmark scores look better than what I experience day to day?

Published benchmarks are frequently run using a model’s maximum effort setting, while the default experience for most users runs at a lower effort level to save on speed and compute cost. That gap alone can account for a noticeable difference without any change to the model itself.

Can I control these settings myself?

Effort level is often adjustable in apps and APIs that expose it, so increasing it can close some of the gap between benchmark performance and your actual results, at the cost of slower responses and higher usage.

Is any of this illegal?

Not currently in any comprehensive way. Only a narrow law in Connecticut addresses disclosure, and it covers usage limits rather than changes to model quality or behavior.

Editorial standards

Presented by MindStudio

No spam. Unsubscribe anytime.