How to Choose the Right LLM for Your Business Tasks (Without Overpaying)
A practical framework for picking the cheapest LLM that still gets the job done, including when open-weight models beat frontier ones.
How do you actually pick the right LLM for a business task?
Start with the most capable model you can access, confirm it can do the task well, then deliberately roll back to cheaper and smaller models, one generation at a time, until quality breaks. The model one step before it breaks is the one you should run in production. This sounds obvious, but most teams skip it. They pick a frontier model once, out of habit or brand trust, and never revisit the decision even as cheaper models catch up.
TL;DR
- Most business tasks don’t need frontier intelligence. Writing internal emails, building simple landing pages, or handling routine customer service queries works fine on mid-tier models, open weight or not.
- Price gaps between frontier and open models are large. Flagship Western models can run from a few dollars to tens of dollars per million input tokens, while competitive open-weight alternatives from Chinese labs have been priced as low as roughly $2 per million input tokens, with earlier versions under $1.
- Data control is a real driver, not just cost. Routing sensitive workflows through a third-party API means that provider sees every prompt. Self-hosting open weights keeps intellectual property inside your own infrastructure.
- Open-weight adoption has become a serious ecosystem, not a niche. Base models like Qwen have been downloaded billions of times and spawned well over 100,000 derivative fine-tunes on Hugging Face.
- The decision process should be a ladder, not a single choice. Test the smartest available model, confirm it clears your quality bar, then step down a generation and retest, repeating until performance drops below what the task needs.
- Build swappable infrastructure before you need it. An internal abstraction layer that can redirect requests to any model provider, similar to how OpenRouter works, is what makes the rollback process fast and cheap to execute.
- Self-hosting only pays off at scale. Fine-tuning and running your own GPU infrastructure requires real budget and technical capacity, so it tends to make sense for mid-size and large companies more than solo builders.
- 1Secure
- 2Regulation-compliant
- 3Auditable
- 4Scalable
Why are companies rolling back from frontier models at all?
Frontier models are priced for the hardest problems: novel reasoning, frontier coding, research-grade analysis. Most production workloads inside a company are not that. They’re repetitive, well-scoped, and already have thousands of historical examples sitting in a database. Customer support replies, internal documentation, lightweight code generation, and landing page copy don’t require a model that can solve unsolved math problems. They require a model that’s fast, consistent, and cheap enough to run at high volume.
Once a team accepts that distinction, the economics shift. Paying frontier-model prices for high-volume, low-difficulty tasks is paying for intelligence you never use. That gap is exactly where cheaper models, whether a previous-generation Western model or a competitively priced open-weight model, start to make sense.
What’s driving the shift toward open-weight and Chinese models specifically?
Three forces show up repeatedly: price, performance that’s “good enough,” and control over data.
On price, some open-weight models have been offered through their own hosted APIs at a fraction of flagship Western pricing per million input tokens, with earlier versions priced even lower. That’s a meaningful cost difference when you’re running millions of tokens a day through customer service or coding agents.
On performance, benchmark gaps between frontier closed models and the best open-weight models have narrowed enough that for many applied tasks, the difference is no longer noticeable to the end user. A model doesn’t need to top every leaderboard to generate a solid internal tool’s worth of code or draft a reasonable support reply.
On control, this is the less talked about piece but arguably the more important one for larger companies. Every prompt sent to a third-party API provider is visible to that provider. If your workflows embody proprietary processes, internal data, or hard-won intellectual property, sending that information to an external frontier lab means that lab now has it too. Open-weight models can be downloaded and run entirely on infrastructure you control, meaning sensitive data never has to leave the company, regardless of where the model’s original training happened.
Is self-hosting an open-weight model actually worth it?
It depends almost entirely on scale and sensitivity of data, not on raw model preference.
Self-hosting means acquiring or renting GPU capacity, setting up inference infrastructure, and optimizing the model for speed, then potentially fine-tuning it on your own task-specific data. That’s a real investment in both money and technical capability. It rarely makes sense for a small team running occasional queries, where paying per-token API pricing to a hosted provider is simpler and cheaper overall.
It starts to make sense once volume is high enough that infrastructure costs are smaller than the ongoing API bill, or once data sensitivity makes third-party hosting a non-starter regardless of cost. Large companies with existing engineering teams and compliance requirements are the ones most likely to find the tradeoff worthwhile, which is consistent with the kind of companies that have reportedly made this move: large-scale consumer platforms, automation tooling vendors, and marketplaces with heavy ad or support workloads.
Fine-tuning adds another layer of payoff. A base open-weight model might test slightly behind a top closed model on general benchmarks, but fine-tuned on a company’s own historical support tickets, code style, or workflow patterns, it can outperform a generic frontier model on that specific task. This is the real advantage open weights offer that a closed API can’t: you get to shape the model around your own data, not just prompt it.
How should a team actually run the model-selection process?
A practical version of the ladder approach looks like this:
First, build (or buy into) an abstraction layer that lets any request get routed to any model provider through a consistent interface. This is the same idea OpenRouter provides at the market level: one API format, swappable backend. Without this, testing different models for the same task is slow and annoying, and teams default to whatever they set up first.
Second, start every new task evaluation at the top. Send it to the most capable model currently available, regardless of cost, and confirm it can do the task well. This sets your ceiling for quality.
Third, step back one generation (or one tier down in cost and size) and run the same test. If quality holds, step back again. Keep going until output quality drops below what the business actually requires. The model one step before that drop is your production candidate.
Fourth, once you’ve found the cheapest model that clears the bar, check where it sits relative to the best available open-weight alternatives. If an open model matches or beats it, that’s the signal to consider self-hosting, assuming your company has the infrastructure or budget to support it.
This isn’t a one-time exercise. New model generations ship frequently, and a task that needed a frontier model six months ago may now run fine on something two tiers cheaper. Revisiting the ladder periodically is part of the cost-optimization work, not a one-off project.
Frequently Asked Questions
Do open-weight models require sending data to another country?
No. The model weights can be downloaded and run entirely on infrastructure a company controls, even if the model was originally developed and open-sourced by a lab based elsewhere. Using a hosted chat interface or API from that lab does send data externally, but running the downloaded weights locally keeps data in-house.
How do I know when a cheaper model is “good enough”?
Set a quality bar specific to the task before testing, such as acceptable error rate, tone consistency, or code correctness, and test candidate models against real examples from that task rather than general benchmarks. General leaderboard scores don’t always predict performance on a narrow, repetitive business workflow.
Is fine-tuning necessary to get good results from open-weight models?
Not always, but it usually improves results on narrow, high-volume tasks where a company has substantial historical data. Fine-tuning requires more upfront work than prompting a hosted API, so it tends to be worth it once a task is well-defined and run at scale.
What’s the risk of relying entirely on a single closed-model provider?
Pricing, rate limits, or access policies can change at any time, and the provider sees all data sent through its API. Building swappable infrastructure and keeping an open-weight fallback reduces dependency on any single company’s decisions.
Are Chinese open-weight models reliable for production use?
● Reading src/server.ts
● Editing 3 files
+ import { limit } from 'rate'
- app.use(cors())
✓ Tests passed · ready for PR
They’ve seen rapid adoption, with some base models drawing billions of downloads and spawning large ecosystems of fine-tuned variants. As with any model choice, teams should test them directly against their own tasks and data rather than relying on general reputation alone.
