Small models, big decisions: choosing the right AI for the job

There is a natural tendency, when a new capability arrives, to reach for the most powerful version of it. If a large model can do almost anything, why would you use a smaller one? The answer is that capability is only one dimension of a good system.

In practice, the most effective AI architectures are rarely built on a single model. They are built on a set of deliberate choices about where intelligence is needed, how much, and at what cost.

Four questions before choosing a model

What does the task actually require? Classification, extraction, summarisation, and open-ended reasoning are different problems. Some need deep reasoning; many need reliable pattern matching. Matching the model to the task often produces better results than simply using the biggest available option.

How often will it run? A task that runs once a day has a very different cost profile from one that runs on every customer interaction. Volume changes the economics, and the economics should shape the design.

How bad is a mistake? Some errors are annoying; others are expensive, unsafe, or irreversible. The tolerance for error determines how much human review, guardrailing, and verification the system needs — and whether a simpler, more predictable approach is better.

How fast does it need to be? Latency is part of the user experience. A slower but more capable model may be the right choice for a background analysis and the wrong choice for an interactive tool.

The case for smaller models

Smaller models have real advantages. They are often cheaper, faster, and easier to run in constrained environments. They can be fine-tuned for a narrow domain. They may be easier to audit and to keep within a predictable operating envelope.

For many business tasks — routing a request, extracting fields from a document, classifying a message, checking a policy — a well-designed smaller model with good prompting and good evaluation can outperform a larger general model that has not been adapted to the context.

The larger model is not obsolete. It is simply not the default. A common and effective pattern is to use a smaller model for the bulk of routine work and escalate the difficult or ambiguous cases to a more capable one. That keeps cost and latency under control while preserving quality where it matters.

Architecture is a series of trade-offs

The most useful mental model is not “which model is best?” but “where in this workflow should intelligence sit, and what kind?”

That question leads to a set of trade-offs:

• Accuracy versus cost. More capability usually costs more.
• Autonomy versus control. More autonomy is efficient until it is not.
• Generality versus fit. General models are flexible; specialised ones are predictable.
• Speed versus depth. Fast responses help flow; deeper processing helps quality.

There is no universal answer. There is only the answer for this process, with these constraints, for these users.

The goal is not to use the most advanced model. It is to build the simplest system that reliably does the job.

Keep room to change

Model choice is not a permanent decision. The landscape is moving quickly, and what is expensive or slow today may be cheap and fast tomorrow. Design the system so the model is a replaceable component rather than the foundation everything else depends on.

That means clean interfaces, strong evaluation, and a clear understanding of what the model is responsible for. It also means resisting the temptation to over-engineer around the quirks of a single provider.

Choosing the right AI for the job is less about benchmarks and more about judgement: understanding the work, the constraints, and the cost of being wrong. That judgement is becoming one of the most valuable skills in technology — and it has very little to do with which model is currently at the top of the leaderboard.