Proprietary or open source LLMs: the selection criteria for companies

The question arrives in every adoption journey, usually framed as a matter of allegiance: better the proprietary models of the large providers, consumed via API, or the open source models (more precisely open weight, since what is open are the weights and rarely everything else) run under one's own control? The shape of the question is the first thing to correct, because it presupposes a single answer for the company, while the serious answer is given per use case, and almost always in the plural.

From a vendor-agnostic position like ours, with no models to sell and no alliances to honour, the criteria we see actually deciding are six, and it pays to walk through them in order of weight.

Required capability and data sensitivity

The first criterion is the distance between what the use case requires and what the two families offer. On frontier tasks, complex reasoning and reliability across difficult domains, the leading proprietary models keep an edge, while on a growing share of corporate tasks, classification, extraction, summarisation on bounded domains, mid-sized open models reach indistinguishable results at a fraction of the cost per token. The practical consequence is that paying frontier capability for routine tasks is the most common waste we find in clients' bills.

The second criterion is the data the model will see. For ordinary information, the enterprise contracts of serious providers today offer robust contractual guarantees on non-training and data residency, and they should be read rather than presumed. There remain the cases where data cannot leave, by regulatory constraint, industrial secrecy or client mandate, and that is where the open model on one's own infrastructure stops being an ideological preference and becomes the only option on the table.

Total cost, with the hidden lines

The third criterion is TCO, and the honest comparison looks beyond the price per token. The proprietary route charges for usage and gives everything else away: updates, infrastructure, serving security, scalability. The open route zeroes the licence cost and opens the lines that enthusiastic business cases always miss, the GPUs or cloud for inference, the engineering to serve the model at acceptable latency, the monitoring, and above all the people able to do all this, who are the scarcest and most expensive item on the list. The rule of thumb that emerges: below a certain usage scale the API almost always wins on full cost, and the break-even point should be computed on your real traffic, never on the GPU supplier's slides.

The fourth criterion is control, in its concrete forms: the ability to pin a model version and decide its lifecycle (with APIs, the provider can deprecate what you built on), to specialise it in depth through fine-tuning on your own data, to inspect its behaviour for audit needs. Where these needs weigh, open weights weigh with them.

Compliance, skills and the portfolio logic

The fifth criterion is the regulatory profile. The AI Act assigns specific obligations to whoever provides general-purpose models, and consuming via API leaves those obligations with the provider, while whoever takes an open model and modifies it or puts it into service must understand which part of the responsibility chain it is assuming, an evaluation to run with legal counsel beforehand and with written evidence, never after. The sixth criterion, skills, is the simplest to state: the open route is a build choice, and build choices are made with the people you have or are certain you can hire and retain.

Lined up, the six criteria almost always lead to the same target architecture, the portfolio: leading proprietary models where frontier capability pays for itself, open or smaller proprietary models where the task is routine and volumes bite, and a technical abstraction layer that allows routing and swapping without rewriting the applications. The portfolio, besides optimising costs, is the best insurance against lock-in, a topic deserving its own article and left here as a contractual reminder: the exit clause is negotiated at the entrance.

Model choice, seen from this distance, is a business decision disguised as a technical one, and as such it enters the AI Strategy work we do with clients, inside explicit criteria rather than departmental preferences: calendly.com/fabiolalli/zerofive, or hello@zerofive.ai. The quick test to see whether your current choice was a decision or a habit fits in one question: could you say, use case by use case, why it runs on the model it runs on?