Build, Buy, or Assemble: The Enterprise AI Platform Decision

The build-or-buy question is usually posed at the wrong level. An AI platform is not one thing to be built or bought — it is six or seven layers with very different rates of change, very different differentiation potential, and very different costs of being wrong. The useful decision is made per layer, and the answer is almost always a mixture.

Decompose before deciding

Two rows deserve emphasis because they are consistently misallocated. Evaluation is the layer teams most often expect a vendor to supply, and it is the one that cannot be bought — a general-purpose evaluation harness cannot know what a correct answer looks like in your business. Conversely, serving infrastructure is the layer teams most often build, and it is improving so quickly in open source that any in-house version is obsolete before it is finished.

The commoditisation test

There is a straightforward criterion for whether building a layer can possibly pay, and it disposes of most bad decisions without further analysis.

Applied to inference serving in the current environment, T_advantage is measured in months and T_build in quarters. The inequality fails before the payback term is even considered, and it fails for every layer where an active open-source ecosystem is moving faster than an internal team can. Applied to a retrieval layer over a proprietary corpus with domain-specific access rules, T_advantage is effectively unbounded — no vendor will build it because no vendor has the corpus.

Cost the engineering honestly

Build cases routinely compare a vendor's licence fee against an engineering estimate that omits most of the engineering. The comparison must be fully loaded and must extend across the operating life, not the build.

Two terms carry the argument. Maintenance is perpetual — a platform component needs roughly 15–25% of its original build effort every year merely to stay current, and that continues for as long as it is in service. And opportunity cost is usually the largest number in the calculation: the four engineers maintaining an internal serving layer are four engineers not building the domain application that is the only place your advantage could come from.

Buy without being captured

The correct objection to buying is lock-in, not price. The correct response is not to build — it is to buy behind an interface you control. Define your own abstraction for each rented capability, keep vendor concepts strictly behind it, and the vendor becomes an implementation detail rather than an architectural commitment.

class LLMProvider(Protocol):
    def complete(self, req: CompletionRequest) -> CompletionResponse: ...
    def embed(self, texts: list[str]) -> list[Vector]: ...

# Application code depends on LLMProvider only. Provider-specific retry,
# rate-limit and token-accounting behaviour lives in the adapter, never
# above it. Swapping vendors is an adapter plus an eval run.
class VendorAdapter(LLMProvider):
    def complete(self, req): ...

This is cheap when done from the start and expensive to retrofit. It costs a day of design and a thin layer of code. Retrofitting it after vendor-specific idioms have spread through the application is a multi-month project, which is precisely why organisations end up captured — not because switching is technically hard, but because the coupling was never contained.

The portability that matters most is the evaluation set. If you can measure quality on your own tasks, you can evaluate any provider in an afternoon and negotiate from a position of demonstrated substitutability. Without it, you cannot tell whether a cheaper model is adequate, so you keep paying for the one you have — which is a commercial disadvantage created by a missing engineering artefact.

Where in-house building genuinely earns its cost

• The retrieval and data layer over your proprietary corpus, including access control that reflects your entitlement model — no vendor can build this because no vendor has the corpus or the rules.
• Evaluation for your domain: task definitions, graders, and a maintained set of real cases with correct answers. This is the asset that compounds.
• Domain application logic — the workflows, constraints and decisions that constitute the product.
• Integration with your systems of record, which is specific to your estate by definition.

Notice that none of these is infrastructure. The pattern across the organisations that are getting real value from this technology is consistent: they rent aggressively at the bottom of the stack, they build deliberately at the top, and they keep a clean interface between the two so that the rapid improvement happening below them is something they benefit from rather than something they have to keep up with.