The Library · Perspective
The model is the cheap part
In an AI integration the model is a commodity and the seams are the project. Budgets that treat it the other way round overrun in a predictable place.
Every AI integration budget we have seen underestimates the same thing, and it is never the model. It is the seams: the places where a probabilistic component has to meet a system that was built on the assumption that its inputs were correct.
Where the money actually goes
A model call is a line of code and a per-token price. Both are falling, both are predictable, and neither is where an integration gets expensive.
The expensive parts are the ones that have no equivalent in conventional software:
- Deciding what happens when the output is wrong. Not if — when. Conventional components fail loudly and rarely; this one fails quietly and regularly, and every consumer of its output needs a position on that.
- Evaluation. You cannot improve what you cannot grade, and building the grader is frequently a larger job than building the feature. It is also the part most often cut first.
- The human review path. Someone has to see the uncertain cases. That means a queue, an interface, a person's time, and a decision about what they are accountable for.
- The data boundary. What leaves the building, under what agreement, with what retention. This is a legal and architectural question, and it is answered badly when it is answered late.
A deterministic system above, a probabilistic one below, and the turbulent strip where they meet. Neither system is the project. The strip is the project.
The failure mode
The characteristic failure is not a system that does not work. It is a system that works well enough in demonstration to be approved, and then degrades in a way nobody notices for a quarter.
This happens because the demonstration is run on the cases the builder had in mind, and production is mostly cases nobody had in mind. Without evaluation, the difference is invisible until a customer finds it.
A budget that survives
The shape that holds up in practice allocates roughly as follows, and the proportions matter more than the absolute numbers:
| Component | Share of effort |
|---|---|
| Model selection and prompting | Small |
| Evaluation and grading | Large |
| Failure handling and fallbacks | Large |
| Human review path | Moderate |
| Data boundary and governance | Moderate |
| Ongoing monitoring | Continuous, not one-off |
The line that most often gets cut is evaluation, because it produces nothing a stakeholder can see in a demonstration. Cutting it does not remove the cost; it defers it to the point where the cost is paid in customer trust instead of engineering time.
What this means for scoping
Two practical consequences.
The first is that "which model" is close to the least important decision in the project, and it is reversible. Architect so that it stays reversible; anything that makes the model hard to swap is buying a commitment you do not need to make.
The second is that the integration is worth doing only where you can state the acceptable error rate. Where the honest answer is "it has to be right," the correct design is not a better model but a smaller claim: use the system to prepare work for a person rather than to replace the person's judgement.