Is the model the bottleneck?
No. The spec is.
- Hypothesis
- When AI-built software misses, the cause is more often an unclear or ungrounded spec than a lack of model capability.
- Method
- Three kinds of observation from production work. We measured where model usage went over a working day, across spec drafting sessions and DayStorm Factory builds. We built the same features once with a cheaper and once with a stronger model, and had every version reviewed by both. And we moved the step that grounds a spec in the codebase from the drafting session into the Factory, right before the build, then compared the specs that followed.
- Finding
- Drafting and grounding specs used far more than all Factory builds combined, and most of that went into agents reading code to understand the codebase. For building, the cheaper model kept up with the stronger one; the differences showed up in review, not in construction. Grounding the spec right before the build made specs markedly cheaper to prepare.
Most discussions about AI coding start with the model: which one, how large, how expensive. Our measurements kept pointing somewhere else.
When a build went wrong, the model had rarely failed at writing code. It had built what the spec said, and the spec had left room for interpretation, or did not know enough about the codebase it was meant to change. The expensive part of the work was not the build. It was understanding the existing code well enough to say precisely what should change.
That is why the Factory sharpens every spec against your codebase before a line is written, and asks when intent is unclear. It is also why we spend the stronger model where it pays off most: in review.
We keep measuring this as models improve.