What's the most expensive bug in an AI pipeline?
The one that reports green.
- Hypothesis
- In an automated pipeline, the costliest failures are not the loud ones but the checks that stop working while they keep reporting success.
- Method
- Post-mortems on our own pipeline. Whenever a stage turned out not to do its job, we traced how long it had been broken, what it reported in the meantime and how we found it.
- Finding
- The pattern repeated in separate parts of the system. A test-first stage became unreachable while its status said it was running. A health watchdog stopped firing and kept reporting itself as active. Our flaky-test detector read only the start of a large report and showed a perfect record. None of them failed loudly.
Loud failures are cheap. A red build stops the line, somebody looks, it gets fixed. The expensive ones look like success.
In spring we measured our test-first stage on a benchmark: the generated tests were correct, and they failed before the code existed, exactly as they should. Later, a code change made that stage unreachable, while the status line kept saying it was running. We found it during a probe on a new tech stack, because a step we expected in a real build simply was not there.
The same shape showed up elsewhere. A watchdog that should have alerted us had stopped running and still reported itself active. A detector for flaky tests read only the first part of a much larger report and showed a perfect record.
The rule we took from it
A stage counts as working only when its step shows up in a real build, not when it reports itself. Silence is a signal, and it gets its own check.
In the Factory
In the DayStorm Factory, we check that every stage’s step appears in real builds, including test-first. Silence from a stage that should be talking is now something we look for on purpose.