Clean data is not a control

What the paper argues

There are two questions hiding inside “can I trust this AI”. The first is whether the inputs are sound: provenance, lineage, accuracy, freshness. The second is whether the transformation is sound: whether the process turning that input into a shipped decision behaves the way you think it does, under pressure, when no one is watching. Data quality answers only the first. A model can take immaculate data and produce a confidently wrong, biased or unexplainable output, and the lineage diagram will not have moved an inch.

On the Cognitive Delivery Index five-level scale, a clean data pipeline is a Level 4 move. It is documentable and auditable: you can point at a catalogue, a lineage diagram or a quality score and say the control exists. The model, the review and the point where a human is meant to catch what the agent could not see sit one level up. Whether any of that demonstrably changes the outcome is the four-to-five jump, and it is exactly the jump being skipped.

Key points

• Garbage in, garbage out addresses the first question only. Securing the ingredients tells you nothing about whether the chef behind the closed door is doing anything sane.
• The CDI scale is not linear. The hardest jump, and the one that matters, is from Level 4, complete and satisfying to auditors, to Level 5, substantive and visibly shaping decisions.
• The category error is mistaking the existence of a clean dataset for control over what the AI does with it. A document is mistaken for a control, and that mistake is where most governance failures live.
• You can hold immaculate provenance and a review step that has never once sent anything back, on the same day. One is documented. Neither is governed.
• Governed means you can show, with examples from the last quarter, that a control changed what shipped: something was sent back, a scope was defended, a decision was challenged.
• Data quality is necessary, but it was never sufficient. For any control you claim, ask whether it demonstrably changed what shipped. If not, it is documentation, not governance.