How we test governance: the CDF test bench

A living methodology document, version 0.1. It describes how Cognitive Delivery tooling, the plugin today and the kernel as it develops, is tested. It will grow as the bench does.

The principle

A governance control that has never been attacked is a record of intent, not a control. So the test bench is built on one rule: every enforcement claim gets an adversarial test that tries to break it, alongside the conventional tests that show it working.

That gives each control three layers of evidence:

1. Conformance tests. The happy path: a governed workflow that follows the rules completes, produces valid artefacts, and records provenance and audit entries with correct hashes.
2. Enforcement tests. The refusals: every gate has tests proving it rejects what it must reject. A spec cannot advance without review evidence; approval language that is really work-direction language is refused; provenance hash mismatches block completion. If a rule exists, there is a test that tries to disobey it and asserts the failure.
3. Adversarial agent simulation. The realistic threat: scripted agent sessions that attempt to route around governance the way a capable but unaligned agent would, editing artefacts directly instead of through tools, reusing stale approvals, batching task states, claiming work without decision evidence. The bench asserts the bypass is caught, refused, or surfaced in the audit trail.

The bench itself

The bench runs governed workflows against a disposable workspace fixture: a repository seeded with steering, config and specs in known states. Each scenario is a sequence of tool calls with an expected outcome (allowed, refused, or flagged), and every run also validates the two invariants that make evidence trustworthy:

• Audit integrity: the journal is append-only and each entry carries the reasoning that produced it.
• Provenance integrity: artefact content hashes match their stamps, and any drift is detected.

What the kernel changes

For the kernel, the same bench moves down a layer: policy evaluation becomes property-tested (the same policy and state always produce the same decision), the evidence log gains corruption and truncation tests, and adversarial scenarios run against the kernel’s interface directly, without an editor in the loop, so no integration can soften a refusal.

What is not tested yet

Honesty section, kept current: cross-model review diversity is asserted structurally but not yet exercised against live model pairs in CI, and long-horizon session-recovery scenarios are covered manually rather than by the bench. Both are on the bench’s own backlog.