Why Pass/Fail Reviews Hide the Risks That Actually Cause Incidents

A pass/fail verdict feels like closure. It isn’t: it compresses everything reviewed, and everything that was never looked at, into a single binary that carries almost no information about actual exposure.

The verdict says pass. The incident happens anyway.

Every post-incident investigation follows the same arc. The review was done. The checklist was signed. The drawing had a tick against it. And yet something failed, someone was harmed, or a Gateway 2 submission came back with fundamental queries that derailed programme by months.

This is the core problem with how design review works in AEC today. Until the industry names it clearly, the pattern will keep repeating.

The two incident causes a verdict buries.

Post-incident analyses repeatedly surface the same two root causes. Dame Judith Hackitt’s 2018 Independent Review of Building Regulations and Fire Safety was blunt: the system had a habit of recording compliance without recording understanding.

Those two causes are:

1. Things no one ever looked at. Not checked against the requirement. Not cross-referenced. Not caught at design stage because the review was sampled, not comprehensive. The absence of a finding is not the same as the absence of a problem.
2. Confidence that outran the evidence. Someone looked and concluded everything was fine, but their confidence was not calibrated to what they actually verified. The tick said done. The evidence said assumed.

Blind spots that were never looked at.

Design packages are dense. In a complex higher-risk building, the kind now subject to Gateway 2 scrutiny, a submission can run to hundreds of documents, drawings, and specifications, with requirements layered across statutory, regulatory, client brief, and operator standards.

No human reviewer working at normal commercial pace reads all of it. They sample. They focus on what they know well. That’s not a criticism; it’s physics.

But the pass/fail mechanism makes that sampling invisible. The output looks identical whether 80% of the package was reviewed or 30%. The tick doesn’t record where the reviewer ran out of time, which requirements fell outside their expertise, or which drawings were superseded on revision 4 and never re-checked.

Those untouched areas aren’t gaps that got a clean result. They’re blind spots, and a binary verdict actively conceals them.

Being confidently wrong.

Blind spots are one problem. The other is subtler and more dangerous. Reviewers who did look can still be wrong, and the pass/fail format gives no signal about how strongly the conclusion is held, or on what basis. A confident tick from deep expertise and a confident tick from a quick skim look identical on the record.

Honest uncertainty is safer than false certainty. Pass/fail collapses that distinction by design.

Overconfidence is a known failure mode in complex review work. When a team records a risk as managed, it stops watching it. The false all-clear removes the signal that would have prompted further scrutiny. A graded system that records why a reviewer is confident, or marks something as not yet verified, creates a healthier epistemic culture.

Why a tick count feels safe but isn’t.

There’s a reason the industry defaults to pass/fail: it feels manageable. You can count ticks. You can report percentages. It has the texture of rigour.

But a tick count measures area, not volume. It tells you how many items were reviewed, not how deeply, not with what confidence, and not what remains unresolved underneath a conditional pass.

Consider a fire safety review that ticked every item, but the checklist was built three years ago, before the client brief changed, before a new occupier was added, before the sprinkler specification was revised. Every item passes. The underlying exposure appears nowhere on the record.

Graded assurance as the alternative.

Rather than forcing every finding into pass or fail, graded assurance records quality as a volume: across the things that matter, how confident are we, and why? It captures four things a binary verdict cannot:

• What was covered, with evidence: citations to the documents, drawings, and requirements actually checked.
• What the confidence level is, explicitly graded and backed by reasoning, not asserted.
• What remains open, unresolved items held in view, not buried under a conditional tick.
• Where exposure sits, the risk the review couldn’t resolve, kept visible under the signature.

Pass/fail is the degenerate case of graded assurance, where every confidence level is forced to 0 or 1. That compression destroys the information that matters most. Graded assurance doesn’t lower the bar; it raises it, because it requires reviewers to show their working.

Reframing review around exposure.

The fundamental reframe: design review’s purpose is not to produce a verdict. It is to bound exposure.

A verdict answers: did this pass? Exposure-framing asks: what risk remains after review, and where is it?

That shift changes what gets recorded, what the signing dutyholder can defend personally, legally, and evidentially, and what a Gateway 2 submission can demonstrate: a traceable record of what was assured and what residual risk was consciously acknowledged, not just a set of ticks.

The Building Safety Act’s dutyholder regime is pushing in exactly this direction. Personal accountability requires something a checkbox can never provide: a record of why the decision was made, against what evidence, and with what uncertainty left visible. This is what Assurance Engineering names, treating quality as a calibrated, evidence-backed volume, not a binary verdict stamped at the gate.