Act
Human-in-the-Loop
Who can review or override this decision?
Lets authorised people review, change, or override an AI decision, with every intervention on the record.
- Evidence: research-based (Amershi et al. 2014, Santoni de Sio & van den Hoven 2018)
- Tested: concept only, mock data
Overview
Why it matters
AI recommends. People are responsible.
However accurate a system is, some cases need a person to step in: the case is unusual, the stakes are high, the data is wrong. This pattern makes that intervention possible, traceable and legitimate. A decision no one can override is not a UX problem. It's a governance failure.
Unlike Contestability & Appeal, which the affected person uses after the decision, this is used by authorized staff inside the process.
What does it look like?
Standard caseworker override
Standard caseworker override
- Caseworker reviews a case with AI recommendation: "Decline."
- Caseworker checks attribution and notices the income figure is stale.
- Caseworker clicks "Override" → selects "Data appears incorrect."
- Adds note: "Income figure is from 18 months ago. Applicant has provided updated payslips."
- Caseworker approves the case with justification logged.
- Override is recorded in the audit trail.
Mandatory review routing
Mandatory review routing
- Application submitted by a self-employed applicant.
- System flags the case as out of scope. Mandatory human review required.
- Case enters the specialist review queue.
- Specialist reviewer is notified and reviews the case with full context.
- Final decision is made by the specialist, not the AI.
Deferral with additional information request
Deferral with additional information request
- Caseworker reviews a borderline case.
- Rather than approving or overriding immediately, selects "Defer: request additional documents."
- System sends document request to the applicant.
- Case is placed in a pending state until documents are received.
- Caseworker reviews again with complete information.
When does it fit?
Use it when
- The AI recommends, and a person makes the final call
- High-stakes decisions need documented human judgment
- Regulation requires human oversight
Skip it when
- Human review isn't feasible at this scale (document that choice)
- People would rubber-stamp an optional override: use a mandatory review step instead
Design
How do I apply it?
-
Make override a first-class action.
As visible as "accept", not hidden on a second screen.
-
Never harder to override than to accept.
Extra steps for overriding quietly discourage human judgment.
-
Require a reason.
At least a reason category, a short text for high-stakes cases. Undocumented overrides can't be audited.
-
Keep recommendation and decision apart.
"AI recommended X, caseworker decided Y, because Z." Never merge them.
-
Let confidence and scope set the review level.
Low-confidence and out-of-scope cases go to mandatory review.
For your governance team
Meaningful human control needs two conditions (Santoni de Sio & van den Hoven, 2018). Tracking: the system visibly responds to the reasons and facts the reviewer cares about. Tracing: every AI-assisted decision can be traced to a person who understood the system and accepted responsibility. An override button alone meets neither. Interface, documentation and training all have to support it.
How can I show it?
Override action control
A visible way to reject or change the recommendation.
A clearly labelled button or action allowing an authorized user to reject or modify the AI recommendation. Not hidden. Not secondary. The primary action after reviewing the AI output in high-stakes workflows.
Override reason selector
Predefined reasons for overriding.
A structured dropdown or selection of pre-defined override reasons: - Data appears incorrect - Case is atypical / outside model scope - Additional context changes the assessment - Professional judgment differs - Other (with mandatory text input) Structured reasons are more auditable than free-text and reduce the time required per override.
Free-text justification field
The reviewer's own reasoning, on the record.
A mandatory or optional text field for the override rationale: the human reviewer's documented reasoning. Becomes part of the case record and audit trail.
Mandatory review flag
Risky cases routed to a person automatically.
Cases that meet certain criteria (out-of-scope, very low confidence, flagged attributes) are automatically routed to human review. The mandatory nature of the review is visible in the interface.
Deferred review queue
Ask for more information instead of a forced yes or no.
A system state allowing a human reviewer to defer the decision, by requesting additional information, escalating to a specialist, or setting a review date, rather than forcing a binary immediate decision.
Confirmation step for high-stakes overrides
A confirmation before irreversible overrides.
For irreversible or high-impact overrides (approving a previously declined application, clearing a flagged record), a confirmation dialog: "You are overriding an AI decline recommendation. This will be logged. Confirm?"
Override history / activity
Who reviewed the case, when, and what changed.
A visible record in the case of previous overrides: how many times this case was reviewed, who reviewed it, what changes were made. Prevents redundant reviews and provides context for subsequent reviewers.
What does it cost?
Oversight vs. efficiency
More review means less throughput. Match review intensity to risk.
Accountability vs. automation bias
People tend to accept AI recommendations even against evidence. A button alone isn't oversight.
Documentation vs. usability
Too much paperwork and overrides get avoided. Find the minimum that stays traceable.
Connections
Which patterns work with it?
Agent Action Transparency
What actions is the AI agent taking?
HITL applies to recommendation systems where a human reviews before a final decision. Agent Action Transparency applies to agentic systems where the AI has already acted. Both are human oversight mechanisms, but at different points in the action flow.
Contestability & Appeal
How can I challenge this decision?
HITL is exercised by the operator or caseworker within the process. Contestability is exercised by the person affected by the decision. Different roles, different mechanisms, both necessary.
Audit Trail & Logging
What is the complete record of this decision?
Every override is an audit event. HITL and audit trail are inseparable: the human decision is only as accountable as its documentation.
Confidence & Uncertainty
How certain is the system?
Low confidence should trigger HITL. The two patterns are directly connected: uncertainty in the AI output is the primary signal for mandatory human review.
Actionable Recourse
What can I do to change this outcome?
Recourse actions taken by the affected person, such as submitting documents or correcting data, should trigger re-evaluation through the HITL mechanism: a human reviews the case with the new information.
Sources (4)
Sources (4)
Amershi et al. (2014) — Power to the People: The Role of Humans in Interactive Machine Learning
establishes the conceptual and empirical basis for human participation in AI decision loops. Foundational for the design of human oversight mechanisms. Published in AI Magazine, 35(4)
Gaube et al. (2021) — Do as AI say: susceptibility in deployment of clinical decision-aids
empirical study in which physicians received deliberately inaccurate diagnostic advice: diagnostic accuracy dropped significantly when the advice was wrong, regardless of whether it was labelled as coming from an AI or a human expert. Documents how susceptible even trained professionals are to incorrect decision-aid output, and motivates why override design must actively support, not just permit, independent human judgment. Published in npj Digital Medicine, 4, 31
European Union (2024) — EU AI Act, Article 14
requires human oversight measures for high-risk AI systems, including the ability of natural persons to override AI outputs. The regulatory specification this pattern implements
Santoni de Sio & van den Hoven (2018) — Meaningful Human Control over Autonomous Systems: A Philosophical Account
proposes a framework (the tracking and tracing conditions) for what "meaningful" human oversight requires beyond nominal override capability. Provides the normative basis for designing override mechanisms that are genuinely empowering, not performative. Published in Frontiers in Robotics and AI, 5
Your AI feeds real decisions. Can a person actually override it?
And is that override documented? That's where this pattern starts. Let's look at your interface together.
Work with me