Human-in-the-loop design: where approval steps belong in AI workflows

Most teams add a human approval step to an AI workflow for the same reason: nobody is quite ready to let the model act on its own. That instinct is right. The trouble is where the step ends up. Put it everywhere and people start clicking "approve" on autopilot within a fortnight. Put it in the wrong place and the person approves a nice-sounding plan while the risky part (the email, the refund, the record update) happens later with nobody watching.

This post is about placing approval steps deliberately: which actions need one, what the reviewer should see, how to build the pause, and how to tell whether your reviewers are still reviewing.

An approval step is a control, and controls fail#

It helps to treat a human checkpoint like any other safety control. It has a job, it has failure modes, and you should be able to test it.

The best-known failure mode has a name: automation bias. A 2012 systematic review by Goddard, Roudsari and Wyatt describes it as "the tendency to over-rely on automation". They screened 13,821 papers and included 74. Their mediators will sound familiar to anyone who has watched an approval queue in a busy office: workload, task complexity and time pressure, plus the reviewer's trust in and confidence about the system. The review was about clinical decision support, but the mechanism is the same when the suggestion comes from an LLM instead of a diagnostic tool.

Regulators have picked this up. The UK Information Commissioner's Office, in its audit toolkit on human review, says plainly that "non-meaningful human review is caused by automation bias or a lack of interpretability". In other words, a person who can't understand the output, or has stopped questioning it, isn't really a control. The EU AI Act's Article 14 on human oversight goes further and requires that people overseeing high-risk systems can stay aware of automation bias, especially where the system gives "information or recommendations for decisions".

Most business automations aren't high-risk systems under the AI Act, so Article 14 probably doesn't bind you. It's still a useful checklist, because it describes what a working human checkpoint has to allow.

Which actions need a human#

Start from consequences, not from how confident the model seems. Four questions do most of the work:

1. Can the action be undone cheaply? A draft saved to a folder can. An email to a customer, a payment or a deleted record can't.
2. Does it leave the organisation? Anything a customer, supplier or regulator will see deserves more care than an internal note.
3. Does it move money, change access or touch personal data?
4. How many things does one call affect? Updating one ticket is different from bulk-closing 400 of them.

OWASP's entry on excessive agency in its 2025 Top 10 for LLM applications puts the rule simply: "require a human to approve high-impact actions before they are taken". It lists excessive autonomy as one of three root causes, alongside excessive functionality and excessive permissions. That ordering matters. Before you design an approval screen for an action, ask whether the agent needs that action at all. OWASP's own example is an assistant that only has to summarise email but has been given a plugin that can also send it. The fix it suggests is a read-only tool and a read-only OAuth scope first, then a person who reviews and hits "send" on anything the assistant drafts.

A rough default table for a small business workflow:

The middle row is where you should expect to change your mind. Approval there is temporary, and it should come off once you've measured the step properly. The bottom two rows usually stay.

Approve the action, not the plan#

A common mistake is asking a person to approve what the agent intends to do ("I'll refund the customer and close the ticket") and then letting it run. By the time the tool call happens, the amount, the customer ID or the account may differ from what the reviewer read. Prompt injection makes this worse, because text the agent reads after approval can steer it.

The checkpoint belongs directly in front of the side effect, showing the exact arguments that will be sent: this amount, to this account, for this order. OWASP's advice to build the approval into the extension that performs the "post" operation says the same thing in security language.

Two related points. First, the approval step doesn't replace normal authorisation. OWASP calls this complete mediation: the downstream system should check whether the user is allowed to do the thing, "rather than relying on an LLM to decide if an action is allowed or not". Second, a well-designed draft-and-send split often removes the need for a separate approval screen altogether. If the agent can only create drafts and a person sends them from their normal inbox, the approval is built into work they already do.

Design the screen so "no" is easy#

A reviewer can only catch an error they can see. Among the mitigators the Goddard review found were training, making users accountable for decisions, and design choices: where the advice sits on the screen, updated confidence levels attached to the output, and whether the system gives information or a recommendation. Translated to an LLM workflow:

• Show the source next to the proposed action: the email the agent read, the invoice it extracted from, the policy it applied. A reviewer who has to open three tabs to check will stop checking.
• Show what will change, as a diff if you can, not as a paragraph of the model's reasoning.
• Make reject and edit as quick as approve, and ask for a short reason on reject. Those reasons become your next test cases.
• Don't pre-tick anything, and don't make "approve all" the biggest button on the page.
• Give people a way to stop the whole run. Article 14 describes this as a "'stop' button or a similar procedure" that halts the system in a safe state.

Workload is a design input too. The ICO toolkit expects organisations to "assign human reviewers a manageable caseload" and to give them enough time for the task. If one person is approving hundreds of items an hour, you have built an expensive delay rather than a check.

Building the pause: interrupt, persist, resume#

Technically, an approval step means the workflow has to stop, wait for someone who may not reply until tomorrow, and pick up where it left off. Agent frameworks now ship this directly. The OpenAI Agents SDK's human-in-the-loop guide is a good example of the shape:

• You mark a tool with needs_approval, either True to always ask or a function that decides per call (so you could require approval only for refunds above a threshold).
• When a call needs approval, the run pauses and the pending calls appear in interruptions with the tool name and arguments, which is what you show the reviewer.
• You approve or reject each item on the run state and resume. The state can be saved with to_json() and restored later, so the pause can outlast the process.

Two details from that guide are worth copying whatever tool you use. Approval rules "fail closed" when the SDK can't safely inspect the arguments, so a malformed call goes to a person instead of slipping through. And there are sticky always_approve and always_reject options that persist a decision for later calls in the same run. Use the sticky approve sparingly. It is exactly the shortcut a busy reviewer reaches for, and it turns one considered decision into a blanket one. The guide also warns that serialised state includes your app context, so keep secrets out of anything you park in a queue or database.

If you're not using an agent framework, the same pattern works with a database table of pending actions, a status column and a worker that only executes rows marked approved.

Check that the checkpoint still works#

Approval steps decay quietly. The signal to watch is the decision log. The ICO toolkit asks organisations to "maintain a log of when AI decisions are overridden by a human reviewer, including the reasons why", and suggests mystery shopping, which in practice means feeding known-bad items into the queue and seeing whether reviewers catch them.

Read the log in both directions. A queue where reviewers almost never reject anything means one of two things: the step is ready to come off for that action type, or people have stopped looking. A planted error tells you which. A queue with frequent rejections tells you the automation isn't ready for that task, and the rejection reasons tell you why. Either way you learn something you can act on, which is the point of having a person there.

At TheAILAB we treat each approval step as having an owner and a review date, so it gets either removed or kept on purpose rather than by habit.

A practical starting point#

Pick one workflow and list every action the AI can take. Strip out the tools it doesn't need. For what's left, put approval in front of anything external, financial, destructive or bulk, and show the reviewer the exact arguments and the source material. Log every decision with a reason on rejections. After a month, read the log, plant a few known errors, and decide action by action whether each approval stays, becomes a spot check, or goes.

Sources

1. LLM06:2025 Excessive Agency — OWASP Gen AI Security Project (Top 10 for LLM Applications 2025) (checked 2026-10-16)
2. Automation bias: a systematic review of frequency, effect mediators, and mitigators — Goddard, Roudsari and Wyatt, Journal of the American Medical Informatics Association (2012) (checked 2026-10-16)
3. Human review (AI and data protection audit toolkit) — Information Commissioner's Office (UK) (checked 2026-10-16)
4. Article 14: Human oversight — European Commission, AI Act Service Desk (Regulation (EU) 2024/1689) (checked 2026-10-16)
5. Human-in-the-loop — OpenAI Agents SDK for Python documentation (GitHub) (checked 2026-10-16)