Human-in-the-loop for agentic systems: where people belong, and where they only slow things down
"Keep a human in the loop" is easy to say and usually done badly: either every step waits for someone, or nothing does. This is how we decide which decisions get a human checkpoint, how the checkpoint is built so it is actually used, and how the loop shrinks over time without losing control.
Two ways to get it wrong
Everything waits. The system prepares work and a person approves each item. Handling time drops a little, the review queue becomes the bottleneck, reviewers start approving without reading, and the checkpoint stops meaning anything. The company paid for automation and got a rubber stamp.
Nothing waits. The system acts end to end. The first bad outcome that reaches a customer, an auditor or a bank account ends the project, regardless of how well the other thousand items went.
The useful version sits between the two: humans decide exactly where the consequence of a wrong action is real and hard to reverse, and nowhere else.
Classify decisions by consequence, not by difficulty
The instinct is to send "hard" decisions to people. Difficulty is the wrong axis; models are often better than tired humans at hard-but-bounded judgement. The right axis is what happens if the decision is wrong:
| If wrong… | Example | Who decides |
|---|---|---|
| Nothing leaves the system; easy to redo | Extract fields from a PDF, classify a ticket, draft a reply | Model, verified by rules |
| Something changes internally, reversible | Stage an invoice, assign a ticket, create a draft vendor | Model, logged, reviewable after the fact |
| Something leaves the company or moves money, below a threshold | Send a standard reply, post a matched invoice under the limit | Model, with sampling review (e.g. 5% audited weekly) |
| Above the threshold, or novel, or affects a relationship | Post a large invoice, reply to an escalated customer, onboard a new vendor | Human, presented with the prepared decision |
The threshold is a business rule written by the process owner (amount, customer tier, first-time counterpart, confidence below a bar). It is never the model's own opinion of its confidence alone.
Design the checkpoint so it gets used
A checkpoint people skip is worse than none, because it creates the appearance of control. Rules we apply:
- Show the decision, not the transcript. The reviewer sees the invoice fields, the matched PO, the variance, the reason it was flagged and the exact action that will happen on approval. Not a chat log.
- One click, typed outcome. Approve / Reject / Correct. The outcome is data the orchestrator reads, not a message it interprets.
- In the tool they already use. Slack, Teams, the helpdesk, the ERP inbox. A new dashboard is a new place to forget.
- Deadline and fallback. Every request has an owner, a due time and a defined outcome when nobody answers (escalate, or return to requester). Silence must not mean approval.
- Server-side enforcement. The system that performs the irreversible action checks that an approval exists. The agent cannot talk itself past the gate; neither can a prompt injected through a document.
- Every decision is logged with the reviewer's identity. This is what an auditor asks for, and it is what lets you widen the safe path later.
Reject is a first-class path
Most designs handle approval and forget rejection. A rejection should return the item to the requester with the reviewer's note, leave the systems of record untouched, and be visible in metrics. Recurring rejections are the most valuable signal in the whole system: each one is either a rule you have not written yet or a spoke you have not built.
Shrink the loop deliberately
Start conservative and widen the automated path on evidence, not on optimism:
- Read-only phase. The system prepares; humans commit everything. You collect the approval log.
- Auto-commit under the threshold. Only items that pass every rule and fall under the limit go straight through. Measure: straight-through rate, corrections, complaints.
- Raise the threshold or add rules when the approval log shows reviewers approving a category without changes for a defined period (say 200 items, zero corrections).
- Keep sampling. Even fully automated categories get a random audit. That is how you notice drift when a vendor changes its invoice format or a model version changes behaviour.
What to measure
- Straight-through rate (no human touch) per category
- Review queue age and time-to-decision
- Approval rate without changes vs. corrected vs. rejected
- Items reversed after auto-commit (the number that must stay near zero)
- Reviewer minutes per item, which is the cost the whole design is trying to reduce
The principle
People should make decisions, not do work. A well-designed loop presents a person with a prepared, consequential choice a few times a day and keeps everything else moving. If reviewers are reading PDFs or re-keying numbers, the loop is in the wrong place.
Frequently asked questions
Does a human have to approve every AI action?
No. Approvals belong where a wrong action is consequential and hard to reverse: money above a limit, customer-facing messages in sensitive cases, new counterparties. Reads, drafts and low-risk commits run without approval and are audited by sampling.
Who sets the approval threshold?
The process owner, as a written business rule (amount, customer tier, novelty, confidence bar). The system enforces it server-side; the model cannot lower it.
How do we keep reviewers from rubber-stamping?
Show a prepared decision rather than a transcript, keep the queue short by automating the safe path, give each request a deadline and fallback, and measure the approve-without-change rate; when it is near 100% for a category, that category is ready to automate.
We look at the process, propose an architecture, and decide together whether it is worth building. No packages, no price list.
Get in touch → Book a call