Designing Human-in-the-Loop Workflows

This guide helps you decide where humans belong in the loop, choose the right approval shape, and set escalation thresholds before you encounter an incident in production. For a reference on the mechanics of creating approval requests — how to write AOP instructions, respond via Slack, or manage Requests — see Human-in-the-Loop.

When to Add an Approval Step

Not every step needs human review. Use this risk framework to decide where approval gates add genuine value and where they only add friction.

Risk tier framework

Tier	Action type	Examples	Default stance
1 — Irreversible external	Sends something that cannot be recalled or undone	Outbound email, publish to public channel, submit a payment	Always gate
2 — Destructive internal	Removes or overwrites data	Delete records, overwrite fields, archive items	Always gate
3 — High-value transaction	Financial or compliance impact above a threshold	Approve purchase order, grant access, sign contract	Gate above threshold
4 — Ambiguous interpretation	Agent cannot reliably determine intent	Categorizing a one-off case, routing to the right team	Gate on low-confidence matches
5 — Routine and reversible	Can be corrected without consequence	Tagging a record, updating a status, creating a draft	Skip the gate

A practical starting point: start with an approval gate on all Tier 1 and 2 actions. Remove gates only after you have observed that a branch never produces surprises.

Decision criteria by action category

Category	Gate?	Notes
Send email to external recipient	Yes	Include recipient, subject, and full body in the request
Post to a public Slack channel	Yes	Include the channel name and draft text
Submit a financial transaction	Yes	Include amount, recipient, and reference number
Modify or delete a customer record	Yes	Include the record ID and the proposed change
Reply to an internal thread	Usually not	Gate only if the reply is policy-sensitive
Tag or label a record	No	Reversible; iterate without a gate
Create a draft (not sent)	No	Show in the approval request instead of gating during creation

Choosing the Right Approval Shape

The Human-in-the-Loop connection supports three request types. Picking the right one keeps operators efficient and prevents ambiguous responses.

Approval (confirm or reject)

Use when the agent has already determined what to do and just needs a go/no-go before acting.When it fits: The agent is about to send an email, submit a payment, or delete a batch of records.AOP pattern:

Before sending the email, request approval. Set the title to
"Send to [recipient] — [subject line]". Include the full email body
in the description. Only send after approval. If denied, ask what to
change and revise.

Question (choose from options)

Use when the right next step depends on context the agent does not have, and the options are well-defined.When it fits: The agent needs to route a case, choose a tone, or decide between two valid policies.AOP pattern:

If the invoice currency does not match the vendor's default, ask:
(a) Convert to USD at today's rate and continue
(b) Flag for manual review by the finance team
(c) Reject and return to sender with a note

Design rules for good options:

Keep option labels short and action-oriented.
Make options mutually exclusive — if they overlap, the operator will guess.
Do not add an “Other” option unless you also tell the agent what to do with a free-text answer.

When to use free-text instead

Free-text operator input makes sense when the operator needs to supply content, not just choose a path: dictating a reply, providing context about an anomaly, or overriding a specific field value.Risk: Free-text answers must be parsed by the agent in its next step. If the agent expects a decision but gets prose, it may misinterpret. Only use free-text when the agent’s AOP explicitly handles open-ended input.Common mistake: Using a free-text question where a structured Question would do. Structured options are faster for operators and produce more reliable downstream behavior.

Designing Fallback and Escalation Behavior

Every approval gate needs an explicit answer to “what happens if no one responds?” Without a defined fallback, the agent stalls indefinitely or makes an unsafe assumption.

Timeout → escalate or fail-safe

Decide up front: if the operator does not respond within a reasonable window, should the agent escalate or default to the safe action? Escalate pattern:

Request approval with a 4-hour timeout. If not approved within 4 hours,
ask the account manager: [same options]. If still no response after 24 hours,
cancel the action and notify the original requester.

Fail-safe default pattern:

Request approval before sending the refund. If not approved within 2 business
days, cancel the refund and log the reason as "Approval timeout".

Use the escalate pattern when delay is costly (outbound communications, time-sensitive transactions). Use fail-safe defaults when the safe outcome is to do nothing and revisit manually.

Repeated rejection → halt

If the operator rejects the same action multiple times, the agent has likely misunderstood the requirement. Continuing to loop wastes operator time.

If the approval is denied twice in a row for the same case, stop and
add a note: "Halted after two rejections — requires manual review."
Mark the case as Failed.

Multi-tier escalation

For high-stakes workflows, define a clear escalation chain so a request never hangs waiting for a single person.

Request approval from the team lead. If not answered within 2 hours,
escalate to the department head. If not answered within 24 hours,
escalate to the account manager and halt processing until resolved.

Keep escalation chains short. More than three tiers signals a process that is not well-defined.

Using Queue to Manage Exceptions at Scale

When an agent processes many cases, Requests becomes a bottleneck. The Queue is the right tool for managing exceptions across high-volume workflows.

Triage pattern

Consumer agent processes cases automatically.
Cases that need a human decision surface as Needs Input in the queue.
An operator reviews the Needs Input filter, responds in the case detail panel, and the agent resumes.
Cases the agent cannot resolve at all land as Failed — the operator reviews, updates the case data, and retries.

This separates routine processing from exception handling: the agent handles volume, the operator handles judgment calls.

Bulk delegation for surges

When a backlog of Needs Input cases accumulates, you can select multiple cases and delegate them to a specialist agent — one designed specifically for exception handling — rather than asking one person to respond to dozens of individual HITL requests.

Aging and escalation logic

For cases that sit in Needs Input beyond an acceptable window:

After processing the case, if it moves to Needs Input, postpone it for 24 hours.
On retry, check whether the human has responded. If not, escalate to the team
lead by asking a question: (a) Proceed with the default option, (b) Assign to
a specialist, (c) Close the case.

Ramping Toward Safe Autonomy

The goal is not maximum oversight — it is the right level of oversight. As an agent matures, you should expect to remove approval gates where they no longer provide signal.

Start with gates on every risky branch

When you first deploy an agent, add approval gates on all Tier 1-2 actions and any branch where you are unsure what the agent will do.

Measure the approval rate

After a few weeks of production traffic, review the pattern of approvals and denials in Requests or the Runs list. Estimate the approval rate per gate.

Approval rate above ~95%: The agent is getting it right consistently. Consider removing the gate and trusting the output directly.
Approval rate below ~70%: The agent is frequently wrong. Refine the AOP before removing the gate.
High denial rate with a consistent pattern: The agent is doing the same wrong thing repeatedly. Update the AOP to correct the root behavior.

Remove gates selectively

Remove gates one at a time, in order of confidence. Monitor the next two weeks of output. If quality holds, the gate can stay removed.Do not remove a gate if:

The action is irreversible and errors are high-cost.
Volume is too low to measure a meaningful approval rate.
You have changed something in the connected systems recently.

Re-introduce gates when conditions change

Re-add approval gates when:

You modify the AOP in ways that could affect the gated branch.
The agent gains access to a new system.
Volume increases significantly (edge cases appear at scale that were rare before).
Approval rate drops during a quarterly review.

Quarterly cadence: Review approval rates for all agents in production. Prune gates with consistently high approval rates. Re-add gates on branches that have drifted.

Worked Examples

Expense report approval — threshold-based gate

The agent auto-handles low-value expenses and escalates high-value ones through a structured flow.

Process each expense report:
— Under $200: approve automatically and update the status.
— $200–$1,000: request approval. Title: "Approve expense — [employee] — $[amount]".
  Description: employee name, department, line items. If denied, return to the
  employee with the reason.
— Over $1,000: ask whether to (a) approve, (b) deny, or (c) escalate to the
  finance director. If no response within 1 business day, escalate automatically.

See the full tutorial: Expense Report Approval

Customer response email — tone check before send

The agent drafts a reply and holds it for review before sending.

Draft the reply email. Request approval before sending.
Title: "Send reply to [customer name] — [ticket ID]".
Description: the full draft email body.
If approved, send immediately.
If denied, ask: (a) Revise the tone to be more formal, (b) Revise the tone
to be more empathetic, (c) Escalate to the account manager.
Regenerate based on the selected option and request approval again.

See a related example: Reviewing Drafts Before Sending

Supplier follow-up — timeout escalation

The agent follows up on outstanding purchase orders and escalates if no response arrives.

Send the follow-up email to the supplier. Request approval from the operations
lead before sending.
Title: "Follow-up: PO [number] — [supplier] — [days overdue] days overdue".
If the approval request is not answered within 4 hours during business hours,
escalate to the account manager with the same request.
If still no response after 24 hours, send the email using the default template
and log: "Sent without explicit approval after escalation timeout".

Human-in-the-Loop

Reference for request types and responding via Slack, Teams, and Requests.

Requests

Managing and responding to pending approval requests.

Queue

Queue-based exception handling for high-volume workflows.

Guardrails for High-Risk Automations

Risk classification, hard caps, allow/deny lists, and kill switch procedures.

Refining Your Agent

Using HITL approval feedback to improve your AOP.

Expense Report Approval

Full tutorial showing threshold-based approvals.

Reviewing Drafts Before Sending

Full tutorial showing draft review before outbound communication.

​When to Add an Approval Step

​Risk tier framework

​Decision criteria by action category

​Choosing the Right Approval Shape

​Designing Fallback and Escalation Behavior

​Timeout → escalate or fail-safe

​Repeated rejection → halt

​Multi-tier escalation

​Using Queue to Manage Exceptions at Scale

​Triage pattern

​Bulk delegation for surges

​Aging and escalation logic

​Ramping Toward Safe Autonomy

​Worked Examples

​Related

Human-in-the-Loop

Requests

Queue

Guardrails for High-Risk Automations

Refining Your Agent

Expense Report Approval

Reviewing Drafts Before Sending

When to Add an Approval Step

Risk tier framework

Decision criteria by action category

Choosing the Right Approval Shape

Designing Fallback and Escalation Behavior

Timeout → escalate or fail-safe

Repeated rejection → halt

Multi-tier escalation

Using Queue to Manage Exceptions at Scale

Triage pattern

Bulk delegation for surges

Aging and escalation logic

Ramping Toward Safe Autonomy

Worked Examples

Related