> ## Documentation Index > Fetch the complete documentation index at: https://docs.duvo.ai/llms.txt > Use this file to discover all available pages before exploring further. # Make Every Run Auditable > When someone asks "what did the Agent do, and was it right?", the answer should be a link, not an investigation. What Duvo records for every Run, how automatic Evaluations work, and what you must design into the AOP yourself. Sooner or later, someone will question a Run: an auditor, a customer, a teammate, or you at 9am after a night of scheduled Runs. An auditable Agent answers that question in minutes — what happened, what was produced, and whether it was right — from the record the platform already keeps. This page covers what Duvo records automatically, how to read it, and the parts the platform cannot do for you: durable outputs, stable identifiers, and alerting. ## What the Platform Records for Every Run Every Run keeps a durable record you can open later from the [Runs List](/user-guide/running-assignments/jobs-list): * **The full transcript** — every message and tool call, kept on the Run permanently. * **Status and timestamps** — one of eight statuses (Completed, Failed, Stopped, Needs Input, and the in-flight states), with start and completion times. * **A screenshot replay** for browser work — after the Run ends, the [Live View](/user-guide/running-assignments/live-view) tile is replaced by a step-by-step replay, one frame per browser action. * **The Case timeline** for Queue work — every claim, update, settlement, and approval on the Case, with each Run as a card (see below). * **An Evaluation** — an automatic quality verdict on every completed Run (next section). * **Workspace files** — everything the Run wrote in its sandbox, browsable and downloadable from the Run detail. **These are temporary** (see "Outputs Must Outlive the Sandbox"). One honest caveat: Duvo does not write a prose summary of what a Run did. The transcript is the record of *what happened*; the Evaluation is the verdict on *whether it went well*. If you want a human-readable recap per Run, instruct it in the AOP — end every Run with a short report of what was done, where results were written, and under which identifiers. For Queue work, have the Agent write that recap into the Case data, where it outlives everything else. ## Evaluations: Your Built-In QA Every Run that completes is evaluated automatically — no setup, no opt-in. An LLM judge (separate from the model that did the work) reads the complete Run trace and scores it against two sets of criteria: * **Platform evaluations** — a fixed catalog of failure checks every Run is scored on, grouped by cause: Agent Misbehavior, AOP Gap, Platform Issue, and User Access Gap. * **Agent specific evaluations** — up to five Pass/Fail questions derived automatically from your AOP, scoped to the Revision the Run actually ran against. Each criterion gets a verdict (Pass, Failed, or Not applicable), a severity (**Critical** — blocks the outcome, **Medium** — a defect needing intervention, **Low** — verified harmless), and a short plain-language reason. The judge also writes an **AI Analysis**: the single most useful change to make the Agent faster or more reliable next time — a recommendation, not a recap. Where to read them: * **Runs List** — each row carries a badge: **Success**, **Issues** (colored by worst severity), **Evaluating…**, or **N/A**. Filter the list with the **Issues** dropdown (All issues / Critical / Medium / Low). * **Run detail** — the evaluation banner opens the **Run Evaluation Analysis** dialog with the per-criterion breakdown and reasons. * **The Agent's Evaluations tab** — issue rates across Runs ("Issues found", Today / Last 7 days) with a per-criterion breakdown, scopable to a specific revision, plus **Issues analysis**, an AI summary of recurring problems across Runs. **Failed and stopped Runs are not evaluated.** The evaluation is a verdict on completed work, not a diagnosis of crashes. A Failed Run's record is its transcript and error — start there, and see [Retries and Failures](/user-guide/reliability/retries-and-failures) for the triage path. Two practices make Evaluations dramatically more useful: * **Write measurable success criteria into the AOP.** The Agent specific evaluations are derived from it. "Reply to every review in its own language, using the matching template" derives a sharp Pass/Fail question; "handle reviews appropriately" derives nothing testable. This is the same advice as [Writing Effective AOPs](/best-practices/writing-effective-aops), paying off a second time. * **Curate the Agent specific evaluations for Agents you rely on.** They are editable via the API, MCP tools, or the CLI (`duvo agents eval-rubrics list | add | update | remove`) — each is a short title plus a one-to-two sentence Pass condition, at most five per Revision. Replace a vague generated question with the check your business actually cares about. You can always open evaluations on your own Runs. Reading a teammate's Run evaluation requires the team Owner or Administrator role — factor that into who does the reviewing. ## Turn Evaluations into Alerts By default, evaluations are pull, not push: nobody is emailed when a Run fails its checks. Two opt-in switches change that: On the Agent's **Evaluations** tab, enable **"Email me when Runs on this Agent fail evaluations"**. Do this for every production Agent whose output matters. Under **Settings → Notifications**, the **Evaluation failure emails** card controls delivery: immediately, every 6 hours, daily, or weekly. Choose the digest cadences for high-volume Agents so one bad pattern doesn't become forty emails. For programmatic monitoring, the public API exposes the same signal: the Runs list endpoint supports `has_issues=true` and `issue_severity=critical|medium|low` filters, `getRunEvaluation` returns the full per-criterion result, and `getEvalScores` returns issue counts and flag distribution over a time window — enough to chart a trend or wire a check into your own monitoring. See [Monitoring Runs via API](/user-guide/running-assignments/api-monitoring). ## Outputs Must Outlive the Sandbox The files a Run produces live in its sandbox workspace, and the sandbox is reclaimed roughly **12 hours** after the Run. Until then you can browse and **download them as a ZIP** from the Run detail; after that, the Files panel reports the sandbox no longer exists and the files are gone. The transcript and evaluation survive; loose files do not. The rule that follows: **a deliverable is not delivered until it's in a system of record.** Make the AOP's final steps write the output somewhere durable through a Connection — the spreadsheet, the drive folder, the ERP, the email — and then record *where it went*: * **Give every item a stable identifier** (order number, ticket ID, invoice number) and use it in everything the Agent writes to external systems, so a result can be traced back to its Run and Case. * **Have the Agent state its outputs at the end of the Run**: what was written, to which system, under which identifier. For Queue work, into the Case data; otherwise, as the Run's closing message. ## The Case Is the Audit Trail for Queue Work For Agents working a [Queue](/user-guide/assignment-features/case-queue), the Case is the record with the longest memory. Its **Activity** timeline records every event — created, updated, completed, failed, postponed, re-processed, priority changes, manual interventions, and every approval requested and resolved — and each Run appears as a card carrying its status, trigger, timestamps, and evaluation verdict. When someone asks "why was this order handled that way?", the Case answers without opening a single transcript. Cases also get their own evaluation: when a Case settles as completed or failed (including manual settlements), it is judged against outcome-oriented Pass/Fail criteria derived from the AOPs of every Agent wired to the Queue — did the Case end with the required fields, the correct final decision, and the settlement those AOPs call for. The verdict appears in a **Case evaluation** panel on the Case detail, each criterion with the judge's reason, always against the criteria that were in force when the Case ran. That record is only as good as what the Agent writes into it. Update the Case data with the structured results of each stage — the fields checked, the values computed, the decision taken — not just a final status. [Designing Work Around Queues and Cases](/best-practices/queues-and-cases) covers the settlement rules; the audit angle is the same discipline with a second payoff. ## Know How You'll Hear About Problems Be deliberate about your alerting surface, because the defaults are quieter than most teams assume: | Signal | How it reaches you | | --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | A Run fails its evaluation | Email — **only if you opted in** (see above) | | A Run ends in Failed | Nothing is sent. Check the Runs List, or watch `run_failed` via an API webhook | | A scheduled Run is skipped for a missing Connection | Email to the schedule's owner on each skip | | Skips keep repeating | The schedule or trigger is **auto-paused** (after 5 missed Runs over 5+ days, or 20 missed Runs) and the owner gets an auto-paused email | | A Run started from Slack or Teams finishes or fails | A reply in the originating thread — **only** for Runs that started there | Two consequences worth designing for: * **Scheduled and API Runs fail silently by default.** If a failure must page someone, either opt into evaluation emails, poll the API, register a `webhook_url` when starting Runs via the API (`run_completed` / `run_failed` / `run_interrupted` events), or make notification part of the work: give the Agent a Slack or email Connection and an AOP step to report failures to your ops channel. * **A "quiet week" can mean an auto-paused schedule.** The [circuit breaker](/best-practices/reliable-connections) protects you from infinite skips, but the resume is manual — reconnect, then switch the schedule back on. For the human rhythm around all of this in an Agent's early days — daily checks, spot-checking outputs, anomaly response — see [Monitor the First Week](/user-guide/getting-started/monitor-first-week). For the compliance-grade record of who did what in the workspace, see the [Audit Log](/user-guide/security/audit-log). ## Related Statuses, filters, evaluation badges, and severity colors. Polling, webhooks, and evaluation filters for programmatic oversight. The daily-check rhythm for a freshly promoted Agent. The workspace-level record of actor, builder, and Run events.