Evals explained: how to test an AI system before you trust it
A plain-English guide to AI evals: test sets, graders, repeated trials and why a 75 percent success rate fails more often than it sounds.
Most AI pilots are judged by impressions: a demo that looked good, a few prompts that worked. An eval replaces impressions with a repeatable test. It is the single most useful habit for anyone who intends to put an AI system in front of customers or into a business process, and it does not require a research team.
What an eval is made of
Anthropic's engineering team, in a January 9 post, defines the pieces in terms that apply well beyond coding agents [1]:
Task: a single test with defined inputs and success criteria.
Trial: one attempt at a task. Outputs vary, so you run several.
Grader: the logic that scores the result. A task can have more than one.
Transcript: the full record of a trial, including outputs and tool calls.
Outcome: the final state of the environment, such as whether a booking exists in the database.
Note the last item. For an agent that takes actions, the right question is often whether the world ended up in the correct state, not whether the text of its reply sounded right [1].
A business example: an insurance back office wants an assistant that extracts policy numbers and renewal dates from emailed documents. A task is one email with the correct answers written down. A trial is one run of the assistant on it. The grader compares the extracted fields with the answers. The transcript is the assistant's full working, which you read when a trial fails.
Three kinds of grader
Grader | Strengths | Weaknesses |
|---|---|---|
Code-based (exact match, unit tests, state checks) | Fast, cheap, objective, reproducible | Brittle to valid variations, weak for subjective work |
Model-based (an AI scores against a rubric) | Flexible, scalable, handles open-ended output | Non-deterministic, costs more, needs calibration against humans |
Human (expert review, spot checks) | Highest quality, calibrates the other two | Expensive and slow |
Anthropic recommends deterministic graders where possible, AI graders where needed, and human review to calibrate them [1]. It also recommends grading outcomes rather than rigid step sequences, since valid solutions often take different paths [1].
Why you run each task more than once
AI output varies from run to run, so one success does not establish reliability. The post distinguishes two measures: pass@k, the chance of at least one success in k attempts, and pass^k, the chance that all k attempts succeed [1]. Its example is a 75 percent per-trial success rate, which passes three trials in a row about 42 percent of the time [1].
Two ways to count success at a 75% per-attempt rate
Illustrative calculation using the definitions in the source
View data
| pass@k (at least one success) | pass^k (every attempt succeeds) | |
|---|---|---|
| k=1 | 75% | 75% |
| k=2 | 93.8% | 56.3% |
| k=3 | 98.4% | 42.2% |
| k=5 | 99.9% | 23.7% |
| k=10 | 100% | 5.6% |
Which measure matters depends on the job. If one success is enough, such as generating a draft you will pick from, pass@k is relevant. If every interaction must work, as with a customer-facing agent, pass^k is the honest number [1].
A starting plan
Anthropic's suggested path, condensed [1]:
Start small and early. Twenty to 50 tasks drawn from real failures is a good start.
Write unambiguous tasks. Two domain experts should reach the same verdict on each. A 0 percent pass rate across many trials often means the task is broken, not the system.
Balance the set. Test cases where a behavior should happen and where it should not.
Use a clean environment for every trial, so runs do not contaminate each other.
Read the transcripts to confirm graders are fair and measuring the right thing.
Watch for saturation. A suite at 100 percent gives no improvement signal and needs harder tasks.
Keep suites alive. Assign ownership and let subject-matter experts add tasks.
The post also separates capability evals, which should start with a low pass rate to give a team something to climb, from regression evals, which should sit near 100 percent and catch breakage when you change a model or prompt [1].
A reminder from real usage data
Anthropic's Economic Index report found that task success differs by context: in its sample, success was 67 percent on Claude.ai versus 49 percent in API usage, and 78 percent for personal tasks versus 61 percent for software development [2]. These are Claude's own assessments of whether it completed tasks, which the report calls directionally accurate, so they are estimates rather than audited outcomes. They illustrate the point: average performance says little about performance on your tasks.
The practical implication
Before the next AI pilot goes live, write down 20 to 50 real tasks with known-correct outcomes, run each several times, and decide in advance what pass rate is acceptable. Re-run the same set whenever the model, the prompt or the data changes.
Sources
- Demystifying evals for AI agents Anthropic Engineering, 2026-01-09. www.anthropic.com/engineering/demystifying-evals-for-ai-agen
- Anthropic Economic Index: Economic Primitives Anthropic, 2026-01-15. www.anthropic.com/research/anthropic-economic-index-january-
