How much test data an AI pilot needs depends on the decision the evidence must support. There is no universal sample size. A pilot that drafts reversible internal notes requires different coverage from an agent that changes access or communicates externally. The test set is adequate when it represents eligible work, exposes critical failure modes, and supports the next bounded permission.
Start with a coverage map rather than a target number. List workflow variants, user roles, source types, languages, tools, boundary conditions, prohibited requests, dependency failures, and recovery paths. Mark which cells carry high consequence even when they are rare.
Build the set from operating strata
Sample ordinary traffic to understand common performance, then deliberately add boundary and adverse cases. Historical frequency alone underweights new products, uncommon permissions, malicious instructions, missing evidence, and situations where the correct behavior is to abstain.
Separate unit-like cases from full journeys. A retrieval set can test whether the right source appears. A workflow set must also test authorization, model use, tool arguments, human review, destination state, and confirmation. Count and report these layers independently.
The article on company memory beyond vector search explains why source identity and policy matter to retrieval. It cannot determine how many cases another workflow needs without its data distribution and risk map.
Protect the decision from tuning leakage
Use a development set for prompt, routing, and control changes. Reserve a held-out acceptance set whose expected outcomes were defined separately. If the team repeatedly inspects and tunes against the acceptance cases, those cases become development material and should not support an independent release claim.
Measure label confidence. Some tasks have a clear source-backed answer; others require professional judgment. Ask independent reviewers to label a sample and record disagreement. More examples do not repair an undefined outcome or an inconsistent policy.
Size each critical stratum for the decision. A team may need enough cases to observe repeated behavior across major workflow variants, but it should not manufacture statistical certainty from a small pilot. Report counts, coverage, misses, uncertainty, and exclusions directly.
The confident-failure trust checklist can generate adversarial questions. It remains a hypothesis source; actual cases must reflect the agent’s permitted inputs, tools, and consequences.
Stop when evidence answers the next question
Define the acceptance and stop rules before the run. A critical unauthorized action can block the pilot regardless of total volume. Stable performance on low-risk cases may justify a narrow human-reviewed trial, while weak coverage or reviewer disagreement calls for more data or a clearer process.
Use learning curves cautiously. If additional cases keep revealing new failure classes, the set is not mature. If results stabilize across meaningful strata and reviewers agree, the team may have enough evidence for the stated permission. Neither pattern proves readiness for a broader population.
Preserve every evaluated configuration: source snapshot, dataset version, labels, model, prompt, tools, thresholds, grader, and date. The analysis of agent metrics and denominators is useful because a percentage without eligible case counts can mislead.
After deployment, sample live cases and compare them with the test distribution. Add novel incidents and near misses to a regression set. Re-evaluate when the workflow, policy, model, source, tool, population, or failure cost changes materially.
Plan the labeling workload as part of the pilot. Record who can define expected outcomes, how disputes are resolved, and how long evidence remains valid. If expert review is scarce, prioritize high-consequence strata and narrow the workflow instead of filling the set with easy examples. The budget should include secure preparation, redaction, review, adjudication, storage, and later deletion.
Frequently Asked Questions
No. The required set depends on workflow diversity, failure consequence, label quality, subgroup coverage, and the decision being made.
It reflects common traffic but may miss rare critical failures, new users, attacks, tool outages, and cases that require abstention.
No. Keep a held-out decision set so tuning choices are not evaluated only on cases the team has already studied.
Add cases when new failure types, weak subgroups, changed policies, model updates, tools, or production drift alter the decision boundary.
For a risk-weighted coverage map, held-out set, labeling protocol, and evidence-based stop rule, use AI governance and agent audit. The appropriate volume remains specific to the workflow and decision.