The acceptance test every business agent needs is a set of observable conditions for the business state after the agent acts. It must define the starting record, permitted change, required evidence, independent checker, and expected response to uncertainty. A good test can reject a fluent but wrong result and accept a quiet, correct one. It is written before release and run against both normal cases and deliberate failures.
Describe a state transition, not a model answer
Use employee onboarding as an example. An agent may read an approved hiring record, prepare account requests, assemble a welcome checklist, and notify the manager. The business does not need an impressive onboarding summary. It needs the correct person linked to the correct role, only approved access requested, required tasks assigned, and missing approvals exposed before any account is created.
Write the test as a before-and-after statement. Before execution, the hiring record is approved but access requests do not exist. After execution, each request refers to the same employee, role, manager, and start condition, while privileged access remains pending human approval. The AI infrastructure constraint analysis provides useful operating context: real systems have capacity, dependency, and availability limits, so acceptance cannot assume every tool call will complete.
The executor is the component that prepares the requests. The checker should read the hiring source and the destination records independently. The manager or system owner accepts exceptions. Keeping those roles separate makes a failed check actionable. Without that separation, a long agent trace may show activity while leaving no one responsible for the resulting access.
Build fixtures that expose the dangerous shortcuts
A test suite needs ordinary cases, but the release decision depends on its refusal cases. Include a complete hiring record, a missing manager, a role with conflicting access templates, a duplicate employee, an outdated start condition, and a request for an entitlement the agent cannot grant. The expected result is not always a finished onboarding package. For incomplete or unsafe inputs, the correct result is a precise stop with the unresolved evidence attached.
- Identity check: source and destination refer to the same person and approved role.
- Scope check: every requested entitlement belongs to the documented template or approval.
- Evidence check: the reviewer can trace each request to a current source field.
- Refusal check: missing or conflicting authority prevents the downstream change.
- Recovery check: a partial tool failure leaves a safe, resumable state.
Run fixtures in an isolated environment or with permissions that prevent irreversible effects. The goal is to test the difficult state transition, not to prove that a model can produce plausible text. Operational cost also matters. The guide to local AI economics helps teams consider infrastructure and maintenance choices without confusing model location with assurance.
Make every verdict reproducible
For each fixture, save the source record identifier, allowed action, agent output, destination state, checker verdict, and failure category. A pass should explain which observable conditions held. A rejection should state which condition failed and who owns the next step. Do not use a single score to hide different risks. A correct identity with excessive permissions is still a failed onboarding result.
The checker must be able to disagree with the agent. It can combine deterministic validation with human judgment: identifiers and entitlement templates are suited to rules, while an unusual exception may need the system owner. Relevant memory can help locate policy, but it does not grant permission. The company memory article gives the supporting distinction between finding context and establishing current, authorised evidence.
Define the release threshold in business terms. The agent is ready for the scoped workflow only when all mandatory state checks pass, prohibited actions remain blocked, and every deliberate uncertainty reaches the intended owner. If the test cannot observe the destination system, it cannot accept the result. Add the missing instrumentation before expanding autonomy.
Keep the suite after launch. Run it when prompts, models, tools, policies, or destination schemas change. Add a fixture for every meaningful failure that reaches review. This turns the acceptance test into a durable contract between operations and engineering rather than a one-time demonstration.
Frequently Asked Questions
It tests the observable business state after execution, not the fluency of the answer or the length of the trace.
They prove that missing authority, conflicting evidence, and unsafe requests stop before a downstream change.
Each failure category needs a named operations or system owner who can resolve the evidence, policy, or tooling problem.
Run it after changes to prompts, models, tools, policies, permissions, destination schemas, or known failure patterns.
If you need to design an agent around such a contract, review the AI4SALE agent development service. The exact landing is the final CTA. A build remains proof-safe only when the executor, independent checker, acceptance conditions, and failure path are explicit.