Test your agent before it talks to a customer

Why this matters

Traditional software fails loudly. A null reference throws, a test goes red, a pager fires. Agents fail quietly and plausibly. They answer confidently with the wrong account balance, they refuse a request they should have handled, they take an action against the wrong record. Nothing throws. The customer just gets a worse answer than a human would have given, and you find out in a QBR six weeks later.

That asymmetry is why agent evaluation cannot be an afterthought. You are not testing whether the code runs. You are testing whether the judgment holds.

Prerequisites

• A deployed agent (Agentforce, or Claude against your own orchestration layer) with at least one production-shaped use case
• Read access to a year of real support tickets, chat transcripts, or call summaries
• A staging org or sandbox with representative data, not empty test records
• Somewhere to store eval results over time: a Data Cloud DMO, a Snowflake table, or at minimum a versioned spreadsheet

Step 1: Build the eval set from tickets you actually received

Do not write test cases from imagination. Pull 150 to 300 real customer interactions from the last twelve months and label each one with the outcome a good human agent produced. Weight the sample toward the messy end: partial information, two questions in one message, the customer who is angry about something unrelated.

Three buckets matter more than volume:

• Handle: the agent should resolve this without a human
• Escalate: the agent should recognize its limits and hand off
• Refuse: the agent should decline, because the request is out of policy or the data is not available

Teams overfit to the first bucket and then act surprised when the agent confidently handles something it should have escalated.

Step 2: Score four dimensions, not one

A single pass or fail hides the failure mode you need to see.

Retrieval accuracy. Did the agent pull the right records? Check this independently of the answer. An agent can retrieve the wrong account and still produce a plausible sentence.

Action correctness. If the agent wrote anything (case update, field change, refund, email), did it write the right thing to the right object? This is the dimension with real blast radius.

Boundary behavior. Given an escalate or refuse case, did it escalate or refuse? Measure this separately. An agent that scores well on handle cases and poorly here is the dangerous configuration.

Tone and length. Cheapest to fix, easiest to measure, and the one executives notice first in a demo.

Step 3: Use a model as the grader, then audit the grader

Grading 300 cases by hand once is fine. Grading them on every prompt change is not, so use Claude as the judge with an explicit rubric per dimension and a required justification for each score.

Then audit it. Hand-grade a random 20 of the graded cases yourself and compare. If model and human agree on fewer than about 90 percent, your rubric is ambiguous, not your agent. Fix the rubric and rerun. Skipping this audit is how teams end up trusting an eval number that measures nothing.

Step 4: Put the eval in CI, gated on the prompt

Agent behavior changes when anyone edits an instruction, adds a knowledge article, swaps a model version, or changes a retrieval filter. All of those are code changes in every respect except that they usually bypass code review.

Run the eval set on every change to the prompt, the tool definitions, or the retrieval configuration. Fail the build on a regression in boundary behavior or action correctness. Warn, but do not block, on tone. Store every run so you can answer the question that always gets asked later: did this get worse, and when.

Step 5: Set thresholds you will actually enforce

Pick numbers before you see results, otherwise you will rationalize whatever you get. A reasonable starting bar for a customer-facing agent:

• Action correctness at or near 100 percent. Anything that writes to a record and gets it wrong is not shippable.
• Boundary behavior above 95 percent, weighted toward escalation. Missing an escalation costs more than an unnecessary one.
• Retrieval accuracy above 95 percent.
• Tone above 85 percent, because taste is genuinely subjective and chasing the last points is not worth the prompt complexity.

Gotchas

Your eval set will go stale. Products change, policies change, the ticket mix changes. Refresh a quarter of it every quarter or it quietly stops representing your business.

Sandbox data lies. Agents that perform well against clean seeded records fall over on production data with duplicate contacts, half-populated fields, and the account someone created in 2019 with a typo in the name. Test against a sanitized production copy.

Do not tune on the test set. Keep a holdout you never look at while iterating. If you tune against everything, you learn your eval set rather than improving your agent.

When this breaks

The pattern strains in two places. First, multi-turn conversations, where the failure is often in turn four and depends on state from turn two. Single-turn eval cases miss this entirely, so you need conversation-level cases with a simulated customer, which are meaningfully more work to build and maintain.

Second, agents with real side effects. You cannot run 300 eval cases against an agent that issues refunds without either a mocked action layer or a genuinely isolated environment. Build the mock early. Teams that defer it end up testing in production, which is exactly the thing the eval was supposed to prevent.

Where this leaves you

An eval harness is the difference between shipping an agent and hoping. It is also the artifact that makes the governance conversation tractable, because it turns "is this agent safe" into a number your risk team can read.

The build is a week of work for the first agent and a day for every one after, since the harness is reusable and only the eval set is domain-specific. That is a cheap insurance premium against the failure mode where nobody notices for six weeks.