How to Test an AI Voice Agent Before It Talks to Customers

What does it actually mean to test an AI voice agent?

Testing an AI voice agent means placing a fixed set of deliberately difficult telephone calls against the agent, over the same telephony path a real customer would use, and scoring each call against rules that were written down before anyone decided what the agent should say. The rules are mostly prohibitions: what the agent must never confirm, never quote, never promise, and never continue past. The set is re-run in full before any change reaches a live number.

That is the entire discipline. Everything below is why each part of it is there, and what quietly breaks when a part is missing.

It is worth being precise about what it is not. It is not a person ringing the number, asking three sensible questions and being pleased with the answers. That is a demonstration. A demonstration proves the agent can succeed. Verification is the deliberate attempt to make it fail, on the specific calls where failure would cost the business something, plus the record showing it did not.

How the agent was verified is not a question that tends to come up in a vendor conversation, because buyers are rarely told it is one they can ask. The evaluation frameworks in circulation, our own vendor evaluation checklist and evaluation guide, spend most of their questions on languages, integrations, contracts and uptime. Very few ask what stands between a change to the agent and your customers hearing it. That gap is the subject of this article.

Why is calling the number a few times not a test?

Three reasons, and they compound.

The person testing knows what the agent can do. Somebody who has read the script asks well-formed questions in a sensible order, because they are trying to be understood. Real callers are not. They are sorting out a thing while doing something else. The caller who breaks an agent is the one who says: it is for my mum, well no, it is for me, Tuesday if you have got it, actually make it Thursday, and does the first one cost anything. Nobody phrases a test call like that on purpose.

The happy path is the part that was easy to build. An agent that handles a clean booking request from a cooperative caller is close to a solved problem, which is why it is the call a demonstration tends to be built around. The calls that decide whether the system is an asset or a liability are the ones where the caller wants something the business does not do, where two facts in the conversation contradict each other, or where the right answer is to stop and fetch a human.

A conversational system does not answer the same way twice. Work measuring repeated identical runs of the same language model under settings that are supposed to be deterministic reported "accuracy variations up to 15% across naturally occurring runs with a gap of best possible performance to worst possible performance up to 70%", and found that none of the models tested delivered repeatable accuracy across all tasks. One successful call is a single sample from a distribution. It proves the distribution contains a success. It says nothing about how much of it does.

What gets written down before anyone writes what the agent says?

This is the step most teams skip, and it is the one that makes everything downstream possible.

A conversation has no single correct output. There is no string you can compare the agent's reply against, because a dozen different replies are all fine and the thirteenth is a disaster for reasons that have nothing to do with wording. This is a named problem in software engineering rather than a new problem in AI. It is the oracle problem: a test oracle decides whether an execution revealed a fault, usually by comparing observed output to expected output, and for some programs predicting the correct output in advance is, as the standard survey of the field puts it, "non-trivial and error-prone". The established response is metamorphic testing: rather than checking one output against a known answer, you check that defined relations between inputs and outputs continue to hold.

A voice agent sits squarely in that territory, and the practical form those relations take is a list of invariants written as prohibitions. Never confirm a booking that has not been written into the diary. Never quote a price that does not exist in the price list. Never name a service the business does not offer. Never keep talking once the caller has asked for a person. Never collect card details on a line that is being recorded. Never proceed without telling the caller they are speaking to an AI system, which since 2 August 2026 has been an obligation under Article 50 of the EU AI Act rather than a matter of taste. What that obligation looks like in a natural opening line is on our page on telling callers they are speaking to AI.

Written first, that list is a specification and the tests are derived from it. Written afterwards, it is a description of what the agent already happens to do, which is a different artefact and is not a test of anything.

The regulator phrases it the same way, which is a useful sanity check. The EU AI Act is explicit about the order for high-risk systems: "Testing shall be carried out against prior defined metrics and probabilistic thresholds", performed at any time throughout the development process and, in any event, before the system is placed on the market or put into service. Most front-desk agents are not high-risk systems under the Act, so this is not a duty that binds a dental practice or a garage. It is, though, the clearest published statement anywhere of what the word "tested" is supposed to mean: the thresholds exist before the test does. Which obligations actually attach to a voice agent, and which do not, is covered in our EU AI Act compliance checklist.

Which calls actually break an agent?

Not the ones in the demo. The catalogue below is the shape of an adversarial suite: each row is a scenario, what it is probing, and the failure it is there to expose.

The important property of that list is where it comes from. A generic scenario library shipped with a platform tests a generic business. The scenarios that matter are derived from the policies the front desk already follows: who gets seen the same day, what is never quoted over the phone, which requests always go to a named person, what happens when somebody wants to cancel inside the notice period. The edge cases live in the gap between the written policy and what the experienced person on the desk actually does, and finding that gap is most of the work.

This is a different exercise from security testing, and both are needed. Probing for prompt injection, data exfiltration and social engineering is its own discipline, covered in our voice AI security audit guide and in the controls listed on our security page. The suite described here is about whether the agent does the business's job correctly under pressure, not about whether an attacker can bend it.

Why does a test have to be a phone call and not a text transcript?

Because the failures live in the speech, and a text harness cannot fail the way a call fails.

Most agent testing that exists is text testing: the scenario is typed in, the reply is read back. It is fast and cheap, and it skips the layer where the expensive damage happens. The size of that gap has its own research literature. SpokenWOZ, a benchmark built precisely because existing dialogue datasets were written by annotators and therefore left "a gap between academic research and real-world spoken conversation scenarios", reports a dialogue state tracking model reaching 25.65% joint goal accuracy and an end-to-end model completing the user's request in 52.1% of dialogues on genuinely spoken task-oriented calls. There is an entire shared task, the speech-aware track of DSTC11, whose stated purpose is to investigate "the performance gap between the written and spoken forms of input".

Recognition accuracy is also not uniform across the people who will actually ring you. In a 2020 PNAS study, Racial disparities in automated speech recognition, five commercial speech recognition systems averaged a word error rate of 0.35 for Black speakers against 0.19 for white speakers on interview audio matched for age and gender, and on the authors' own threshold for an unusable transcript, 23% of the audio from Black speakers fell below it against 1.6% of the audio from white speakers. Those figures describe US English, five US cities and the systems as they stood in 2020, so they are not a measurement of any system today. The structural lesson survives them intact: a test set recorded by your own team, in your own accent, on your own good handset, measures your team. It does not measure your callers. The same problem shows up in multilingual deployments, which is why a multilingual agent has to be tested per language rather than once in English and assumed.

Then there is everything that exists only on a real line. How long the silence runs before the agent replies. Whether it hears the caller who begins speaking during its second sentence. Whether a transfer to a human connects or drops into dead air. Whether the codec a mobile network picked mangles a postcode. Whether keypad digits arrive at all. Whether the number even rings, which across borders is its own subject and is covered on our multi-country telephony compliance page. None of that appears in a text harness, and all of it is obvious to a customer inside ten seconds.

How do you grade a conversation that has no single right answer?

In two layers, and the boring layer matters more.

Layer one is deterministic assertions about side effects. After the call, is there a row in the diary, with the right date, against the right record. Was the transfer actually placed. Was the disclosure spoken before anything was collected. Was the note written back to the system of record. These are true or false, they need no judgement, and they catch the failures that cost real money, because an agent that says "you are booked for Thursday" and writes nothing is indistinguishable from a working one until Thursday. Where the write-back lands is a design decision in itself, which is why what the agent is connected to is part of the test and not a separate project.

Layer two is judgement about the transcript, scored against the prohibition list rather than against a model answer. Did it quote a price that is not in the price list. Did it commit the business to a service it does not offer. Did it keep going after a request for a person. Automated scoring can make the first pass over volume, and it is not the release decision: a person reads a sample, because an automatic grader shares failure modes with the thing it is grading, and a grader that has been quietly agreeing with a broken agent for three weeks is worse than no grader.

What you do not grade on is whether the agent sounded good. Sounding good is the property this technology is best at, and the property least correlated with whether the business was harmed. Optimising for it is one of the recurring mistakes businesses make with voice AI, along with measuring nothing at all after launch.

What happens between changing the agent and the change going live?

This is the question that separates verification from good intentions.

An agent is not built once. Somebody adjusts an instruction on a Tuesday because a caller got confused on Monday, and that adjustment changes the system's behaviour everywhere, not only at the point it was aimed. This is the oldest finding in production machine learning and it long predates voice agents. Google's engineers named it in 2015: "We refer to this here as the CACE principle: Changing Anything Changes Everything", and they were explicit that it applies not only to input signals but to hyper-parameters, learning settings, sampling methods, data selection "and essentially every other possible tweak". A prompt edit is a tweak. The agent that got better at the thing you were watching may have got worse at four things you were not.

Google researchers later published a production-readiness rubric of 28 tests, and one of them states the release gate in a single sentence. Before a model affects real traffic, "an automated system needs to inspect it and verify that its quality is sufficient; that system must either bless the model or veto it, terminating its entry to the production environment". Between "we changed the agent" and "the change is live on the number" there has to be a step that is capable of saying no.

NIST's AI Risk Management Framework describes the same structure at organisational level: test, evaluation, verification and validation tasks run across the whole lifecycle rather than at the end of the build, covering design, development, deployment and ongoing operations, and, pointedly, "Ideally, AI actors carrying out verification and validation tasks are distinct from those who perform test and evaluation actions." Its generative AI profile adds red-teaming, defined as "a structured testing exercise used to probe an AI system to find flaws and vulnerabilities", and notes that it can be performed before or after a system reaches the broader public, while concentrating its own guidance on the pre-deployment case. Neither document treats verification as a launch-day formality: it has to happen before exposure and keep happening after it.

Without that gate, every change goes straight to the vendor's customers, and their customers are the test set. That is how a system which was fine in March quietly becomes a liability by June, and why nobody ever finds out from a report. They find out from a complaint.

What should you ask a vendor about how they test?

One question does most of the work. Ask them to describe, step by step, what happens between the moment they change your agent and the moment the change is live on your number.

If the answer is a single sentence with no step in it, there is no verification. Everything after that is detail, and these six follow-ups are the detail worth having:

1. Who wrote down what the agent must never do, and when? Before the script or after it. The order tells you whether it is a specification or a description.
2. Are the tests phone calls or typed text? If text, ask what covers interruption, accents, transfers and dead air.
3. Where did the scenarios come from? Your policies, or a library that ships with the platform and knows nothing about your business.
4. When you change one thing, do you re-run everything or only the thing you changed? Only the thing you changed is not regression testing.
5. Who or what can veto a release, and has it ever? A gate that has never stopped anything is a formality.
6. How fast can you roll back, and who decides? Ask for the number of minutes and the name of the person.

There is an honest limit to what a page can prove here, ours included: you cannot verify a vendor's testing from their website. You can only make them describe the gate and notice whether the description contains anything load-bearing. The shorter, structural version of this argument is on our page about how an agent gets verified before it speaks to a customer, and the architectural question underneath it, why some agents fail on the calls that matter and others do not, is on our page on voice agent reliability. Earlier than that sit the switching checklist and how it works.

What we offer next is a working session rather than a demo. Forty-five minutes on your actual call flow: we take the policies your front desk already follows, work through the edge cases that break them, and you keep the written version at the end whether or not anything else happens between us. If that is useful, send us the flow and we will talk it through. Where an engagement follows, it is one workflow, missed calls and after hours, roughly two weeks, before anything else moves.