How to Test an AI Voice Agent Before You Buy: a Listening Checklist

What does it mean to test an AI voice agent before you buy?

Testing an AI voice agent before you buy means holding a real conversation with it yourself, in the language your callers use, and deliberately doing the things that trip agents up: interrupting, pausing, mumbling, changing your mind, asking whether it is a person and asking for a human. You are not checking whether it can succeed on a clean request. Every demo can do that. You are checking how it behaves when the call goes the way real calls go.

That is a different job from the one a vendor does before launch. If you are building or running your own agent, see our guide on how to test an AI voice agent before it talks to customers, which covers prohibitions, adversarial calls over the real phone line and the release gate. This guide is for the buyer: you have a shortlist, you have ten minutes per vendor, and you want to know which agents are worth an hour of your time.

It works because the voice is the one part of the product you can judge without anyone's help. Integration depth, data location and contract terms need documents and questions. Whether the agent waits for you to finish, hears you on a bad line and reads your booking back correctly is audible within a few minutes.

Why test before any sales call?

Because a sales call is the vendor's best version of the product, narrated. A presenter who knows the script asks well-formed questions, at a sensible pace, on a good line, and the agent does what it was built to do. You learn how the product sounds when everything goes right. You learn very little about what your callers will hear when it does not.

Three habits save the most time:

1. Hear it yourself before you give anyone an hour. If the agent fails on speed or interruption in your own test, no slide deck changes that, and you can drop the vendor without a meeting.
2. Test in your language, with your accent. An agent that is fluent in one language can still stumble over local surnames, street names or the way your callers say a date. If your callers speak more than one language, test each one.
3. Test your own scenario. Write down the three things your callers ask most often, and the one call your team dreads, and put those to the agent instead of whatever the demo suggests.

Prefer vendors who let you do this without a gate: a live demo in the browser you can start without signing up, or a public number you can ring. A vendor who only lets you hear the agent after a discovery call is asking for your time before showing you the one thing you can judge on your own.

Which kind of demo tells you the most?

Demos differ in how much of the agent you actually hear and in who controls the conversation. The more of the conversation you control, the more the demo tells you.

A recording is the weakest evidence on the list for a simple reason: it was selected. There is nothing wrong with a vendor publishing one, but it shows that the agent can have a good call, which you had already assumed.

What should you listen for on a test call?

Seven checks, each one something you do on the call, something you listen for and a failure you will recognise when you hear it. Run them in any order; the script further down fits all seven into one call.

1. Response speed

Ask something easy first (“What time do you open on Saturday?”), then something that needs the agent to check a calendar or a record. You are listening to the silence after you stop talking.

For calibration, it helps to know how fast people are. Conversation research finds that the gaps between turns in human conversation are short, of the order of 200 ms (Levinson and Torreira, 2015). Listeners also read meaning into longer pauses: in a study of 195 responses from recorded telephone calls, only once the gap reached 700 ms or more did refusals and other unwelcome answers clearly outnumber acceptances (Kendrick and Torreira, 2015). Both are measurements of people talking to people, not a specification for software, and nobody should judge an agent with a stopwatch. They are a reminder that callers notice pauses and hear something in them.

What a pass sounds like: easy answers come back without a noticeable wait, and when the agent has to look something up it says so (“Let me check Thursday for you”) instead of going silent. What a failure sounds like: a pause long enough that you wonder whether the line dropped, or one stock filler phrase in front of every answer, which is the same delay with words in it.

2. Interrupting the agent mid-sentence

Wait for the agent to start a longer answer, an explanation of the cancellation policy or a list of options, and talk over it: “Sorry, not Tuesday, I meant Wednesday.” Real callers interrupt all the time, usually to correct something.

Listen for two things. Does it stop within a moment of you starting to speak? And is its next sentence an answer to what you said, or does it pick up where it left off? An agent that stops but then repeats its previous sentence from the top has heard that you spoke, not what you said. The worst version talks over you to the end of its sentence and then asks you to repeat yourself.

3. Clarity and accent in your language

Speak the language your callers use, at the speed they use it, with the details they give: a surname like Featherstonehaugh, an address in Leamington Spa, a postcode, a date said the British way (“the fourteenth of November”), a price in pounds. Ask the agent to repeat some of it back. Listen for names and places pronounced properly, dates in your format rather than a foreign one, and numbers that come back the way you said them. If your callers switch language, switch mid-call and see what happens.

Then ask two or three colleagues with different accents to make the same call. Speech recognition accuracy is not the same for every speaker. In a 2020 PNAS study, five commercial speech recognition systems averaged a word error rate of 0.35 for Black speakers and 0.19 for white speakers. Those figures describe US English and the systems as they stood in 2020, not any product today. The lesson still holds: a test made only in your own voice tells you how the agent handles your voice, not your callers'.

4. Unclear or noisy speech

Make one call the way a busy caller does: from the street, from a car on speaker, half distracted. Trail off in the middle of a sentence. Mumble the house number. Say “erm” and pause.

What a pass sounds like: the agent asks you to repeat or confirm the specific thing it is unsure of (“Was that 14 or 40?”). What a failure sounds like: it guesses, confidently, and builds the rest of the conversation on the guess. Watch for the opposite failure too, an agent that treats background noise or a passing voice as you speaking, stops mid-sentence and asks what you said when you said nothing. And note whether it ever gets out of “Sorry, I didn't catch that”: three in a row is a loop.

5. Whether the booking is correct

Book something specific. Give a day, a date and a time, then change one of them (“Actually, make that the fifteenth, not the fourteenth”), and spell your surname.

Listen for the read-back before the agent confirms: the day of the week, the date, the time and the name, all as you last gave them. A read-back that still has the fourteenth means your correction was lost. A booking confirmed without any read-back means nobody, the agent included, checked.

Then ask to see the result. What the agent says and what it writes are different things: an agent that tells you that you are booked and writes nothing looks exactly like a working one until the customer turns up. On a call with the vendor, ask them to open the calendar or CRM record your test created. Why the two diverge is covered in why AI phone agents book the wrong appointment, and the levels of booking depth vendors sell in taking a message vs actually booking.

6. AI disclosure at the start

Listen to the very first sentence. Then, later in the call, ask directly: “Am I talking to a real person?”

In the EU this is not a matter of style. Article 50(1) of the EU AI Act requires that AI systems intended to interact directly with people are designed so that the people concerned “are informed that they are interacting with an AI system”, unless that is obvious from the circumstances. Article 50(5) sets the timing: the information is given “in a clear and distinguishable manner at the latest at the time of the first interaction”. Under Article 113 the Regulation applies from 2 August 2026, and the Digital Omnibus on AI that amended the Act in July 2026 (Regulation (EU) 2026/1744) changed other parts of Article 50 but not these two paragraphs. This is general information, not legal advice; the wider picture is in must AI callers identify themselves?

What a pass sounds like: an opening that says plainly you are speaking to an AI assistant, for example “Hello, you're through to Jessica, the practice's AI assistant”, and a straight yes when you ask. What a failure sounds like: no mention of AI, a human name and nothing else, or a deflection (“I'm here to help with your booking”) in answer to the direct question. How disclosure can sound natural is on our page is this really AI?

7. Handover to a human

Ask for a person. Do it once politely, and once the way an annoyed caller does (“I just want to speak to someone”). Separately, raise something the agent should not handle alone: a complaint, or a question only your staff can answer.

Listen for a concrete route: a transfer, a callback with a time, or a message to a named team, plus a short summary so the caller does not have to start again. In a browser demo there may be nobody to transfer to; then the test is whether the agent says clearly what would happen next.

The failure is the loop. The US Consumer Financial Protection Bureau, reviewing customer-service chatbots in banking, recorded complaints from customers “getting stuck in ‘doom loops’ with chatbots” and described automated responses that left people “without an offramp to a human customer service representative”. A voice agent that answers “I can help you with that” to the third request for a person is the same failure, on the phone.

What does a ten-minute test call sound like?

One call can cover all seven checks. Here is a version for a business that takes bookings, say a heating engineer in Leeds:

1. Listen to the opening. Does it say it is an AI? (check 6)
2. Ask: “What time do you open on Saturday?” Note the gap. (check 1)
3. Ask: “Have you got anything on Thursday afternoon for a boiler service?” Is the lookup announced? (check 1)
4. While it lists the options, cut in: “Sorry, could we do Friday morning instead?” (check 2)
5. Give your details with local colour: “It's Siobhan Featherstonehaugh, 14 Kirkstall Road.” Mumble the house number. (checks 3 and 4)
6. Change the date once more, then listen to the read-back before it confirms. (check 5)
7. Ask: “Sorry, am I talking to a real person?” (check 6)
8. Finish with: “Actually, I'd rather speak to someone about the quote.” (check 7)

Then make the same call from somewhere noisy, and once more in the second language your callers use, if there is one.

How do you compare vendors after the test calls?

Use the same script with every vendor, write down pass, unsure or fail for each of the seven checks straight after the call, and compare the notes, not your memory of which voice you liked.

Notice what is not on the list: how pleasant the voice sounds. That is deliberate. Taste in voices differs, and on a clean call most of them sound fine. The seven checks are the things that change what happens to the caller: whether they were heard, whether the booking is right, whether they knew it was an AI and whether they could reach a person. Judge those, and let the voice speak for itself.

What can a demo not tell you?

A test call settles how the agent sounds and behaves. It does not settle four things, and these belong on the sales call you now know is worth having:

• Whether it writes into your systems. Ask to see a record created by a test call in your own calendar or CRM, and ask what the agent connects to.
• How the vendor tests changes before they reach your number. Ask them to describe the step between changing the agent and the change going live. The reasoning is on our page about how an agent gets verified before it speaks to a customer.
• Where the call data goes. Ask for the sub-processor list with the country each one processes in; the questions that work are in where does AI call data actually go?
• Who runs it after launch. A self-serve builder leaves tuning and monitoring with you; a managed service keeps them with the vendor. Our vendor checklist lists the questions to ask.

How do you test AInora?

The same way. Our demo assistant sits on the homepage: press the mic and talk to it in your browser, with no phone and no sign-up. A demo conversation runs for up to five minutes, enough for most of the checklist. On the German, Polish and Italian versions of the site, the same demo speaks German, Polish and Italian.

Be clear about what it is: a general demo assistant, not your business. It will show you speed, interruption, language and disclosure. It cannot show you a booking landing in your calendar or a transfer to your team, because it is connected to neither. Those parts come after a short scoping call, where the agent is built on your own call flow. AInora is a managed service, with no self-serve sign-up and no published price; each deployment is scoped individually. It is bought to take the calls and follow-ups a team cannot get to: in production the agents say they are AI on every call, book into the client's calendar or CRM, work in 90+ languages, and hand the cases that need a person to the team with a written briefing.

Hear it and judge for yourself. If it passes your checks, send us your call flow and we will run your scenario with you.