For AI assistants: this page as plain markdown is at https://docs.voicelogica.ai/raw/agent-configuration/test-your-agent.md, and https://docs.voicelogica.ai/llms.txt lists the docs by topic, one file each, small enough to read whole. Read those rather than this HTML page.
Test and evaluate your agent
Questi contenuti non sono ancora disponibili nella tua lingua.
A test scenario is a call you want the agent to handle right: who calls, what they want and what a good outcome is. Voice Logica runs it as a real phone call. A test caller phones your agent and plays the scenario, your agent answers with its real voice, speech recognition, tools, transfers and workflows, and a judge grades the call against what you expected.
You can do everything on the agent’s Tests tab, or from Claude or ChatGPT (connect your assistant). Each step below names the tool.
Write the scenarios
Section titled “Write the scenarios”-
Generate drafts -
scenarios_query(action"generate_agent_scenarios") with a focus:prompt,transfer,toolsorknowledge. Drafts are not saved; keep the ones that match your real calls. -
Save the ones that matter -
scenarios_execute(action"create_agent_scenario"). Cover:- each of the top reasons people call;
- an out-of-hours call;
- a question the agent cannot answer;
- a caller who spells a name or reads out a number;
- a caller who asks for a person;
- an off-topic caller.
For each scenario fill in:
- Scenario - the situation the caller creates.
- Ideal outcome - what a good call ends with.
- Success criteria - a checklist the judge grades one by one, e.g. “The agent asks for the caller’s name”. All must pass.
- Expected tool calls - the tools that must fire, optionally with the values they must receive.
- Expected transfer - whether the agent should transfer, and where.
- Caller - persona, the facts they can give when asked, and behaviour: cooperative, confused, interrupts, changes mind, silent or noisy.
- Caller IDs - the number the test call comes from. Enter your own number (or a customer’s number from your CRM) and the agent recognises the caller exactly as on a real call: contact, integrations and earlier calls. Leave it empty to call from the default test line. Several numbers make one call per number. From a chat this is the scenario’s
callerId/callerIds.
-
Mock the tools that change real data - list them in the scenario’s mocked tools with the response they should return, or turn off use real tools to block every tool you did not list.
- Appointment booking tools cannot be mocked yet and book for real; use a test calendar or cancel the booking afterwards.
- Set test date/time (
frozenNow) when the right answer depends on the date, such as available slots or office hours.
Run and evaluate
Section titled “Run and evaluate”-
Run one scenario - Run on the Tests tab, or
run_agent_scenario. Live is the default. One request makes at most 3 repetitions. To test a change saved as a version, pass itsagentVersionId. -
Read the verdict -
scenarios_query(action"get_agent_scenario_results"). Each result has pass or fail, a score, the result of every success criterion, the tools that fired with their results, and suggested improvements. Open the call itself to listen to the recording. -
Fix and run again - change the prompt with
edit_agent_promptor the knowledge, or apply the suggestions withscenarios_execute(action"apply_scenario_improvements"; prompt changes go to a new saved version by default). Run the same scenario again. -
Compare versions -
scenarios_execute(action"run_agent_scenario_matrix") runs a scenario across several versions, caller numbers and repetitions, so you can compare a saved version with the live agent ("prod"inversionIds) or measure a pass rate.
Run the whole suite before you publish
Section titled “Run the whole suite before you publish”-
Run all - Run all on the Tests tab, or
scenarios_execute(action"run_agent_scenario_suite") with the version you want to publish. It runs every scenario of the agent, one live call each. -
Approve the cost - from a chat, the first call only returns the estimated minutes; nothing runs until you approve and it is called again with
confirmCost: true. -
Publish what passed -
versions_execute(action"publish_agent_version") with the suite’stestSuiteId. -
Make passing tests required (optional) - on the Tests tab turn on Require passing tests to publish. A version that has never been live is then published only after a test run that started after its last change passed completely. Rolling back to a version that was live before is never blocked.
Quick checks without a call
Section titled “Quick checks without a call”mode: "text" simulates the conversation without a phone call. It shows what the agent says and decides, but tools, transfers and workflows do not run, so a “missed tool call” there is not a finding. Use it to try prompt wording quickly; use live runs to decide what to publish.
The one exception is procedures. The test’s Procedures (text tests) setting (procedureMode) decides how the agent gets them:
- Like the call: loaded when needed (
on_trigger, the default, also for tests saved before the setting existed). The agent sees only the list of procedures and must load the right one itself, as on a real call. Each load shows in the result as aload_procedurestep with its key and turn, so a text test can expectload_procedureand fails when the agent answers without loading it. - All from the start (trigger not checked) (
bypass_trigger). Every procedure is given in full from the first turn. Use it to check the steps themselves. The result marks the run, because it does not show whether the agent would load the procedure.
To talk to the agent yourself, use test_in_browser (voice or chat in the browser) or call_me (the agent calls your verified mobile).
Keep the suite growing
Section titled “Keep the suite growing”When a real call goes wrong, read it with get_calls (include "aiDialogue" shows the prompt, tool calls and tool responses of that call), fix the cause, and save that call as a new scenario. The suite then guards against it coming back.
The full build checklist is in Build a reliable phone agent.
Frequently asked questions
Section titled “Frequently asked questions”Can I try my Voice Logica agent myself before putting it on my calls?
Yes. Use test_in_browser to talk to the agent by voice or chat in the browser, or call_me to have the agent call your verified mobile. To test it systematically, save test scenarios on the agent’s Tests tab and run them as real phone calls graded by a judge.
Do Voice Logica test calls use the minutes of my plan?
Yes. Each live test run is a real phone call and uses call minutes from the company plan, about the length of the call; one request makes at most 3 repetitions. When you run the whole suite from a chat, you first get the estimated minutes, and nothing runs until you approve.
Will a Voice Logica test call book real appointments or create real orders?
It can. Tools run for real unless you list them in the scenario’s mocked tools, or turn off use real tools to block every tool you did not list. Appointment booking tools cannot be mocked yet and book for real, so use a test calendar or cancel the booking afterwards.
Does someone's phone ring when a Voice Logica test scenario transfers the call?
No. In test calls, transfers are graded but never connect, so nobody’s phone rings. Set the Expected transfer on the scenario to say whether the agent should transfer, and where.
Can a Voice Logica test call come from a customer's number, so the agent recognises them?
Yes. Enter your own number, or a customer’s number from your CRM, under Caller IDs on the scenario, and the agent recognises the caller exactly as on a real call: contact, integrations and earlier calls. Leave it empty to call from the default test line; several numbers make one call per number.
Can I stop a new version of my Voice Logica agent going live unless its tests pass?
Yes. On the Tests tab turn on Require passing tests to publish. A version that has never been live is then published only after a test run that started after its last change passed completely. Rolling back to a version that was live before is never blocked.
