Skip the hype. Learn how to build a production voice AI agent with real tools like Retell AI, n8n, and Claude. No frameworks. No bloat.
Most voice AI agent tutorials start with a framework, end with a demo that doesn't work in production, and never mention the parts that actually cost money. Here's how to build one that takes calls, qualifies leads, and doesn't hallucinate mid-conversation.
The real problem isn't the AI. It's orchestration. Most teams confuse "using a voice agent platform" with "building a voice AI agent." They're different. One is picking a SaaS tool. The other is wiring together a stack so the agent actually stores data, integrates with your CRM, and handles edge cases before your ops team has to.
Teams fail at how to build a voice AI agent before they even pick a tool. The failures are consistent.
First: they pick a platform before defining scope. "We want a voice AI agent." For what? Lead qualification? Appointment setting? Customer support? Each one needs different guardrails, different prompts, different integrations. Vague scope kills 30% of projects in week two.
Second: they ignore latency. A 300ms delay between the caller speaking and the agent responding feels dead. Most tutorials don't test for it. They run a demo on localhost and ship it. Production breaks.
Third: they skip the data layer. Where do call notes live? How do they flow to your CRM? Does the agent have context about the caller before answering? This is where 70% of the value lives, and it's also where 70% of projects break. The agent works. The integration doesn't.
Fourth: they conflate "building an agent" with "calling an agent API." Real work is orchestration. It's webhooks. It's retry logic. It's deciding what happens when Claude times out at 12:47 AM on a Saturday.
Forget agent frameworks. Skip LangChain and AutoGen. They add latency and abstraction you don't need for voice calls.
Here's the stack that works:
Retell AI handles the voice layer. It takes inbound and outbound calls, manages DTMF input, handles real-time interruptions. It's not an LLM. It's the glue between the caller and your reasoning layer. Cost: ~$0.10–0.30 per minute depending on volume and geography.
Claude (via API) powers the reasoning. It's faster than GPT-4 for structured tasks. Cheaper. And it doesn't hallucinate call scripts as often as open-source models do. Use tool_use to give it discrete actions: store_lead_info, schedule_callback, escalate_to_human.
n8n orchestrates the flow. Call comes in → Retell captures the transcript → n8n receives a webhook → Claude processes the data → n8n stores it in Supabase → a second webhook fires to your CRM or calendar system. This happens while the caller is still on the line or right after they hang up.
Supabase holds the data. Call logs. Lead records. System prompts. Your CRM link. Use it as source-of-truth, not an afterthought.
This stack costs roughly $0.80 per call all-in at moderate volume. A Vapi alternative or bundled platform costs 2–3x that once you factor in per-minute charges on top of LLM costs.
Create a Retell account and configure a phone number. Takes 15 minutes.
Then do the hard part: write a single paragraph that defines what the agent does and doesn't do. Example:
"This agent qualifies inbound leads for our SaaS. It asks three questions: annual budget, timeline for implementation, primary use case. If the lead answers all three and signals interest, the agent books a meeting with a sales rep and stores the call notes in Supabase. If the caller asks about pricing or presses for a discount, the agent transfers to a human. If the caller is hostile, it logs the call and ends it."
That's it. Not vague. Not aspirational. Executable.
Write system prompts that include this scope, a knowledge cutoff date, and a hard stop rule. Vague prompts create calls that drift. At 2 AM, the agent starts debating philosophy with a caller instead of qualifying them.
Before you go live, test latency with a test call. Call your own number. If you hear more than a 400ms gap before the agent responds, the orchestration layer is too slow. Fix it now.
Don't dump the entire call transcript into Claude and hope. Use tools.
Give Claude discrete actions: store_lead_info, schedule_callback, escalate_to_human. Define the schema for each. If Claude calls store_lead_info, it must include: lead_name (string), lead_email (string), lead_phone (string), qualification_score (1–10 integer).
Keep your system prompt under 500 tokens. Longer prompts create latency spikes and inconsistent behavior. Use examples instead. Show Claude two good calls and two bad calls. Pattern-match beats lengthy instructions.
Test the Claude integration with 50 mock conversations before going live. Use the Messages API batch feature if you're iterating fast. Batch costs 50% less than regular API calls.
Set a token limit per call. A 30-minute call will leak money and hallucinate if you don't cap it. Use ~4,000 tokens per call as your starting point. Most lead qualification calls finish in 3,000 or fewer.
Create an n8n workflow that listens for Retell webhooks: call_started, call_ended, transcript_ready.
On call_ended, parse the transcript and send it to Claude for structured extraction. Have Claude pull out lead_name, lead_email, qualification_score, and any key objections. Store the result in Supabase.
Add a conditional: if qualification_score > 7, fire a webhook to your CRM or calendar system. If the score is < 5, add the lead to a follow-up queue instead of the hot queue.
Log every call and every Claude decision. You'll need this data to debug hallucinations and improve prompts. Without logs, you're flying blind.
Your first live call will expose gaps you didn't think of.
Silent fails: What happens if Claude times out? Your n8n workflow should retry once, then log the failure and send you an alert. Without retry logic, you lose lead data.
Hallucinated call notes: Claude sometimes fabricates details. A caller mentions "Q3" and Claude decides that means "third quarter of next year" when the caller meant "third quarter of this quarter." Add a post-processing step that only stores answers to your three scripted questions. Ignore Claude's inferences.
Escalation confusion: If your agent transfers to a human, does the human see the call context? The lead's name, the three answers they gave, the reason for transfer? If not, fix it now. Test a live transfer before you launch.
Cost overruns: Run a 100-call pilot. Track Retell cost, Claude API cost, and operator time. Most teams underestimate Claude costs on high-volume calls. If you're running 1,000 calls a week, you'll spend $600–800 on Claude alone.
A qualified voice AI agent (inbound lead qualification) costs roughly $0.80 per call all-in. Retell runs ~$0.15–0.25. Claude ~$0.30–0.40. n8n and Supabase ~$0.10–0.15.
At 100 calls per week, that's $80 per week. At 400 calls per week, you're at $320 per week. The cost scales linearly until you hit volume discounts.
Now conversion. If your agent books 8 qualified meetings per week and your sales team closes 25% of those, you're generating 2 deals per week. At $0.80 per call, you're at $0.25 per closed deal. That's the math that matters.
Iteration time: your first version takes 2–4 weeks. Refinements (better prompts, tighter scope, CRM sync) take another 4 weeks before you're confident enough to hand off to ops.
Vapi bundles everything into a black box. Fine if you want a demo in a day. Bad if you need custom integrations, data governance, or cost control.
They charge per minute on top of LLM costs. At scale, you'll pay 2–3x what the Retell + Claude stack costs.
Their system prompts are limited. You can't easily add real-time data lookup. If the caller is an existing customer, you want the agent to fetch their support history before answering. Vapi doesn't support that. Retell + Claude does.
Use Vapi if you need a proof-of-concept in 24 hours. Use Retell if you're in production and your margins matter.
Total cost per call is roughly $0.80 at moderate volume (100–400 calls per week). That includes Retell AI (~$0.15–0.25/call), Claude API (~$0.30–0.40/call), and n8n + Supabase (~$0.10–0.15/call). Most teams spend $200–400 on initial setup and configuration.
Yes, but it's harder than lead qualification. Support calls are longer, more complex, and require real-time access to customer history, order data, and support tickets. You'll need a larger context window, more sophisticated routing logic, and human escalation paths. Start with lead qualification to learn the stack, then expand.
Building gives you full control. You own the data, the integrations, the cost. A platform like Vapi trades control for speed. You get a working agent faster, but you pay per-minute charges and depend on their API. For a production system where your margins matter, building is cheaper. For a quick demo, a platform is faster.
---
If you want to talk through applying this to your stack, book a strategy call at cognival.co/book.
30-min strategy call. No pitch, real look at your stack.
Book a strategy call →