Voice Agent Stack Benchmark — Task 1

Building and Optimizing a
Low-Latency Voice Agent

A comparative benchmark evaluating a sequential Google Gemini baseline against an ultra-low latency Deepgram + Groq streaming pipeline.

Baseline (P50)
15.3s
Sequential Gemini
Optimized (P50)
1.18s
Streamed Pipeline
Net Reduction
-92.3%
13x Speedup
Groq TTFT
357 ms
gpt-oss-20b
Sequential ExecutionP50: ~15.3s
Baseline Voice Agent
Processes turns sequentially: Browser RMS VAD → Audio File Upload → Gemini 3.5 Transcribe → Gemini 3.7 Flash LLM → Gemini 3.1 Flash TTS Audio Synthesis.
VAD (Silence Window):660 ms
STT Model:gemini-3.5-transcribe (~3.2s)
LLM Model:gemini-3.7-flash (~2.3s)
TTS Model:gemini-3.1-flash-tts (~7.5s)
Streaming PipelineP50: ~1.18s (13x Faster)
Optimized Voice Agent
Overlaps execution pipelines: Deepgram Live Nova-3 STT (WebSocket) → Groq LPU (openai/gpt-oss-20b) → Deepgram Flux TTS (/v2/speak) with Web Audio PCM streaming.
VAD Endpointing:150 ms (-77%)
STT Streaming:Deepgram Nova-3 (0ms perceived)
LLM Inference:Groq LPU (357ms TTFT)
TTS Audio Engine:Deepgram Flux (/v2/speak PCM)

Pipeline Architecture Breakdown

Contrast between blocking sequential roundtrips vs continuous overlapped stream execution.

Baseline (Sequential)Total: ~15.3s
1. Silence VAD Wait669 ms (4.4%)
2. File Upload & Gemini STT3,195 ms (20.9%)
3. Gemini 3.7 Flash LLM Gen2,333 ms (15.3%)
4. Gemini Full-Text TTS Synthesis7,495 ms (49.1%)
Blocking Bottleneck: The browser sits idle for ~15 seconds until the entire audio synthesis completes.
Optimized (Streaming)Total: ~1.2s
1. Adaptive VAD Endpointing150 ms
2. Deepgram Live WebSocket STT0 ms (Concurrent)
3. Groq LPU First Token (TTFT)346 ms
4. Deepgram Flux First Audio (TTFA)719 ms (Streaming)
Instant Playback: Audio begins playing immediately through Web Audio API while the LLM streams remaining tokens.

20-Turn Benchmark Measurements

Full turn-by-turn latency data collected during empirical conversation testing.

Median Total (P50)
1,178 ms
P95 Total Latency
1,649 ms
Mean Total Latency
1,215 ms
Mean LLM TTFT
346 ms
TurnVAD (ms)STT (ms)LLM TTFTLLM TotalTTS TTFATotal LatencyUser Transcript
#115003615468801031 ms"Can you tell me about the capital of India?"
#21500329432773923 ms"of France?"
#31500213425741892 ms"places are there in France?"
#4150037270410251176 ms"that I can do in France?"
#5150041686011761326 ms"what are some special things I can do in India?"
#615003215358561007 ms"And I'm also planning to visit Japan do you have any suggestions for me?"
#715003656139071057 ms"what would a Friday trip look like for Japan?"
#81500329123514861637 ms"I'm looking for I'm not talking about Friday. I'm talking about a five day trip."
#91500334017361886 ms"Japan?"
#101500355519815965 ms"What are the visa requirements for Japan?"
#11150040876911531303 ms"what are the special places to visit in Tokyo?"
#12150035667510291180 ms"how many like, what are the food options like in Tokyo?"
#131500357479791941 ms"Got it. And how long does it take from India to Tokyo in terms of the flight?"
#14150037399113671517 ms"And what would a five day trip to Japan cost in rupees?"
#1515003625328661016 ms"how can you reduce this cost? Let's say if your budget is 1.5 lakhs, what changes would you have to make?"
#16150032467410391189 ms"some souvenirs that I could buy?"
#17150043276310761226 ms"Is what's the best time to visit Japan?"
#18150039377911281278 ms"Got it. And any other suggestions that you might have for me?"
#19150034486912191369 ms"What are the best transportation methods over there?"
#20150035769310251175 ms"And any other final tips?"

Assignment Discussion & Technical Evaluation

In-depth answers to core latency questions, architectural trade-offs, and scaling strategies.