ChatGPT Voice Mode explained

When you tap the waveform icon in the ChatGPT app and start talking, you are using the closest thing the mainstream market has produced to a digital human you can hold a conversation with in real time. ChatGPT Voice Mode – specifically the "Advanced Voice" experience powered by GPT-4o's native audio – collapses the pause between what you say and what the model says back to something close to human conversational rhythm. OpenAI has stated a response latency averaging around 320 milliseconds for GPT-4o audio, comparable to a person's reaction time in dialogue, though real-world figures vary with network conditions and load. That single number is why Voice Mode matters to anyone building embodied AI: it moves voice from a novelty transcript feature into a genuine interface.

This piece is a capability and latency analysis of Voice Mode treated as a building block. What can it actually do, how does the architecture achieve it, where does it break, and how does it compare to the alternatives a developer faces when shipping voice-enabled AI? Every specific figure below – price, latency, model version – is time-sensitive in a field that changes monthly, so treat each as accurate as of writing and verify against the vendor's own documentation before you rely on it.

What Voice Mode actually is

ChatGPT Voice Mode is a real-time spoken conversation interface inside the ChatGPT mobile and desktop apps. You speak; the model listens, reasons, and speaks back in a synthetic voice. There are two distinct things wearing the "voice" label, and the distinction matters.

The older Voice Mode, sometimes called Standard Voice, is a pipeline. Your speech is transcribed to text by a speech-to-text system (OpenAI's Whisper family), that text is fed to a language model, the model's text reply is then synthesised back into audio by a separate text-to-speech engine. It works, but each stage adds delay and each hand-off discards information. Whisper turns your rising, sarcastic intonation into flat words on a page; the LLM never hears that you were laughing.

Advanced Voice Mode, introduced alongside GPT-4o in May 2024 and rolled out to users over the following months, is architecturally different. GPT-4o – the "o" stands for omni – accepts audio directly as input and produces audio directly as output. There is no transcription bottleneck in the middle. The model hears the audio and speaks the response, and because a single network handles the whole exchange, it can preserve and generate the paralinguistic layer: tone, pacing, breath, emphasis, laughter. This is the version that hits conversational latency and the version worth analysing seriously.

What is ChatGPT Voice Mode?

ChatGPT Voice Mode is a feature in the ChatGPT app that lets you have a spoken, back-and-forth conversation with the model instead of typing. The current Advanced Voice Mode uses GPT-4o's native audio processing, which OpenAI announced in May 2024, to listen and reply directly in speech with a response time OpenAI reports averaging roughly 320 milliseconds, close to human conversational latency.

How Voice Mode works architecturally

The reason Advanced Voice feels different from every voice assistant before it comes down to what OpenAI removed from the stack.

The death of the STT → LLM → TTS pipeline

For a decade, voice AI meant three components chained together. Speech-to-text (STT) converted audio to a transcript. A language model or intent classifier processed the transcript. Text-to-speech (TTS) converted the reply back to audio. Every commercial voice assistant, from smart speakers to phone-tree bots, worked this way, and the architecture imposed two hard costs.

The first cost is latency. Each stage must largely finish before the next begins. STT often waits for a pause to decide you are done talking; the LLM then generates its full reply; only then does TTS start speaking. Even with streaming optimisations, the accumulated delay lands in the one-to-three-second range – long enough that the interaction feels like radio comms, not conversation.

The second cost is information loss. A transcript is a lossy compression of speech. "Fine." spoken flatly and "Fine!" spoken with delight collapse to nearly the same characters. The LLM in a pipeline is reasoning over a stripped-down text shadow of what you actually communicated, and its TTS voice, generated from text alone, has no principled way to match the emotional register of the exchange.

GPT-4o's native multimodal design folds all three functions into one model trained end-to-end across text, audio and vision. It tokenises audio directly and generates audio tokens directly. Because there is no serial hand-off, response can begin almost as soon as the model has heard enough to reason, and because the model perceives the raw audio, it can register that you sounded uncertain and respond in a reassuring tone. This is the architectural reason native-multimodal is becoming the default design for serious voice agents rather than a curiosity.

The trade-offs underneath

Native audio is not free. Training a single model to be excellent at spoken interaction, text reasoning and vision simultaneously is far harder than optimising three specialised components independently, and there is evidence that heavy multimodal training involves capability trade-offs against a pure text model of the same size. Serving audio-in, audio-out is also more compute-intensive per interaction than serving text, which shows up in the per-minute pricing developers pay.

There is also a controllability cost. In a pipeline, you can inspect the transcript, log it, run it through a content filter, swap the TTS voice, and audit each stage. A native audio model is more of a black box: the audio goes in and comes out, and intermediate representations are not human-readable. For regulated industries that need transcripts and deterministic guardrails, that opacity is a genuine engineering concern, which is why many production systems still run a parallel transcription for logging even when using native audio for the live exchange.

Server-side, a great deal happens that users never see: voice activity detection deciding when you have stopped speaking, interruption handling that cuts off the model's speech when you start talking over it, safety classifiers screening both input and output, and session state management. The perceived smoothness of Voice Mode is as much about this invisible orchestration as about the model itself.

Voice Mode capability map

Conversational dynamics

The headline capability is turn-taking that approximates human dialogue. You can interrupt Advanced Voice mid-sentence and it stops and adapts, rather than plowing through its scripted reply. It handles back-channels – the "mm-hm" and "right" that keep a conversation moving – with more grace than pipeline systems, and it can modulate pace and tone on request, speaking faster, slower, more dramatically, or in a whisper. It can sing, hum and produce non-speech vocalisations, though OpenAI has restricted some of these behaviours over time for safety and rights reasons.

Languages, accents and emotional expression

GPT-4o supports a broad range of languages for spoken interaction, with strongest performance in high-resource languages and more variable quality in lower-resource ones – check OpenAI's current documentation for the supported list, as it expands. The model handles many accents in its input and can adopt different speaking styles in output. Emotional expression is a real strength relative to older assistants: it can sound cheerful, sombre, excited or calm, and it responds to the emotional content it perceives in your voice. This is precisely the territory that specialists like Hume AI have staked out, and we cover the emotional-voice angle in depth in our analysis of Hume AI and its EVI system.

The hard limits

Voice Mode is not a solved problem, and the limits matter for anyone designing around it. It is built for one-on-one conversation and does not reliably separate or track multiple simultaneous speakers – drop it into a meeting with four people and it struggles to attribute who said what. It has no persistent memory of your voice characteristics across sessions in the way a human remembers a friend's voice; voice-specific personalisation is limited by what the broader ChatGPT memory feature stores as text. Its knowledge is still bounded by training data and whatever tools it can call, so it can be confidently wrong out loud, which feels more authoritative and therefore more dangerous than a wrong line of text.

Camera and vision integration

OpenAI has extended Advanced Voice with live video and screen-sharing, letting the model see through your phone camera while you talk to it – you can point the camera at a broken appliance or a maths problem and discuss it in real time. This is the clearest signal of where the product is heading: a multimodal presence that hears, sees and speaks. It is also the feature that most directly delivers on the embodied-AI thesis, moving from a voice in a box toward something closer to a perceiving conversational agent.

Can ChatGPT transcribe audio?

Yes, ChatGPT can transcribe audio, but through different mechanisms depending on how you access it. In Voice Mode the app converts your spoken words for the conversation, and OpenAI's Whisper model – available through the API – is a dedicated speech-to-text system that produces text transcripts from uploaded audio files. For accurate, logged transcription of recordings, developers typically call the Whisper or the current audio-transcription API endpoint rather than relying on Advanced Voice, which is optimised for live interaction rather than transcript fidelity. Verify current transcription endpoints and file-size limits in OpenAI's API documentation.

Voice Mode for builders

The consumer app is one thing; building on the same capability is another. The relevant product is the OpenAI Realtime API, which exposes low-latency, speech-to-speech interaction over a persistent connection (WebSocket or WebRTC) so developers can put GPT-4o-class voice into their own applications with interruption handling and streaming built in.

Pricing and what to check

Realtime API pricing is charged separately for audio input and audio output tokens, and audio is materially more expensive per minute than text. As of writing, real-time audio runs to a level that makes always-on voice a genuine cost line to model before launch, and OpenAI has introduced cheaper mini variants and cached-input discounts to bring it down. Because these numbers move, price your specific expected usage against the official OpenAI pricing page rather than any figure quoted in an article, and stress-test your assumptions about average call length. If you scale hard, budget for rate limits too – our guide to 429 Too Many Requests errors covers what happens when you hit the ceiling.

How it compares to the alternatives

A developer building voice AI in 2025 is not choosing between OpenAI and nothing. The field has several serious players, each with a different centre of gravity.

ElevenLabs built its reputation on high-fidelity text-to-speech and voice cloning, and now offers a Conversational AI product that stitches STT, an LLM of your choice, and its own excellent TTS into a managed low-latency stack. Its advantage is voice quality and flexibility – you pick the language model, you get outstanding synthetic voices, and you can use cloned or custom voices within its policy. The trade-off is that it is fundamentally an orchestrated pipeline rather than a single native-audio model, so it carries pipeline latency characteristics, mitigated by heavy engineering. We analyse the company in detail in our piece on ElevenLabs and the TTS market.

Hume AI specialises in emotional intelligence. Its Empathic Voice Interface (EVI) is designed around reading and responding to the emotional prosody in a speaker's voice, which makes it compelling for wellbeing, coaching and companion use cases where feeling understood matters more than raw factual throughput.

Sesame has drawn attention with its Conversational Speech Model and an open approach, offering a route for teams that want more control and the ability to self-host or fine-tune, at the cost of running more of the stack themselves.

Cartesia and its Sonic model compete hard on latency and efficiency, targeting the developers for whom the deciding factor is the fastest possible time-to-first-audio at scale.

The honest guidance: build on the Realtime API when you want the tightest native speech-to-speech loop, strong general reasoning in the same model, and vision in the same session, and you can accept a black-box audio path and OpenAI's pricing. Reach for a TTS-led stack like ElevenLabs when voice quality and voice choice are paramount, for Hume when emotional nuance is the product, and for Sesame or Cartesia when control, self-hosting or raw latency economics dominate. Many teams also compose these – Whisper or a specialist STT, a reasoning model, and a best-in-class TTS – which is exactly the pattern the voice-agent platforms are built around. We map that ecosystem in our coverage of voice agents on Bland, Retell and Vapi.

Real applications

In customer service, Voice Mode-class technology handles the top of the funnel well – answering common questions, triaging, gathering context – but stumbles on the same things every voice agent stumbles on: noisy lines, strong regional accents, callers who ramble or talk over the system, and the moment a conversation needs a hard guarantee about account state that only a deterministic backend can provide. The interruption handling helps enormously here, but the confident-but-wrong failure mode is a real liability when the wrong answer is spoken aloud with a reassuring tone.

Tutoring and language learning are where the native-audio approach genuinely shines. A model that hears your pronunciation, responds patiently, adjusts its pace, and role-plays a conversation partner in the target language is a materially better practice tool than a transcript-based drill. The emotional register makes the difference between a tool that corrects you and one that encourages you.

Accessibility is an underrated application. Real-time spoken interaction with vision – describing a scene through the camera, reading a label aloud, walking a user through a form – is transformative for people with low vision or limited mobility, and it is here that the combined hear-see-speak loop earns its keep.

Companion and character applications are the most culturally charged. A voice that expresses warmth, remembers context and responds emotionally is exactly what makes a digital companion feel present, and it is also where the risks of over-attachment and manipulation are sharpest. We treat that terrain, and the design responsibilities it imposes, in our survey of AI assistants and AI companions.

The voice-cloning question

OpenAI does not permit users to clone the voice of a real person in ChatGPT Voice Mode. The Advanced Voice offering ships with a fixed set of curated voices produced with paid voice actors, and the product is deliberately not a general voice-cloning tool. OpenAI has separately previewed a Voice Engine capability able to reproduce a voice from a short sample, but has restricted its release precisely because of misuse concerns.

The episode that crystallised the stakes was the "Sky" voice. When GPT-4o's Advanced Voice was demonstrated in May 2024, one of its voices, Sky, struck many listeners as resembling Scarlett Johansson – who had voiced an AI companion in the film Her, and who said she had declined OpenAI's request to lend her voice. OpenAI paused the Sky voice and stated that it was not an imitation of Johansson and belonged to a different professional actor. The controversy, widely reported at the time, did more than any policy document to surface the core tension in synthetic voice: even a voice that is legally someone else's can evoke a specific person, and evocation alone carries reputational and consent weight. We examine the law, technology and ethics of this in our dedicated piece on AI voice cloning.

The Sky incident is a useful lesson for any builder: the constraints on voice AI are not only technical but reputational and legal, and the safest defensible position is explicit consent and clearly synthetic, non-impersonating voices.

Voice Mode in the broader voice-AI landscape

Set the players side by side on transparent criteria – architecture, voice quality, emotional range, control, and cost – and a picture emerges. OpenAI's Advanced Voice leads on integrated reasoning and native speech-to-speech latency with vision in the loop, but gives you a fixed voice set and a black-box audio path. ElevenLabs leads on voice fidelity and choice and gives you your pick of language model, at the cost of a pipeline architecture. Hume AI leads on emotional interpretation. Sesame and Cartesia lead on openness and latency economics respectively. None is strictly best; each is best for a defined job.

The larger trend is unmistakable. Native-multimodal, end-to-end audio is becoming the reference architecture for conversational AI, and the pipeline approach is increasingly a fallback for cases that need its transparency and component flexibility. As Google, Meta and others ship their own native-audio models, the 320-millisecond conversational loop OpenAI demonstrated will stop being remarkable and start being the baseline expectation. For those of us tracking the emergence of digital humans – agents you can see, hear and talk to as naturally as a colleague – Voice Mode is the clearest early evidence that the interface is arriving faster than the norms and guardrails around it. The technical question of how to make a machine converse is largely being answered. The harder questions, about consent, attachment, disclosure and trust, are the ones still open, and they are the ones that will decide how these systems are actually allowed to live in the world.