How voice dictation works: the complete guide

You press a key, you speak, and a few seconds later a clean sentence appears on screen. A great deal more happens in between than most people imagine: a sound wave becomes a sequence of numbers, those numbers become an image, that image becomes text, and that text finally becomes a properly written sentence. This guide walks through each of those stages, without unnecessary jargon but without oversimplifying either. By the end you will understand why your phone makes you say "comma" while other tools handle it themselves, why the same model understands your neighbour perfectly and stumbles over your client's name, and where your voice actually travels along the way.

The four stages at a glance

Every modern voice dictation tool rests on the same chain. What separates one tool from another is almost never the first stage: it is whether or not they perform the third.

Dictation built into operating systems and office suites stops at stage 2. That is true of Windows voice typing, macOS Dictation, iOS Dictation, Gboard voice typing on Android and Google Docs voice typing. Tools described as "AI" add stage 3, and that is the whole difference between a transcript and text you can send as is. We covered each of them in our guides to iPhone, Android, Word and Google Docs.

Stage 1: your voice becomes numbers

Sampling

When you speak, your vocal cords make the air vibrate. That vibration is a continuous variation in pressure. Your microphone's membrane follows those variations and converts them into an electrical voltage, and the sound card then measures that voltage at regular intervals. Each measurement is called a sample.

For speech, the standard sampling rate is 16,000 samples per second, or 16 kHz. One minute of dictation therefore amounts to roughly 960,000 numbers. That 16 kHz figure is not arbitrary: the Nyquist-Shannon sampling theorem requires measuring at least twice as fast as the highest frequency you want to preserve. Since most of the information that distinguishes an "s" from an "f" sits below 8 kHz, 16 kHz is enough. It is also why dictation has no need for CD quality at 44.1 kHz: the extra data would add nothing to recognition and would only inflate computation and transfer. This is not an informal convention either, the Whisper research paper states that all audio is resampled to 16,000 Hz before processing.

The spectrogram: the image the model actually "sees"

A raw sequence of 960,000 numbers is very hard to work with directly. So it is turned into something far more readable for a neural network.

The principle: the recording is chopped into small overlapping windows of 25 milliseconds, advanced by 10 milliseconds each time. For each window, a Fourier transform works out which frequencies are present and at what intensity. Stacking all those results side by side produces an image: the spectrogram. The horizontal axis is time, the vertical axis is frequency, and brightness is the energy present at that frequency at that instant. Those 25 and 10 millisecond values are exactly the ones Whisper uses.

One last subtlety: the frequency axis is not linear, it is compressed towards the high end. This is the mel scale, modelled on human perception, which distinguishes the gap between 200 and 300 Hz far more finely than the one between 8,000 and 8,100 Hz. In practice the final image holds only a few dozen frequency bands: Whisper uses 80, and 128 on its large-v3 version. That is a massive reduction in information, but it preserves precisely what allows speech sounds to be told apart.

Stage 2: transcription, from image to text

Whisper, the de facto standard

Until around 2020, speech recognition relied on systems assembled from several separate components: an acoustic model, a pronunciation dictionary, a language model. They worked, but they required painstaking tuning per language and degraded quickly as soon as the audio fell outside expected conditions.

The shift came from end-to-end models, the best known of which is Whisper, released as open source by OpenAI in late 2022. Its significance lies less in its architecture than in its training. The original version was trained on 680,000 hours of audio collected from the web along with the matching transcripts, of which 117,000 hours cover 96 languages other than English, as detailed in the associated research paper. The following generation changed scale again: the large-v3 version draws on one million hours of weakly labelled audio, supplemented by four million hours transcribed automatically by the previous model.

At that scale, the model has seen so many voices, accents, mediocre microphones and background noises that it stays robust on everyday audio, not just studio recordings. It now covers around a hundred languages within a single model. That is what made it the standard a good share of recent dictation tools build on.

Encoder, decoder and tokens

The model is made of two halves that work one after the other.

• The encoder reads the spectrogram in 30-second slices and condenses it into an internal representation. Think of it as a dense numerical summary that keeps, for each instant, what matters from a speech standpoint and discards the rest.
• The decoder produces the text piece by piece. At each step it looks at both that audio summary and everything it has already written, then predicts the most likely next fragment of text.

Those fragments are called tokens. A token is not a word: it is closer to a chunk of a word, often three or four characters. Very frequent words form a single token, while a rare term takes up several. That splitting lets the model handle any language and any previously unseen word with a vocabulary of reasonable size.

Why context eliminates homophones

This is where most of the last decade's progress sits. Because the decoder takes into account what it has already written, it does not pick its words on sound alone. Take "their", "there" and "they're", or "to", "too" and "two": these forms are acoustically identical. No amount of signal processing will ever separate them. A model that has read millions of sentences, on the other hand, knows that "two hundred euros" is far more likely than "too hundred euros".

The same mechanism explains the errors. The model is not searching for truth, it is searching for the most likely continuation. Faced with a rare proper noun or an industry term thinly represented in its data, it will confidently propose a common word that sounds the same. That is why custom vocabulary is a genuine topic, handled at the next stage rather than this one.

How quality is measured: WER

The reference metric is WER, or word error rate. The produced transcript is compared against a reference transcript written by a human, three kinds of error are counted, and the total is divided by the number of words in the reference.

• Substitutions: a word was replaced by another.
• Insertions: a word was added that was never spoken.
• Deletions: a spoken word was omitted.

A WER of 5 % means one word in twenty is wrong. One caution is essential, though: WER means nothing in the abstract, it depends entirely on the test corpus. The same model can post 3 % on audiobook readings recorded with a headset and 20 % on a multi-speaker meeting in a reverberant room. Two WER figures are only comparable if they come from the same evaluation set, something marketing material rarely specifies.

Stage 3: AI cleanup, what separates a transcript from text

Suppose stage 2 were perfect: every spoken word correctly recognised. You still would not have usable text, for a very simple reason: you do not speak the way you write. A faithful transcript reproduces your hesitations, your restarts, your filler words and your sentences that change direction halfway through. It is faithful, and that is precisely the problem.

That transformation is the work of a large language model sitting behind the transcription. Concretely, it handles:

• Punctuation and sentence segmentation, which you never spoke and which therefore has to be inferred from the structure of what you said.
• Removal of hesitations and false starts, without altering the meaning of what you intended.
• Grammar and agreement, often loose in speech.
• The expected formatting: an email does not look like an internal note or a short message. The most capable tools let you define your own formatting rules, applied to every dictation.
• Industry vocabulary and proper nouns, from a list supplied by the user. This is the antidote to the homophone problem described above, and it works better than you might expect: your client's name does not need to be known to the transcription model, it only needs to be known to the correction stage, which restores the exact spelling from a phonetic approximation.

Stage 4: insertion into your application

The least spectacular stage, and yet the one that decides whether the tool is usable day to day. The final text has to land where you are writing, without a detour through a clipboard or an intermediate window.

Two families of tools compete here. Dictation tied to a given application, such as Word's or Google Docs voice typing, only works inside that application, sometimes only inside one specific browser. System-level tools install themselves as a layer above everything else: they listen for a global keyboard shortcut, then write the result at the cursor position, whatever window is active. A search field, an email, a ticket, a terminal: anything that accepts text accepts dictation.

Technically, that insertion happens either by simulating keystrokes or through the operating system's accessibility interfaces. This difference, invisible in use, explains why a tool can work perfectly in a word processor and fail in an application with unusual graphics rendering, such as certain games or remote desktop environments.

How long does all this take?

Contrary to a widespread intuition, processing time does not map onto perceived waiting, because the stages do not all happen after you stop speaking. Capture and spectrogram preparation run continuously while you talk. What remains is transcription and cleanup, which execute when you release the shortcut.

Three factors dominate the result. First, available compute: datacentre graphics cards process a multi-minute recording in a fraction of the time a laptop would take. Second, the number of network round trips, which a well-designed chain keeps to a minimum by merging transcription and cleanup rather than chaining them as two separate calls. Third, how the audio is split: since the model reasons in 30-second slices, a long recording can be processed in parallel chunks rather than strictly sequentially.

One counter-intuitive consequence follows: waiting time does not grow in proportion to dictation length. On a well-designed chain, doubling the length of your dictation does not double the wait, which makes long dictations considerably more worthwhile than isolated phrases.

Where do these models run: your computer or a server?

The chain described here is the same everywhere. What changes from one tool to another is where each stage executes, and that choice has direct consequences on what you get.

• Everything local. The audio never leaves your machine, which is the maximum in privacy terms and works offline. In practice you get stages 1, 2 and 4, rarely 3, for want of the compute a large cleanup model demands.
• Transcription and cleanup on a server. Your audio travels to infrastructure built for these computations, which makes the full chain possible on any computer, even a light laptop with no dedicated graphics card. In exchange, a connection is required and you depend on the provider's commitments regarding retention and data location.

That trade-off deserves treatment of its own: we detailed it in local vs cloud voice dictation.

The vocabulary, straightened out

Four terms are regularly used interchangeably even though they refer to different things.

What really drives quality

With the same model, results can range from excellent to poor. The factors that weigh most, in order:

• The ratio between your voice and ambient noise. What counts is not the volume of the noise but its gap with your voice. A headset microphone, or the built-in microphones of a recent laptop at a sensible distance, are plenty; a microphone on the far side of a large room is the single biggest source of degradation.
• Reverberation. A bare room with a high ceiling sends your voice back with a delay, which blurs the spectrogram's patterns far more than steady fan noise does.
• Pace and articulation. Speaking fast is not a problem in itself, swallowing word endings is.
• Proper nouns and jargon. The structural weak point explained above, and the reason custom vocabulary lists exist.
• Language and language mixing. Models are markedly better in English than elsewhere, simply because training data is more abundant. Alternating two languages within a single sentence remains the hardest case.
• Passage length. More context helps the model resolve ambiguity: a dictation of several sentences often produces better results than three isolated words.

What happens to your voice along the way

Once the chain is understood, the data question becomes far more concrete. It is not about whether a tool is "secure" in the abstract, but about what happens to three clearly identified objects: the audio recording, the raw transcript and the final text.

Three questions are enough to frame the topic, and they apply to any tool:

• Are these items retained after processing, and if so for how long?
• Are they reused to train models? That is a separate question from retention, and it is settled in the terms of service.
• Under which jurisdiction does processing take place? A server physically located in Europe but operated by an entity subject to US law does not answer the same question as processing carried out in Europe by a European entity.

A voice recording is personal data under the GDPR, and its content can be a great deal more sensitive still depending on what you dictate. These three points are detailed in our guide to voice dictation and GDPR. Professions bound by confidentiality obligations will find specific guidance in the guides dedicated to lawyers and accountants.

Key takeaways

Voice dictation is not a black box. It is a chain of four clearly identified operations, and almost every question people ask about it traces back to one specific stage of that chain.

• A disappointing result in a noisy environment happens at stage 1, and is fixed with a better microphone rather than better software.
• An error on a proper noun or an industry term happens at stage 2, and is fixed with a vocabulary list.
• Missing punctuation, hesitations left in place and text you have to rework happen at stage 3, which is simply absent from most built-in tools.
• Being unable to dictate in a given application happens at stage 4, and depends on the tool's architecture, not on the quality of its recognition.

That leaves the question running through the whole chain: the trade-off between the compute stages 2 and 3 demand and the confidentiality of what you dictate. That is the real underlying subject, and it is decided when you choose a tool, not afterwards.

Frequently asked questions