Word-level timestamps

Word-level timestamps from any video, no code.

Word-level timestamps give every word in a transcript its own start and end time, instead of one time per subtitle line. To get them from a video without writing code, transcribe it in suubbs and export Word-level JSON: every word with its start and end in seconds, a confidence score, and a flag saying whether its timing was measured or estimated. In the Mac app those boundaries measure at about 3 ms against known speech onsets; the free browser version is looser, roughly 60 to 140 ms, and covers the first 60 seconds of a video.

Get the Mac app — $9.99 → Try it free in your browser
1 · The steps

From a video to word timings in four steps.

Everything runs on your own machine. The video is read off your disk and never uploaded.

  1. Drop the video into the studio — MP4, MOV or WebM — in the Mac app, or in a browser tab for the free version (Chrome or Edge are the fast ones).
  2. Press Generate. In the Mac app, leave DTW word alignment and Refine boundaries from the waveform switched on; the next section explains what each one buys you. In the browser the equivalent switch is Refine word timing from audio.
  3. Correct any misheard words. Click the line and retype the wrong word. The words you leave alone keep their measured timing. A corrected word was never measured, so it gets an estimated time within its line and is exported with edited: true and aligned: false; adding or removing a word re-estimates the later words in that line too. Fix words in place where you can.
  4. Export Word-level JSON from the ··· menu. The same transcript can also go out as SRT, VTT or ASS at any point, so you are not choosing one or the other.
2 · How they are made

Why raw Whisper word times run late.

A speech model like Whisper is trained to write down what was said. When each word was said is worked out afterwards, and that is where the error comes from.

Whisper produces text one token at a time. To place each word in time, the usual technique looks at which slice of the audio the model was attending to as it wrote that word, and lines the two sequences up with dynamic time warping (DTW). It works, and it is a good first answer. But attention is a proxy for timing, not a measurement of it, and it tends to be late at exactly the places a caption needs to be right: the start of a phrase after a pause.

suubbs measured this on its own test audio. DTW alone came out at a 143 ms mean error against known speech onsets. DTW followed by a second, acoustic pass came out at about 3 ms. That second pass is the Refine boundaries from the waveform switch, and it does three things:

The Mac app also runs Silero voice-activity detection in front of the model, which is what stops long silences being transcribed as hallucinated speech — and a hallucinated word is a word with a timestamp that points at nothing.

The browser version runs smaller Whisper models through Transformers.js, which is why its word timings are rougher. They are fine for a transcript you want to search, and for deciding where a caption line should break. For anything that has to land on a syllable, use the Mac app. Whisper on a Mac compares the whole pipeline against running whisper.cpp yourself.

3 · The file

What the Word-level JSON contains.

The words are grouped into the same lines you see in the editor, so the file carries both levels at once.

{
  "version": 1,
  "language": "en",
  "style": { … },
  "cues": [
    {
      "id": "…",
      "start": 2.1,
      "end": 4.4,
      "words": [
        { "text": "You",   "start": 2.1,  "end": 2.26, "conf": 0.97, "edited": false, "aligned": true },
        { "text": "could", "start": 2.26, "end": 2.48, "conf": 0.95, "edited": false, "aligned": true },
        …
      ],
      "lineBreaks": [],
      "origin": "asr",
      "locked": false
    }
  ]
}

The numbers in that example are illustrative; the field names are the real ones.

If you want a flat word list, take every words array in order. Read language before you process text: the Mac app can detect it, and it is the language the transcript is actually in.

4 · What people build

What word timings are for.

Anything that needs to point at a moment in the video by what was said.

5 · The limits

Where word timings are only estimates.

Each of these is visible in the file, which is the point of the aligned flag.

If you would rather script it, open-source Whisper implementations can return word timings too. Check each project's own documentation for the current option, and measure the output against the audio before relying on it — late boundaries at phrase starts are the thing to look for.

What people ask next

Word timing questions.

Including the ones where the answer is a limit.

What are word-level timestamps?

A start and end time for every individual word in a transcript, rather than one start and end per subtitle line. They are what word-by-word captions, editing by text and search inside video are built on.

Does Whisper give word-level timestamps?

It can infer them, usually by aligning the model's attention with the audio. Those raw times tend to run late at the start of phrases: suubbs measured 143 ms mean error for alignment alone and about 3 ms once boundaries were snapped to real pauses in the waveform.

How accurate are suubbs' word timestamps?

About 3 ms against known speech onsets in the Mac app, and roughly 60 to 140 ms in the browser version. Inside continuous speech, where there is no pause to measure against, a boundary is left where the model placed it.

Can I get word timestamps from an SRT file?

Not real ones. SRT only records when each line starts and ends. suubbs can import one and spread the words evenly across each line, but those are estimates, marked aligned: false in the JSON. Generate from the video for measured timings.

Which export keeps the word timings?

Word-level JSON keeps them as data. The ASS export uses them to draw a highlight that moves word by word. SRT and VTT keep only line timings.

Is it free?

The browser version is free, needs no account, and transcribes the first 60 seconds of any video, with export working on that minute. The Mac app is $9.99 once and transcribes the whole video, however long.

Keep reading

Related

Every word, with its own clock.

"start": 2.26, "end": 2.48