Extracting subtitles

Extract subtitles from a video, or find out there are none.

If the subtitles can be switched off in a player, they are a track, and one ffmpeg command copies them out: ffmpeg -i video.mkv -map 0:s:0 subtitles.srt. If they cannot be switched off, they are burned into the picture. There is no text left to extract, so you either read them off the frames with OCR or make a new file from the audio, which is what suubbs does.

Get the Mac app — $9.99 → Try it free in your browser
First, which kind

A track can be copied out. A burn-in cannot.

Subtitles reach a video file in one of three ways, and only one of them leaves text to take back out.

A text track. The subtitles sit inside the file as their own stream, beside the video and the audio, as timed text. MKV files usually carry SubRip or ASS tracks; MP4 and MOV files carry mov_text, the MPEG-4 timed-text format. A player lists them in its subtitle menu and lets you turn them off. These extract perfectly, because the words are still words.

An image track. DVDs and Blu-rays store subtitles as pictures of text (VobSub and PGS), and rips often keep them that way. They are a track, so they can be switched off and copied out, but what comes out is images. Turning those into an SRT is optical character recognition, not extraction.

Burned in. The words were drawn into the video frames before encoding, which is what social clips and most downloaded videos have. Nothing in the file knows there are subtitles. There is no stream to select and no command that lifts them out.

To see which you have without guessing, ask ffprobe, which comes with ffmpeg:

ffprobe -v error -select_streams s -show_entries stream=index,codec_name:stream_tags=language -of csv=p=0 video.mp4

Each line it prints is one subtitle track: its stream number, its format, and its language if the file says. On a test MP4 with English and Spanish tracks it printed 2,mov_text,eng and 3,mov_text,spa. If it prints nothing, the file has no subtitle track, and any subtitles you can see are part of the picture.

What ffprobe shows What it is How to get an SRT
subrip, ass, mov_text, webvttA text trackCopy it out with ffmpeg, below
hdmv_pgs_subtitle, dvd_subtitleAn image trackNot with ffmpeg: OCR it, or regenerate from the audio
Nothing at allNo track. Visible subtitles are burned inNothing to extract: regenerate from the audio
1 · Text tracks

Copying a subtitle track out with ffmpeg.

Every command below was run on test files with ffmpeg 6.1. On a Mac, ffmpeg installs through Homebrew.

The first subtitle track, as SRT

ffmpeg -i video.mp4 -map 0:s:0 subtitles.srt

-map 0:s:0 means "from the first input, the first subtitle stream". ffmpeg picks the output format from the extension, so the same command with subtitles.vtt writes WebVTT instead. Change the last number to take the second or third track: 0:s:1, 0:s:2.

One language, by its tag

ffmpeg -i video.mp4 -map 0:s:m:language:spa spanish.srt

This selects by the language tag instead of the position, which is handy when a file has a dozen tracks. Tags are three-letter codes such as eng, spa or fra. ffprobe shows which ones the file actually uses.

An ASS track, with its styling kept

ffmpeg -i video.mkv -map 0:s:1 -c copy subtitles.ass

-c copy writes the track out as it is stored. For an ASS track that keeps the styles, the positions and the override tags. Converting it to SRT instead works, but SRT has nowhere to put most of that. ffmpeg keeps the style's font, size, colour and bold as SRT tags and drops the rest. In our test, with a white, non-bold style, that came out as <font face="…" size="…"> on every line. Keep the .ass if you mean to burn it back in later.

When it says "matches no streams"

Stream map '0:s:0' matches no streams means just that: the file has no subtitle track. You have a burned-in video, or a video with no subtitles at all, and should skip ahead to the section on burned-in subtitles.

2 · Check what came out

An extracted file is only as good as the track was.

Extraction copies faithfully, including what was already wrong with it.

To fix any of it, open the extracted SRT or VTT in a subtitle editor rather than a text editor. suubbs imports SRT and WebVTT files directly: lines can be retimed against the waveform, the whole file shifted at once, and the text corrected and restyled before you export again. One thing to know: an SRT has no word timings, so suubbs spreads each line's words evenly across it on import. The line timings are exactly as extracted.

3 · Burned in, or images

No track to copy? Make the subtitles again.

Two routes, and which one fits depends on whether the words you need are on screen or in the audio.

From the pictures: OCR

Optical character recognition reads the text off the frames, which is the only way to recover on-screen text that was never spoken: a translated foreign-language film, signs, lyrics on a music video. It is also how image tracks from DVDs and Blu-rays become SRT. ffmpeg will not do this conversion. Asked to write a text format from an image track, it stops with "Subtitle encoding currently only possible from text to text or bitmap to bitmap", as it did in our test going the other way. Dedicated subtitle tools with OCR exist for this. suubbs is not one of them: it does not read text from video frames.

From the audio: speech recognition

If the subtitles were a transcript of what is said, which is true of nearly every burned-in caption on a talking-head clip, an interview or a tutorial, you do not need the old text. You need the words, timed, and those are in the soundtrack. That is what suubbs does:

  1. Open the video in suubbs and press Generate. On the Mac app the speech model runs on your Mac, so the video is not uploaded anywhere.
  2. Fix the names and the jargon the model got wrong. Every word carries its own timing, so correcting one does not disturb the lines around it.
  3. Export SubRip (.srt), WebVTT (.vtt) or ASS, whichever the next step wants.

The result is often better than the burned-in original would have been, because it is a fresh transcript with word-level timing rather than text copied from another cut. It is not a copy of the original wording, though. If the burned-in captions were edited, abridged or translated, only OCR gets that text back. The free browser version captions the first 60 seconds of a file, which is enough to judge the transcript on your own footage. The steps, settings and SRT details are on the SRT generator page, and audio-only files on audio to SRT.

A burned-in caption also stays in the picture. A new subtitle file cannot remove it. Putting a second set on top only makes sense if you position the new set elsewhere; see subtitle position.

Nothing to extract? Generate it.

[stream map matches no streams]

Questions

What people ask next.

How do I extract subtitles from an MP4?

If the MP4 has a subtitle track, run ffmpeg -i video.mp4 -map 0:s:0 subtitles.srt. Check first with ffprobe: if it lists no subtitle stream, the subtitles you see are burned into the picture and cannot be extracted.

How do I extract subtitles from an MKV file?

The same way: ffmpeg -i video.mkv -map 0:s:0 subtitles.srt. If the track is ASS and you want to keep its styling, copy it as it is with -map 0:s:0 -c copy subtitles.ass.

Can you extract burned-in subtitles?

Not as text, because they are part of the image. Either read them off the frames with OCR, or, if they were a transcript of the speech, generate a new subtitle file from the audio.

Why does ffmpeg say the stream map matches no streams?

The file has no subtitle track at the position you asked for, usually because it has none at all. Run ffprobe to list the subtitle streams the file does have.

Can ffmpeg convert PGS or DVD subtitles to SRT?

No. Those tracks are images, and ffmpeg only converts text to text or bitmap to bitmap. Converting them to SRT needs OCR in a dedicated subtitle tool.

Can I edit the subtitles after extracting them?

Yes. An extracted SRT or VTT opens in any subtitle editor. suubbs imports both, so you can retime, correct and restyle the lines and export again.

Keep reading

Related

No track? Make one from the audio.

[transcribed on your Mac]