I’m setting up a service to transcode input videos, generally MP4, for HLS with MPEG-TS as the container format. This currently works fine when starting from scratch, but I would like the transcode to be resumable in case the service crashes while working on a longer video.

I’m running into two issues with resuming the transcoding process:

• ffmpeg generates too many segments: if the processing is halted at, say, 10 segments for a video that should be 20 segments long and processing is resumed later, the manifest contains 10 + 20 segments, where the final 10 segments don’t actually play back properly
• Adjacent to this, running the command again when the video is fully transcoded results in the playlist being appended with the entire video once more, with the numbers above that would result in 20 additional segments being added to the playlist every time the conversion is ran. In other words, ffmpeg doesn’t realize the transcoding is already complete

The desired result here is that running the ffmpeg command again simply continues the transcode after the last completed segment and appends these segments to the m3u8 playlist. If all the segments are already present ffmpeg should not output or modify anything. I’ve tried to coax ffmpeg into this behaviour, but no combination of -start_number, -ss and append_list gets me there.

Here’s my current command that is executed when a transcode should be resumed (I’ve ommited some filters for brevity):

ffmpeg -i input -map 0:v -map 0:a -map 0:v -map 0:a -map 0:v -map 0:a 
-c:v libx264 -preset veryfast -c:a libfdk_aac 
-maxrate:v:0 1500K -b:a:0 128k 
-maxrate:v:1 3000K -b:a:1 192k  
-maxrate:v:2 6000K -b:a:2 256k 
-flags +cgop -g 60 -hls_time 5 -hls_list_size 0 
-hls_flags independent_segments+temp_file+append_list 
-var_stream_map "v:0,a:0,name:480p v:1,a:1,name:720p v:2,a:2,name:1080p" 
-y -f hls -master_pl_name master.m3u8 ./%v.m3u8

Here is the folder structure that ffmpeg would work in for a resumed transcode, assuming the service crashed after the first two chunks:

.
├── 1080p.m3u8   
├── 1080p0.ts    
├── 1080p1.ts   
├── 480p.m3u8    
├── 480p0.ts     
├── 480p1.ts     
├── 720p.m3u8    
├── 720p0.ts     
├── 720p1.ts     
├── input        
└── master.m3u8  

Here is the contents of 480p.m3u8:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:5.000000,
480p0.ts
#EXTINF:5.000000,
480p1.ts
#EXT-X-ENDLIST

However, as explained in the prelude, running the command against this folder results in too many chunks being appended to the playlist.

This is the expected 480p playlist:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:5.000000,
480p0.ts
#EXTINF:5.000000,
480p1.ts
#EXTINF:5.000000,
480p2.ts
#EXTINF:5.000000,
480p3.ts
#EXTINF:5.000000,
480p4.ts
#EXTINF:5.733333,
480p5.ts
#EXTINF:5.000000,
480p6.ts
#EXTINF:5.000000,
480p7.ts
#EXTINF:5.000000,
480p8.ts
#EXTINF:5.000000,
480p9.ts
#EXTINF:5.000000,
480p10.ts
#EXTINF:5.000000,
480p11.ts
#EXTINF:5.000000,
480p12.ts
#EXTINF:3.950000,
480p13.ts
#EXT-X-ENDLIST

And this is the actual playlist:

#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:5.000000,
480p0.ts
#EXTINF:5.000000,
480p1.ts
#EXT-X-DISCONTINUITY
#EXTINF:5.000000,
480p2.ts
#EXTINF:5.000000,
480p3.ts
#EXTINF:5.000000,
480p4.ts
#EXTINF:5.000000,
480p5.ts
#EXTINF:5.000000,
480p6.ts
#EXTINF:5.733333,
480p7.ts
#EXTINF:5.000000,
480p8.ts
#EXTINF:5.000000,
480p9.ts
#EXTINF:5.000000,
480p10.ts
#EXTINF:5.000000,
480p11.ts
#EXTINF:5.000000,
480p12.ts
#EXTINF:5.000000,
480p13.ts
#EXTINF:5.000000,
480p14.ts
#EXTINF:3.950000,
480p15.ts
#EXT-X-ENDLIST

I’ve already surmised the #EXT-X-DISCONTINUITY tag can’t be disabled, which is fine, as I can remove that with post-processing steps. The two extra segments are problematic however, as they break the video at the end of its actual runtime.

To bring this to a point: is there a way to get ffmpeg to resume transcoding from the last segment, such that it does not append the entire video to the last transcoding attempt?