I am trying to build a web player that is able to stream MPEG DASH, but with SRD information in the manifest. I have a problem at the “gluing” of the tiles part.
How the data is set:
I have a video of a resolution of XxY, which I’ve divided in tiles of NxM, meaning that I now have the NxM videos of the same duration as the original one, but which is reduced to a smaller resolution, a portion from the original (seeing only the upper left corner for example).
I have fragmented those tiles for the MPEG DASH streaming format, meaning an init.mp4 and the m4s fragments for each tile. To note that I have the tiles in multiple bitrate, in order to imitate some adaptive streaming.
The “gluing” part:
The gluing part consists of gluing the tile togheter into 1 big tile, aka the original video. The SRD information allows me to know where each tile should be position to reconstruct the original video shape. Once I have the spatial information from the manifest, I use this command to glue the tiles togheter:
ffmpeg.exe -i v1 -i v2 -i v3 -i .v4 -i v5 -i v6 -i v7 -i v8 -i v9 -filter_complex "[0:v][1:v][2:v][3:v][4:v][5:v][6:v][7:v][8:v]xstack=inputs=9:layout=0_0|w0_0|w0+w1_0|0_h0|w3_h0|w3+w4_h0|0_h0+h3|w6_h0+h3|w6+w7_h0+h3" out.mp4
The command above allows me to “glue” together tiles from a grid of 3×3 tiles, into 1 big tile.
The web streaming method:
To stream, I use JavaScript MediaSource, and I am appending the fragments to the buffer continuously, like this reference
The issue:
The issue is that the FFMPEG gluing command only works on proper mp4 videos, not with fragments (m4s). I cannot glue the wanted m4s tiles back into a bigger one. However, the JavaScript player only takes fragments, meaning that I cannot just send continuous mp4 video.
I have tried many things with fragments and full videos, but the only solution I found for this chicken-egg problem is to:
- Create video clips with smaller duration for each tile, at each bitrate.
- Glue the desired video clips tiles together with the FFMpeg command.
- Convert that glued video in one big fragment, and send it.
However, I feel that it is not a clean method and I fear that I am missing something.
Is there a better solution that I am not aware of?
Any idea suggestions are welcome.
Thank you for your time.