I have received a very specific requirement from a client and am not sure what solution to use.
My client would like to play an online streaming video while overlaying texts and images in a sort of augmented reality fashion. These texts and images would be tempered with/animated to fit accordingly in the video (stretched, skewed, rotated, alpha’ed etc) and these assets could possibly have to be animated TO “FOLLOW” on the videos themselves.
Therefore it would be way more than just the “average” video player. The solution is probably to write a component that uses OpenGL but I have no “web” AR experience.
Are there any materials/research on this subject I could start with?
Last but not least, there are a few additional constraints:
– Compatibility with mobiles (Android / iOS) which prevents using Flash.
– We are just talking “web” here (no opening for mobile apps).
– Finally, I’d prefer to avoid the installation of plug-ins (i.e. Firefox plug-ins)
Thanks!
4
WebGL
WebGL could be a solution, but mobile support is going to be rather sparse, and requires a plugin activation even for not so distant versions of modern browsers.
A Custom Video Renderer
You could probably do away with implementing a video renderer using a Canvas, though I’m not sure how much fun this would be… (Well this would quite fun as a programmer I guess, but probably not so great within the timeframe and constraints of a commercial project).
Video with Canvas Overlay
Maybe look at these blog posts for inspiration, that’s mostly what I had in mind when I read your question:
- http://html5doctor.com/video-canvas-magic/
- https://developer.mozilla.org/en-US/docs/Web/HTML/Manipulating_video_using_canvas
It’s definitely doable, though it’s going to take some work. Also, make sure to clarify all these platform requirements and see what will or won’t work, and to see if you can have dumbed-down alternatives for platforms you can’t support (e.g. maybe just have video support and no interactivity, instead of nothing at all, etc…).
Tag Support
Tag support should be decent enough on modern platforms for <canvas> and <video>, as documented here and here, but make sure you test this properly with a quick prototype first. For instance, WebGL support isn’t so great (as of October 2014), and canvas blending might also be a bit deficient (as of October 2014, it kills IE support for you). Beware of the video format you use as well.
So depending on how you want to approach this, you may have to roll out different solutions for different platforms.
1