HTML5 <video> and <canvas> on mobile devices

[Mikkel]

I've been exploring what you can do with HTML5, and the new options like canvas and video, but unfortunately I've run into a showstopper: Canvas can capture still frames from a video running in a <video> tag quite easily, but I haven't been able to do this successfully on a mobile device - tried it on an Asus TF300 (Android 4.0.3) and a HTC Desire (Android 2.2) so far.

The example is straight from w3schools (link: http://www.w3schools.com/html5/tryit.asp?filename=tryhtml5_canvas_drawimage_video), and works beautifully on a PC.

Is there any way to get similar functionality to work on mobile devices too? Both devices support both the <video> and the <canvas> tag, but seem unable to use them both.

[zcorpan]

I think <video> support on mobiles is generally bad even if it's there. However this will most likely get better over time (to be equivalent to desktop computers basically).

[Mikkel]

I think <video> support on mobiles is generally bad even if it's there. However this will most likely get better over time (to be equivalent to desktop computers basically).

Yeah that's my experience as well, <video> support is mostly ok (it plays), but could be better. What I was hoping for was *some* way it could work now, but that might just be too soon, more's the pity.

[zcorpan]

Well, Apple goes to great lengths to work around this problem: https://docs.google.com/document/pub?id ... jQHR_d1WsI

[JAB Creations]

This is a political issue you've run in to. Large corporations trying to create gardened pay walls. Both Apple (through Safari) and Microsoft (through Internet Explorer) intentionally don't support open source license/royalty-free codecs because they invested money in H.264 along with other companies. Because these companies have large amounts though not absolute control over everything that is why you see H.264 in most if not all electronics. Then there is the web which the last time a walled-garden tactic was successful Netscape disappeared and it took over a decade to get rid of Internet Explorer 6.

If you're working on an independent project abandon video on the iPhone until it's properly supported using open source free codecs. If you're contracting you should be charging a premium for supporting something that is non-standard. No, that is not debatable, it's not a standard and it's not acceptable that WHATWG refuses to enforce proper policies. If you're on a corporate ladder advice those you work for the folly of supporting something that will not be successful in the long term and shift your strategies to something viable.

[Mikkel]

This is a political issue you've run in to. Large corporations trying to create gardened pay walls. Both Apple (through Safari) and Microsoft (through Internet Explorer) intentionally don't support open source license/royalty-free codecs because they invested money in H.264 along with other companies. Because these companies have large amounts though not absolute control over everything that is why you see H.264 in most if not all electronics. Then there is the web which the last time a walled-garden tactic was successful Netscape disappeared and it took over a decade to get rid of Internet Explorer 6.

IE6 *shudder*
Politics and big businesses *double shudder*

If you're working on an independent project abandon video on the iPhone until it's properly supported using open source free codecs. If you're contracting you should be charging a premium for supporting something that is non-standard. No, that is not debatable, it's not a standard and it's not acceptable that WHATWG refuses to enforce proper policies. If you're on a corporate ladder advice those you work for the folly of supporting something that will not be successful in the long term and shift your strategies to something viable.

Thank you for your post. I work in-house, on a project that will be used both in-house and by external contractors, but I do thankfully have a relatively large amount of influence on what devices and platforms we choose to support - but the more, the better. That said, if iOS is out, at least for the time being, I can live with that - but I would really like to support Android. Any ways to get <canvas> and <video> to play nice on Android yet?

[zcorpan]

Codecs is a separate issue. It's not the reason why video can't be painted on a canvas on mobile...

I think the best thing you can do is file bugs with browser vendors pushing for this to be fixed.

[Mikkel]

Codecs is a separate issue. It's not the reason why video can't be painted on a canvas on mobile...

I think the best thing you can do is file bugs with browser vendors pushing for this to be fixed.

You're right re: codecs, since all the videos play fine (using whichever they prefer of webm, mp4 or ogv), even on the mobile devices in question.

I reported it to Google via the Report an Issue function in the Android version of Chrome.

[Mikkel]

Oh, additional and possibly very relevant information:

I don't actually need the video to play real-time inside the canvas, I just need canvas (or something else!) to be able to take still-frames from a paused video playing in a <video> tag, and save those still-frames on the server. Later on I will manipulate those still-images with <canvas>, but that part seems doable on most devices already, as <canvas> is relatively well supported, at least on the devices I've experimented with so far.

[JAB Creations]

If you're in a corporate environment with enough resources then (as much as I sip resources on a server while others are doing college frat drinking parties chugging until they choke to death) I think if you're looking for a cross-browser/phone/platform solution you may want to consider having this executed at the server side? I don't specialize in code mixing with media so I'm not sure what the approach would be specifically. If you're trying to capture a frame from a specific point in the video then if there is at least a cross-whatever method to determine which frame in the video has been loaded then you could make a request to the server for a URL that will pull the video, find the frame and return that (from the server) as the background-image for the Canvas element?

[zcorpan]

Yeah, doing it server-side seems like a sane option. This was demonstrated at the FOMS workshop, actually.

http://www.foms-workshop.org/foms2012/p ... deoEditors
http://lac.linuxaudio.org/2012/papers/13.pdf (email the author of this paper if you're interested in his implementation)