The ultimate performance leap could come from... about reel

[pisi]

Interesting idea indeed! I tried Google some Javascript video decoding techniques to at least look into what possibilities may lay in there, but couldn't find anything reasonable. Maybe you know some links I should look at.

The main question here is whether it will be able to advance step by step both forwards and backwards. There also are sometimes rather drastic changes in playback speed when manipulated by the user. From my rather limited knowledge of video codecs, they utilize keyframes (which are saved as an entire image) and in between these keyframes, regular frames are calculated based on what has changed compared to the preceeding frame. And this doesn't quite allow playback in reverse direction. Please correct me, if I'm wrong (I'm not into video much).

I also kinda fear the added complexity for the user for the actual movie creation - even the compositing work on sprites baffle many users.

But again, a very nice idea!

[FlixFlix]

• "[...] even the compositing work on sprites baffle many users"
  Yeah, that includes me, too. I mean it's not a complex thing to do, but it involves significant manual labor.
• "[...] the added complexity for the user for the actual movie creation"
  Not only do most video editors handle image sequences gracefully, there are plenty of command-line encoders that do the same. In fact, this truly is easier than sprite compositing.
• "[...] advance step by step [...] changes in playback speed [...] video codecs utilize keyframes [...] reverse direction [...] correct me if I'm wrong."
  Actually, you're 100% correct and your concerns are all valid. However, my original suggestion was not to handle a video object as a whole; instead, decode it into memory and then use the resulting image sequence exactly the same way as you're currently doing it.
• some links I should look at."
  When I said that "it's been done before", I didn't realize that what I found ("JSONVID") was essentially just a MJPEG implementation. It's still helpful as a video player whereby I can force a clip to autoplay on an iPhone Safari inline (not fullscreen), but as far as compression efficiency, MJPEG is inadequate by definition.

I think if you would ask someone working on https://github.com/libav or otherwise connected with the libavcodec, you would get some good answers. As creator of the now-famous Reel, I'm sure you'd have a better chance of finding people willing to help. Remember, "all you need" is a stripped-down, single-format, ultra-limited and totally non-flexible video decoding algorithm.

[pisi]

Thanks for the suggestions. A really appreciate it! Although very appealing idea in terms of saving bandwidth, I kind of don't like it as a solution just to clear off the manual labour. There already is a way how to eliminate the task of compiling the sprite - using a sequence of individual images, which doesn't need any post-processing like this and is easy on memory.

I'd much rather see tools for mitigating this tedious labour, then introduction of another format, which doesn't have this labour associated. Something like already existing Josh's Python script. I long have a plan to port it into Ruby.

[SimonEast]

Yeah, videos traditionally are not great at playing backwards. And I think you'll find that you'd get BANDWIDTH improvements from using encoded video, but not PERFORMANCE benefits. I think the CPU usage might actually be higher from trying to play frames in reverse (where the computer has to calculate multiple frames just to display 1, because of intra-frame encoding). Decoding H.264 video is already very CPU intensive.

[pisi]

Exactly, using images is the lightest solution so far, so I took the liberty to close this issue (albait fairly interesting one). But the discussion can keep on in here.