Wikipedia has comparison of various common video container formats, but while it serves as a great overview, I think that it falls short in reviewing the quality of real-world implementations. I realize that such information would not only be quite out of scope of Wikipedia, but also inevitably quite subjective. Nonetheless, this leaves a bit of a gaping hole and perhaps quite a false impression.

They correctly assess AVI as problematic on almost all fronts: B-frames aren’t in the specification and thus need third-party modifications, audio at a variable bit rate isn’t supported by Microsoft’s preferred handler (ACM), and so on. Implementations that follow the spec too closely (such as older versions of QuickTime) ironically suffer as most people expect implementations to mimic each other’s custom extensions and behaviors. Effectively, AVI file playback is unreliable and hack-ish. Despite that, AVI is without question still among the most popular container formats, probably due to its very ubiquity. Virtually any computer comes with some player that will work with the container; it’s only a matter of having the right codecs as well.

Microsoft has been far less successful pushing newer container formats through. ASF, for example, or the almost unheard-of AAF. They failed to sell the format to the industry, who saw no advantages over the already widespread QuickTime MOV format. Meanwhile, consumers had no reason to care – as far as they could tell, the existing AVI was just fine and dandy.

QuickTime’s container format, MOV, is similarly old to AVI, but has been designed with much foresight – as a look at Wikipedia’s table will reveal, there is virtually nothing it can’t do. Due to its capability to edit in-place (and due to third parties writing software and hardware around QuickTime), it became hugely popular in the 1990s’ production industry, and hasn’t had that position challenged much at all. This was hardly relevant for consumers, who didn’t have anywhere near the hardware specs (or inclination) to edit movies. What hurt the adoption as a playback format much more, though, was Apple’s incompetent Windows port, which right from the start gave Apple an (undeserved, but not unsurprising) reputation that they deliberately cripple their Windows apps as to make Mac OS look superior. Even today, QuickTime for Windows has various inexcusable flaws and annoyances that discourage people from installing it, resorting to semi-legal alternatives instead.

Ogg Media hasn’t gained much adoption outside Xiph.org’s own formats, such as FLAC, Vorbis and the even more rare Theora. Worse, it also doesn’t support H.264/AVC/MPEG 4 Part 10, which many vendors are basing codecs on these days (succeeding MPEG 4 Part 2).

MP4 and 3GP, both based mostly on MOV, inherit many of its features, but rely on restrictive licensing and a predefined, very limited selection of codecs. The same goes for VOB (not based on MOV), which has mostly remained in the domain of DVD-Video discs. NUT, MCF, Adobe Flash Video, and others have thankfully remained in their respective places as well, and better yet, RealNetworks’s RealMedia has been fading away. DivX (not to be confused by the DivX codecs themselves) is frequently listed as its own media format but is actually just a proprietary extension to AVI.

In terms of flexible, scalable, future-proof formats, this leaves us – aside from MOV – with only one: Matroska, which has better license than every single alternative: the specification is entirely in the public domain. Certainly a huge advantage over MOV (and also all others), though in comparison, it lacks in-place, making it decidedly a playback format, not an editing one, not that there’s anything wrong with that. Like MOV, Matroska allows for any codecs, regardless of their complexity. Whereas QuickTime provides its own component system (since there only is one official implementation: Apple’s own), each actual implementation of Matroska has to take care of this on its own, but that should not be problematic in practice.

However, performance and stability are currently an issue. The two major implementations on Mac OS X are in the form of VLC media player (which goes into detail here) and Perian, a QuickTime component, which apparently wraps libmatroska into a MOV importer: effectively, a QuickTime application works with a MOV file that was generated from the original. It goes without saying that this method is slow; I’ve also found it to achieve low framerates especially while the importing still takes place, but even afterwards. With various files I’ve tested, all using H.264, I did not end up with a full MOV representation of the original – while no recoding takes place (and therefore, no lossy conversion either), many frames were dropped. To illustrate the difference, from one such 1.7 GBs test file, the importer created a result at about 800 MBs: a loss of about half the frames. The Perian team chose to do an import (rather than hacking the media handler to work with the Matroska file directly) in order to work around Matroska’s indexing limitations, thus enabling better support for alternate tracks.

VLC media player works natively with the files instead, but while it doesn’t suffer performance problems to anywhere near the same extent, it fails in another way: it’s crashy. Watching a movie with pausing and scrubbing is a major pain when you don’t know if the next un-pausing might crash the application, at which point you better remember where you left off and scrub to the same position, in hopes that doing so won’t crash it yet again.

I do love the concept of having nobody own a container format description, of course. But the reality is a little more complicated. Beyond the implementation issues illustrated above (the situation may be better on Windows or other operating systems; haven’t checked, but it’s clearly in its infancy), there’s also commercial adoption. Companies have traditionally been reluctant to accept something with no involvement of ISO, SMPTE or MPEG, and while the open nature of the specification will without doubt help, it might not be enough.

I’m hopeful that Matroska will spread. It is by far the most promising offering for playback, and for recording and editing, MOV will continue to serve us well for years to come.

Update: Yuvi provides some insight into particular problems. Thanks!