You don't need to understand online video to consume it, but anyone intent on producing the stuff has to know the basics. Simon Frost, technical architect for the BBC's iPlayer and a speaker at the upcoming Velocity conference, fills in some of the most common web video knowledge gaps in the following Q&A.
Q: What is buffering and why does it happen so often?
Video files are very large compared to what is normally distributed on the Internet. To avoid having the user download the whole thing before viewing, you can allow the user to start watching while it downloads or streams. If the data coming in is interrupted in any way, you end up with "buffering" and the video stream stops.
The root causes of this are: inadequate bandwidth to deliver the video (at both the client and server end), poor network latency, and no prioritization of network traffic. The Internet's designers made a choice to create a network that would ensure that data sent between two points would eventually be delivered, with the trade-off that it might be slow. The alternative would have been a network that could deliver data faster, but with the risk that it would sometimes go missing.
Broadcast TV networks control their available bandwidth, and it's one-way. On the Internet, adding new "receivers" increases bandwidth demands on both sides of the connection so bandwidth quickly becomes limited.
Network latency is an issue as each data packet can take its own route, and there's no guarantee of the video data going the shortest possible route, as happens in TV networks. When you turn on your TV, the data hasn't been routed from one end of the country to the other and back, as it might be on ISPs' backbones.
The final problem is all traffic is equal on the Internet, irrespective of what it is. That email that won't be required by the other party for a few hours is in contention with the video that you're watching. TV networks, on the other hand, are run on guaranteed radio spectrum where they're the only data streams.
Compared to broadcast networks, the Internet is not an information superhighway. It's more like a public transit system, where journeys can take a longer roundabout route. But the Internet enables us to provide services that broadcast networks can't, from true on-demand video and the short-form content of YouTube and Vimeo, to interactive TV experiences.
Q: What do users expect out of online video?
Video is a linear, narrative experience. If a video stream stops, you break the narrative in the user's head, something they're unlikely to forgive for very long.
Research suggests that if content doesn't start to play quickly or buffers, users will click away in large numbers. It also reports that users watch more content if the video is transcoded at high, near-television bitrates.
Users also want control over playback, but I'm not sure there's a clear indication yet as to what is best. Many users tell me they like the DVR-style concept of stopping and starting again at a later date from the same point in the content. So when they come back to a web page, it starts from when it left off. Scrubber bars are another common control you see, and there's some great implementations where these are marked up with important points in the content.
Q: How is online video delivered?
The common methods are download and streaming and, but there are variants of each.
Download is what we've come to expect from a web experience, but users don't want to wait, so platforms like Flash have support for progressive download. This is what you'll see on YouTube and Vimeo, where you can watch content as you download. Downloading doesn't make the best use of resources at the server. Content will download irrespective of whether a user is watching, so if you pause a progressive download video it continues to stream. Streaming allows a server and client more control over how data is sent.
Peer-to-peer (P2P) is another option. It removes centralized download servers by creating on-the-fly distribution among users' computers. It hasn't quite taken off as people expected. Content delivery networks (CDNs) have driven down the cost of centralized distribution, and users don't like how peer-to-peer can make use of their connections beyond what a simple download does.
Q: What kind of infrastructure is needed to serve video to thousands of users?
Streaming and downloading requires keeping server connections open for a long time, unlike the bursty traffic of traditional web media.
You would hope that a modern server could handle around a thousand concurrent streams, so that's the first point you have to look to scale at. A thousand streams of TV-quality video is going to require around 1500 Mbps of bandwidth. Add in the complexity of delivering that data to multiple locations worldwide at low latency and you can quickly hit scalability issues at small numbers of users, never mind tens or hundreds of thousand of users.
In reality, the best option is to go with one or more CDNs. They can offer large amounts of bandwidth, and massive numbers of geo-located servers, which are all available in a elastic way to help you deal with peaks and troughs. They typically offer download and streaming options, and will cache your files.
A CDN will only get you part of the way, though. You're still dependent on equipment and network connections downstream from the edge servers, and this is where there is currently plenty of companies offering techniques such as adaptive bitrate and technologies such as last-mile caching to ISPs.
Q: What are the big issues with mobile video?
The problems of mobile video are the same as the mobile web. Proliferation of multiple devices, with different codecs and screen sizes, and, outside the major smartphones, a limited upgrade path for software capabilities.
Mobile networks also have their own bandwidth and latency issues, if the network operator even allows video in the first place. Many video services currently favor downloading or Wi-Fi-based streaming.
Q: What affect will HTML5 have on online video?
The iPad and iPhone provide a great place to experiment by providing a standard platform and engaged users. There's still quite a way to go before HTML5 becomes standard for the desktop, though.
Codecs are still an issue. HTML5 doesn't mandate a codec and support is currently split between H.264 and Ogg Theora. Some browser developers such as Mozilla and Opera don't like H.264, as it's patent-encumbered. Currently, no one has to pay licence fees to decode H.264 on the web, but that doesn't mean they won't in the future. Google and Apple have already paid licences, so they're happy to include support in their browsers. Mozilla and Opera do support the open source codec Ogg Theora, but this is not as good as H.264 and it's still unclear if Ogg Theora is subject to the same patent claims. There are rumors that Google will open up the VP8 codec it acquired when it bought On2, so this might give a third option for people to agree on. I'm sure patent holders will scrutinize this closely as well. [Note: The rumors were true. Google recently open-sourced VP8.]
Finally, digital rights management (DRM) is still a major requirement from content producers for many services. HTML5 obviously doesn't support this. Until we see a standard here, or unless the industry follows the music business and drops requirements for it, this is going to be a major barrier to adoption for some.