How 'Apple Streaming' Works

One of the major issues with video delivery over the internet is that it is based on patents going back a decade or more, and the industry has already seen its fair share of lawsuits regarding patent infringements.

This, perhaps, is the reason that Apple took a very different approach to delivering video to its iOS devices.

If you're looking to do live or adaptive streaming you have to use a format called 'M3U8'. Tis started life as an indexing file for MP3 video files, but was adapted by Apple as a means of splitting up and then reforming something called an MPEG-2 transport stream.

MPEG2 is the traditional codec used by broadcasters, and much of the television you will watch will be in this format. It isn't widely used in internet TV since the file sizes tend to be very big and the compression isn't very effective at lower bitrates (hence the popularity of MP4, or MPEG-4 for internet delivery).

Another objective that Apple seem to have set was to ensure that video delivery can be handled over port 80 using http. In other words, that it could travel using the same route as an old web page, hence making it more widely available in a world where 'reserved ports', such as 1935 for rtmp, are blocked by firewalls and the like.

The way it works is that an input stream is transcoded, or rather 'split up, into thousands of individual files with the extension '.ts'. These can carry the signal at multiple data rates, hence allowing for the data rate to be dynamically increased or decreased according to the viewer's available bandwidth. All of these files are then 'managed' by the index, or M3U8 file, which knows how to reassemble them and to adapt the stream.

It's actually very similar to the way the delivery of traditional browser based content works over the world wide web using http, but it is a very strange and rather crude approach to video delivery.

However, it seems to work pretty well. There are exceptions - sporting events, or events with a lot of movement, or when the adaptive rates are set too close together and the stream cannot 'adapt' quickly enough.

Another issue is that the setup of equipment to output M3U8 is far from easy - most of the tools available to do this are command based LINUX applications that require a fair degree of experience to use effectively. But gradually I'm sure we'll see these being automated and built into existing encoding solutions.

Delivering Live Mobile Video

Mobile video is a difficult subject, and mobile live video is even more difficult. The range of devices, networks and supported protocols are immense.

The first thing to do is to be pragmatic. Outside of very good 3G coverage, which is sparse, and 4G networks, which are rare outside of Asia and some American conurbations, WiFi and WiMax are really basic requirements for delivering any kind of reliable video to mobile devices.

Another basic problem with mobile phones is that they are, er..., mobile, and reliably handing over data services from cell to cell (or from one WiFI connection to the next) is a tough thing to do. So, the second things is to be stationary.

So, presuming you're standing still with a decent connection, the next thing to worry about is the device you're holding in your hand. If it's more than a couple of years old, and isn't a 'smartphone', then you should again probably give up. There is a very steep diminishing law of returns in trying to reach all mobile devices.

Older handsets primarily used the rtsp protocol and a format called 3GP, or H.263. The rtsp protocol isn't good at high data rates and the 3GP codec means you have to support yet another format. (You can stream H.264 over rtsp, but the devices often have no means of playing this back).

So, let's move back into the modern age.

Today there are five broad platforms you need to support:

• iOS - the Apple operating system used on iPads and iPhones
• Android - the wildly popular open source platform from Android
• Symbian - the underlying OS used by Nokia (and certain other manufacturers)
• Windows Mobile 9 - the Microsoft mobile OS used by a variety of manufacturers
• Blackberry OS - the platform for the eponymous devices favoured by businessmen and texters alike

Unfortunately, each one of these devices requires a slightly different approach for delivering live video, which ostensibly means taking a live feed or simulcast and encoding it into a number of formats and then delivering it over a number of different networks. Then comes the really tough bit - scaling the solution. Finally, there are the issues of detecting the device being used and providing a playout environment - a player, navigation and associated functions and applications such as voting, tweeting and sharing.

Over the coming weeks we're going to cover delivery to each of these device platforms, as well as looking at on demand delivery of video.