How 'Apple Streaming' Works

One of the major issues with video delivery over the internet is that it is based on patents going back a decade or more, and the industry has already seen its fair share of lawsuits regarding patent infringements.

This, perhaps, is the reason that Apple took a very different approach to delivering video to its iOS devices.

If you're looking to do live or adaptive streaming you have to use a format called 'M3U8'. Tis started life as an indexing file for MP3 video files, but was adapted by Apple as a means of splitting up and then reforming something called an MPEG-2 transport stream.

MPEG2 is the traditional codec used by broadcasters, and much of the television you will watch will be in this format. It isn't widely used in internet TV since the file sizes tend to be very big and the compression isn't very effective at lower bitrates (hence the popularity of MP4, or MPEG-4 for internet delivery).

Another objective that Apple seem to have set was to ensure that video delivery can be handled over port 80 using http. In other words, that it could travel using the same route as an old web page, hence making it more widely available in a world where 'reserved ports', such as 1935 for rtmp, are blocked by firewalls and the like.

The way it works is that an input stream is transcoded, or rather 'split up, into thousands of individual files with the extension '.ts'. These can carry the signal at multiple data rates, hence allowing for the data rate to be dynamically increased or decreased according to the viewer's available bandwidth. All of these files are then 'managed' by the index, or M3U8 file, which knows how to reassemble them and to adapt the stream.

It's actually very similar to the way the delivery of traditional browser based content works over the world wide web using http, but it is a very strange and rather crude approach to video delivery.

However, it seems to work pretty well. There are exceptions - sporting events, or events with a lot of movement, or when the adaptive rates are set too close together and the stream cannot 'adapt' quickly enough.

Another issue is that the setup of equipment to output M3U8 is far from easy - most of the tools available to do this are command based LINUX applications that require a fair degree of experience to use effectively. But gradually I'm sure we'll see these being automated and built into existing encoding solutions.

Delivering Live Mobile Video

Mobile video is a difficult subject, and mobile live video is even more difficult. The range of devices, networks and supported protocols are immense.

The first thing to do is to be pragmatic. Outside of very good 3G coverage, which is sparse, and 4G networks, which are rare outside of Asia and some American conurbations, WiFi and WiMax are really basic requirements for delivering any kind of reliable video to mobile devices.

Another basic problem with mobile phones is that they are, er..., mobile, and reliably handing over data services from cell to cell (or from one WiFI connection to the next) is a tough thing to do. So, the second things is to be stationary.

So, presuming you're standing still with a decent connection, the next thing to worry about is the device you're holding in your hand. If it's more than a couple of years old, and isn't a 'smartphone', then you should again probably give up. There is a very steep diminishing law of returns in trying to reach all mobile devices.

Older handsets primarily used the rtsp protocol and a format called 3GP, or H.263. The rtsp protocol isn't good at high data rates and the 3GP codec means you have to support yet another format. (You can stream H.264 over rtsp, but the devices often have no means of playing this back).

So, let's move back into the modern age.

Today there are five broad platforms you need to support:

• iOS - the Apple operating system used on iPads and iPhones
• Android - the wildly popular open source platform from Android
• Symbian - the underlying OS used by Nokia (and certain other manufacturers)
• Windows Mobile 9 - the Microsoft mobile OS used by a variety of manufacturers
• Blackberry OS - the platform for the eponymous devices favoured by businessmen and texters alike

Unfortunately, each one of these devices requires a slightly different approach for delivering live video, which ostensibly means taking a live feed or simulcast and encoding it into a number of formats and then delivering it over a number of different networks. Then comes the really tough bit - scaling the solution. Finally, there are the issues of detecting the device being used and providing a playout environment - a player, navigation and associated functions and applications such as voting, tweeting and sharing.

Over the coming weeks we're going to cover delivery to each of these device platforms, as well as looking at on demand delivery of video.

Codec Roundup

Perhaps the most confusing aspect of online video is the formats to use.

Here is a quick roundup of those currently available:

H.264 - also called 'MPEG4 and 'MP4'. This is a very long standing standard that has recently become de facto for the delivery of on demand content over the internet. It is a codec (MP4 is a 'wrapper' or 'format' and can also be played in other formats such as .FLV, MOV and even .WMV), and has good encoding, especially at high data rates. There are other advantages: this is the only format Google currently indexes (although they have recently announced that they will not play this format in future releases of their Chrome browser in order to favour their own 'WebM' codec). This format plays on iPads and iPhones and can be made to play on Android devices. It is also now largely supported by IPTV devices such as set top boxes. Overall, it is by far and away the best codec to use if you want widespread distribution. The downside is that it is 'owned' by a consortium of very large companies, although they recently agreed to make it freely available to all but the largest commercial users.

H.263, also called 3GP, is a codec that was developed for mobile devices. It is still used on older phones and on platforms supported by Blackberry and Nokia.

Ogg Theora - is a very old and largely unloved codec that is 'open source'. The quality is poor and support is patchy at best.

WebM - In 2009 Google purchased a company called On2, who were responsible for the codec at the core of the delivery of most Flash video (FLV). These codecs carry the VPx lable and VP6 and VP7 are what you will largely find as the formats of most current videos on the web. However, these are quickly being deprecated to H.264 (see above) and to VP8, or WebM, which is a codec that Google has ostensibly 'open sourced', but which is almost totally unsupported at the moment by anyone else.

VPx - On2 was the company that developed the core codecs used by Adobe Flash and as such has had a core role in the development of video over the internet. On2 was acquired by Google in 2009 and its codecs are no longer supported, although widely used.

VC1 - this is the core codec developed by Microsoft, and was, for a long time very popular since it was the only codec that came with a digital rights management specification. When Microsoft went on to develop the Silverlight development platform it deprecated VC1 and now sees H.264 as its core codec.

There are a number of other, largely proprietary, codecs available, for example those from Move Networks, whose assets were recently sold EchoStar, but none have achieved mainstream adoption.

Welcome to About Internet TV

This blog, or wiki, has been set up to provide a source of advice for anyone involved in setting up, using, or managing TV or video services over the internet.

The team at About Internet TV come from a broadcasting background, and we've been involved in delivering video over the internet since technology first made this possible in the nineties.

As a result we have a wealth of experience, and we hope that this will be of benefit to you.