The Importance of Torrents
Up until now most video on the Internet has been postage stamped
sized. A small 320x240 or 240x180 window at 15 frames per second. We
had to do this because we simply had no good CODEC to compress with
and no good way to distribute larger files. With the advent of h.264
we’re able to distribute full 1080p HD resolutions online. The
problem is that these files are HUGE! One 25 minute show can be around
2GB. If there were 10,000 downloads of that show, it would be 20,000GB
of information transferred. That’s a lot of information for one server
to deal with. By using BitTorrent we’re able to distribute that load
among everyone interested in the video, reduce the strain on the
server, drastically reduce costs, and improve the user experience. In
my mind this will end up being the standard way of delivering HD
content online.
What is BitTorrent?
Remember the days of yesteryear when we had the Napsters of the world? Napster was a peer-to-peer model which is where you’re sharing your files with those around you, no centralized server was needed. Alas, most of the files on Napster were illegal music and the industry was not too keen on having all their work stolen. Apple came to the rescue with the poorly branded iTunes Music store (it should be called iMedia now that they do video) and gave everyone a client-server based model for downloading media. By that I mean Apple has a bunch of servers with music on it, and when you purchase a song you download from their server. That is what most people are used to, a client-server model. If the server goes down, you can’t access your resources.
The client-server model is great if you have a bunch of small music files. Where it breaks is when you have a series of very, very large files, for example a 25 minute HD video interview. With the small files the strain on the server and bandwidth is minimal, but with a 1GB HD file, well, the available network goes away very quickly. The cost to increase the network is very high. We need a way to distribute the bandwidth load. This is the problem with the iTunes Music Store when used for video. There’s no way to use a peer-to-peer model like Napster to share bandwidth among users who are interested in our video. Everyone downloads the file from our server, and that costs us a lot in system resources.
BitTorrent solves this problem by separating out the bandwidth needs, a lot like Napster, but done legally. Just as Napster shared music files among peers, BitTorrent shares bandwidth among peers. The best part is… the more users we get on a BitTorrent session, the faster it becomes!
How does BitTorrent work?
Torrents are broken into three basic parts: the peer, the seeder and the tracker. The peer would be a user who would like to download one of our videos. They don’t have the video yet, all they have is a small .torrent file that they got from a friend, off of a web site, or in an RSS feed. The seeder is someone who has already downloaded the entire video and is sharing the bandwidth to all of the peers. And the tracker is what manages all of these connections in the background, telling all of the software how to connect with one another… Lets break this down and actually go through the entire process. I’ll use our Frank Gruber 480p file as an example.
First, we need to get a torrent client such as Azureus. As the content creator I use Azereus to create a torrent file. I tell the client where the file is on my hard drive, where the tracker is located, and if I want users to enter a password or not. From there Azureus scans the file, chops it up into much smaller segments, and create a text file with a .torrent extension. This .torrent file is usually very small, typically just a few k. Inside of this file is the information on the URL of the tracker, information on how to segment the video file, and a verification method to ensure you’re getting the proper information for each segment (basically a way to ensure someone does not add a virus to the file after the fact). I take this file and give it to everyone that I want to share my video with. Since we want to share the video with everyone, we have posted the .torrent file on our web server and added it to our RSS feed. So now all we have is a small 20k .torrent file sitting on a web server (keep in mind, so far this is the person wanting to share the content, none of this is required if you just want to view it). In the background we’re also running our own tracker server. That server just sits there telling the clients who is connected, how much bandwidth is available, etc. In most cases users don’t need to deal with the tracker, we just wanted to use our own so we could get detailed stats.
Now we get to the good part: getting the video downloaded. Anyone who wants our HD video will download this .torrent file we just created off of our web site. All they need to do is double-click on the .torrent and the rest will happen automatically. We have a couple of seeders sitting around the Internet that already have the entire video available to them, and all they are doing is uploading. When the .torrent file connects to the tracker, the tracker responds saying “connect to these 5 seeders, and these 50 peers”. Since the file has been broken up into thousands of virtual chunks, you’re able to download one part from seeder #1, a different part from seeder #2, and so forth and so on. As you download these chunks of the video, you start uploading them to other peers around you as well. Torrents are smart too. If there’s a chunk of the video that not a lot of peers have, the Torrent client knows to request that first from the seeders so that your upload can help balance the load among all the peers. The load on our network is minimal since we only need to get the seeders started. From there the peers start sharing the bandwidth between themselves and the more peers we get on the network, the faster the .torrent becomes. Once all the parts have been downloaded, you then become a seeder, doing nothing up uploading. While the torrent file broke the video into virtual parts, to you the video is just one single file. Nothing was really broken apart, different parts of the video were given to different seeders and peers. Your Torrent client made sure that you got everything needed to reconstruct the file, and put it all back in order for you.
What is BitTorrent?
Remember the days of yesteryear when we had the Napsters of the world? Napster was a peer-to-peer model which is where you’re sharing your files with those around you, no centralized server was needed. Alas, most of the files on Napster were illegal music and the industry was not too keen on having all their work stolen. Apple came to the rescue with the poorly branded iTunes Music store (it should be called iMedia now that they do video) and gave everyone a client-server based model for downloading media. By that I mean Apple has a bunch of servers with music on it, and when you purchase a song you download from their server. That is what most people are used to, a client-server model. If the server goes down, you can’t access your resources.
The client-server model is great if you have a bunch of small music files. Where it breaks is when you have a series of very, very large files, for example a 25 minute HD video interview. With the small files the strain on the server and bandwidth is minimal, but with a 1GB HD file, well, the available network goes away very quickly. The cost to increase the network is very high. We need a way to distribute the bandwidth load. This is the problem with the iTunes Music Store when used for video. There’s no way to use a peer-to-peer model like Napster to share bandwidth among users who are interested in our video. Everyone downloads the file from our server, and that costs us a lot in system resources.
BitTorrent solves this problem by separating out the bandwidth needs, a lot like Napster, but done legally. Just as Napster shared music files among peers, BitTorrent shares bandwidth among peers. The best part is… the more users we get on a BitTorrent session, the faster it becomes!
Torrents are broken into three basic parts: the peer, the seeder and the tracker. The peer would be a user who would like to download one of our videos. They don’t have the video yet, all they have is a small .torrent file that they got from a friend, off of a web site, or in an RSS feed. The seeder is someone who has already downloaded the entire video and is sharing the bandwidth to all of the peers. And the tracker is what manages all of these connections in the background, telling all of the software how to connect with one another… Lets break this down and actually go through the entire process. I’ll use our Frank Gruber 480p file as an example.
First, we need to get a torrent client such as Azureus. As the content creator I use Azereus to create a torrent file. I tell the client where the file is on my hard drive, where the tracker is located, and if I want users to enter a password or not. From there Azureus scans the file, chops it up into much smaller segments, and create a text file with a .torrent extension. This .torrent file is usually very small, typically just a few k. Inside of this file is the information on the URL of the tracker, information on how to segment the video file, and a verification method to ensure you’re getting the proper information for each segment (basically a way to ensure someone does not add a virus to the file after the fact). I take this file and give it to everyone that I want to share my video with. Since we want to share the video with everyone, we have posted the .torrent file on our web server and added it to our RSS feed. So now all we have is a small 20k .torrent file sitting on a web server (keep in mind, so far this is the person wanting to share the content, none of this is required if you just want to view it). In the background we’re also running our own tracker server. That server just sits there telling the clients who is connected, how much bandwidth is available, etc. In most cases users don’t need to deal with the tracker, we just wanted to use our own so we could get detailed stats.
Now we get to the good part: getting the video downloaded. Anyone who wants our HD video will download this .torrent file we just created off of our web site. All they need to do is double-click on the .torrent and the rest will happen automatically. We have a couple of seeders sitting around the Internet that already have the entire video available to them, and all they are doing is uploading. When the .torrent file connects to the tracker, the tracker responds saying “connect to these 5 seeders, and these 50 peers”. Since the file has been broken up into thousands of virtual chunks, you’re able to download one part from seeder #1, a different part from seeder #2, and so forth and so on. As you download these chunks of the video, you start uploading them to other peers around you as well. Torrents are smart too. If there’s a chunk of the video that not a lot of peers have, the Torrent client knows to request that first from the seeders so that your upload can help balance the load among all the peers. The load on our network is minimal since we only need to get the seeders started. From there the peers start sharing the bandwidth between themselves and the more peers we get on the network, the faster the .torrent becomes. Once all the parts have been downloaded, you then become a seeder, doing nothing up uploading. While the torrent file broke the video into virtual parts, to you the video is just one single file. Nothing was really broken apart, different parts of the video were given to different seeders and peers. Your Torrent client made sure that you got everything needed to reconstruct the file, and put it all back in order for you.
The next step in this whole crazy process is integrating
a torrent client with a video player to completely automate the video
download and viewing process (a lot like iTunes, but with torrents).
There are a couple of players out there today, and I'll touch on my
favorite player in tomorrows post. I firmly believe that the next big
revolution in television will be with these torret video players and it
will forever change the face of media.
