Pulled Apart - Part VI: A simple download manager

Note: This is part of a series, you can find the rest of the parts in the series index.

A podcatcher, like Pull, really is just a RSS reader with a download manager built in to download RSS enclosures. This means that both parts should work and work very well. The RSS reader side is fairly easy to do, however the downloader is anything but easy.

WebClient

Initially I thought about a simple download manager would be enough and I built it around a .NET class called WebClient. This has a nice async method (DownloadFileAsync) and a few events you can subscribe to which does a good job at downloading a file easily.

WebClient webclient = new WebClient();
webclient.DownloadFileCompleted += new System.ComponentModel.AsyncCompletedEventHandler(webclient_DownloadFileCompleted);
webclient.DownloadProgressChanged += new DownloadProgressChangedEventHandler(webclient_DownloadProgressChanged);
webclient.DownloadFileAsync(episode.EpisodeUri, this.Episode.Local_Path);

However in exchange for this nice simple implementation, you lose a ton of features that a more powerful download manager may have, in particular ways of dealing with errors. For me, living in South Africa where the bandwidth isn’t great, dealing with errors during downloads is essential and so I eventually dropped WebClient as the system I used.

HttpWebRequest

.NET also includes a fully featured HTTP request/response system built around HttpWebRequest and HttpWebResponse. These classes offer a ton of features which WebClient doesn’t. However using it much more code than WebClient. Below is the code I use and some interesting bits in are:

• Lines 2 & 3: Setting a connection group name and setting UnsafeAuthenticationConnectionSharing to true means that the system will attempt to reuse an existing pipe to the server. This can have some great performance improvements when hand shaking is done.
• Line 4: Setting the user agent means that various stats programs can identify your client specifically.
• Line 8: Changing the range means that I can start at a different point in the file. This enables me to resume downloads which are broken because of errors or closing the application. An important aspect I learnt is that some web servers hate the start point to be set to 0 (which means the beginning), which is why I do a check.
• Line 15: This gets the response from the server, but not the data.
• Line 25: This is where we actually start to get the data.
• Line 37: This is where we write the data to disk.

HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(uri);
webRequest.ConnectionGroupName = "Pull (pull.codeplex.com)";
webRequest.UnsafeAuthenticatedConnectionSharing = true;
webRequest.UserAgent = "Pull (pull.codeplex.com)";

if (startPointInt > 0)
{
    webRequest.AddRange(startPointInt);
}

webRequest.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse webResponse = null;
try
{
    webResponse = (HttpWebResponse)webRequest.GetResponse();
}
catch (WebException err)
{
    // Error handling

    return;
}

Int64 fileSize = webResponse.ContentLength;
webFileStream = webResponse.GetResponseStream();
if (!fileInfo.Exists)
{
    localFileStream = new FileStream(fileInfo.FullName, FileMode.Create, FileAccess.Write, FileShare.None);
}
else
{
    localFileStream = new FileStream(fileInfo.FullName, FileMode.Append, FileAccess.Write, FileShare.None);
}

int bytesSize = 0;
byte[] downBuffer = new byte[2048];
while ((bytesSize = webFileStream.Read(downBuffer, 0, downBuffer.Length)) > 0)
{
    localFileStream.Write(downBuffer, 0, bytesSize);
    onUpdate(Convert.ToInt32((localFileStream.Length * 100) / (fileSize + startPointInt)));
}

As you can see it is more complex, but you can do so much more, like resuming downloads.

Final Thoughts

I want to point out an article by Andrew Pociu who wrote a fantastic article on building a download manager which inspired a lot of my code.

In the end changing to HttpWebRequest was a much better idea, but this is an area where I am still finding new types of errors and trying to put in error handling to cope with the WIDE variety of errors you can get on the web.