Pulled Apart - Part VI: A simple download manager

Published on 20 August 2010 • Estimated reading time: 3 minutes

[]

Note: This is part of a series. You can find the rest of the parts in the series index.

A podcatcher, like Pull, really is just an RSS reader with a download manager built in to download RSS enclosures. This means that both parts should work—and work very well. The RSS reader side is fairly easy to implement, however the downloader is anything but.

WebClient

Initially, I thought about using a simple download manager and built it around a .NET class called WebClient. This has a nice async method (DownloadFileAsync) and several events you can subscribe to, making it easy to download files:

WebClient webClient = new WebClient();
webClient.DownloadFileCompleted += new System.ComponentModel.AsyncCompletedEventHandler(webclient_DownloadFileCompleted);
webClient.DownloadProgressChanged += new DownloadProgressChangedEventHandler(webclient_DownloadProgressChanged);
webClient.DownloadFileAsync(episode.EpisodeUri, this.Episode.Local_Path);

However, in exchange for this simple implementation, you lose a ton of features that a more powerful download manager may offer—particularly in error handling. For me, living in South Africa where bandwidth isn’t great, dealing with errors during downloads is essential. So, I eventually dropped WebClient as the system I used.

HttpWebRequest

.NET also includes a fully featured HTTP request/response system built around HttpWebRequest and HttpWebResponse. These classes offer many features that WebClient lacks. However, using them requires more code. Below is the code I use, along with some key details:

• Lines 2 & 3: Setting a connection group name and UnsafeAuthenticatedConnectionSharing to true allows the system to reuse an existing connection to the server, improving performance during handshakes.
• Line 4: Setting a user agent helps stats programs identify your client specifically.
• Line 8: Changing the range enables me to resume downloads by starting from a different point in the file—critical if errors occur or the application is closed. Notably, some web servers dislike the start point being set to 0 (the beginning), which is why I perform a check.
• Line 15: Fetches the server’s response but not the actual data.
• Line 25: Begins retrieving the data.
• Line 37: Writes the data to disk.

HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(uri);
webRequest.ConnectionGroupName = "Pull (pull.codeplex.com)";
webRequest.UnsafeAuthenticatedConnectionSharing = true;
webRequest.UserAgent = "Pull (pull.codeplex.com)";

if (startPointInt > 0)
{
    webRequest.AddRange(startPointInt);
}

webRequest.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse webResponse = null;
try
{
    webResponse = (HttpWebResponse)webRequest.GetResponse();
}
catch (WebException err)
{
    // Error handling
    return;
}

Int64 fileSize = webResponse.ContentLength;
webFileStream = webResponse.GetResponseStream();
if (!fileInfo.Exists)
{
    localFileStream = new FileStream(fileInfo.FullName, FileMode.Create, FileAccess.Write, FileShare.None);
}
else
{
    localFileStream = new FileStream(fileInfo.FullName, FileMode.Append, FileAccess.Write, FileShare.None);
}

int bytesSize = 0;
byte[] downBuffer = new byte[2048];
while ((bytesSize = webFileStream.Read(downBuffer, 0, downBuffer.Length)) > 0)
{
    localFileStream.Write(downBuffer, 0, bytesSize);
    onUpdate(Convert.ToInt32((localFileStream.Length * 100) / (fileSize + startPointInt)));
}

As you can see, it’s more complex but far more capable—for example, it supports resuming downloads.

Final Thoughts

I want to point out an article by Andrew Pociu, who wrote an excellent guide on building a download manager that inspired much of my code.

In the end, switching to HttpWebRequest was a much better decision—but this is an area where new error types still emerge, and robust error handling remains a challenge given the wide variety of issues that can arise on the web.