๐ Fast and reliable multipart downloader with .Net Core 3.1+ supporting ๐
Downloader is a modern, fluent, asynchronous, testable and portable library for .NET. This is a multipart downloader with asynchronous progress events.
This library can added in your .Net Core v3.1
and later or .Net Framework v4.5
or later projects.
Downloader is compatible with .NET Standard 2.0 and above, running on Windows, Linux, and macOS, in full .NET Framework or .NET Core.
- Simple interface to make download request.
- Download files async and non-blocking.
- Download any type of files like image, video, pdf, apk and etc.
- Cross-platform library to download any files with any size.
- Get real-time progress info of each block.
- Download file multipart as parallel.
- Handle all the client-side and server-side exceptions non-stopping.
- Config your
ChunkCount
to define the parts count of the download file. - Download file multipart as
in-memory
orin-temp files
cache mode. - Store download package object to resume the download when you want.
- Get download speed or progress percentage in each progress event.
- Get download progress events per chunk downloads.
- Pause and Resume your downloads with package object.
- Supports large file download.
- Set a dynamic speed limit on downloads (changeable speed limitation on the go).
- Download files without storing on disk and get a memory stream for each downloaded file.
- Serializable download package (to/from
JSON
orBinary
) - Live streaming support, suitable for playing music at the same time as downloading.
- Ability to download just a certain range of bytes of a large file.
Installing via NuGet
PM> Install-Package Downloader
dotnet add package Downloader
var downloadOpt = new DownloadConfiguration()
{
ChunkCount = 8, // file parts to download, default value is 1
OnTheFlyDownload = true, // caching in-memory or not? default values is true
ParallelDownload = true // download parts of file as parallel or not. Default value is false
};
var downloadOpt = new DownloadConfiguration()
{
BufferBlockSize = 10240, // usually, hosts support max to 8000 bytes, default values is 8000
ChunkCount = 8, // file parts to download, default value is 1
MaximumBytesPerSecond = 1024 * 1024, // download speed limited to 1MB/s, default values is zero or unlimited
MaxTryAgainOnFailover = int.MaxValue, // the maximum number of times to fail
OnTheFlyDownload = false, // caching in-memory or not? default values is true
ParallelDownload = true, // download parts of file as parallel or not. Default value is false
TempDirectory = "C:\\temp", // Set the temp path for buffering chunk files, the default path is Path.GetTempPath()
Timeout = 1000, // timeout (millisecond) per stream block reader, default values is 1000
RangeDownload = true, // set true if you want to download just a certain range of bytes of a large file
RangeLow = 273618157, // floor offset of download range of a large file
RangeHigh = 305178560, // ceiling offset of download range of a large file
RequestConfiguration = // config and customize request headers
{
Accept = "*/*",
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,
CookieContainer = new CookieContainer(), // Add your cookies
Headers = new WebHeaderCollection(), // Add your custom headers
KeepAlive = false, // default value is false
ProtocolVersion = HttpVersion.Version11, // Default value is HTTP 1.1
UseDefaultCredentials = false,
UserAgent = $"DownloaderSample/{Assembly.GetExecutingAssembly().GetName().Version.ToString(3)}"
}
};
var downloader = new DownloadService(downloadOpt);
// Provide `FileName` and `TotalBytesToReceive` at the start of each downloads
downloader.DownloadStarted += OnDownloadStarted;
// Provide any information about chunker downloads, like progress percentage per chunk, speed, total received bytes and received bytes array to live streaming.
downloader.ChunkDownloadProgressChanged += OnChunkDownloadProgressChanged;
// Provide any information about download progress, like progress percentage of sum of chunks, total speed, average speed, total received bytes and received bytes array to live streaming.
downloader.DownloadProgressChanged += OnDownloadProgressChanged;
// Download completed event that can include occurred errors or cancelled or download completed successfully.
downloader.DownloadFileCompleted += OnDownloadFileCompleted;
string file = @"Your_Path\fileName.zip";
string url = @"https://file-examples.com/fileName.zip";
await downloader.DownloadFileTaskAsync(url, file);
DirectoryInfo path = new DirectoryInfo("Your_Path");
string url = @"https://file-examples.com/fileName.zip";
await downloader.DownloadFileTaskAsync(url, path); // download into "Your_Path\fileName.zip"
Stream destinationStream = await downloader.DownloadFileTaskAsync(url);
The โDownloadService
class has a property called Package
that stores each step of the download. To stopping or pause the download you must call the CancelAsync
method, and if you want to continue again, you must call the same DownloadFileTaskAsync
function with the Package
parameter to resume your download!
For example:
Keep Package
file to resume from last download positions:
DownloadPackage pack = downloader.Package;
Stop or Pause Download:
downloader.CancelAsync();
Resume Download:
await downloader.DownloadFileTaskAsync(pack);
So that you can even save your large downloads with a very small amount in the Package and after restarting the program, restore it again and start continuing your download. In fact, the packages are your instant download snapshots. If your download config has OnTheFlyDownload, the downloaded bytes โโwill be stored in the package itself, but otherwise, only the downloaded file address will be included and you can resume it whenever you like. For more detail see StopResumeDownloadTest method
Note: for complete sample see
Downloader.Sample
project from this repository.
For easy and fluent use of the downloader, you can use the DownloadBuilder
class. Consider the following examples:
Simple usage:
await DownloadBuilder.New()
.WithUrl(@"https://host.com/test-file.zip")
.WithDirectory(@"C:\temp")
.Build()
.StartAsync();
Complex usage:
IDownload download = DownloadBuilder.New()
.WithUrl(@"https://host.com/test-file.zip")
.WithDirectory(@"C:\temp")
.WithFileName("test-file.zip")
.WithConfiguration(new DownloadConfiguration())
.Build();
download.DownloadProgressChanged += DownloadProgressChanged;
download.DownloadFileCompleted += DownloadFileCompleted;
download.DownloadStarted += DownloadStarted;
download.ChunkDownloadProgressChanged += ChunkDownloadProgressChanged;
await download.StartAsync();
Resume the existing download package:
await DownloadBuilder.Build(package);
Resume the existing download package with a new configuration:
await DownloadBuilder.Build(package, new DownloadConfiguration())
.StartAsync();
Serialization is the process of converting an object's state into information that can be stored for later retrieval or that can be sent to another system. For example, you may have an object that represents a document that you wish to save. This object could be serialized to a stream of binary information and stored as a file on disk. Later the binary data can be retrieved from the file and deserialized into objects that are exact copies of the original information. As a second example, you may have an object containing the details of a transaction that you wish to send to a non-.NET system. This information could be serialised to XML before being transmitted. The receiving system would convert the XML into a format that it could understand.
In this section, we want to show how to serialize download packages to JSON
text or Binary
, after stopping download to keep download data and resuming that every time you want.
You can serialize packages even using memory storage for caching download data which is used MemoryStream
.
To serialize or deserialize the package into a binary file, just you need to a BinaryFormatter of IFormatter and then create a stream to write bytes on that:
DownloadPackage pack = downloader.Package;
IFormatter formatter = new BinaryFormatter();
Stream serializedStream = new MemoryStream();
Serializing package:
formatter.Serialize(serializedStream, pack);
Deserializing into the new package:
var newPack = formatter.Deserialize(serializedStream) as DownloadPackage;
For more detail see PackageSerializationTest method.
Serializing the package to JSON
is very simple like this:
var packageJson = JsonConvert.SerializeObject(package);
But to deserializing the IStorage Storage property of chunks you need to declare a JsonConverter to override the Read method of JsonConverter
. So you should add the below converter to your application:
public class StorageConverter : Newtonsoft.Json.JsonConverter<IStorage>
{
public override void WriteJson(JsonWriter writer, IStorage value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override IStorage ReadJson(JsonReader reader, Type objectType, IStorage existingValue, bool hasExistingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.Null)
return null;
var obj = JObject.Load(reader); // Throws an exception if the current token is not an object.
if (obj.ContainsKey(nameof(FileStorage.FileName)))
{
var filename = obj[nameof(FileStorage.FileName)]?.Value<string>();
return new FileStorage(filename);
}
if (obj.ContainsKey(nameof(MemoryStorage.Data)))
{
var data = obj[nameof(MemoryStorage.Data)]?.Value<string>();
return new MemoryStorage() { Data = data };
}
return null;
}
}
Then you can deserialize your packages from JSON
:
var settings = new Newtonsoft.Json.JsonSerializerSettings();
settings.Converters.Add(new StorageConverter());
var newPack = Newtonsoft.Json.JsonConvert.DeserializeObject<DownloadPackage>(serializedJson, settings);
For more detail see PackageSerializationTest method
Welcome to contribute, feel free to change and open a PullRequest to develop branch. You can use either the latest version of Visual Studio or Visual Studio Code and .NET CLI for Windows, Mac and Linux.
For GitHub workflow, check out our Git workflow below this paragraph. We are following the excellent GitHub Flow process, and would like to make sure you have all of the information needed to be a world-class contributor!
The general process for working with Downloader is:
- Fork on GitHub
- Make sure your line endings are correctly configured and fix your line endings!
- Clone your fork locally
- Configure the upstream repo (
git remote add upstream git://github.com/bezzad/downloader
) - Switch to the latest development branch (eg vX.Y.Z, using
git checkout vX.Y.Z
) - Create a local branch from that (
git checkout -b myBranch
). - Work on your feature
- Rebase if required
- Push the branch up to GitHub (
git push origin myBranch
) - Send a Pull Request on GitHub - the PR should target (have as base branch) the latest development branch (eg
vX.Y.Z
) rather thanmaster
.
We accept pull requests from the community. But, you should never work on a clone of master, and you should never send a pull request from master - always from a branch. Please be sure to branch from the head of the latest vX.Y.Z develop
branch (rather than master
) when developing contributions.
Licensed under the terms of the MIT License