lambwheit / cyberdropdownloader

.NET downloader for albums hosted on cyberdrop

.Net CyberdropDownloader

This is a Cyberdrop album downloader, written in .Net

NOTES

• more threads = more instances of the downloader will be run, which means pottentially faster downloads at higher cpu use
• supports multiple album links

Changelog

• Users can now change number of threads used <- added in v1.1
• Updated Regex <- added in v1.2
• Changed download link scraping to use json instead of regex = increased reliability <- added in v1.3

How to use

• Just add your album links in a file called links.txt located in same folder as the executable
• Run executable
• Set threads
• Profit???

cyberdropdownloader's Issues

QOL recommendations

Nice tool

I recommend downloaders grab the download link from id="file" as it can go full speed. The clickable link onsite (class="image") has a bandwidth rate limit. (the fast link has a /s/ inbetween the filename and domain name)

CyberdropDownloader/CyberdropDownloader/Program.cs

Line 99 in a12dba6

string pattern = @"<a class=""image"" href=""(.*?)"" target=""_blank"" title=""";

You can also get the full original filename in the elements title attributes instead of parsing the download link

CyberdropDownloader/CyberdropDownloader/Program.cs

Lines 121 to 129 in a12dba6

	string FileExtention = PicLink.Substring(PicLink.Length - 3);
	if (PicLink.Contains("cyberdrop.nl"))
	{
	FileName = Substring(PicLink, "https://f.cyberdrop.nl/", FileExtention);
	}
	else
	{
	FileName = Substring(PicLink, "https://f.cyberdrop.cc/", FileExtention);
	}

Whats this about

CyberdropDownloader/CyberdropDownloader/Program.cs

Lines 131 to 132 in a12dba6

	var wreq = WebRequest.Create("http://mywebsite.com/file1.pdf");
	wreq.Timeout = 15000;

Bug

The URLs.txt contains

https://cyberdrop.me/a/jr9vwBBy
https://cyberdrop.me/a/KZGXDQjB

To Help Reproduce
Steps to help reproduce the behavior:

Do above
Setup Information

Crash with special characters

The files I am trying to download have special characters, such as "(☞ﾟヮﾟ)☞ ?". The application throws an error:

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.NullReferenceException: Object reference not set to an instance of an object.
at CyberdropDownloader.Program.<>c.

b__5_0(String PicLink)
at System.Threading.Tasks.Parallel.<>c__DisplayClass31_02.<ForEachWorker>b__0(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass17_01.b__1()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.b__0(Object )
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Func4 bodyWithLocal, Func1 localInit, Action1 localFinally) at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IList1 list, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally) at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable1 source, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally) at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable1 source, Action`1 body)
at CyberdropDownloader.Program.d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at CyberdropDownloader.Program.()

not working with different dowload urls

download urls were not matching with the if/else cases so I modified the file to match all cases for urls.

modified the script to make it work:

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;

namespace CyberdropDownloader
{
    public class MyWebClient : WebClient
    {
        protected override WebRequest GetWebRequest(Uri address)
        {
            var req = base.GetWebRequest(address);
            req.Timeout = 10000000;//ms
            return req;
        }
    }
    class Program
    {
        private static void Title()
        {
            while (true)
            {
                Console.Title = "Downloading: " + AlbumName + " | Pic: " + Downloaded + "/" + AlbumPics.Count;
            }
        }

        private static string Substring(string self, string left, string right, int startIndex = 0,
            StringComparison comparison = StringComparison.Ordinal, string fallback = null)
        {
            if (string.IsNullOrEmpty(self) || string.IsNullOrEmpty(left) || (startIndex < 0) ||
                startIndex >= self.Length)
                return fallback;
            int num1 = self.IndexOf(left, startIndex, comparison);
            if (num1 == -1)
                return fallback;
            int startIndex1 = num1 + left.Length;
            if (string.IsNullOrEmpty(right))
            {
                return self.Substring(startIndex1);
            }
            else
            {
                int num2 = self.IndexOf(right, startIndex1, comparison);
                return num2 == -1 ? fallback : self.Substring(startIndex1, num2 - startIndex1);
            }
        }

        private static string AlbumName = "";
        private static int Downloaded = 0;
        private static List<string> AlbumPics = new List<string>();

        private static async Task Main()
        {
            int threads = 5;

            Console.Write("Threads(Default 5): ");
            try
            {
                threads = int.Parse(Console.ReadLine());
                if (threads <= 0)
                {
                    threads = 5;
                }
            }
            catch
            {

            }
            string[] lines = { };
            try
            {
                lines = File.ReadAllLines(@"links.txt");
            }
            catch
            {
                Console.WriteLine("[Error] links.txt not found, press enter to exit");
                Console.ReadLine();
                Environment.Exit(0);
            }
            Thread thread = new Thread(new ThreadStart(Title));
            thread.Start();
            if (lines?.Length == 0)
            {
                Console.WriteLine("[Error] no album links found, press enter to exit");
                Console.ReadLine();
                Environment.Exit(0);
            }
            foreach (var AlbumLink in lines)
            {
                string htmlCode = "";
                using (WebClient client = new WebClient())
                {
                    try
                    {
                        htmlCode = client.DownloadString(AlbumLink);
                    }
                    catch (Exception e)
                    {
                        if (e.ToString().Contains("(404) Not Found"))
                        {
                            continue;
                        }
                        else
                        {
                            Console.WriteLine("Unknown Error");
                            break;
                        }
                    }

                }
                //@"<a  
                string pattern = @"<a class=""image"" href=""(.*?)"" target=""_blank""";
                RegexOptions regexOptions = RegexOptions.None;
                Regex regex = new Regex(pattern, regexOptions);
                string inputData = htmlCode;
                AlbumName = Substring(htmlCode, "<h1 id=\"title\" class=\"title has-text-centered\" title=\"", "\"");
                if (Directory.Exists("Downloads\\" + AlbumName)) continue;
                Directory.CreateDirectory("Downloads\\" + AlbumName);
                AlbumPics?.Clear();
                Downloaded = 0;

                foreach (Match match in regex.Matches(inputData))
                {
                    if (match.Success)
                    {
                        string MatchParse = Substring(match.ToString(), "<a class=\"image\" href=\"", "\"");
                        AlbumPics.Add(MatchParse);
                    }
                } //get all links
                ThreadPool.SetMaxThreads(threads, threads);
                ThreadPool.SetMinThreads(threads, threads);

                Parallel.ForEach(AlbumPics, (string PicLink) =>
                {

                    var url = new Uri(PicLink);
                    var filename = url.LocalPath;


                    while (true)
                    {
                        try
                        {
                            using (WebClient webClient = new MyWebClient())
                            {
                                webClient.DownloadFile(PicLink, "Downloads\\" + AlbumName + "\\" + filename);
                                Downloaded++;
                                Console.WriteLine("Downloaded: " + AlbumName + " | " + filename);
                            }
                            break;
                        }
                        catch
                        {
                        }
                    }


                });
            }
            Console.WriteLine("Done");
            Console.ReadLine();
        }
    }
}