Persistent Incomplete Download about cyberdropdownloader

[Jules-WinnfieldX]

A logs.log file from a run where this happens would probably be best. the logs.log file in the original post happened on a consequent run when the files already existed.

[Jules-WinnfieldX]

This problem is usually indicative of network issues on the users end. Check for yourself if the links that were scraped (can be found in the logs.log file you uploaded) work in your browser. Delete the bad files, turn off any antivirus, vpn or proxy and try again.

[MrxHellsing]

I did what you said, but it seems like the problem lies in the download limit. Apparently, bunkr has a download limit for every IP address. I was indeed getting a network error on my end, even on the links that had been downloaded fully. After I turned on VPN and reloaded the same link, it immediately displayed correctly, and the app was downloading too. However, ran into another problem. I am not sure exactly what is the issue here but probably it has something to do with multiple files downloading simultaneously and this is quite random I think. When I downloaded a cyberdrop link, it downloaded about 10 files simultaneously. In bunkr's case, it downloaded 2-4 files at once, and maybe due to multiple downloads at the same time, it is running into download limit for all files or at least 3-4 or more files. How about adding a feature to limit the number of simultaneous downloads that most downloaders have. That way, if it does run into download limit, then at least it will be for that file and will not proceed to the next one before finishing the current one. This way, it will be easy to check which file it ran into limit instead of opening all files and checking if it had been downloaded fully or not.
.
Also, since the download is somehow IP bound, how about adding proxy support? Kind of like what Megabasterd has, even though that app doesn't work most of the time and not sure how that app works at all or if it even makes use of proxy at all caz you run the app, it downloads about 3 GB and then just stop, naturally, it should run the next proxy to continue downloading but it doesn't. Not sure if this will be feasible for your app or not, but I'm just putting out ideas.
.
EDIT: I forgot to mention one thing, I even tried turning off anti-virus, still the same result.

[Jules-WinnfieldX]

Bunkr does not have a download limit. Bunkr only restricts to 2 simultaneous downloads. The program already restricts to 2 simultaneous downloads for bunkr either way. You are not hitting a limit with bunkr, you are having issues on your end with part of your configuration.

There already is a feature to restrict how many simultaneous downloads are taking place, and that's the threads setting in settings.py.

Addressed downloads above, they are IP bound, but not in the way you are intending. To re-iterate, you are not hitting a limit with bunkr.

[MrxHellsing]

Well then, how do I fix this problem that you're referring to that I have on my network configuration. I am at a loss here. Back then when I opened the auto-closing issue, after your fix, it downloaded everything well and didn't run into these kinds of problems, well at least not this often. Since then I didn't change anything in my configuration, it is what it was back then. Not sure, what is causing the issue. I don't run VPN unless I have to and since bunkr or cyberdrop isn't blocked here, so don't need to run VPN for that either. I don't run any proxy. My DNS is Cloudflare. I do run a Dynamic IP with Double NAT restriction though and because of that I run into seeding problems in Private torrent, other than that, nothing much that I know of.

[Jules-WinnfieldX]

There is no hard set fix I can give you for that. As for your logs.log file you provided, The files already existed as not .download files, so there is no way for me to see if there were any errors that occurred. Normally when a user faces an issue like this in the past it's been because of anti-virus like avast or proxies. But everyone's setup is different and I have no idea how your setup would impact things.

Important to remember the program cannot see if the download is incomplete if the files are not the partial .download files it creates. If you had network issues (or bunkr did) and it somehow managed to create video files containing html text, and marked them as complete, it won't try and fix them automatically. You have to remove them first.

I wish I could be of more help.

[MrxHellsing]

Could it be something to with bunkr itself? I tried JDownloader which as far as I know can download from bunkr as well. But when I add links in that, it runs into a download error. I had downloaded Gofile files with it before. But even though it shows files being online for most links, the ones that are present in the log file, I tried adding those and all showed as offline with a Red Minus icon.
.
EDIT: Also, if you open some links in browser, sometimes has IDM notification containing full MP4 and a default 1.73 KB of MP4 file and sometimes it only shows the default 1.73 KB MP4 file.

[Jules-WinnfieldX]

It could, but the fact that I haven't had anyone else come forward with this issue is what is making me assume that the issue is on your end. Without physical access or information (that I don't even know what to ask for) there isn't any way for me to help beyond this point unfortunately.

[MrxHellsing]

Well, I download a lot on a daily basis from Mega, CyberDrop, Bunkr. Maybe that's why I am experiencing more issues. So far I haven't tried downloading from CyberDrop yet with your app. I'll try a few downloads from CyberDrop and see if the issue persists and let you know about it.

[MrxHellsing]

Update on CyberDrop site downloads: I'm currently downloading 2 links from cyberdrop and the total amount of videos is about 300+. So far 299 have been completed. I am seeing new downloads are being written as .download and all videos so far have been downloaded fully, not partially. The thread count is set to "1" to download 1 file at a time. After cyberdrop completes, I'll add 2 links from bunkr as well. As far as cyberdrop is concerned, I am not seeing any downloads incomplete as mp4 files.
.
Edit: All files from cyberdrop downloaded without any issues, will try bunkr and let you know.

[DeadManLogan]

Just run into the same problem. There are not any .download files left but most of the files (.mp4) are not opening. It probably has to do something with your network. If you try to download the files by clicking the "download media" button on site, it will work perfectly. If not maybe the files are uploaded corrupted.

[Jules-WinnfieldX]

I've yet to be able to recreate this, any additional info would be handy.

Just run into the same problem. There are not any .download files left but most of the files (.mp4) are not opening. It probably has to do something with your network. If you try to download the files by clicking the "download media" button on site, it will work perfectly. If not maybe the files are uploaded corrupted.

[MrxHellsing]

It probably has to do something with your network. If you try to download the files by clicking the "download media" button on site, it will work perfectly. If not maybe the files are uploaded corrupted.

The download media button sometimes doesn't work either, normally my download manager catches the download but sometimes a pop-up opens but no action. When I am downloading a file from bunkr and at the same time if I try to open another link, I sometimes get a Network error.

[MrxHellsing]

UPDATE on BUNKR: Yeah, just as I suspected. This problem keeps occurring only with Bunkr files based on my observation. A lot of files are in mp4 files with few bytes and a vast majority of the files are full mp4 files with 60-70% data missing. I am attaching log file of today, the download was in progress from 2nd March, 2022 nighttime I think and finished today.
.
logs.log

[Jules-WinnfieldX]

So two things. The first being I'm confused as for a vast majority of the files that you attempted to download, the files already existed on your computer (whether they were good or bad is another thing).

The second being, as you stated yesterday you said that the download media button often doesn't work for you on the website itself. These two things are going to be connected in the issue for you. If you open those few kb .mp4 files in a text editor you will see that they most likely contain html code (a webpage, not actual a video file). What this program does is grabs the links from that download button and just downloads them. If that button does not work for you, for whatever reason it may be, the program will also not function.

[MrxHellsing]

1. I am talking about the whole package, I didn't check until all 30+ GB was finished downloading. After checking, I found that a lot of the files are downloaded partially but showing as a full mp4 file and you can even play those but they only have contents of a few mins. For example: A video that is initially 10 mins, the downloaded file would only have about 2-3 or sometimes up to 8-9 mins, 1 min or more would be missing.
   .
2. The download media button is kinda like a paradox, it works and then it doesn't. The timing is very random. If I am to give an example that would be this, suppose I am downloading a link that has about 20 files. Now, 5 of them completed fully but 15 of them failed ( by fail, I mean downloaded as a full mp4 file but has mins left out ). Now, at that very moment, if I open one of the failed files on the browser and attempt to download by clicking on it, it will open a pop-up but immediately close. I confirm that the button doesn't work by seeing my IDM pop up. If that doesn't have another full video listed below the 2KB mp4 file, then for that moment, the button wasn't working. Also, if I open multiple links at the same time, then in 2-3 tabs the button works and IDM captures the download, in the rest of the tabs, the buttons initiates a pop-up but nothing happens, and in those tabs, there is no full video listed below the 2KB mp4 file in IDM pop-up either. So, the working part is very random.

[Jules-WinnfieldX]

We are essentially talking about the same thing. With the retry function, if the download button is intermittent for you, the exact same result you are experiencing is going to happen. If the first attempt works for the first 9 minutes for example, and then the download stops and it has to try and resume, no the second attempt the button doesn't work, it's going to thing the download is complete. The header on the second attempt is going to be kb's in size, where you will have hundreds of mb's or gb's, it's going to drop the .download and assign it as a .mp4.

[Jules-WinnfieldX]

We use a resume point (range) in the header information, I have no idea what that would return in your case where the button isn't working. It could be 0, it could be something completely random, but not the rest of the file. Your issue isn't with my downloader, it's working as intended. Your issue is with Bunkr.

[MrxHellsing]

Yeah, I think that too, the problem is with Bunkr. It has some sort of limit in place that prevents a full download or prevents multiple download request or something similar to that. There are sites where download with IDM becomes impossible due to something called request being denied, basically, the browser sends a download request and at the same time IDM sends one too, so some sites are configured to deny the same request two in a row. I think bunkr has some sort of limit in place. Is there any way to bypass the limit through the app? Based on my observation, the app has no issue with CyberDrop as it has no limit in place.

[Jules-WinnfieldX]

There is no limit in place besides the simultaneous download limit of 2. That is the only limit. If you are downloading with your browser or IDM while using my program you will run into issues. There is no other limit. I’ve talked to the owner and he confirmed that.

[hellhond]

Have the same issue, the incomplete files contain html error content, specifically, 503 Service Temporarily Unavailable

I'm guessing the HTTP response status is not reflected (returning 200 [success]), or does CyberDropDownloader not check the HTTP status code ?

So it's definately an issue at Bunkr. Either rate limiting, which you mention should not be the case, or the specific server is overloaded.

bunkr-error.txt

[Jules-WinnfieldX]

It doesn't explicitly check. I'll fix that on the downloaders side in a few minutes.

[Jules-WinnfieldX]

https://github.com/Jules-WinnfieldX/CyberDropDownloader/releases/tag/2.3.5

Try this.

[hellhond]

Ok, seems to work, now I get a lot of

.. likely exceeded download attempts
Re-run program after exit to continue download.

Thanks for the effort and quick assistance!

[Jules-WinnfieldX]

Not an optimal solution. But better than having dummy or broken files. If you want you can change the Download_Attempts number in settings.py to something higher and see if the 503 errors eventually pan out on their own.

I'm going to find a way to better track which files were successful and which weren't. I'll likely just add a tally to the end of the script and display "Archive Name - x/y successful - w/z partially downloaded - r/t missing" Or something to that effect.