Comments (8)
@masteoo Don't worry I have seen your issue. There is no need to ping me in another issue :D But you can contact me at any time via mail.
from moodle-dl.
I think we could introduce a feature to check the size of most external files. Currently we don't check the size of external files in advance, so we can't tell how big the files (mostly videos from external sources) are before we start downloading them. Sometimes we can not even tell the size if we start downloading the files, because some video formats or platforms do not tell how large the file is, so there is no way to estimate it in any way.
For now we could tell the user somehow that the estimated size is only for moodle files and not external files.
If someone is willing to implement this, we can have a discussion how I imagine this feature, and how it would be best to implement it. I personally would like to implement it, but have only little free time currently.
from moodle-dl.
After downloading 357GB which supposedly are 87% of 407 GB I had to kill it.
It says 246 Failed. So I wanted to look in MoodleDL.log which ones failed, but that file is empty. Is it supposed to be empty ?
Where can I learn which ones have failed so I can hunt them up manually and see if I can somehow get them ?
After freeing some space, can I just rerun 'moodle-dl' and it will resume where it left off without having to download again what's already downloaded, and fetching what's still missing and maybe retrying what has failed before ?
from moodle-dl.
moodle-dl was designed in such a way that it never downloads files that have already been downloaded. If it does, it is a bug, or the database has been changed manually.
Since a few versions, the log file is only used if you use the option --log-to-file
.
Where can I learn which ones have failed so I can hunt them up manually and see if I can somehow get them ?
This information is only present in memory, if you kill moodle-dl you will not get informed about failed tasks. But you could scroll up the terminal output.
Failed tasks will be re-downloaded if you re-run moodle-dl
That also brings me to a good idea that we could implement. We could catch kill signals in the download loop, and stop downloading on that signal but inform and notify the user as it would do normally.
from moodle-dl.
Thank you
from moodle-dl.
If the size estimation is not really possible due to external sources, should I close this issue? Also, the percentage given there is clearly not 100 * downloaded / current_estimate.
from moodle-dl.
We can leave the issue open for now, until we find a proper way to tell the user what is going on. It will help users that see that the estimation is increasing while downloading. Also we can discuss here about this topic and find solutions / ideas that we could implement.
from moodle-dl.
The percentage is currently the best we could do (at least in a simple way). Some downloads like HLS Videos or other video formats, have irregular partition sizes, thats why sometimes we add bytes to the estimation, and sometimes we need to remove bytes from the estimation. Actually we should remove the total estimation at the end of a download of such a file and replace it with the actually downloaded size, that way the percentage should be a little better.
from moodle-dl.
Related Issues (20)
- Download fails on some courses HOT 1
- Unable to lock cache on Moodle v4 HOT 4
- Retry on timeout
- config.json should be created unreadable to others HOT 2
- Do not download large files HOT 3
- Usage of -nt -u -pw just hangs moodle-dl HOT 1
- Same file with identical content gets downloaded again every time Moodle-DL is executed HOT 5
- NoneType has no len() - not downloading HOT 1
- Rework optional downloads
- Is there anyway to use it to DL all "Books" on a course? HOT 7
- The Moodle System rejected the Request [invalidprivatetoken] HOT 1
- Quizzes are not properly downloaded HOT 9
- Calendar mod fails with 'dict' object has no attribute 'append' HOT 1
- question re pip install HOT 9
- Add index to subfolders and files
- Encountering this SSL error whilst running --init HOT 4
- Download global grades from a course
- Use pyinstaller to build moodle-dl packages in CI/CD
- Investigate if we can download parts of h5p Modules
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from moodle-dl.