halfwhole / luminus-downloader

Downloads files and folders from LumiNUS

LumiNUS Downloader

This downloader compares your local directory to LumiNUS, and pulls all new files and folders from LumiNUS into the appropriate location in your local directory.

🌅 Sunsetting LumiNUS Downloader

This project will no longer be supported as of June 2022. I'll be leaving NUS soon, and so will lose my access to LumiNUS and my ability to test its features. More importantly, NUS will be migrating to Canvas in AY2022/23, leaving both LumiNUS and this downloader largely obsolete.

Nevertheless, feel free to continue using this project if it remains useful to you. If there are any new issues, I'll try to see if I can fix them; otherwise, feel free to fork this repository to make any necessary changes.

I'm glad to know that others have found this downloader useful, and am grateful to everyone who's contributed to this project. Good luck, and best wishes to everyone in NUS!

Setting up

User credentials (optional)

In the config directory, create a file credentials.yaml with the following details:

• username: NUSNET username
• password: NUSNET password

If credentials.yaml is absent, the user will be prompted for these details, and the file will be saved automatically.

Module mapping

In the config directory, copy modules_example.yaml into a new file modules.yaml. It should have the following:

• The top line should be the path of your desired local directory, relative to your home directory. This is where all your LumiNUS modules will be stored.
• Each of the following lines should map one LumiNUS module to its corresponding sub-folder in your local directory. A line should have the format <Local sub-folder name>: <LumiNUS module name>.

Running the script

Ensure that you have 7-Zip available, through sudo apt install p7zip-full or brew install p7zip.

Run npm install, then run the main script using node main.js.

Options:

• --silent or -s: No printing (apart from errors)
• --timeout or -t: Timeout per request in ms, defaults to 5000
• --no-save or -n: Will not save your login credentials to a local file
• --reset-credentials or -r: Reset login credentials

Contributions

Contributions are welcome! If you have any issues or suggestions, feel free raise them in the issue tracker, or open a pull request directly.

luminus-downloader's Issues

Better error messages

For lacking directory_path in configuration settings, or for timeout.

Thank you.

No issues. Works incredibly well. Thanks for making this!

Explore making this a CLI

Luminus Folder Names With Slashes

My friend came across this rare problem where if the folder name contains forward slashes, the code breaks (on MacOS and Linux, not sure for Windows).

For example, a path on Luminus can look like this:
My Modules > PS2240 > Files > Tutorial Readings > Tutorial 1-Admin affairs/grouping/case discussion > Tutorial.pdf
'Tutorial 1-Admin affairs/grouping/case discussion' is a permissible folder name on Luminus.

However, when fs parses the path, the path looks like this: 'PS2240/Tutorial Readings/Tutorial 1-Admin affairs/grouping/case discussion', so the program thinks that 'grouping' and 'case discussion' are additional folders to be created.

More specifically, the code stops working at line 68 in downloader.js, when writeFile() call is made in downloadFolder().

Possible solutions:

1. Creating the folders locally, even though they don't exist on Luminus - possibly a hacky solution
2. Adding an additional check in parent folder 'Tutorial Readings' to check if sub-folders have slashes, and if they do, to replace them with some other character for Unix systems

Any ideas on how to resolve this?

Downloaded .zip files in are in Windows format instead of Unix

Backslashes in Windows paths screw up the downloaded .zip file, and so it can't be extracted properly. This also happens when you download the .zip file as a folder from LumiNUS directly; it gives the same bad result. Dammit, LumiNUS. 😡

Sigh, have to find some way to either write the downloaded buffer as a Unix-formatted .zip file, or convert it after it's been written.

Make username and password optional

Missing npm package

Hi, just tried out this luminus-downloader (good stuff btw). I noticed that I was missing the yargs package even after npm install. This was solved by running an additional command in terminal to install it: npm i yargs. Maybe it would be good to add this to README.md under the "Running the script" section in case others encounter this issue? Cheers!