luux / arbsmapdo Goto Github PK
View Code? Open in Web Editor NEWAdvanced Ranked Beat Saber Map Downloader
Home Page: https://luux.dev/projects/arbsmapdo/
License: MIT License
Advanced Ranked Beat Saber Map Downloader
Home Page: https://luux.dev/projects/arbsmapdo/
License: MIT License
Due to ARBSMapDo getting more complex, some automatic tests would be nice to prevent breaking bugs. This includes configuration stuff as well as filtering and downloading functions.
For example:
7B9DAB9DDAD1BD3234C1E12D83E6BC2D4A48EC5B_Skeelie--AaltopahWi_Splatter-Party.zip cannot be unzipped by the zipfile package and returns BadZipFile("File is not a zip file"). This results in an error message, the song doesn't get extracted and the tmp dir is not cleared automatically.
The windows explorer isn't able to extract the zip as well. 7zip seems to do the job. However, checking the archive reveals that the included "SplatterVocal.egg" is erroneous.
Oof. Idk if I can fix this one easily. But at least a workaround should be implemented in order to proceed normally.
Filtering by additional info from beastsaber. For example, allows filtering by genre (looking for non-Anime songs? Here you go!)
This will of course be limited by the information that is currently present on bsaber. Many songs don't have genre tags for example. Could be useful nevertheless. However, missing tags must be taken into consideration.
Specify a playlist in parameters/config and add downloaded filtered songs into that. Seems like a natural addition. Goes hand in hand with #4
Check if the user input is valid when asking for options, for example at vote_ratio_min.
Currently, detecting existing songs works on directory name level, which works just fine for maps downloaded with ARBSMapDo. However, maps from other locations don't follow ARBSMapDo's naming structure, so they won't be detected.
This can be solved by generating hashes of other maps and comparing them against a maintained list.
Here's the error:
Scanning already existing maps...
Traceback (most recent call last):
File "D:\Documents\bs\ARBSMapDo\arbsmapdo.py", line 331, in <module>
downloader = advanced_downloader(config_handler.config)
File "D:\Documents\bs\ARBSMapDo\downloader.py", line 61, in __init__
self.cache = cache.Cache(config)
File "D:\Documents\bs\ARBSMapDo\cache.py", line 35, in __init__
self.update_levelhash_cache()
File "D:\Documents\bs\ARBSMapDo\cache.py", line 143, in update_levelhash_cache
levelhash = utils.calculate_Level_hash_from_dir(self.download_dir.joinpath(entry.name))
File "D:\Documents\bs\ARBSMapDo\utils.py", line 62, in calculate_Level_hash_from_dir
info_data = json.loads(info_binary, encoding="utf-8")
File "C:\Python39\lib\json\__init__.py", line 359, in loads
return cls(**kw).decode(s)
TypeError: __init__() got an unexpected keyword argument 'encoding'
The executable seems to be picked up as trojan by virustotal. No idea why, just thought I'd let you know. Malwarebytes also seems to immediately remove the file upon download, and windows smartscreen doesn't like it either
When using multiple presets, download_dir currently has to be saved in each one seperately. A hierarchical preset architecture would solve that. (Note for myself: do not allow cycles)
For uploading maps for the Quest, files are uploaded as *.zip. Implement an option for not extracting downloaded files.
Since BeatDrop2 doesn't work at all since months and doesn't seem to get updated and this is already a Beat Saber download tool - why not support downloading songs given by .bplist files directly?
The Error:
Found Levels: 52/200 | | # | 52 Elapsed Time: 0:00:15 Traceback (most recent call last): File "arbsmapdo.py", line 208, in <module> File "downloader.py", line 47, in start levels_to_download = self.fetch_and_filter() File "downloader.py", line 150, in fetch_and_filter if self._filter_level_with_beatsaver_info(level) is True: File "downloader.py", line 206, in _filter_level_with_beatsaver_info bs_metadata = bs_info["metadata"] KeyError: 'metadata' [11716] Failed to execute script arbsmapdo
This is checking for all ranked songs between 6 and 7 stars. The same thing also happened while filtering between 7 and 10 stars.
I'll leave that here. andruzzzhka/BeatSaberScrappedData#6
It seems there have been major changes in both https://github.com/andruzzzhka/BeatSaberScrappedData/ and the beatsaver API.
As there are some major changes needed in order to get it running again and I'll be quite busy the next days, ARBSMapDo currently does not work.
I'll try to fix these issues as soon as I have some spare time, however this could take a while as we really need some deeper changes here. Sorry :/
89219da contains an experimental first approach of filtering maps by game mode.
This allows searching specifically for OneSaber or 360° maps for example. However, this first implementation has flaws which is why it won't go into master right now: a) it's slow, b) it spams the BeatSaver API with hundreds of requests to find a few songs, since OneSaber maps are quite rare. This is not optimal and cannot be changed directly right now because of how the BeatSaver API works.
However, I could download a regularly updated scrape of the BeatSaver data regularly and perform the search locally on that copy. Daily updated scrapes are already available at https://github.com/andruzzzhka/BeatSaberScrappedData
The problem is that at the moment, the scraped data from BeatSaver doesn't contain the much needed characteristic infos which is where the game mode information lies. If andruzzzhka includes this information in the future, that would be the solution as long as the BeatSaver API is as it is.
andruzzzhka/BeatSaberScrappedData#3
Making it pip-ready. :)
At least direct upload to BMBF could be supported in the future. I'll have to check if there is any way to see the already present maps there.
Handling multiple requests at the same time, starting downloads in parallel to searching songs that meet the criteria.
For example, input URI https://bsaber.com/songs/new/?uploaded_by=ARBSMapDoDummy or https://scoresaber.com/?search=ARBSMapDoDummy instead of a specific map would result in downloading all maps uploaded by the specified user (just as it already works with playlists)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.