Prototype of HakuNeko based on Electron/NW.js + TypeScript + Vite
manga-download / haruneko Goto Github PK
View Code? Open in Web Editor NEWPrototype of HakuNeko based on NW.js + TypeScript
Home Page: https://haruneko-docs.pages.dev
Prototype of HakuNeko based on NW.js + TypeScript
Home Page: https://haruneko-docs.pages.dev
Prototype of HakuNeko based on Electron/NW.js + TypeScript + Vite
Download Jobs are managed with throttling shared at the connector level.
Since its a given that we have to mimic browser behavior in order not to trigger server errors and Cloudflare, make sure all requests sends at least the website url as referer by default.
Having the possibility to override this for "pages" without having to overload a function (ie. using the plugin requests options when downloading pages)
Default behavior should be at least sending truncated uri from the request itself.
Ex : CartoonMad : Chapter list is https://www.cartoonmad.com/comic/1152.html.
To fetch one chapter html page i need to send the chapter list url as referer. Which i cant.
Some websites put tokens in picture links that expires after some times. If for some reason you add chapters in bulk / you download volumes, and your internet is crappy, there is a chance that when the time come to download a chapter/volume queued pages links are expired and you got 403.
Some users encountered that in Hakuneko, when testing MyAnimeList, but similar website can be affected.
I think the issue could ofc come in haruneko
"http trafic tracking":
https://developer.chrome.com/docs/extensions/reference/devtools_network/
https://developer.chrome.com/docs/extensions/reference/webRequest/
https://developer.chrome.com/docs/extensions/reference/debugger/
Check if devtools api are acessible from the UI code.
//The following code logs URLs of all images larger than 40KB as they are loaded:
chrome.devtools.network.onRequestFinished.addListener(
function(request) {
if (request.response.bodySize > 40*1024) {
chrome.devtools.inspectedWindow.eval(
'console.log("Large image: " + unescape("' +
escape(request.request.url) + '"))');
}
}
);
Could be a nice advanced reader solution (for both vertical & horizontal scrolling)
https://github.com/ollm/OpenComic
Solution must be reviewed first.
Is it possible to add connectors for doki1001.com, manga1000.top, manga1001.top and comick.top . All the ๆผซ็ปใฐใใ sites ?
Confirm that fedora's rpm doesn't have the same sandboxing issue than electron
manga-download/hakuneko#4449
On some machines, hardware acceleration will make apps look blurry or fuzzy. Adding a hardware acceleration toggle option in the settings would be great.
Context : people keep thinking Haku hasnt been updated for years when its false.
Users feedback is important and it should be considered. That is a simple way to tell people "we are working on it".
Ofc user must be able to disable it.
Hello
Japscan website add recently a cloudflare protection that broke scrapping and your scrapping doens't work anymore on haruneko
Implement the "how to use hakuneko" pages accessible from the home screen
To be sorted
To be completed (review from hakuneko issue list that could be adressed by the haruneko rework)
cause of the multiple titles translations problem
so currently HN will not get
Tensei Kizoku no Isekai Boukenroku - Jichou wo Shiranai Kamigami no Shito
if the site is
Tensei Kizoku no Isekai Boukenroku - Jichou o Shiranai Kamigami no Shito
the only differance is the missing w
but its a pretty common differance
fuzzy would deal with this quite handily
english example:
She Professed Herself Pupil of the Wiseman
She Professed Herself the Pupil of the Wise Man
and exact matches would still be at the top of the list, so its not gonna make it any worse
Todo:
Retrieve the previous color scheme used in hakuneko
Following #7 on macOS the maximized state is not detected at start.
Its not caused by a bad CSS selector in Haru.
There are 2 chapters 3.1, with the same id.
When selecting the manga it cause errors, spinning stays, and subsequent click on others mangas keeps the spinning thingie, chapters are added BELOW IT
THIS IS NOT A FILE RENAMING PROBLEM BUT A REQUEST PROBLEM.
Images links ends with webp. Browser downloads WEBP. Haruneko downloads files and rename as jpg as content is really JPG.
Browser uses
Accept : text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Haru uses
Accept : */*
In Common.ImageDirect, try to guess mime type of the link using MangaPlugin.mimeFileExtension dictionary and modify request headers accordingly.
header buttons
Case:
URL 1 : https://mangayeh.com/chapter/cheat-kusushi-no-slow-life-isekai-ni-tsukurou-drugstore/chapter-44
This url redirect (302) to
Then there is another redirection on URL2 (302 again) which is
URL3 : https://gamesk.pro/games/d/serial-cleaners-459051 (randomized)
In browser, URL3 behave differently if you open it in a new tab than if you followed redirections.
Direct access : ads for videogames
Redirected access : page with chapters pictures.
If the last request, Firefox got JSESSION in cookies, which Haru didnt. Cookie which should have been added according to the 'Set-Cookie' directive in PREVIOUS Request response, even if the response code was 302.
This may definitely be this JSESSION cookie that we miss. That and maybe referer (but i added it and it doesn't change anything) or the cloudflare cookie (that will be a problem).
In any case, we should have seen JSESSION cookie in last request i guess.
Intercept the requests response (with a handler) and set the cookie.
Requests to website (get mangas, get chapters, get img) are throttled at the connector level
Remember window size & position on startup
manga-download/hakuneko#3046
We have kitsu, anilist would be good!
Something like
https://codepen.io/ycpand/pen/mwrZXg
git clone https://github.com/manga-download/haruneko.git
cd haruneko
npm install
npm run serve:web-dev
npm run launch:app-dev
npm run launch:app-dev
receive error messageCopied from Haku issue but relevant for haru too :
Currently we have to manually import one videostream mjs per video hoster we want to use. And perform checks on the urls in the connector itself to know which one we need to use.
It would be handy if we could just instanciate one single dispatcher to get the stream url we need.
let streamwhatever = new StreamDispatcher("[https://ok.ru/embed").getstreams(preferered_quality]
Or better : Just returning the embeded url as "page" in _getpages and HN does the job itself (maybe wrapping it inside anobject)
console log alongside jobs
Provide a multiple step post command generator including examples provided by the community.
Common requested use cases needs to be listed.
Case ; Animepahe uses Kwix. We are able to get Kwix playlist.
Downloading the video is ok it works through FFMpeg.
However, its not possible to preview the video. Fragments in the playlist cant be fetched to be previewed in Haku because we need a custom referer to get them, and HLS.min doesn't allow custom referer afaik (and its not planed to be added). While its understandable in a browser context its problematic for us.
Not sure what could be the solution tho.
Use cases : On some websites you have to mix different methods for fetching infos : using urls, their own api, etc..
I.e you have to chose an identifier for each manga : sometimes an internal id (for their api or not) sometimes a pathname.
And sometimes you need both or various reasons. The information is lost between "getmangas, getchapters and getpages" procedures
Storing it inside as class variable (In Plugin/Connector) is not a good idea as its a per-item thing and we can, for example, update manga list while downloading pages. I cant name a particular manga or website but it happened when i was working on connectors.
Since the Rename string is applied to both the view and the file name, it would be convenient to add a language tag. Multi-language sites have their files all renamed to the same name despite one being English, one being Russian, etc. (especially obvious on MangaDex).
A simple tag to indicate the language (even if just the two-letter code EN, ES, RU, IT, etc.) would be great.
Once we got the playlist URL/video link we could allow opening it in an external player.
This could be a nice feature, and a possible workaround in case downloading / previewing with Haru doest work.
I'd see that as an option available on right click on a anime chapter .
Along with throttling,
something like https://github.com/Nitij/HtmlImageDownloadProgressBar
More and more people are asking for it
(open for debate)
So i was porting another legacy website .
Happened on other websites i tried to port too.
Exemple : https://manga-chan.me/online/220423-curtain_v1_ch5.html#page=1
Pictures are AVIF files. Haru display them properly, but downloading save them with JPEG extension that trouble external picture viewers.
Apparently the correct mime type is "image/avif".
Originally posted by korason7117 June 25, 2022
a
it toggles what mcomix call "manual zoom mode" it just display the real deal of resolution of the image i.e : if you have a 1080p screen and the image resolution is 2200 x 1500, you'll be seeing more detail of the image, and thats quite neat. Manga double paged images doesnt go way overhead in resolution. Having a keybind for toggling this "manual zoom mode" which is actually the original resolution would be a big plus C:Recommend trying MComix for getting a general idea of the reader, for some, it could be a bad reader because you have to scroll with the spacebar.
Here my configs for MComix
preferences.conf: https://pastebin.com/Swzh5GGL
keybindings.conf: https://pastebin.com/6iXtg1w0
Dev edit:
Todo:
Creating and updating connectors are recurrent issues present in the hakuneko repository, as the community like to see their favourite sources integrated into the application.
A feasible first solution would base itself on a Github bot that runs a query over the given URLs (main portal, content list, content details, and content reader URLs) and compare styling, structure, and a overall score for each one of these pages against an already present cache for the set of connectors. These scores could be easily calculated by https://github.com/matiskay/html-similarity. Considering that we already have ground thruth connectors parent class this solution seems fair enough.
Further developments could base theirselves at integrating new pages directly at the client, giving our newcomer developer users the power to contribute to the haruneko codebase.
Theres only epub for e-readers and its not supported in kindles. I have to download a .cbz and I need to convert using kcc(kindle comic converter), there i can even enable manga mode to flip through the pages from right to left.
Also i can use stretch/upscale so the pages panels will fit kindle's screen.
Thats it, mobi format!
You can leave some feedback here. Make sure to explain the UI flaws you find out or the good points that should be worked more and any suggestions on how to improve a specific part.
This is not a feature request topic.
This is not about HakuNeko but HaruNeko the WIP/POC of the HakuNeko rework (and probably called HakuNeko 10)
Take note that haruneko is currently in early developement stage
Some random thoughts on how we could have better language support for medias.
Set a default language for a website plugin . Deduced from website unique language tag or manually?
By default children inherit the language property from their parent (no need to modify already done plugins). May add a default arguments in constructors?
What about Extractors? For now they return only ID and / or title. They should return language too. I think about making them return JSON array
{ id : 1234, title : One Piece, language : en_US }
instead of
{ 1234, "One Piece", en_US }
for clarity. This could help avoid errors because i think further. We may have to extract other things in the future and messing with values order will have disastrous effects.
This could event help in other cases : when you have to return a custom ID in copy/paste decorators, you are forced to rewrite FetchManga to match the rest of the code (think "API"). If extractor value got "id", use it instead of default (pathname or pathname+search according to arguments)
This way is more flexible as we can further expand it with stuff for animes : { audio : jp_JP, subtitles : [])
TLDR : Extractors revamping.
Now there are questions :
What to do with multilingual tag?
Language normalization : website use custom codes or values. Exactly what are we using, when , what to store,, how to we translate ? dictionary? where to put it? and many other questions
UI and settings : Having setting by default for subtitles and audio or per plugin?
Dont really have ideas here, i need to think further.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.