[Feature Request] HTML Source: More ways to filter links? about obtainium

[ImranR98]

A HEAD request gets redirected somewhere else entirely.

That's unfortunate - it would have been simpler to use this (could do it in a way that would be re-usable for #771).

[Rhys-T]

Yeah… I was trying to come up with approaches that wouldn't be specific to that site, but I ran across that issue, and figured I'd mention it so you wouldn't have to go through the same "it works for the browser, why doesn't it work for me?" stuff that I did.

If I do curl -I https://kustom.rocks/download/KLWP/939, I get redirected to https://help.kustom.rocks/search?q=download/KLWP/939. If I add -XGET, it sends me to the correct azureedge URL for the actual APK instead. My guess is that whatever web server they're using only has a route configured for that download URL for GET, not for HEAD, and it's falling back to a default 'file not found' route that redirects to a help search.

The filename on the final APK url is kapp-aosp-klwp-prod-release.apk - not enough to get a version number from, but enough to tell it's the AOSP version. (Still not sure why that one says release when it's listed in the beta section…)

I suppose one option would be to include a toggle switch for "Don't use HEAD requests". It looks like curl -I -XGET just does a full GET request like normal, then closes the connection as soon as the response body starts coming in. I don't know if the HTTP client library you're using gives you enough control to be able to do that. It might involve switching to the lower-level send/StreamedResponse APIs? I'm not sure if that would be worth it or not, though. I don't know if there are any other app sites that have this problem.

(The situation here is a little bit different from #771. Here, a given download URL always redirects to the same version of the app, and you still need the HTML to find new versions.)