Suggestion for version detection in HTLM sources about obtainium

[ImranR98]

Really good idea 👍
If I can implement it, it should become the default instead of hashing the version string.

[TotallyAvailable]

How about starting in the opposite end.
If you have a company proudly carrying the Open Source tag, providing you several ways of obtaining their Apps, why not try reaching out and ask.

• Are they willing to change those things to contain actual useful information.
• Are they willing to "adopt" certain ways the Open Source community has come up with to get their updates in consistent and trustable ways.
• Are they willing to give you tips on how to make your preferred way of updating work.

Even if you end up going to implement a feature to allow coverage for a few more use cases, just like with #946, you'll eventually run into issues of keeping all those things working.
Not saying don't do or try it, more resilience is always better of course.
More of a "don't try to brute force it" before kindly asking.

Also related to #1097, if they start seeing behavior they might not like, that would be 1 more available option removed from the "could work" bucket for making it work.

[hurried6158]

@TotallyAvailable while I agree with a lot of what you're saying, in the case of Proton, there are numerous people who have done such inquiries already with no positive outcome. They only support protonapps.com and google play store as channels for downloading their apps. They do however post Proton VPN and Proton Mail (most often a few weeks behind) on Github releases.

[hurried6158]

It seemingly works. Got the Proton Calendar update that was released today. Thanks again for implementing this!

[hurried6158]

I'm glad you enjoy the idea. I think the ideal way of doing this is with a HTTP range request. Not all servers hosting files support this, but many (or most?) do (this is the case with protonapps). If they do, they should return HTTP response code 206 wheb the HTTP range request is processed. Since not all do, it might be worth having the version string support left as a fallback. And for scenarios with no support for HTTP range request, and no presense of a version string, it would be unsupported by Obtainium I suppose?

[hurried6158]

(Sorry for accidentally pressing "Close with comment")

[hurried6158]

So I did some experimenting by curling the Proton Calendar APK with a HTTP range requiest using curl:

> curl -r 0-29 -O https://protonmail.com/download/CalendarAndroid/ProtonCalendar-Android.apk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    30  100    30    0     0    110      0 --:--:-- --:--:-- --:--:--   109
> xxd -e ProtonCalendar-Android.apk
00000000: 04034b50 00000000 08210000 00000221  PK........!.!...
00000010: 00000000 00000000 00000000     0066  ............f.

Looking at the (APK) ZIP file header, unfortunately the relevant fields in the header such as "File last modification date" and "CRC32 of uncompressed data" are all set to 0. This would probably mean that checksumming/hashing these 30 bytes won't really provide enough information that differs between two releases.

Perhaps one approach would be to checksum/hash something like the first 1KB of the APK. They should reasonably differ I suppose, but I can't say for sure.

EDIT:

Interestingly, this does not seem to be the case with Proton Pass, which has the CRC32 field in the ZIP header set accordingly:

> curl -r 0-29 -O https://proton.me/download/PassAndroid/ProtonPass-Android.apk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    30  100    30    0     0    131      0 --:--:-- --:--:-- --:--:--   130
> xxd -e ProtonPass-Android.apk
00000000: 04034b50 00000000 08210008 18d90221  PK........!.!...
00000010: 00340e63 00380000 00390000     0000  c.4...8...9...
> xxd -e -s 14 -l 4 ProtonPass-Android.apk  # Retrieve CRC-32 of uncompressed data
0000000e: 0e6318d9

[ImranR98]

Added: https://github.com/ImranR98/Obtainium/releases/tag/v0.14.35-beta

It's optional but is on by default (for any newly added HTML apps, not for existing ones) since it seemed to work well. Hopefully I don't regret that.

[hurried6158]

Thanks! I will let you know if it works in the case of protonapps as soon as they release an update for any of the applicable apps.