Support scanning AAR/JAR about apkid

[ajinabraham]

If APKiD does the scanning only at DEX level, it makes sense to not support additional format out of the box.

However, here is an example of missed detection from a converted DEX using d8.

(venv-py3.10) ➜  ./d8 --release MTestApp-1.0.0-Android-release-r8.jar
(venv-py3.10) ➜   ls classes.dex 
classes.dex
(venv-py3.10) ➜ apkid classes.dex 
[+] APKiD 2.1.5 :: from RedNaga :: rednaga.io
[*] classes.dex
 |-> compiler : unknown (please file detection issue!)
(venv-py3.10) ➜  ./d8 --release MTestApp-1.0.0-Android-release-r8.aar 
(venv-py3.10) ➜  apkid classes.dex                                                         
[+] APKiD 2.1.5 :: from RedNaga :: rednaga.io
[*] classes.dex
 |-> compiler : unknown (please file detection issue!)

[strazzere]

What are you trying to net out of this? If you build the dex yourself, you would know what created it and you wouldn't be interested in the binary level details of it. If you just want to grep for strings, you can do so or run the string yara rules over the jar contents/aar. I'm not sure exactly what you're trying to get or what failed in your dex file creation.

…

On Fri, Sep 1, 2023, 11:18 Ajin Abraham ***@***.***> wrote: APKiD currently supports APK and DEX files. Can the support be also extended to AAR/JAR. I tried converting AAR/JAR to DEX and scanning with APKiD, but the binary features are not being detected. — Reply to this email directly, view it on GitHub <#360>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEYIRW7JWOTYHVE5BGXHXLXYIKE7ANCNFSM6AAAAAA4H3CBOU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ajinabraham]

Trying to ID AAR/JAR just like we do on DEX or APK. I did the DEX conversion to see if APKiD can work with the converted DEX format.

(venv-py3.10) ➜  testfiles apkid MTestApp-1.0.0-Android-release-r8.apk 
[+] APKiD 2.1.5 :: from RedNaga :: rednaga.io

[*] MTestApp-1.0.0-Android-release-r8.apk!classes.dex
 |-> anti_vm : Build.FINGERPRINT check, Build.HARDWARE check, Build.MANUFACTURER check, Build.MODEL check, Build.PRODUCT check, possible Build.SERIAL check
 |-> compiler : r8
(venv-py3.10) ➜  testfiles apkid MTestApp-1.0.0-Android-release-r8.aar
[+] APKiD 2.1.5 :: from RedNaga :: rednaga.io
(venv-py3.10) ➜  testfiles apkid MTestApp-1.0.0-Android-release-r8.jar 
[+] APKiD 2.1.5 :: from RedNaga :: rednaga.io

All these binaries share pretty much same code and compiler but are built differently as APK, JAR and AAR.

[strazzere]

Right. So what is missing? Can you show an example of the missed detection? I don't see the use on supporting the extra formats as we specifically are looking for binary differences in a dex file specifically. If you're just looking to leverage the tool to scan what is wasn't intended to scan.... Then it sounds like you found your solution - convert to the format that is supported...

…

On Fri, Sep 1, 2023, 10:44 Ajin Abraham ***@***.***> wrote: Trying to ID AAR/JAR just like we do on DEX or APK. I did the DEX conversion to see if APKiD can work with the converted DEX format. (venv-py3.10) ➜ testfiles apkid MTestApp-1.0.0-Android-release-r8.apk [+] APKiD 2.1.5 :: from RedNaga :: rednaga.io [*] MTestApp-1.0.0-Android-release-r8.apk!classes.dex |-> anti_vm : Build.FINGERPRINT check, Build.HARDWARE check, Build.MANUFACTURER check, Build.MODEL check, Build.PRODUCT check, possible Build.SERIAL check |-> compiler : r8 (venv-py3.10) ➜ testfiles apkid MTestApp-1.0.0-Android-release-r8.aar [+] APKiD 2.1.5 :: from RedNaga :: rednaga.io (venv-py3.10) ➜ testfiles apkid MTestApp-1.0.0-Android-release-r8.jar [+] APKiD 2.1.5 :: from RedNaga :: rednaga.io All these binaries share pretty much same code but are built differently as APK, JAR and AAR. — Reply to this email directly, view it on GitHub <#360 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEYIRWOPC77MYY6I3WLDSTXYINJJANCNFSM6AAAAAA4H3CBOU> . You are receiving this because you commented.Message ID: ***@***.***>

[strazzere]

Correct. If you'd like to profile the dex file of the d8 compiler, a PR would be greatly appreciated for that!

…

On Fri, Sep 1, 2023, 11:07 Ajin Abraham ***@***.***> wrote: If APKiD does the scanning only at DEX level, it makes sense to not support additional format out of the box. However, here is an example of missed detection from a converted DEX using d8. (venv-py3.10) ➜ ./d8 --release MTestApp-1.0.0-Android-release-r8.jar (venv-py3.10) ➜ ls classes.dex classes.dex (venv-py3.10) ➜ apkid classes.dex [+] APKiD 2.1.5 :: from RedNaga :: rednaga.io [*] classes.dex |-> compiler : unknown (please file detection issue!) (venv-py3.10) ➜ ./d8 --release MTestApp-1.0.0-Android-release-r8.aar (venv-py3.10) ➜ apkid classes.dex [+] APKiD 2.1.5 :: from RedNaga :: rednaga.io [*] classes.dex |-> compiler : unknown (please file detection issue!) — Reply to this email directly, view it on GitHub <#360 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEYIRVHM37MOSBDQW26CB3XYIP5TANCNFSM6AAAAAA4H3CBOU> . You are receiving this because you commented.Message ID: ***@***.***>

[enovella]

If APKiD does the scanning only at DEX level, it makes sense to not support additional format out of the box.

However, here is an example of missed detection from a converted DEX using d8.

(venv-py3.10) ➜  ./d8 --release MTestApp-1.0.0-Android-release-r8.jar
(venv-py3.10) ➜   ls classes.dex 
classes.dex
(venv-py3.10) ➜ apkid classes.dex 
[+] APKiD 2.1.5 :: from RedNaga :: rednaga.io
[*] classes.dex
 |-> compiler : unknown (please file detection issue!)
(venv-py3.10) ➜  ./d8 --release MTestApp-1.0.0-Android-release-r8.aar 
(venv-py3.10) ➜  apkid classes.dex                                                         
[+] APKiD 2.1.5 :: from RedNaga :: rednaga.io
[*] classes.dex
 |-> compiler : unknown (please file detection issue!)

Hi @ajinabraham,

It would be great to add this compiler in APKiD. I don't think I will have the time to work on this in the near future, but maybe someone else wants to dig into it. Thanks for opening the ticket!

Best,
Edu

[dustty0]

R8 (and / or d8, L8) compiler leaves string artifacts inside dex.

eg.

~~R8{"backend":"dex","compilation-mode":"release","has-checksums":false,"min-api":24,"pg-map-id":"5820188","r8-mode":"full","version":"8.0.46"}

Such artifacts can be retrieved by using r8's ExtractMarker feature.

Refs:

• https://r8.googlesource.com/r8/+/refs/heads/main/src/main/java/com/android/tools/r8/ExtractMarker.java
• https://stackoverflow.com/questions/67158123/how-to-determine-which-dx-d8-version-was-used-for-apk-creation

Matching signatures like ~~R8 can detect newer dex compilers (I think).

[CalebFenton]

We already do something like this.

https://github.com/rednaga/APKiD/blob/master/apkid/rules/dex/compilers.yara#L178

Anyone want to take a stab at it? I may have time next week.