Always showing 0 issues about mariana-trench

[yuhshin-oss]

Hi @abhineetsagar , thanks for trying out Mariana Trench!

I'm not able to repro this. Could you share the output logs of the run? Also check if there are any specific errors that show up in those logs.

If following the steps in https://mariana-tren.ch/docs/getting-started, check that the pre-requisites are satisfied. Also check that the $ANDROID_SDK is set, and that the full path --system-jar-configuration-path=$ANDROID_SDK/platforms/android-30/android.jar exists.

[abhineetsagar]

Will it be the same like - --system-jar-configuration-path=$ANDROID_SDK/platforms/android-30/android.jar ?

…

On Tue, Oct 12, 2021 at 12:39 AM yuhshin-oss ***@***.***> wrote: Hi @abhineetsagar <https://github.com/abhineetsagar> , thanks for trying out Mariana Trench! I'm not able to repro this. Could you share the output logs of the run? Also check if there are any specific errors that show up in those logs. If following the steps in https://mariana-tren.ch/docs/getting-started, check that the pre-requisites are satisfied. Also check that the $ANDROID_SDK is set, and that the full path --system-jar-configuration-path=$ANDROID_SDK/platforms/android-30/android.jar exists. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#53 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF3XSVM5K4QY7KZVXC2PT2LUGMY5TANCNFSM5FYTL7EQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[yuhshin-oss]

That option looks right to me. Can you check if that file ($ANDROID_SDK/platforms/android-30/android.jar) exists in your local setup too?

Also, it would help if you can share the output logs to see if there are any errors reported, something like this:

(mariana-trench) sample-app % mariana-trench \
  --system-jar-configuration-path=$ANDROID_SDK/platforms/android-31/android.jar \
  --apk-path=sample-app-debug.apk \
  --source-root-directory=app/src/main/java
INFO Extracting `/Users/yuhshin/mariana-trench/documentation/sample-app/sample-app-debug.apk`...
INFO Extracted APK into `/var/folders/ng/3z2pkhqs59dg_q9lrypm1g_80000gn/T/tmpl37w3i1p_apk` and DEX into `/var/folders/ng/3z2pkhqs59dg_q9lrypm1g_80000gn/T/tmp5apjxlx__dex`
INFO Running Mariana Trench: /Users/yuhshin/.venvs/mariana-trench/bin/mariana-trench-binary --system-jar-paths /Users/yuhshin/Library/Android/sdk/platforms/android-31/android.jar --apk-directory /var/folders/ng/3z2pkhqs59dg_q9lrypm1g_80000gn/T/tmpl37w3i1p_apk --dex-directory /var/folders/ng/3z2pkhqs59dg_q9lrypm1g_80000gn/T/tmp5apjxlx__dex --rules-paths /Users/yuhshin/.venvs/mariana-trench/share/mariana-trench/configuration/rules.json --repository-root-directory /Users/yuhshin/mariana-trench/documentation/sample-app/ --source-root-directory /Users/yuhshin/mariana-trench/documentation/sample-app/app/src/main/java/ --apk-path /Users/yuhshin/mariana-trench/documentation/sample-app/sample-app-debug.apk --output-directory /Users/yuhshin/mariana-trench/documentation/sample-app/ --maximum-source-sink-distance 7 --model-generator-configuration-paths /Users/yuhshin/.venvs/mariana-trench/share/mariana-trench/configuration/default_generator_config.json --model-generator-search-paths /Users/yuhshin/.venvs/mariana-trench/share/mariana-trench/configuration/model-generators
Trace settings:
TRACEFILE=
SHOW_TIMESTAMPS=
SHOW_TRACEMODULE=
TRACE_METHOD_FILTER=
INFO Initializing Redex...
INFO Redex initialized in 0.46s.
INFO Building source index...
...

[leeweiche20]

I have the same issue.
$ANDROID_SDK/platforms/android-30/android.jar can not found on my computer.
I guess it's because my $ANDROID_SDK doesn't pointed to the location of the SDK correctly.
Would you please kindly share more information about this part?
Thanks in advance.

[abhineetsagar]

Hey @yuhshin-oss Please find below the logs to the performed test to one demo apk names 'via'.:-

INFO Extracting /home/ubuntu/mariana-trench/documentation/sample-app/via.apk...
INFO Extracted APK into /tmp/tmp_mefjdkt_apk and DEX into /tmp/tmprsad6vuq_dex
INFO Running Mariana Trench: /home/ubuntu/.venvs/mariana-trench/bin/mariana-trench-binary --system-jar-paths /platforms/android-30/android.jar --apk-directory /tmp/tmp_mefjdkt_apk --dex-directory /tmp/tmprsad6vuq_dex --rules-paths /home/ubuntu/.venvs/mariana-trench/share/mariana-trench/configuration/rules.json --repository-root-directory /home/ubuntu/mariana-trench/documentation/sample-app/ --source-root-directory /home/ubuntu/mariana-trench/documentation/sample-app/ --apk-path /home/ubuntu/mariana-trench/documentation/sample-app/via.apk --output-directory /home/ubuntu/mariana-trench/documentation/sample-app/ --maximum-source-sink-distance 7 --model-generator-configuration-paths /home/ubuntu/.venvs/mariana-trench/share/mariana-trench/configuration/default_generator_config.json --model-generator-search-paths /home/ubuntu/.venvs/mariana-trench/share/mariana-trench/configuration/model-generators
Trace settings:
TRACEFILE=
SHOW_TIMESTAMPS=
SHOW_TRACEMODULE=
TRACE_METHOD_FILTER=
ERROR Unable to find system jar /platforms/android-30/android.jar
INFO Initializing Redex...
INFO Redex initialized in 0.10s.
INFO Building source index...
sh: 1: hg: not found
WARNING Source directory is not a mercurial repository. Trying find to discover files.
INFO Built source index in 0.02s.
INFO Inferring types...
INFO Inferred types in 0.02s.
INFO Created class properties in 0.00s.
INFO Building class hierarchies...
INFO Built class hierarchies in 0.00s.
INFO Building fields cache...
INFO Built fields cache in 0.01s.
INFO Creating life-cycle wrapper methods...
INFO Created lifecycle methods in 0.00s.
INFO Building override graph...
INFO Built override graph in 0.02s.
INFO Building call graph...
INFO Processed 10000/12545 methods.
INFO Built call graph in 0.33s.
INFO Generating models...
INFO Building method mappings for model generation over 12559 methods
INFO Generated method mappings in 0.04s
INFO Running model generator ActivitySourceGenerator (1/14)
INFO Running model generator AndroidBroadcastSourceGenerator (2/14)
INFO Running model generator AndroidIntentLaunchSinkGenerator (3/14)
INFO Running model generator CodeExecutionSinkGenerator (4/14)
INFO Running model generator content_provider_generator (5/14)
INFO Running model generator ContextFeature (6/14)
INFO Running model generator DataCastFeatureGenerator (7/14)
INFO Running model generator ExplicitIntentFeatureGenerator (8/14)
INFO Running model generator FileSinkGenerator (9/14)
INFO Running model generator ImplicitIntentSourceGenerator (10/14)
INFO Running model generator IntentDataFeatureGenerator (11/14)
INFO Running model generator service_sources (12/14)
INFO Running model generator SQLiteDatabaseSinksGenerator (13/14)
INFO Running model generator WebViewLoadGenerator (14/14)
INFO Generated 428 models in 0.16s.
INFO Initializing models...
INFO Initialized 12559 models in 0.01s.
INFO Initializing rules...
INFO Initialized 5 rules in 0.00s.
INFO Removing unused Kinds
WARNING Kind ProviderExitNode is not used in any rule! You may want to add one for it.
WARNING Kind SQLMutation is not used in any rule! You may want to add one for it.
WARNING Kind ArrayAllocation is not used in any rule! You may want to add one for it.
INFO Removed 3 kinds in 0.01s.
INFO Building dependency graph...
INFO Built dependency graph in 0.02s.
INFO Building the analysis schedule...
INFO Built the analysis schedule in 0.01s.
INFO Analyzing...
INFO Computing global fixpoint...
INFO Global iteration 1. Analyzing 12559 methods... (Memory used, RSS: 0.09GB)
INFO Processed 10000/12559 methods.
INFO Global iteration 2. Analyzing 5500 methods... (Memory used, RSS: 0.09GB)
INFO Global iteration 3. Analyzing 1592 methods... (Memory used, RSS: 0.10GB)
INFO Global iteration 4. Analyzing 829 methods... (Memory used, RSS: 0.12GB)
INFO Global iteration 5. Analyzing 590 methods... (Memory used, RSS: 0.13GB)
INFO Global iteration 6. Analyzing 425 methods... (Memory used, RSS: 0.13GB)
INFO Global iteration 7. Analyzing 312 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 8. Analyzing 256 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 9. Analyzing 183 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 10. Analyzing 124 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 11. Analyzing 7 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 12. Analyzing 9 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 13. Analyzing 9 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 14. Analyzing 12 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 15. Analyzing 13 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 16. Analyzing 21 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 17. Analyzing 30 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 18. Analyzing 28 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 19. Analyzing 25 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 20. Analyzing 30 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 21. Analyzing 37 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 22. Analyzing 63 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 23. Analyzing 77 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 24. Analyzing 63 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 25. Analyzing 55 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 26. Analyzing 26 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 27. Analyzing 29 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 28. Analyzing 23 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 29. Analyzing 16 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 30. Analyzing 23 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 31. Analyzing 29 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 32. Analyzing 27 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 33. Analyzing 12 methods... (Memory used, RSS: 0.14GB)
INFO Global iteration 34. Analyzing 6 methods... (Memory used, RSS: 0.14GB)
INFO Analyzed 12559 models in 21.84s. Found 0 issues!
INFO Augmenting positions...
INFO Augmented positions in 0.00s.
INFO Writing models to /home/ubuntu/mariana-trench/documentation/sample-app/.
INFO Wrote models to 2 shards.
INFO Wrote models in 0.85s.
INFO Writing metadata to /home/ubuntu/mariana-trench/documentation/sample-app/metadata.json.

Please help...

Also please tell me how to run this with local system not by making a virtual environment. how to setup android SDK path and run via local system only not via VENV. Thanks in advance.

[yuhshin-oss]

Thanks for the logs! Yeah, it looks like the issue you're facing is due to $ANDROID_SDK path not being set:

 ERROR Unable to find system jar /platforms/android-30/android.jar

First, check that you have the Android SDK installed. It is usually installed in a location like ~/Library/Android/sdk.

• If it does not exist, you can get it from: https://developer.android.com/studio
• If you already have Android studio installed, but the SDK path is somewhere else, you can find the path by opening Android Studio > Preferences > Search for "sdk". The location will be in "Android SDK Location".

Set the variable in your shell. In my case, this was:

(mariana-trench) ~ % export ANDROID_SDK=/Users/yuhshin/Library/Android/sdk

The commands in the documentation point to $ANDROID_SDK/platforms/android-30/android.jar. However, when I used a fresh installation of Android studio, I had to use this instead: --system-jar-configuration-path=$ANDROID_SDK/platforms/android-31/android.jar (31 instead of 30). My fresh installation didn't come with platforms/android-30/android.jar.

As for running outside of the venv, you can build and run from source as well. Instructions are here: https://mariana-tren.ch/docs/contribution

Hope that helps!

[abhineetsagar]

Thanks @yuhshin-oss I am now able to get the issues as well as see them.

Just last one thing, as I am viewing those issues, I am never getting file snippets where the keywords are present. what can I do for it? Attaching the screenshot..

[yuhshin-oss]

Glad you can see the issues now!

Regarding the file snippets, the setup in the getting started guide will show only source code in app/src/main/java. These are specified in the --source-root-directory and --source-directory options of the following commands:

(mariana-trench)$ mariana-trench \
  --system-jar-configuration-path=$ANDROID_SDK/platforms/android-30/android.jar \
  --apk-path=sample-app-debug.apk \
  --source-root-directory=app/src/main/java

(mariana-trench)$ sapp --database-name=sapp.db server --source-directory=app/src/main/java

In your screenshots, those snippets come from the Android SDK. Those files do not live under app/src/main/java, so they will not show up in the UI.

However, you should see the snippets for issues containing flows within app/src/main/java. The sample app should have produced some issues in "com/example/myapplication/MainActivity.java".

[abhineetsagar]

Hello @yuhshin, Thanks for answering me. I just wanted to know if there is some options to get snippets visible for all the issues cause How come I get to know that the specific folder containing the issue. Can we remove --source-root-directory=app/src/main/java And get all the snippets we want? Regards, Abhineet

…

On Thu, 14 Oct 2021, 01:56 yuhshin-oss, ***@***.***> wrote: Glad you can see the issues now! Regarding the file snippets, the setup in the getting started guide will show only source code in app/src/main/java. These are specified in the --source-root-directory and --source-directory options of the following commands: (mariana-trench)$ mariana-trench \ --system-jar-configuration-path=$ANDROID_SDK/platforms/android-30/android.jar \ --apk-path=sample-app-debug.apk \ --source-root-directory=app/src/main/java (mariana-trench)$ sapp --database-name=sapp.db server --source-directory=app/src/main/java In your screenshots, those snippets come from the Android SDK. Those files do not live under app/src/main/java, so they will not show up in the UI. However, you should see the snippets for issues containing flows within app/src/main/java. The sample app should have produced some issues in "com/example/myapplication/MainActivity.java". — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#53 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF3XSVNAJRS52IOYO3TEZ4DUGXTNXANCNFSM5FYTL7EQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[yuhshin-oss]

Hi @abhineetsagar, removing --source-root-directory=app/src/main/java will result in having no snippets at all, not even for issues reported in MainActivity.java. The --source-root-directory option is meant to point to the source code of the APK being analyzed.

Snippets cannot show up if the source code for them does not exist locally and that is the case with calls into the Android SDK.

[abhineetsagar]

Hey, So what is the use of only .apk file. The tool should have generated the activities files locally and also figure the path by itself, doesn't it?

…

On Tue, 19 Oct 2021, 01:14 yuhshin-oss, ***@***.***> wrote: Hi @abhineetsagar <https://github.com/abhineetsagar>, removing --source-root-directory=app/src/main/java will result in having no snippets at all, not even for issues reported in MainActivity.java. The --source-root-directory option is meant to point to the source code of the APK being analyzed. Snippets cannot show up if the source code for them does not exist locally and that is the case with calls into the Android SDK. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#53 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF3XSVJTOS423JOTU65N4V3UHR2LVANCNFSM5FYTL7EQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[serrapa]

Hey @yuhshin-oss, I have the same problem as abhineetsagar, I cannot see the snippets. For sure I am wrong but I'm trying to analyze an apk and don't have the source code, so what I did is trying to get the sources from jadx-gui and give the path (got from the "save project" of jadx.gui) to the --source-root-directory and --source-directory option. I know that the sources generated are not the same as the real ones but very similar, however, isn't there a possibility to get the snippets without having the source code? Doing as I described, what I receive is a blank page after clicking on the "traces" link of the issue X.

here the server's logs:

021-11-03 18:42:00,315 [INFO] Serving `run/1/issue/1`...
2021-11-03 18:42:00,315 [INFO] Resource not found. Falling back to `index.html`
2021-11-03 18:42:00,316 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "GET /run/1/issue/1 HTTP/1.1" 304 -
2021-11-03 18:42:00,351 [INFO] Serving `static/css/2.4f61bb1c.chunk.css`...
2021-11-03 18:42:00,351 [INFO] Found static resource.
2021-11-03 18:42:00,352 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "GET /static/css/2.4f61bb1c.chunk.css HTTP/1.1" 304 -
2021-11-03 18:42:00,352 [INFO] Serving `static/css/main.6df5d19a.chunk.css`...
2021-11-03 18:42:00,353 [INFO] Found static resource.
2021-11-03 18:42:00,354 [INFO] Serving `static/js/2.2d91596b.chunk.js`...
2021-11-03 18:42:00,354 [INFO] Found static resource.
2021-11-03 18:42:00,355 [INFO] Serving `static/js/main.455a0275.chunk.js`...
2021-11-03 18:42:00,355 [INFO] Found static resource.
2021-11-03 18:42:00,355 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "GET /static/js/2.2d91596b.chunk.js HTTP/1.1" 304 -
2021-11-03 18:42:00,356 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "GET /static/css/main.6df5d19a.chunk.css HTTP/1.1" 304 -
2021-11-03 18:42:00,356 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "GET /static/js/main.455a0275.chunk.js HTTP/1.1" 304 -
2021-11-03 18:42:00,456 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "POST /graphql HTTP/1.1" 200 -
2021-11-03 18:42:00,459 [INFO] Serving `manifest.json`...
2021-11-03 18:42:00,459 [INFO] Found static resource.
2021-11-03 18:42:00,461 [INFO] Serving `favicon.svg`...
2021-11-03 18:42:00,464 [INFO] Found static resource.
2021-11-03 18:42:00,465 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "GET /favicon.svg HTTP/1.1" 200 -
2021-11-03 18:42:00,468 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "GET /manifest.json HTTP/1.1" 304 -
2021-11-03 18:42:00,472 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "POST /graphql HTTP/1.1" 200 -
2021-11-03 18:42:00,478 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "POST /graphql HTTP/1.1" 200 -
2021-11-03 18:42:00,493 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "POST /graphql HTTP/1.1" 200 -
2021-11-03 18:42:00,504 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "POST /graphql HTTP/1.1" 200 -
2021-11-03 18:42:00,528 [INFO] 127.0.0.1 - - [03/Nov/2021 18:42:00] "POST /graphql HTTP/1.1" 200 -