Tainted data not flowing through custom propagations about mariana-trench

[arthaud]

Hi @adityavardhanpadala,

You are using signature_match with a regex pattern: "name": "*put*". This does not work, signature_match does a string equality check. You will have to use the signature_pattern constraint.
Also, *put* is not a valid regular expression, it should be .*put.*.

For instance:

       "find": "methods",
       "where": [
         {
          "constraint": "signature_pattern",
          "pattern": "Lorg/json/JSONObject;\\..*put.*"
         }
       ],

[adityavardhanpadala]

Thanks @arthaud! I have modified the propagation according to your comments.

{
    "model_generators" : [
      {
       "find": "methods",
       "where": [
         {
          "constraint": "signature_pattern",
          "pattern": "Lorg/json/JSONObject;\\.<init>.*"
         }
       ],
       "model": {
          "propagation": [
            {
              "input": "Argument(1)",
              "output": "Return"
            }
          ]
       },
       "verbosity": 3
      },
      {
       "find": "methods",
       "where": [
         {
          "constraint": "signature_pattern",
          "pattern": "Lorg/json/JSONObject;\\..*get.*"
         }
       ],
       "model": {
          "propagation": [
            {
              "input": "Argument(0)",
              "output": "Return"
            }
          ]
       },
       "verbosity": 3
      },
      {
       "find": "methods",
       "where": [
         {
          "constraint": "signature_pattern",
          "pattern": "Lorg/json/JSONObject;\\..*put.*"
         }
       ],
       "model": {
          "propagation": [
            {
              "input": "Argument(2)",
              "output": "Argument(0)"
            }
          ]
        },
       "verbosity": 3
      }
    ]
}

But while running mariana trench I get the following log for source model

2023-06-07 13:59:21 INFO Running model generator `TaintedSource` (2/2)
2023-06-07 13:59:21 INFO Method `Lcom/example/flowcheck/Tainted;.source:()Ljava/lang/String;` satisfies all constraints in json model generator TaintedSource
2023-06-07 13:59:21 INFO Method Lcom/example/flowcheck/Tainted;.source:()Ljava/lang/String; generates no new sinks/generations/propagations/sources from 0 for_all_parameters constraints:
Instantiated model: {
  "method" : "Lcom/example/flowcheck/Tainted;.source:()Ljava/lang/String;"
}.
Model template: {}.
2023-06-07 13:59:21 INFO Model generator `TaintedSource` generated 0 models in 0.00s.

Any reason why this is happening? I am using the same model for the source as mentioned before.

This is how the Tainted class looks like.

package com.example.flowcheck;
public class Tainted {
    public String source() {
        return
                "<some string here>";

    }

    public int sink(int mdc) {
        Log.d("FlowCheck", "found: "+ mdc);

        return 0;
    }
}

[arthaud]

I'm really confused here.

According to your first message, it seems like you have 3 model generator files, Source.json, Sink.json and Propagation.json.
The logs in your last message only mention TaintedSource, like if the other model generators were not ran?

Then, your default-model-generator.json only contains TaintedSource and TaintedSink, but not Propagation. That might be one problem.

Another problem is that you forgot the s in sources:

"model": {
        "source": [ # <- here it should be sources not source
          {
          "kind": "TaintedSource",
          "port": "Return"
          }
        ]
      },

Unfortunately, Mariana Trench doesn't detect such typos which lead to a lot of confusion. We should probably improve this.

[arthaud]

fe18ac4 should prevent this error in the future.

[adityavardhanpadala]

Thanks for the commit, should help a lot with debugging. I have fixed the typo in the source model and a few issues with the propagation.

{
    "model_generators" : [
      {
       "find": "methods",
       "where": [
         {
          "constraint": "signature_pattern",
          "pattern": "Lorg/json/JSONObject;\\.<init>.*"
         }
       ],
       "model": {
          "propagation": [
            {
              "input": "Argument(0)",
              "output": "Return"
            }
          ]
       },
       "verbosity": 3
      },
      {
       "find": "methods",
       "where": [
         {
          "constraint": "signature_pattern",
          "pattern": "Lorg/json/JSONObject;\\..*get.*"
         }
       ],
       "model": {
          "propagation": [
            {
              "input": "Argument(0)",
              "output": "Return"
            }
          ]
       },
       "verbosity": 3
      },
      {
       "find": "methods",
       "where": [
         {
          "constraint": "signature_pattern",
          "pattern": "Lorg/json/JSONObject;\\..*put.*"
         }
       ],
       "model": {
          "propagation": [
            {
              "input": "Argument(2)",
              "output": "Argument(0)"
            }
          ]
        },
       "verbosity": 3
      }
    ]
}

This is how it looks now. But the propagation still does not work as expected since I don't get any result for the following sample target class that I am trying to analyze. I have commented the 3 places where I am trying to check for tainted data none of these sinks are getting tainted.

public int GetMotionDetectConfig(int i, MotionDetectConfig motionDetectConfig) {
        HashMap HashMap = new HashMap();
        HashMap.put("cfgType", 1);
        String jSONObject = new JSONObject(HashMap).toString();
        Response response = new Response();
        Tainted tt = new Tainted();
        int sendCommand = 0;
        response.resp = tt.source();
        tt.sink(response.resp); // finds an issue here
        Log.d("FlowCheck", "GET_MOTION_DETECT_CONFIG:" + response.resp);
        if (sendCommand != 0 || TextUtils.isEmpty(response.resp)) {
            return 1;
        }
        try {
            JSONObject jSONObject2 = new JSONObject(response.resp);
            tt.sink(jSONObject2); // Does not find an issue here
            if (jSONObject2.isNull("ret")) {
                return 1;
            }
            int i2 = jSONObject2.getInt("ret");
            tt.sink(i2); // Does not find an issue here. 
            Log.d("FlowCheck", "ret = "+i2);
            if (i2 == 0) {
                if (!jSONObject2.isNull("channel")) {
                    motionDetectConfig.channel = jSONObject2.getInt("channel");
                }
                if (!jSONObject2.isNull("enable")) {
                    motionDetectConfig.enable = jSONObject2.getInt("enable");
                }
                if (!jSONObject2.isNull("linkage")) {
                    motionDetectConfig.linkage = jSONObject2.getInt("linkage");
                }
                if (!jSONObject2.isNull("snapInterval")) {
                    motionDetectConfig.snapInterval = jSONObject2.getInt("snapInterval");
                }
                if (!jSONObject2.isNull("triggerInterval")) {
                    motionDetectConfig.triggerInterval = jSONObject2.getInt("triggerInterval");
                }
                if (!jSONObject2.isNull("moveAlarmEnable")) {
                    motionDetectConfig.moveAlarmEnable = jSONObject2.getInt("moveAlarmEnable");
                }
                if (!jSONObject2.isNull("pirAlarmEnable")) {
                    motionDetectConfig.pirAlarmEnable = jSONObject2.getInt("pirAlarmEnable");
                }
                if (!jSONObject2.isNull("schedule")) {
                    JSONArray jSONArray = jSONObject2.getJSONArray("schedule");
                    motionDetectConfig.schedule = new long[jSONArray.length()];
                    int i3 = 0;
                    while (true) {
                        long[] jArr = motionDetectConfig.schedule;
                        if (i3 >= jArr.length) {
                            break;
                        }
                        jArr[i3] = jSONArray.getLong(i3);
                        i3++;
                    }
                }
                if (!jSONObject2.isNull("sensitivity")) {
                    motionDetectConfig.sensitivity = jSONObject2.getInt("sensitivity");
                }

                if (!jSONObject2.isNull("area")) {
                    JSONArray json_obj = jSONObject2.getJSONArray("area");
                    if (json_obj.length() > 3) {
                        motionDetectConfig.area = new int[json_obj.length()]; 
                        for (int idx = 0; idx < motionDetectConfig.schedule.length; idx++) {
                            tt.sink(json_obj.getInt(idx)); // Does not find an issue here
                            motionDetectConfig.area[idx] = json_obj.getInt(idx);
                        }
                    }
                }
            }
            return i2;
        } catch (JSONException e) {
            e.printStackTrace();
            return 1;
        }
    }

Tainted.java

public class Tainted {
    public String source() {
        return
                "<some string here>";

    }

    public int sink(int mdc) {
        Log.d("FlowCheck", "found: "+ mdc);

        return 0;
    }
    public int sink(String str) {
        Log.d("FlowCheck", "found "+ str);
        return 0;
    }
    public void sink(JSONObject jo){
        Log.d("FlowCheck", "found "+ jo);
    }
}

mariana-trench logs:

INFO Extracting `/workdisk/aditya/app-debug.apk`...
INFO Extracted APK into `/tmp/tmpgodih4hk_apk` and DEX into `/tmp/tmpdfb5t5dk_dex`
INFO Running Mariana Trench: /workdisk/aditya/mtbuild/mariana-trench/build/mariana-trench-binary --system-jar-paths /workdisk/aditya/Android/Sdk//platforms/android-30/android.jar --apk-directory /tmp/tmpgodih4hk_apk --dex-directory /tmp/tmpdfb5t5dk_dex --rules-paths configuration/rules.json --repository-root-directory /workdisk/aditya/mtbuild/ --source-root-directory /workdisk/aditya/mtbuild/ --apk-path /workdisk/aditya/app-debug.apk --output-directory /workdisk/aditya/mtbuild/output/app-debug.apk.results/ --maximum-source-sink-distance 7 --model-generator-configuration-paths /workdisk/aditya/mtbuild/configuration/default_generator_config.json --model-generator-search-paths /workdisk/aditya/mtbuild/configuration/model-generators --lifecycles-paths /workdisk/aditya/mtbuild/configuration/lifecycles.json
Trace settings:
TRACEFILE=
SHOW_TIMESTAMPS=
SHOW_TRACEMODULE=
TRACE_METHOD_FILTER=
2023-06-09 10:32:08 INFO Initializing Redex...
2023-06-09 10:32:09 INFO Redex initialized in 0.34s.
2023-06-09 10:32:09 INFO Storing methods...
2023-06-09 10:32:09 INFO Stored all methods in 0.03s. Memory used, RSS: 0.25GB
2023-06-09 10:32:09 INFO Storing fields...
2023-06-09 10:32:09 INFO Stored all fields in 0.02s. Memory used, RSS: 0.26GB
2023-06-09 10:32:09 INFO Building source index...
2023-06-09 10:32:09 INFO Finding files to index in `/workdisk/aditya/mtbuild/`...
sh: 1: hg: Permission denied
2023-06-09 10:32:09 WARNING Source directory is not a mercurial repository. Trying `find` to discover files.
2023-06-09 10:32:09 INFO Found 586 files in 0.21s.
2023-06-09 10:32:09 INFO Indexing classes...
2023-06-09 10:32:09 INFO Indexed 819 top-level classes in 0.04s.
2023-06-09 10:32:09 INFO Indexing method paths...
2023-06-09 10:32:09 INFO Indexed 372 method paths in 0.01s.
2023-06-09 10:32:09 INFO Indexing method lines...
2023-06-09 10:32:09 INFO Indexed 42736 method lines in 0.01s.
2023-06-09 10:32:09 INFO Built source index in 0.27s. Memory used, RSS: 0.26GB
2023-06-09 10:32:09 INFO Building control flow graphs...
2023-06-09 10:32:09 INFO Built control flow graphs in 0.05s. Memory used, RSS: 0.32GB
2023-06-09 10:32:09 INFO Inferring types...
2023-06-09 10:32:09 INFO Reflection analysis 0.02s. Memory used, RSS: 0.34GB
2023-06-09 10:32:11 INFO Global analysis 1.71s. Memory used, RSS: 0.41GB
2023-06-09 10:32:11 INFO Inferred types in 1.73s. Memory used, RSS: 0.41GB
2023-06-09 10:32:11 INFO Building class hierarchies...
2023-06-09 10:32:11 INFO Built class hierarchies in 0.02s. Memory used, RSS: 0.41GB
2023-06-09 10:32:11 INFO Building fields cache...
2023-06-09 10:32:11 INFO Built fields cache in 0.04s. Memory used, RSS: 0.43GB
2023-06-09 10:32:11 INFO Creating life-cycle wrapper methods...
2023-06-09 10:32:11 INFO Created 6 life-cycle methods for classes inheriting from `Landroid/app/Activity;`
2023-06-09 10:32:11 INFO Created 5 life-cycle methods for classes inheriting from `Landroid/app/Fragment;`
2023-06-09 10:32:11 INFO Created 11 life-cycle methods for classes inheriting from `Landroidx/fragment/app/Fragment;`
2023-06-09 10:32:11 INFO Created 21 life-cycle methods for classes inheriting from `Landroid/app/Service;`
2023-06-09 10:32:11 INFO Created 1 life-cycle methods for classes inheriting from `Landroidx/fragment/app/FragmentActivity;`
2023-06-09 10:32:11 INFO Created lifecycle methods in 0.01s. Memory used, RSS: 0.43GB
2023-06-09 10:32:11 INFO Building override graph...
2023-06-09 10:32:11 INFO Built override graph in 0.08s. Memory used, RSS: 0.44GB
2023-06-09 10:32:11 INFO Building method mappings for shim/model generation over 97217 methods
2023-06-09 10:32:11 INFO Generated method mappings in 0.19s. Memory used, RSS: 0.51GB
2023-06-09 10:32:11 INFO Creating Shims...
2023-06-09 10:32:11 INFO Created Shims in 0.00s. Memory used, RSS: 0.51GB
2023-06-09 10:32:11 INFO Building call graph...
2023-06-09 10:32:11 INFO Built call graph in 0.23s. Memory used, RSS: 0.60GB
2023-06-09 10:32:11 INFO Generating models...
2023-06-09 10:32:11 INFO Found model generator: `TaintedSink`
2023-06-09 10:32:11 INFO Found model generator: `TaintedSource`
2023-06-09 10:32:11 INFO Found model generator: `JSONPropagations`
2023-06-09 10:32:11 INFO Running model generator `TaintedSink` (1/3)
2023-06-09 10:32:11 INFO Model generator `TaintedSink` generated 3 models in 0.00s.
2023-06-09 10:32:11 INFO Running model generator `TaintedSource` (2/3)
2023-06-09 10:32:11 INFO Model generator `TaintedSource` generated 1 models in 0.00s.
2023-06-09 10:32:11 INFO Running model generator `JSONPropagations` (3/3)
2023-06-09 10:32:11 INFO Model generator `JSONPropagations` generated 19 models in 0.07s.
2023-06-09 10:32:11 INFO Generated 23 models and 0 field models in 0.09s. Memory used, RSS: 0.61GB
2023-06-09 10:32:11 INFO Reset MethodToShims and Method Mappings. Memory used, RSS: 0.61GB
2023-06-09 10:32:11 INFO Initializing models...
2023-06-09 10:32:11 INFO Initialized 100101 models and 0 field models in 0.04s. Memory used, RSS: 0.61GB
2023-06-09 10:32:11 INFO Initializing rules...
2023-06-09 10:32:11 INFO Initialized 1 rules in 0.00s. Memory used, RSS: 0.61GB
2023-06-09 10:32:11 INFO Initializing used transform kinds...
2023-06-09 10:32:11 INFO Initialized 0 source/sink transform kinds and 0 propagation transform kinds in 0.00s.Memory used, RSS: 0.61GB
2023-06-09 10:32:11 INFO Removing unused Kinds...
2023-06-09 10:32:11 WARNING Kind `SQLQuery` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `FileResolver` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `ReceiverUserInput` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `SQLMutation` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `ActivityUserInput` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `InputStream` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `SensitiveCookieData` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `IntentCreation` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `CodeExecution` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `WebViewLoadContent` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `ArrayAllocation` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `Partial:LaunchingComponent:a` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 WARNING Kind `Partial:LaunchingComponent:b` is not used in any rule! You may want to add one for it.
2023-06-09 10:32:11 INFO Removed 13 kinds in 0.01s.
2023-06-09 10:32:11 INFO Building dependency graph...
2023-06-09 10:32:12 INFO Built dependency graph in 0.11s. Memory used, RSS: 0.61GB
2023-06-09 10:32:12 INFO Created class properties in 0.18s. Memory used, RSS: 0.61GB
2023-06-09 10:32:12 INFO Building the analysis schedule...
2023-06-09 10:32:12 INFO Built the analysis schedule in 0.14s. Memory used, RSS: 0.61GB
2023-06-09 10:32:12 INFO Analyzing...
2023-06-09 10:32:12 INFO Computing global fixpoint...
2023-06-09 10:32:12 INFO Global iteration 1. Analyzing 100101 methods... (Memory used, RSS: 0.61GB)
2023-06-09 10:32:12 INFO Global iteration 1 completed in 0.28s.
2023-06-09 10:32:12 INFO Global iteration 2. Analyzing 41204 methods... (Memory used, RSS: 0.69GB)
2023-06-09 10:32:13 INFO Global iteration 2 completed in 0.29s.
2023-06-09 10:32:13 INFO Global iteration 3. Analyzing 11628 methods... (Memory used, RSS: 0.71GB)
2023-06-09 10:32:13 INFO Global iteration 3 completed in 0.22s.
2023-06-09 10:32:13 INFO Global iteration 4. Analyzing 7964 methods... (Memory used, RSS: 0.73GB)
2023-06-09 10:32:13 INFO Global iteration 4 completed in 0.21s.
2023-06-09 10:32:13 INFO Global iteration 5. Analyzing 6251 methods... (Memory used, RSS: 0.74GB)
2023-06-09 10:32:13 INFO Global iteration 5 completed in 0.21s.
2023-06-09 10:32:13 INFO Global iteration 6. Analyzing 5431 methods... (Memory used, RSS: 0.74GB)
2023-06-09 10:32:13 INFO Global iteration 6 completed in 0.20s.
2023-06-09 10:32:13 INFO Global iteration 7. Analyzing 4865 methods... (Memory used, RSS: 0.75GB)
2023-06-09 10:32:14 INFO Global iteration 7 completed in 0.19s.
2023-06-09 10:32:14 INFO Global iteration 8. Analyzing 4515 methods... (Memory used, RSS: 0.75GB)
2023-06-09 10:32:14 INFO Global iteration 8 completed in 0.19s.
2023-06-09 10:32:14 INFO Global iteration 9. Analyzing 4053 methods... (Memory used, RSS: 0.75GB)
2023-06-09 10:32:14 INFO Global iteration 9 completed in 0.18s.
2023-06-09 10:32:14 INFO Global iteration 10. Analyzing 3865 methods... (Memory used, RSS: 0.75GB)
2023-06-09 10:32:14 INFO Global iteration 10 completed in 0.18s.
2023-06-09 10:32:14 INFO Global iteration 11. Analyzing 3649 methods... (Memory used, RSS: 0.76GB)
2023-06-09 10:32:14 INFO Global iteration 11 completed in 0.18s.
2023-06-09 10:32:14 INFO Global iteration 12. Analyzing 2784 methods... (Memory used, RSS: 0.76GB)
2023-06-09 10:32:14 INFO Global iteration 12 completed in 0.09s.
2023-06-09 10:32:14 INFO Global iteration 13. Analyzing 1041 methods... (Memory used, RSS: 0.76GB)
2023-06-09 10:32:14 INFO Global iteration 13 completed in 0.04s.
2023-06-09 10:32:14 INFO Global iteration 14. Analyzing 341 methods... (Memory used, RSS: 0.76GB)
2023-06-09 10:32:14 INFO Global iteration 14 completed in 0.01s.
2023-06-09 10:32:14 INFO Global iteration 15. Analyzing 40 methods... (Memory used, RSS: 0.76GB)
2023-06-09 10:32:14 INFO Global iteration 15 completed in 0.00s.
2023-06-09 10:32:14 INFO Global iteration 16. Analyzing 5 methods... (Memory used, RSS: 0.76GB)
2023-06-09 10:32:14 INFO Global iteration 16 completed in 0.00s.
2023-06-09 10:32:14 INFO Global fixpoint reached.
2023-06-09 10:32:14 INFO Analyzed 100101 models in 2.55s. Found 0 issues!
2023-06-09 10:32:14 INFO Removing invalid traces due to collapsing...
2023-06-09 10:32:15 INFO Removed invalid traces in 0.05s.
2023-06-09 10:32:15 INFO Augmenting positions...
2023-06-09 10:32:15 INFO Augmented positions in 0.01s.
2023-06-09 10:32:15 INFO Writing models to `/workdisk/aditya/mtbuild/output/app-debug.apk.results/`.
2023-06-09 10:32:15 INFO Wrote models to 11 shards.
2023-06-09 10:32:15 INFO Wrote models in 0.12s.
2023-06-09 10:32:15 INFO Writing metadata to `/workdisk/aditya/mtbuild/output/app-debug.apk.results/metadata.json`.

[arthaud]

Could you try using --log-method=<full-name-of-your-method> to get the logs of the analysis within the method and send it back to me? Thanks.

[adityavardhanpadala]

Attached the logs for the function.
app-debug.apk.log

[adityavardhanpadala]

I have attached the apk here to make things easier.
app-debug.zip

[arthaud]

Thanks.

I think you made a mistake in the models. For JSONObject.<init> you have:

{
       "find": "methods",
       "where": [
         {
          "constraint": "signature_pattern",
          "pattern": "Lorg/json/JSONObject;\\.<init>.*"
         }
       ],
       "model": {
          "propagation": [
            {
              "input": "Argument(0)",
              "output": "Return"
            }
          ]
       },
       "verbosity": 3
      },

The constructor is actually a non-static method, hence Argument(0) represents the implicit this.
You want to use Argument(1) here to have a propagation from whatever the first argument is (in the example, it will be response.resp) to the return vaule.

[adityavardhanpadala]

Yes, I have tried both Argument(0) and Argument(1) the result remains the same where only this is being detected but the rest of the flows are not detected.

        response.resp = tt.source();
        tt.sink(response.resp);

I have attached mariana trench logs with the propagation input as Argument(1).
app-debug.apk.log

[arthaud]

oh my bad, I think you want a propagation from Argument(1) to Argument(0)

"propagation": [
            {
              "input": "Argument(1)",
              "output": "Argument(0)"
            }
          ]

[adityavardhanpadala]

I am assuming instead of setting the return value as tainted, we need to set Argument(0) i.e this to be tainted and mariana trench propagates the taint to return due to the assignment? Is that how it works? In theory this seemed to be the correct way to define what I wanted to do. Any reason why this was the case?

This is because we analyze the Android bytecode. When you write:
JSONObject o = new JSONObject(response.resp);
It is somewhat translated into:

o = new Object;
JSONObject.<init>(o, response.resp);

The constructor does not really "return" anything, it just updates the implicit this parameter.
We should probably document this. Feel free to submit a PR if you would like to do that.

Why was mariana-trench unable to analyze the underlying function to detect the propagation automatically? Is there a way I can specify mariana-trench to analyze every function it comes across in the flow?

Probably because you did not provide the code for JSONObject. Most likely, it just found a jar with the definitions but not the code.
We usually assume any method without code does taint-in-taint-out (ie., propagation Arg(i) -> Return) and/or taint-in-taint-this (i.e, propagation Arg(i) -> Arg(0)). You should use the TaintInTaintOutGenerator and TaintInTaintThisGenerator which are built-in.
For some reason, it seems like TaintInTaintThisGenerator wouldn't generate propagations for JSONObject, this might be a bug we need to fix.

[adityavardhanpadala]

The android jars that are shipped with the SDK just have this for a lot of similar functions

    public JSONArray put(int index, double value) throws JSONException {
        throw new RuntimeException("Stub!");
    }

Is this what causes the issue of mariana-trench not propagating the taint properly?

[arthaud]

Yes this could be a problem. We would have to use a model generator to cover those.