jakev / dtfmods-core Goto Github PK

Affected modules(s): frameworkres

APKs that are in sub-directories of /system/framework/ are not picked up by frameworkres.

Appears to be a relative path issue.

Affected Module(s): sysappdb, sharedid
Affected Libraries: AppDb

In order to properly implement #7, I'll need to have the Shared user IDs in a database. From here, I can expand AppDb to lookup applications by shared Id, and potentially enumerate shared ID. I will parse this when pulling applications with sysappdb process.

This would also mean converting sharedid to use the new python methods.

Sometimes you might want to create a Java JAR for an application (using dex2jar). A simple module to wrap this will be added.

analyst$ dtf app2jar com.example

Affected Module(s): frameworkdb, sysappdb?

When testing on a gen 3 Kindle Fire, with FireOS 4.5.4, framework files are stored in a strange DEX format, with the magic:

0000000: 6465 790a 3033 3900 4000 0000 20af 0000  dey.039.@... ...

file claims it is:

Dalvik dex file (optimized for host) version 039

But I've never seen version 39 before. It looks like in most cases, the DEX included in the framework JAR is actual DEX (version 35), and not this Amazon ODEX. Options are:

• Hope that baksmali handles this Amazon ODEX
• Attempt to unpack from the JAR first

Affected module(s): sysappdb
Affected libraries: AppDb

Intent filters should be parsed and stored in their own table. When displaying exposed components, the intents can be parsed as objects. This lets us do fun things like:

• Search for and possibly omit launcher intents
• Check if an action is protected
• Run queries based on specific actions or data (show me all secret codes, show me all receivers for action n)

Affected module(s): getframeworks, unframework, frameworkdb. frameworkdexdb

On Lollipop with ART, the directory "/system/framework/" is not a definitive list of frameworks. The "arm(64)?/" sub-directory has the "boot.oat" file, which will have the remaining frameworks.

As it is now, getframeworks will pull everything from "/system/framework/", and unframework is responsible for doing the extraction of the OAT files from "boot.oat", but it does this to the "unframework/" directory.

frameworkdb relies on the *.jar files in the "frameworks/" directory, but this isn't really a good approach, since things like "framework2" do not have a JAR for ART.

There needs to be a operation that actually makes changes to the "frameworks/" directory, much like sysappdb oatextract, or frameworkdb needs to have a better approach for generating it's listing.

classsearch will be restructured to be more flexible and easy to use. It will be renamed as dexsearch going forward.

One such inprovement, is the ability to search frameworks/apps for strings (per capabilities added here: android-dtf/core_dex_db@d136733).

analyst$ dtf dexsearch --apps --hasString "sys\.persist"

Affected module(s): sysappdb
The 'apps' table of the "sysapps.db" has a column for "successfully_unpacked", but it is not used. There should also be a "successfully_pulled" field.

Framework files that have the extension JAR but are actually DEX are not currently handled (Kindle FireOS).

I'll handle these later, but not in 1.3.

11:27:24 /DevTesting/Fire$ file framework/*.jar|grep zip |wc -l
64
11:29:00 /DevTesting/Fire$ file framework/*.jar|grep dex |wc -l
2
11:29:05 /DevTesting/Fire$ 

When runnning sysappdb's process sub-command, you can product a list of components that have permissions specified, but do no actually exist by doing the following hack:

$ dtf sysappdb process |grep "how can this be possible" > missing_perms.txt

This isn't elegant at all. I'd like a '--save-missing' argument to write content to the reports/ directory.

Create a script for handling dependencies on Kali + Ubuntu.

I'd like to parse AF_INET, AF_NETLINK, and AF_UNIX information from /proc/net/.

Some devices are not good about being consistent with who is suppose to be defining a permission. For example, on the Kindle Fire (FireOS), mulitple applications define the same permission, with completely different definitions. An example:

In package com.amazon.kindle.otter:

<permission android:name="com.amazon.CONTENT_PROVIDER_ACCESS"/>

But in com.amazon.kindle it's completely different:

<permission android:description="@string/content_permissions_description" android:label="@string/content_permissions_label" android:name="com.amazon.CONTENT_PROVIDER_ACCESS" android:permissionGroup="android.permission-group.PERSONAL_INFO" android:protectionLevel="signature"/>

I'm not 100% sure how the OS determins which one to use, but Android has picked one of them:

02:29:45 /DevTesting/Fire$ dtf_shell dumpsys package perm |grep CONTENT_PROVIDER_ACCESS -A 2
--
  Permission [com.amazon.CONTENT_PROVIDER_ACCESS] (41f1c2e0):
    sourcePackage=com.amazon.kindle.otter

I'm going to migrate the parse_permissions( .. ) method of the sysappdb module to use dumpsys to determine who the OS has picked for the true permission holder.

Affected module(s): frameworkdiff

Currently, platformdiff associates only the first 'gid' in a permission mapping, but there could be n number of 'gid's per permission.

Affected Module(s): sysappdb

Handle new devices with 64bit arm: /system/priv-app/AppName/arm64/

Affected module(s): platformdiff
Affected binary(s): platform-diff.py

This is the old (dtf 1.1.0 and prior) way of executing python with dtf. It needs to be updated.

Affected module(s): sysappdb

If a MD5 mismatch is detected, the thread will exit, but the program will hang. The behavior needs to be exit.

The newish XML tag in a manifest indicates that the application will likely use the dynamic permission creation. These need to be parsed.

There are quite a few select all statements in AppDb. Since the schema for the AppDb tables keep changing (expanding?), these should be removed. That way, older versions of dtf can still parse databases made by different versions (assuming columns didn't change).

Right now I'm just storing the XML, which isn't really ideal.

Affected module(s): appdexdb, frameworkdexdb

The logic to determine which diffing DB does not work correctly. Like at all.

getpermissions is old and simply grabs the '/system/permissions/' directory. I'm going to add a platformdb module which has the following:

• features
• libraries
• assign_permissions
• gid_mappings

Removed modules: getpermissions, platformdiff
Added module: platformdb

platformdb will support:
*create - Creates the DB (getpermissions + DB)
*diff - Diff the DB (an updated platformdiff)
*dump - Prints information from the DB (new functionality)

Affected module(s): sysappdb
Affected libraries: AppDb

Knowing which protected broadcasts have been added by the OEMs will be useful in determining if receivers are actually exposed.

Some devices, such as HTC add several permissions to the shell user ("android.uid.shell"), some of which may be signature|system protection level. It would be benefitial to:

1. See all (new) permissions assigned to the shell user (this is basically asking for diff support with sysappdb, which used to be a thing)
2. use sysappdb exposed as a given user. A few examples:

As the dummy application com.dtf.client:
$ dtf sysappdb exposed --user-profile com.dtf.client --all

As the shell application:
$ dtf sysappdb exposed --user-profile com.android.shell --all

As a sharedUserId (shell for example):
$ dtf sysappdb exposed --shared-id 2000 --all

Migration from v1.2 to v1.3 progress. I'll update this as I start doing commits.

Modules

• apkget (aae99d7)
• appdexdb (e56266d)
• autosign (aae99d7)
• bindiff (ff7094f)
• classsearch (ff7094f)
• devdb (aae99d7)
• devdiff (aae99d7)
• [Removed] drozer (c9a2275)
• findimp (a171325
• framework2jar (a171325
• frameworkdb (a171325
• frameworkdexdb (e56266d)
• frameworkres (ff7094f)
• getbins (ff7094f)
• getpermissions (aae99d7)
• getsyslibs (aae99d7)
• [Removed] installdrozer (aae99d7) (c9a2275)
• intentsender (aae99d7)
• libinfo (aae99d7)
• mkaosp (25bcbf6)
• newapp (aae99d7)
• oatdextract (a171325
• pbroadcasts (a171325
• permissions (846377e)
• platformdiff (aae99d7)
• process-all (25bcbf6)
• [Added] resetserial (356020d)
• secretcode (aae99d7)
• sharedid (a171325
• sysappdb (356020d)
• syslibdiff (a171325
• sysservicedb (846377e)
• testperm (ff7094f)
• userapps (aae99d7)
• usesperm (a171325

Affected Module(s): apkget

apkget doesn't know to check the arm(64) directory for ODEX files on ART devices.

Traceback (most recent call last):
File "/usr/local/bin/dtf", line 185, in
sys.exit(main())
File "/usr/local/bin/dtf", line 175, in main
rtn = pkg.launch_module(command_name, sys.argv)
File "/usr/local/lib/python2.7/dist-packages/dtf/packages.py", line 188, in launch_module
return __launch_python_module(module_path, cmd, args)
File "/usr/local/lib/python2.7/dist-packages/dtf/packages.py", line 97, in __launch_python_module
return mod_inst.run(args)
File "/usr/local/lib/python2.7/dist-packages/dtf/module.py", line 62, in run
result = getattr(self, 'execute')(args)
File "/home/analyst/.dtf/modules//sysappdb", line 2657, in execute
return self.cmd_process(args)
File "/home/analyst/.dtf/modules//sysappdb", line 2499, in cmd_process
self.do_final_pass(appdb)
File "/home/analyst/.dtf/modules//sysappdb", line 1429, in do_final_pass
self.parse_debuggable(appdb, project_id, manifest_path)
File "/home/analyst/.dtf/modules//sysappdb", line 1163, in parse_debuggable
manifest_f = open(manifest_path)
IOError: [Errno 2] No such file or directory: u'decoded-oem/com.test/AndroidManifest.xml

I'd like to process all the SEAndroid files to help determine what a user can do. These files appear public even on locked down phones.

in order to use dtf effectively, you'll need to run 10+ modules. A process-all wrapper module should contain all the commands needed to:

• pull apps, unpack, populate database, create DEX dbs
• pull frameworks, unpack, populate database, create DEX dbs
• generate system service database
• generate device database

Obviously there might be errors, so error checking is most important here.

Right now, there is no indication of the platform, shared, and media signing keys. These should be mapped and saved.

Add --output json flag to sysservicedb to support parse-able output.

Affected module(s): sysappdb

To satisfy android-dtf/dtf#6, aapt is no longer assumed to be available. Modules should use the dtf_include "dtf_aapt" instead.

A new module called userdb will be added. This creates a database of Linux users on the system, for diffing and list. I might create an API as well.

The first step is to integrate the "makeuserdb" repo into a dtf package.

Affected module(s): permissions

The permissions module should support additional permission capabilities. As of right now, it only supports listing permissions.

List all permissions
dtf permissions

List all OEM added permissions
dtf permissions diff [--diff-db mydb.db] [--filter protection_level,]

Determine which applications use a permission
dtf permissions appuses android.permission.READ_SMS

Determine components that use a permission is used
dtf permissions lookup android.permission.READ_SMS

Affected Module(s): sysappdb, frameworkdb, appdexdb, frameworkdexdb

The diffing modules refer to diffing as being against the AOSP data, which is true, but this is just a default behavior. Most (soon to be all) of these modules actually support arbitrary diffing as is, using the horribly named dtf property 'Local\aosp-data-dir'. This is confusing. Even to me.

The property will be moved, and all modules need to be adjusted to indicate that diffing is default against the AOSP, but can be set to anything. This is really just a huge bit of changing variable names, usage, etc.

The new property is called "Local\diff-data-dir", and all modules will need to be adjusted.

Both devdb and devdiff are old and in need of a refresh.

I'd like to have a single module, that supports the following:

• create - Create the DB (currently as devdb create)
• diff - Diff devices (currently as devdiff)
• exposed - Show exposure of devices (currently as devdiff --exposed)

exposed will be the most powerful, and can take advantage of the new userdb module. Examples:

# Running by default should tell me anything a third-party app can use
$ dtf devdb exposed

# We can also say only non-AOSP devices, like sysappdb
$ dtf devdb exposed --new-only

# But let's say I somehow can access the 'inet' group, what now?
$ dtf devdb exposed --new-only --as-user inet

# We should also support many users:
$ dtf devdb exposed --new-only --as-user inet,log,media

Affected module(s): sysservicedb

It's not possible to simply replicate the "adb shell service list" command without having a device connected.

frameworkdb doesn't check the /system/framework/arm/ directory for ODEX when pulling content.

APK signatures should be parsed to the 'apps.db' database. This will allow you to understand which permissions you gain by compromising an application.

Affected Module(s): sysappdb
Affected Library(s): AppDb

Currently, sysapdb can be used to reveal exposed components. It currently lacks: diffing and dumping capabilities. Here are some examples.

Print all and tags for Phone app
dtf sysappdb dump --filter uses-permissions,permissions com.android.phone

Print all added content providers to Contacts
dtf sysappdb diff --filter providers com.android.contacts

Affected module(s): sysappdb

In the case that the device is no longer detected by adb, the module does not wait until the device is reconnected.

Affected Module(s): oatdextract

The oatdextract module will overwrite the input file if the input file is named the same as the output. By default, a framework called "com.my.framework.odex" will be overwritten if the tool is run as:

$ dtf oatdextract com.my.framework.odex

• sharedUserId permissions are automatically granted, so knowing the listing of perms per group is important
• depending on the shared ID, you might be able to send protected broadcasts
• others im sure

Affected module(s): getframeworks, unframework, frameworkdb, frameworkdiff

The framework modules need to be Python. I was holding off on this until I absolutely needed to, and that time is now (re: #15).

It will be modeled after sysappdb.

Activity alias's possess their own metadata (such as permissions), but are associated with the same underlying implementation. These should be parsed.

http://developer.android.com/guide/topics/manifest/activity-alias-element.html

Affected module(s): ?

Support Lollipop R1.

Affected Module(s): unframework, getsysapps

The Galaxy S5 has a XZ compressed framework ODEX on Lollipop: /system/framework/arm/services.odex.xz. The reason behind this is unknown to me, but these need to be handled.

The S5 on Lollipop also has xz compressed ODEX + ART files:

$ adb shell ls /system/app/WebViewGoogle/arm
WebViewGoogle.odex.art.xz
WebViewGoogle.odex.xz

Traceback (most recent call last):
File "/repos/dtf/modules/sysappdb", line 2042, in
exit(main(argv))
File "/repos/dtf/modules/sysappdb", line 2031, in main
return cmdProcess(argv)
File "/repos/dtf/modules/sysappdb", line 1888, in cmdProcess
parseServices(appdb, project_id, manifest_path)
File "/repos/dtf/modules/sysappdb", line 923, in parseServices
appdb.addServiceIntentFilter(intent_filter, _id)
File "/repos/dtf/lib/AppDb/AppDb.py", line 929, in addServiceIntentFilter
priority = int(intent_filter.getPriority())
ValueError: invalid literal for int() with base 10: '0x7fffffff'