project-artist / artistgui

ARTist's deployment application, for code-injection on an Android device.

ArtistGui - The ARTist deployment app

ArtistGui is the Android application that allows to deploy and utilize ARTist on rooted stock devices. It provides the user interface and management logic while ARTist itself only takes care of instrumenting concrete apps. For more information about the ARTist ecosystem, see the dedicated section below.

Workflow

The current version of ArtistGui ships with a version of ARTist in the form of a specialized dex2oat compiler binary and some library dependencies (e.g., libart-compiler.so), while later versions will be able to download and manage multiple versions of ARTist and several modules.

The user interface exposes a list of all currently installed applications and allows the user to recompile them with the current ARTist modules by tapping on the app name or icon. It is also possible to keep an app instrumented across updates so that whenever an app receives an update, the recompilation is repeated on the new code.

Currently, a CodeLib that provides the methods that are utilized by ARTist needs to be provided by hand, but this will be fixed as a part of our beta release (see beta section below).

Quick Demo

The ARTist ecosystem is complicated to start with, but if you have a rooted device at hand, why not downloading a ready-made ArtistGui version that already ships with the correct ARTist version and demo modules? You can download ArtistGui versions that use our trace module, which writes the names of invoked Java methods to the log (method tracing) here.

Build

After checkout, make sure you initialize the dexterous git submodule:

git submodule update --init

ArtistGui is a regular Android app, so you can build it using gradle:

./gradlew build

In the current version, you need to explicitly plant the compiled ARTist version and CodeLib in the correct asset folders before deploying the app to the device. However, we have tool support to assist you here. In the scripts directory, there are scripts that automate building ARTist in the context of AOSP and copying the corresponding files into the correct folders (dex2oat, libartist-compiler.so, ...). A lot will change here as soon as we reach beta state, i.e. building without AOSP and no more fiddling with binaries and codelibs (see beta section).

Dexterous

ArtistGui relies on the dexterous tool to prepare app apk files before providing them to ARTist. Currently, dexterous is embedded as a git submodule, but in the future, the current release will be pulled from GitHub.

Library Dependencies

All third-tarty code has a valid license (Apache-2.0) and is either included or defined as a gradle dependency.

gradle dependencies

• Android Support Libs:
  • com.android.support:support-v13:25.1.0
  • com.android.support:appcompat-v7:25.1.0
  • com.android.support:design:25.1.0
  • com.android.support.constraint:constraint-layout:1.0.2
• Spongycastle:
  • com.madgag.spongycastle:core:1.54.0.0
  • com.madgag.spongycastle:pkix:1.54.0.0
  • com.madgag.spongycastle:prov:1.54.0.0
• Utilities:
  • co.trikita:log:1.1.5
  • org.apache.directory.studio:org.apache.commons.io:2.4
• Testing
  • junit:junit:4.12

included sourcode

• Kellinwood-ZipSigner (heavily modified and stripped)
  • https://github.com/AllanWang/Kellinwood-ZipSigner
  • https://code.google.com/archive/p/zip-signer/source/default/source
• Android's dex-lib: https://android.googlesource.com/platform/libcore/dex
• Android's dx-tool: https://android.googlesource.com/platform/dalvik/dx

ARTist - The Android Runtime Instrumentation and Security Toolkit

ARTist is a flexible open source instrumentation framework for Android's apps and Java middleware. It is based on the Android Runtime’s (ART) compiler and modifies code during on-device compilation. In contrast to existing instrumentation frameworks, it preserves the application's original signature and operates on the instruction level.

ARTist can be deployed in two different ways: First, as a regular application using our ArtistGui project (this repository) that allows for non-invasive app instrumentation on rooted devices, or second, as a system compiler for custom ROMs where it can additionally instrument the system server (Package Manager Service, Activity Manager Service, ...) and the Android framework classes (boot.oat). It supports Android versions after (and including) Marshmallow 6.0.

For detailed tutorials and more in-depth information on the ARTist ecosystem, have a look at our official documentation and join our Gitter chat.

Upcoming Beta Release

We are about to enter the beta phase soon, which will bring a lot of changes to the whole ARTist ecosystem, including a dedicated ARTist SDK for simplified Module development, a semantic versioning-inspired release and versioning scheme, an improved and updated version of our online documentation, great new Modules, and a lot more improvements. However, in particular during the transition phase, some information like the one in the repositories' README.md files and the documentation at https://artist.cispa.saarland might be slightly out of sync. We apologize for the inconvenience and happily take feedback at Gitter. To keep up with the current progress, keep an eye on the beta milestones of the Project: ARTist repositories and check for new blog posts at https://artist.cispa.saarland .

Contribution

We hope to create an active community of developers, researchers and users around Project ARTist and hence are happy about contributions and feedback of any kind. There are plenty of ways to get involved and help the project, such as testing and writing Modules, providing feedback on which functionality is key or missing, reporting bugs and other issues, or in general talk about your experiences. The team is actively monitoring Gitter and of course the repositories, and we are happy to get in touch and discuss. We do not have a full-fledged contribution guide, yet, but it will follow soon (see beta announcement above).

Academia

ARTist is based on a paper called ARTist - The Android Runtime Instrumentation and Security Toolkit, published at the 2nd IEEE European Symposium on Security and Privacy (EuroS&P'17). The full paper is available here. If you are citing ARTist in your research, please use the following bibliography entry:

@inproceedings{artist,
  title={ARTist: The Android runtime instrumentation and security toolkit},
  author={Backes, Michael and Bugiel, Sven and Schranz, Oliver and von Styp-Rekowsky, Philipp and Weisgerber, Sebastian},
  booktitle={2017 IEEE European Symposium on Security and Privacy (EuroS\&P)},
  pages={481--495},
  year={2017},
  organization={IEEE}
}

There is a follow-up paper where we utilized ARTist to cut out advertisement libraries from third-party applications, move the library to a dedicated app (own security principal) and reconnect both using a custom Binder IPC protocol, all while preserving visual fidelity by displaying the remote advertisements as floating views on top of the now ad-cleaned application. The full paper The ART of App Compartmentalization: Compiler-based Library Privilege Separation on Stock Android, as it was published at the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS'17), is available here.

artistgui's Issues

F-Droid app store

Hi,
I recently browsed through the FOSS app store called F-Droid [0] and I noticed that the ArtistGui app isn't available for download on there. F-Droid is an alternative app store for people that do not use Google and/or the Playstore, so it would be really cool to see your app there.

As F-Droid focuses on FOSS app, they have strict rules of what an app is allowed to do and what not [1], but I believe that the ArtistGui should be fine (but I haven't looked at your source code tbh).

To submit the app to the central F-Droid repo, the fastest way is to clone the metadata repository and create a PR with the new metadata-text file. [2]

Best regards,
gehaxelt

[0] https://f-droid.org/en/
[1] https://f-droid.org/en/docs/Inclusion_Policy/
[2] https://gitlab.com/fdroid/fdroiddata/blob/master/CONTRIBUTING.md

Refresh app list when new app is installed

When we install an application (e.g., via adb) while the view displaying installed apps is already in the foreground, the new app does not show up and we need to close the view and reopen it to see the change.

It would be nice to have this view auto-refresh as soon as an application is installed.

Keep track of which app is instrumented with which module

After importing a module into ArtistGui it can be selected to instrument apps. However there's no way to know with which module a given app was instrumented. We only know if an app is instrumented. This is needed for #70 (Reinstrumentation after app updates broken)

Device compatibility: oat path

We mainly work on Nexus and Pixel devices where oat files are stored in the same location. Other vendors might have changed the folder structure, hence we should test whether ArtistGui works there or if we need to add other oat folder alternatives or even a routine to actively search for the oat path.

Home (description, git log) -> Dashboard (recently instrumented apps, favorite apps, ...)

The app launches the home fragment. It contains a little description and shows information about the git status (current commit, Artist versions, ...). It'd be more useful to show additionally recently instrumented apps, favorite apps (starred in app list) and other things like these.

ModuleImport: Handle failures and edge cases

The ModuleImporter is very naively implemented. It hopes the import succeeds always. The importModule(...) catches IOExceptions and returns null which lead to a NullPointerException in the AsyncTask. The exceptions get logged but the user only sees that the app crashed.

Problems:

• Check if module imported correctly in AsyncTask iterating over the result (null -> NullPointerException)
• During import the module lies in a tmp_extraction folder which get renamed afterwards (don't ignore result, it can fail)
• Handle low space or no space
• ManifestParser can't handle missing/unknown json attributes (app crashes)
• The zip can be corrupted or not a zip at all
• Expected files can be missing in zip
• Notify user that something went wrong (be specific but user friendy)

Modules: Option to reorder module instrumentation order

There are situation where multiple modules want to modify the end of the same method. If one module injects a return statement first the changes of the second module will be ignored.

DexMerger: Replace codelib detection heuristic

DexMerger's current implementation uses a hacky heuristic to detect which dex file is the codelib:
During development, our codelibs were always pretty small, hence the check here tests for the method ID count.
Obviously, this does not work for bigger codelibs and hence we need to replace it.

We can hardly detect codelibs in a generic way since, as of now, they do not have a common format.
Therefore, it might be easier to provide the codelib explicitly and store this information throughout the process.

Reinstrumentation after app updates broken

The reinstrumentation after app updates doesn't work since the introduction of modules. Currently we don't map modules to apps, so we don't know which app was instrumented using which module.

Reinstrument apps after update

Whenever an app is updated, we need to detect this and automatically re-instrument after the installation has finished because the installation triggers a new compilation that replaces our custom oat file for this app.

Crash: Resources.NotFoundException in AppIconCache

ArtistGui/app/src/main/java/saarland/cispa/artist/artistgui/packagelist/view/AppIconCache.java

Line 55 in efa7d3d

appIcon = mdpiContext.getResources().getDrawableForDensity(packageEntry.getAppIconId(),

getDrawableForDensity
might throw a Resources.NotFoundException which causes the App to crash.

Since appIcon already can be null, just catch this Exception too. Might be nicer to return the default-Icon.

Remember user and group for app oat

Before we delete the original oat when recompiling an app, we probe its user and group to later be able to set those accordingly for the freshly compiled, new oat file. However, if compilation fails once (e.g., during debugging) or there is no oat (e.g., we removed it with #10 or the system never created it in the first place), we cannot set user and group properly and the compiled file might be ignored by art.

To fix this, we should be able to determine the correct user and group without having to query the oat each time. This can be solved partially by probing the oat file once in the beginning and persisting the result. However, a superior solution would be to learn per Android OS version (!) what the user and group should be, for example by asking a middleware service such as the package manager or AMS.

constraint-layout 1.1.2: Modules not displayed for some apps

When updating the constraint-layout dependency from 1.0.2 to 1.1.2 and importing a module into ArtistGui, the module is not always shown when tapping on an app. While the list of installed modules appears for some apps, it does not for, e.g., the heise online app (which we use for testing quite often).

Centralized Log Management

While logging verbosity can be set for ArtistGui itself, it does not carry over to the compiler.
Also, as of now, we seem to create log files for each compilation and never delete them ever again.

We want to have a log management that incorporates and controls the logs and their verbosity from ArtistGui AND the artist compiler, takes care of backups and deletes old logs according to a policy. It might be beneficial to allow the user to control this to a certain extent, e.g., decide on the log rotation schedule.

Setup CI

Continuous integration might help to catch bugs, such as:

• version differences (unavailable APIs due to SDK)
• broken assumptions
• problems with dependencies
• UI misbehavior

In the best case, testing should include building the project and running the tests on all supported Android versions. This should be a requirement for landing a pull request.

Travis CI offers free plans for open source repositories, might be worth a look.

UI: Show instrumentation result to user

When I implemented the AppDetailsDialog I had to remove the snackbar. It wasn't possible to move the snackbar into the dialog because it was displayed above the dialog buttons and textviews. It's possible to show a Toast.

Module UI

Exposing the new module management capabilities (#54) to the user requires new UI components and updates to existing ones. In particular, the following is needed:

• module overview UI that displays installed/imported modules
• module details UI that displays available information about a module, such as its version, maintainer, compatibility, ...

Cache last progress update in service

In order to fix the empty progress dialog #32 when the service is handling multiple instrumentation requests we need to cache the latest progress update. Otherwise we can't display the status after the broadcast was fired.

Automatically download/update ARTist

Given that we have ready-made ARTist versions at https://github.com/Project-ARTist/ARTist/releases, it would be nice to directly download the correct ARTist version into ArtistGui instead of requiring it to be compiled into the app as an asset. Effectively, this allows to distribute a slim apk on platforms where there is an ARTist release available.

We should still keep around the possibility for developers to work with self-baked versions so that those have a higher priority than the downloaded ones (if needed). Maybe control this via a settings switch?

UI: Handle multiple instrumentation requests

The InstrumentationController has a queue to handle multiple instrumentation requests but there is no way a user can see how many instrumentation requests are existing. For a while the progress dialog is empty after touching instrument if there is already an instrumentation (triggered by an app update) going on.

Modules: Option to import a module before an instrumentation

It'd be nice to have the an option to import a module before instrumenting an app (during module selection). It's basically a shortcut.

Custom explicit broadcast to auto instrument apps using modules

ArtistGui should import a module, instrument an app and start it triggered by an explicit custom broadcast. This is needed in order to auto deploy modules from an IDE.

Relates to Project-ARTist/meta#21

Add fast scroll to recyclerview in app list (alphabetically)

The app list can be really long. Fast scroll (like in the contacts app/home launcher) makes is easy to jump to a letter (alphabetically) easier.

ArtistConfig.REPLACE_BASE_APK still in LibArtist code

We never touch the original apk so the option to replace the the original apk is useless. The setting doesn't exist anymore but LibArtist checks this.

Port and test on Android 8.0 Oreo

Port and test on Android 8.0 Oreo

Service: Notification stays if started from Broadcast Receiver

The service moves itself to the foreground and creates a notification. In order to reinstrument apps after updates ( #2 ) we start the service from a broadcast receiver which listens for android.intent.action.PACKAGE_REPLACED ( #22 ). After compilation the notification stays in the notification bar.

Invalid last instrumentation data in db after package removal

The app remembers information about instrumented apps. After uninstalling a package the app is still marked as instrumented and present in the db.

Compilation UI redesign

The idea is to redesign the compilation UI to have a fragment/dialog for each app that shows, e.g., whether or when it has been instrumented the last time, and provides functionality to remove instrumentation (#10), reinstrument now or always reinstrument after updates (#9).

Only enable PackageUpdatedReceiver when at least one app marked as "keep instrumented"

The PackageUpdatedReceiver is declared in the AndroidManifest.xml and listens always for android.intent.action.PACKAGE_REPLACED. Using PackageManager.setComponentEnabledSetting()¹ we can enable the BroadcastReceiver only when at least one app is marked as "keep instrument" instead waking up ArtistGui at each app update.

¹ https://developer.android.com/reference/android/content/pm/PackageManager.html#setComponentEnabledSetting(android.content.ComponentName, int, int)

Setting to reinstrument after update

With #8 in mind, we want to have a per-app setting to activate automatic recompilation of apps immediately after they have been updates (and hence the previous instrumentation got lost).

This implies a persistent setting and means to detect updates and trigger compilation accordingly.

Search for app list

It's hard for the user to find a specific app in the list. A search for the app name or package name is more user friendly.

Check for missing ARTist

No ARTist version, meaning dex2oat, libart-compiler.so and the like, is present when ArtistGui is freshly checked out. Hence, we need to check for and warn about missing ARTist binaries.
In the best case, we explain what the problem is and point the user to, e.g., the website.

Loader optimization: Reuse already build AppList for package removal & installation

Currently the BroadcastReceiver (AppListChangedReceiver) which listens for Intent.ACTION_PACKAGE_ADDED and Intent.ACTION_PACKAGE_REMOVED calls Loader.onContentChanged(). This calls forceLoad() on the AsyncTaskLoader. It'd be better to update the existing list instead of rebuilding the entire list.

Replace chmod 777 with reasonable permissions

chmod'ing everything to 777 is an obsolete and dangerous relic from the early times of the project where it was all about getting things done quickly. Now that we want to use ARTist in production systems, we should take care of assigning proper access rights to executable files, such as the resulting instrumented oat file.

The objective is to either dynamically find the correct access rights or find fixed ones that work on a multitude of devices. While the dynamic way is preferable, #11 has shown that it might not always be possible to implement this (easily).

I would assume this is a good first issue if you want to get started with ARTist since you do not have to understand much about the ARTist ecosystem. It is just about finding a way to assign proper access rights to those files that are currently chmod'ed to 777.

ARTist/ARTistGui Support for Android x86/x86_64 Emulator (ranchu)

ARTist/ARTistGui Support for Android x86/x86_64 Emulator (ranchu)

Needs changes in:

• buildscripts
• architecture & platform probing in ARTistGui's functions
• dex2oat's commandline flags

Modules: FAB and DetailView

The primary action of the modules screen is to import a module where a FAB is suitable. A module has metadata like the description which isn't showed anywhere. A module developer can inform the user about something there. That's why we need a detail view for each module.

Visual indication whether app is instrumented

There is no way of seeing if an app is instrumented. It'd be nice to see directly in the app list whether an app is instrumented.

NavigationDrawer: Settings selected after opening and leaving settings

Pick modules for instrumentation

As soon as we have the module management implemented (#54), users should be able to pick the modules that they want to be applied for their apps. The goal is to extend the compilation UI with a module chooser that allows to decide for modules and their ordering (in some scenarios, the ordering matters). The keep-instrumented option also needs to be extended so that the same set of modules is used for the recompilation after updates. For cases where this is not possible (i.e., module not available anymore), the user should be informed to pick a new combination. A simple hint that the auto update did not succeed and she needs to pick a new combination should be fine.

Settings: back navigation broken

In the Settings Activity, the back button in the top left corner is broken.
When navigating to this Activity from the drawer, the arrow will not, as intended, close the Settings Activity but add another instance of the Settings Activity on top.

Undo app instrumentation

With #8 in mind, we want to be able to 'undo' the instrumentation of applications.
The most straightforward way might be to simply delete the new oat file. When executed afterwards, the app will either be compiled again by the system dex2oat or simply interpreted. In any case, the instrumentation should be gone.

Keep apps instrumented (Oreo - implicit broadcast)

We react to updates (#22) with a broadcast receiver which listens for android.intent.action.PACKAGE_REPLACED. This doesn't work in oreo anymore (https://developer.android.com/about/versions/oreo/background.html#broadcasts). We can't listen for implicit events.

Move out DexMerger and Dexterous

Promote Dexterous to a standalone tool so that it can also be used outside ArtistGui.

NavDrawer: Add modules icon (currently using instrumentation icon)

Travis CI doesn't start emulator, highest supported API level is 25

In Travis we run ./gradlew build connectedCheck (line 43). "build" builds the app and runs the unit tests. "connectedCheck" executes the instrumentation tests on an Android device (physical) or an emulator. However we don't create/start an emulator.

Another problem is that Travis CIs highest supported API level is 25 (ArtistGui supports higher versions).
The main problem is that Travis updates/installs its components using the deprecated android binary (in the sdk). I updated the android-sdk cookbook and opened pull request (travis-ci/travis-cookbooks#992).

Module package management

In the context of Project-ARTist/meta#3, we will have different module packages instead of one compiler version that has them hard-coded. So we need to manage those module packages and keep track of them.

Assuming we have a fixed package format, we need logic to

• load modules from the file system and parse them (extract meta information & files)
• persist them in a database
• remove them from the database
• update them from the file system

This issue only focuses on the handling logic, not the corresponding UI.

ProgressDialog: Progress Text (current stage, percent) could be cut off if too long

If the package name of the instrumenting app is too long the progress text could be cut off (below cancel button).

All app updates trigger (re)compilation

PackageUpdatedReceiver does not filter the incoming package names and hence starts instrumentation tasks for every single updated package no matter whether we ever instrumented it before.

We should, however, consult the database and check if we instrumented the app before AND the keep instrumented option is enabled.

Merged APK backup

In the new version of the UI, we dropped the setting to activate merged apk backup.
Instead, it would be nice to always backup the last merged apk and overwrite the previous one.
This way, we make sure to not fill up too much space but still be able to debug efficiently.

ModuleImport: Double import (same package name) can destroy other modules

The ModuleImporter blindly overwrites other modules if both package names match.