Coder Social home page Coder Social logo

bimmergestalt / aaidrive Goto Github PK

View Code? Open in Web Editor NEW
513.0 41.0 89.0 47.75 MB

Implementations of some Android Auto features as unofficial IDrive apps

License: MIT License

Kotlin 99.47% Shell 0.09% Python 0.23% HTML 0.21%
idrive android reverse-engineering android-auto bmw-idrive bmw-connecteddrive

aaidrive's Introduction

AAIdrive

Build Status Code Coverage Crowdin Release Download Download Counter Weekly Users Gitter Buy Me A Coffee MIT Licensed

Get it on Google Play

The BMW/Mini IDrive NBT does not offer native Android Auto integration, but does provide a very powerful Connected Apps convergence option with tight integration points into the car. This project is an effort to implement most of the features of Android Auto as unofficial BMW/Mini Connected Apps.

By relying on the Connected Apps technology, this app greatly extends the functionality of the car without any modifications or hacks. Any MY2014 or newer BMW or Mini equipped with NBT or NBT Evo and the "BMW Apps (6NR)" feature, an active BMW ConnectedDrive subscription, or the "Mini Connected (SA6NM)" option should be compatible.

App List
Gallery

Overview

As part of the Connected Apps feature, when the phone connects to the car over USB (or Bluetooth in 2017+ models), enabled phone apps can show a special dashboard-optimized interface in the car.

AAIdrive, combined with the safety benefits of the tactile IDrive controller, builds on this protocol to allow the user to interact with their incoming notifications and control their phone's music while the phone is safely tucked away.

Getting Started

This app requires that the MyBMW or MINI app for your car is installed and that it can successfully enable your ID5+ car's Apps checkbox, or if your ID4 car has the Connection Assistant option.

Download the APK of the latest stable release from the Releases page. Choose the one that says "sentry" to automatically upload crash reports, or choose "nonalytics" otherwise. It is also available on the Google Play Store. After starting, the app should detect the MyBMW app and start waiting for the car connection.

Also consider trying out the nightly build! It has the latest features and is a preview of the next release, so please consider installing the Sentry build to automatically report crashes. The nonalytics build is available too.

Check out the FAQ if you run into problems.

User Guide

Phone Connection

After connecting the phone to the car, the official MyBMW app should show this car icon in the status bar. When this icon appears, this app should connect and add its functionality to the car.

See this guide for tips on improving the connection reliability of the MyBMW app.

Phone App List

After all the apps are connected, a bunch of new entries will show up in the car's Connected menu. There should be a new Audioplayer or Spotify icon and a book icon with no label. This book icon is the Notifications app, if enabled.

Music App List

Several new entries will be added to the Media section of the control screen. The Audioplayer icon is the one with the main functionality, while the other displayed apps above the Audioplayer are quick shortcuts to switch playback to the respective apps. This screenshot also shows the legacy Spotify app at the bottom of the list.

Development Status

AAIdrive is feature-complete, implementing as many Android Auto features as are possible within the BMW app system, and several more. Beyond the occasional maintenance patch, this project is complete and won't see any more development.

Implemented Features

Integration Points

Besides showing a self-contained remote UI, the IDrive system offers many exciting integration points. Here are a few that this project supports:

  • The UI widgets automatically take on the respective theme to fit the car
  • The Assistants, Map View, Notification List, and Music Playback screens can be assigned to the physical shortcut buttons in the dashboard
  • Car information is retained after disconnect, such as window status and parked location
  • New notifications trigger a statusbar icon
  • New notifications can trigger a popup
  • New notification popups can be disabled if a passenger is detected in the seat
  • Notifications can be read out by the car's TTS when received or when selected
  • The car's navigation system is available to handle Android Navigation Intents
  • The car's location and heading is shown in the custom map
  • The currently-playing app is displayed along the top of the IDrive screen
  • The currently-playing song title is shown in the IDrive4 Multimedia side panel
  • The currently-playing song coverart and progress is shown in the IDrive5+ Home screen and Multimedia side panel
  • On a MY2017+ car supporting Bluetooth Apps, audio focus will be enabled which grants the following extra features:
    • The Media shortcut button opens this app when it is in control of the music
    • Automatically resumes playback when reconnecting to the car
    • Playback pauses when pushing the mute button or during calls
    • The physical back/next buttons can be held down to seek within a track or pressed to skip tracks
    • The steering wheel controls can skip tracks from the instrument cluster
    • Enqueued songs can be scrolled in the instrument cluster, depending on app support

Limitations

This project replicates some of the features of Android Auto using the IDrive interface, using the same APIs that Android Auto uses to talk to the music apps. It cannot currently provide more advanced Android Auto features, such as:

  • Integration with the car's Voice Assistant button
  • Screen-casting of arbitrary phone apps to the car (Google Maps, Waze, or any other apps)
  • Displaying the original Android Auto interface at all

Due to the unofficial reverse-engineered nature of this project, it has some limitations:

  • The main menu entries' icons and text can't be altered, and so do not look exactly correct
  • The individual music source icons sometimes don't open the Audioplayer interface in ID4, but they do switch the active music source
  • Android Oreo disabled Android Open Accessory Protocol 2 audio output, which is required to play audio over the app's USB connection in model years 2014-2017. Please listen over Bluetooth audio and use this app as a control interface.
  • Some Android Auto music apps enforce a list of allowed client apps, preventing this app from launching them or browsing their libraries. However, once they are running, they can be controlled. For example, these popular music apps can not be launched, they must be started manually:
    • Amazon Music
    • Audible
    • Bandcamp
    • CloudPlayer
    • Deezer
    • doubleTwist
    • Google Play Music
    • iHeartAuto
    • TuneIn Radio
    • Pandora
    • Scribd
    • Smart Audiobook Player
    • YouTube Music

Requirements

To communicate to the car, this project relies on the proxy connection that is created by the main MyBMW or Mini app on the phone. Additionally, the legacy Connected and the Connected Classic apps have been tested as compatible for this purpose, but the Connected app is more resilient against Android's memory management.

Developer Guide

To build your own test versions of the app and help contribute to the project, follow this Developer Guide to set up your development environment.

Privacy

This project contains no advertising or user tracking, and is developed entirely for fun and to enhance the usefulness of the BMW/Mini infotainment system.

The app uses the Internet Permission to make a TCP connection to the car, which is reachable through a localhost socket on the main Connected app. Additionally, some cover art and incoming picture notifications (such as from Hangouts) may be fetched from Internet URLs. No other Internet access is required for the app's functionality.

As part of its functionality, the app collects and remembers some information locally on your phone:

  • The Car Information tab remembers information about the most-recently-connected car, including model and available features, specific driving statistics and vehicle settings, and the last location reported by the car while it was connected
  • The music functionality remembers a list of installed music apps and what features they provide
  • The Spotify integration retains a token to access your Spotify Library

The app also needs some other information to function, but does not remember any of it and will never share it:

  • To establish the car connection, the app identifies which BMW and Mini apps are installed
  • The app watches Bluetooth connections to identify when the car is connected
  • Any installed Voice Assistants are discovered to be added to the car
  • Apps which act as AAIdrive Addons are discovered and presented in the Addons tab
  • If enabled, notifications are read from the phone while connected to the car and then forgotten when disconnected
  • If compiled into the app and then enabled, the custom map feature will use the car or phone's location while displaying the map

The analytics-disabled version will never share any information, and no analytics code is compiled into this version of the app. The analytics-enabled version provides an option to share some anonymized information to Sentry to assist with debugging and development: If enabled, besides any rare and unfortunate crashes, the app will share any installed music apps and the capabilities each app provides, as well as the model and capabilities of any connected car for usage statistics and feature prioritization.

Each release provides both an analytics-enabled and analytics-disabled option.

Example Analytics Data

Music App

{
  "appId": "github.daneren2005.dsub",
  "appName": "DSub",
  "controllable": "false",
  "connectable": "true",
  "browseable": "true",
  "searchable": "false",
  "playsearchable": "false"
}

Car Connection

{
  "a4axl": "true",
  "alignment_right": "true",
  "hmi_display_height": "480",
  "hmi_display_width": "1280",
  "hmi_role": "HU",
  "hmi_type": "MINI ID5",
  "hmi_version": "EntryEvo_ID5_1903_Release ID5_1903-490-1837K Build 47 - Rev:203015 2018-11-14 08:39:42",
  "inbox": "true",
  "map": "true",
  "navi": "true",
  "pia": "true",
  "speech2text": "true",
  "speedlock": "true",
  "touch_command": "false",
  "tts": "true",
  "vehicle_country": "US",
  "vehicle_productiondate": "03.00",
  "vehicle_type": "F56",
  "voice": "false"
}

aaidrive's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aaidrive's Issues

Refactor car event tests to be cleaner

Instead of manually triggering events with manual parameters, it would be handy to have some sort of interaction controller to conveniently describe user interactions during the tests.

Needs more translations

With Android and BMW being more popular in Europe than in America, this app could benefit from more strings translations. Anyone who wants to contribute should feel free to send a PR!

Another option is to use the CrowdIn service to quickly edit an existing translation. It also has screenshots to show the strings within the UI. If you want to create a new translation this way, just send me a message to add it to the interface for editing.

Implement navigation rerouting

If the current location has deviated too far away from an existing navigation trace, a new navigation trace should be calculated.

Notifications not working Android 10

Tried the app on 2018 X3 with OnePlus 6t running Android 10. Notifications not working. Able to see list of apps but upon entering app controls do not work. Specifically tried with Amazon Music - music plays but unable to skip forward/back and no artwork.

Implement notification input

Some notifications have a Reply action with a text input. This input could use a short list of common replies (some notifications even provide a list of replies), and a more functional text input screen with word autocomplete (perhaps using slashes and dashes to quickly select words) could be implemented.

YouTube doesn't update metadata when a new song is played

The PlaybackView does not show new information when Youtube starts playing a new song. It's probably because the PlaybackView relies on a changed MediaID as a trigger to update, and Youtube (not being meant to be browsed) does not provide a real MediaID.
Actually no, it does check the entirety previously-shown MusicMetadata against the current one, but something still needs to be looked into

Investigate pipelining RPC calls

Connecting over Bluetooth adds a noticeable latency penalty. However, most of the RPC calls, such as initial setProperty and any setData calls, do not examine a return value, and should be safe to pipeline instead of synchronously running one at a time.
I can't just convert these calls to @Oneway, because the RPC server still sends a response, and the proxy object would send an Unexpected Symbol response to the car when it receives the _result_setData responses.
However, it should be possible to rearrange the Etch library's MailboxManager to send a whole bunch of calls at once and then wait for all of them to return at the end, ignoring any return values and raising (one of?) any exceptions.

[Future req] BMWApps data supply, eg. Weather

HI,

could you try to add connectivity between bmw apps (e.g. weather) and online services. I heard that somebody from Russia developed such an app that restore online connectivity through Bluetooth teetering in case your subscription is over or broken telematic box (sim card problems).
Many apps are still able to get content from ConnectedDrive servers without problems, except the subscriptions (e.g. RTTI) and auth messages.

Other approach could be for example to get weather use different weather source and just give proper xml file to head unit to present it in many places and give alerts (e.g. storm, heave rain, snow, ice on road).

Notification states get stacked up

Clicking into a notification, then clicking a Clear or other action to go back instead of pressing the back button will leave the view state on the stack. After going through a few notifications like this, the stack gets quite large, and the back button requires many presses to get back to the main menu.
Perhaps there's a way to triggerHmiState instead of using hmiAction that would avoid building up the state stack.

Notification actions should have icons

Some notification actions have icons, and it would be nice to show those icons in the car.
Right now it's impossible, because the toolbarView uses imageId instead of RaImage, but a list component can be used instead. It would be great if a layout could be found that has a text box at the top and a list at the bottom. I see several views with some labels followed by some lists, I wonder how to make the list have word-wrap as part of #7.

Can't get Connected Classic app to open

Both connected classic apps stay stuck on loading forever. I know this isn't a problem with your app but wanted to know if you had any tips to get it working

Galaxy s10 Android 10

Implement MapBox mapping and navigation

MapBox should be implemented as an alternative map implementation alongside Google Maps. MapBox does not include a restriction on showing their maps in a car and has a very generous free tier, which means it can be enabled by default in the public builds. It also provides a modern 3D navigation view, unlike Google which disallows replicating the Google Navigation experience.

No visible app

Hi, I have a bmw f45 from 2017.
I have a problem, when I start the app on the vehicle I only see the Youtube app and nothing else (spotify or anything).
Why??
Sorry for my bad english.

Soundcloud sometimes pops up a Notification

When Soundcloud switches music, it seems to post a clearable notification of the paused song before replacing it with a not-clearable notification of the next playing song. Perhaps this can be masked with a buffer period somehow?

GMap sreen issues

Hi,

I compiled the app with the GMaps in it and found some problems:

  1. On the main screen Gmap preview and options to search a place or stop navi (see photo)
  2. After you put search info where is no additional info on the place (e.g. city, address if the is more then one result and is hard to choose correct)
  3. Navigation doesn't work, after you find place and click on it navi doesn't start.
  4. Map doesn't fit the place on the screen (see photos). (Look at the side panel on/off). Changing option from the app to show on the full screen
  5. Refresh rate is 1FPS or less. Disappointment :( Also very often maps just hangs and don't refresh the image for couple of seconds.

Link with the photos: https://photos.app.goo.gl/NTFDtozeVWNhiBD39

Figure out the car's resource caching

There's an rhmi_checkResource system, which seems to allow the car to persist the RHMI resources across connections, which may help speed up reconnections by not needing to upload the resources every time. It looks like it just takes a 16-byte hash and a length, which should be easy to figure out.
It will probably be very helpful for Bluetooth connections too.

Connection issiue

Hi,

I have on two different cars (F48 2017, G30 2018) the same problem that the app lose connection after 10-15sec after the focus.
Normally the bmw connected app is up and running, AndroidAutoIdrive connects and is connected for some time, but then app loose focus (after 30-60s of not touching the app/screen) disconnects and can not connect again.

Implement workaround for Android Oreo implicit broadcast listeners

Android Oreo disables the use of registering a BroadcastReceiver through the AndroidManifest.xml, which makes it harder to listen to the Connected app's car connection announcement. Due to this, the app is currently compiled against Nougat, which prevents it from being published on the Play Store.

Yanosik in the app

Hi,

could you check if it's possible to include popular in Europe app called Yanosik, especially reports about accidents, police patrols, traffic jams, etc.

Multimedia screen shows duplicate items NBT ID4 2015 BMW i3

On a 2015 BMW i3 with NBT iDrive 4, the Multimedia screen shows a bunch of apps, one of which is "Audioplayer". If I choose e.g. CloudPlayer, it doesn't do anything (the car shows a loading animation and nothing happens). But if I go to Audioplayer, the apps list in there works beautifully. This may be unfixable due to the iDrive protocol, so maybe this is a doc/FAQ clarification ticket :).

Switch to lower-quality Map frames during animations

When the Map is animating (zooming or panning) the JPG quality for the frames should be drastically lowered to send faster updates during the animations.
Perhaps the full quality ones should be tuned, perhaps, maybe it is sending unnecessarily high quality maps currently?

Implement PLAY_FROM_SEARCH support

Some apps, such as Youtube, support PlaybackState.ACTION_PLAY_FROM_SEARCH. Perhaps it would be possible to enable the Search input to play the first search result through this, even if the app doesn't implement the MediaBrowser.search api.

Choose which music apps to display

Is there a way to disable some of the apps from showing in the car? For example, I have Plex installed, but do not need it showing up in iDrive.

Implement a better UI for Buffering songs

For example, when DSub is downloading a song, the timer keeps ticking up even though the song hasn't begun playing yet.
Perhaps the time should be --:-- and the progress bar should jump from 25% to 50% to 75% and so on?

Additional info in Debud mode

Hi,
I'd be good to have in Debug version of the app information as follow somewhere at the bottom of the main screen (just above the "connected to car":

  • build version with date and time
  • car screen resolution capabilities["hmi.display-width"] x capabilities["hmi.display-hight"]
  • ability to turn or always on local log file (in device) to get all debug information on the fly -with click shortcut to open it
  • if gmap enabled FPS rate (if possible)

CarProber doesn't figure out InstanceID

When CarProber establishes the connection, because the CAR_ACCESSORY_ATTACHED intent was missed, it uses a hardcoded Instance ID. This is fine, except that the audio context requires that this Instance ID be correct
Can some way be figured out to figure out the correct Instance ID?
CarProber may be important for Android Oreo support, so having full support would be great.

Is there a way to simulate a connected car when developing?

Great work on this project, I've been toying with the idea of something like this but looks like you beet me to the punch!

Would love to contribute but one issue I'm running into is that my car is parked in an underground garage (no cell reception and quite far from my desktop computer) which means running/debugging the app is going to be a bit of a pain, is there any sort of simulator/test harness to I could use to test some of my code? I see you have a branch started for voice input which is something I'd be interested in helping out with.

Android Studio build issue

After loading the project in a fresh install of Android Studio (Windows), I get the following error:

3:06 PM Unsupported Modules Detected: Compilation is not supported for following modules: app, IDriveConnectKitAndroid, AndroidAutoIdrive, IDriveConnectKit. Unfortunately you can't have non-Gradle Java modules and Android-Gradle modules in one project.

Is there any additional setup that needs to be done in order to build?

Take over control of existing music playback

When starting up the app and connecting to the car, it would be cool if any currently-playing music app is used as the initial app instead of showing the AppSwitcher view.
Even if there isn't any currently-playing music, it could use the existence of a (paused) single MediaStyle Notification as a signal to resume playback of that app.

Improve the phone UI about unavailable apps

The list of music apps shown in the phone display does not clearly show that some apps are not available for control, which causes disappointment when users see a list of music apps on the phone and an empty list in the car.
The unavailable apps should be crossed out or greyed out in some way.
Additionally, perhaps it could provide a recommendation to install some working apps.

This would prevent future instances of #37

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.