Android O: Autofill API about mobile

[kspearrin]

Got around the blocker and started working on this in the autofill branch. https://github.com/bitwarden/mobile/tree/autofill

Check out this quick demo :)

[kspearrin]

@JaceHensley We do use the APIs provided by Google. Just the C# version of them.

15.4 is now available for Xamarin, so we'll start looking at this again hopefully sometime soon.

[kspearrin]

Yes, we are beginning to work on this again now.

[kspearrin]

Looks like there is some support being added to Xamarin.Android recently. I'll start looking into it more now but don't expect something to be available as soon as O lands.

[geransmith]

@kspearein I'd love to help test on Android. You should setup a beta channel if you haven't already.

[kspearrin]

This is now live through our beta channel on the play store. Please post any feedback or problems in this issue. Blog post: https://blog.bitwarden.com/bitwarden-the-oreo-autofill-framework-2a8b2e04f29e

[kspearrin]

Xamarin will have official Android 8.0 support in v15.4, so we're waiting on that before we start working on this.

[geransmith]

@Moxville I feel like that is a moot point. It assumes a malicious app on the phone. If you have a malicious app, you can pretty easily get someone to put info in there.

[kspearrin]

I've made significant progress on this task over the past 2 days. Most of the work is now done. Expect a beta test next week.

[hrach]

Great work! Is there any possibility to save the matching mobile app to the entry? Not to search correct entry everytime. LastPass was doing something like this... :-)

[kspearrin]

This is now rolling out to production.

[kspearrin]

Yes, we plan to add support for this as soon as the API is available in Xamarin.Android.

[Moxville]

Password Managers using Android Oreo’s Autofill API are Potentially Vulnerable to Data Leakage
https://www.xda-developers.com/password-manager-autofill-api-data-leak/
https://github.com/commonsguy/AutofillFollies/blob/master/WHITE_PAPER.md
Just sharing the links.

[nicosemp]

@tehspaceg That's true, but it's still good to take some precautions where possible. Like partitioning data, and checking that the app that's being filled is actually the one associated with the entry, as the article suggests.

EDIT: Also it might be wise to wait for "best practices" from Google.

[NanoSector]

Any updates on this? It seems Xamarin has released sample code how to handle this: https://developer.xamarin.com/samples/monodroid/android-o/AutofillFramework/

[geransmith]

I think he means if you saved a site in the web browser, it uses url. The search appears to do pattern matching. Maybe when an item in the mobile app is identified for autofill, add a field automatically (or prompt to do so) that contains the package name.

On Wed, Nov 22, 2017, 8:14 AM Kyle Spearrin ***@***.***> wrote: @hrach <https://github.com/hrach> Not sure what you mean. When you save a new site it should use the mobile app's package name. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#57 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHqNg8qGsKDEKM4lmX8tYJyLsO5omeNNks5s5B4wgaJpZM4M6u-a> .

--

…

-Geran

[kspearrin]

I see, yes, there is room for improvement there which has been the case for autofilling on android for some time now.

[carlylemiii]

Android O is likely to drop within a week or two. Any updated plans on this?

[wjbeckett]

Looking forward to this!

[nicosemp]

@kspearrin This article was recently posted on developer.xamarin.com.
Can't wait for this to happen!

[kspearrin]

@nicosemp Yep, I've been trying to get 15.4 preview 2 working but am having a hard time with it...

[kspearrin]

Blocked by this issue: https://bugzilla.xamarin.com/show_bug.cgi?id=56740

Looks like I'll have to wait for preview 3

[Zer0t3ch]

I'm running O on my Nexus 6P, as well. Would be happy to test this if/when it ends up in a beta branch I can install without having to compile myself.

[anortiz08]

Running Android O on OnePlus 3 OxygenOS, more than willing to test!

[JaceHensley]

Wouldn't best practices be to use the APIs provided by Google?

[vinimk]

can I help with this? is there a specific branch where things are being done? I saw a branch but it got the last commit 2 months ago so i'm not sure

[kspearrin]

I merged that branch into master and starting working on it more.

See https://github.com/bitwarden/mobile/tree/master/src/Android/Autofill

Feel free to stop by our Gitter channel if you want to discuss specifics of how you can contribute to this feature.

[wjbeckett]

With the latest update, When BitWarden is trying to fill a form, my default notification sound is firing constantly.

I can reproduce it every single time. Regardless of whether BitWarden is my autofill or not.

[kspearrin]

@wjbeckett Are you also using the autofill accessibility service? Sounds odd since our autofill framework implementation does nothing with notifications.

[wjbeckett]

@kspearrin ah. yes that's what is doing it.
Disable the accessibility service, and it stops.

I suppose I should log a bug for this then? Happening in all apps and Chrome.

[kspearrin]

@wjbeckett I just reproduced it on my end here as well. I'll look into a fix. No need for a new issue.

[wjbeckett]

@kspearrin Perfect! Thanks mate.

[wjbeckett]

@kspearrin Also seeing that when trying to Autofill in the PayPal app, the BitWarden autofill form appears, I tap it, unlock my vault, select the entry I want to autofill with, and then nothing happens. It doesn't fill in the username/password fields.

[kspearrin]

@wjbeckett I see. Not sure why this is only happening with PayPal app. Will have to investigate more. Unlocked vault can still fill it correctly.

[kspearrin]

@wjbeckett I just tested it and it looks like everything is being done correctly to perform the autofill, but it just doesn't work with that app. Additionally, I even tried 1Password and LastPass apps and they do not autofill with PayPal correctly either. 🤷‍♂️

[ragingsheep]

Do you guys know how 1Password is doing auto-fill within Chrome?

[kspearrin]

I’m using 1password on my Nexus 5X with android 8.0 and the autofill service doesn’t work in chrome at all...??

[ragingsheep]

I'm on Pixel 2 XL with 8.0 with Chrome 62.0.3202.84 and 1Password 6.7.BETA-3.

Only noticed it appearing around a week ago.

[kspearrin]

@ragingsheep

I am running the exact same versions and do not see it working on a few websites that I have tried. Can you give me an example website that it works on?

[ragingsheep]

Actually, I think they might be "faking it", do you have Accessibility turned on for 1Password? It doesn't "autofill" in Chrome if I turn that off but it still autofills in apps.

[kspearrin]

@ragingsheep Yes, that is their accessibility service doing it in Chrome. Their UX is just the same on both methods.

[kspearrin]

@hrach Not sure what you mean. When you save a new login it should use the mobile app's package name.

[IOI-655321]

Implementation works well in most cases a few issues I've found not sure if these are specific to Bitwarden or not:

Google Find Devices (Device manager) doesn't prompt autofill
Amazon Shopping app log in doesn't prompt auto fill

Twitter and some other apps working fine so great job getting this out. Way better than lastpass buggy separate app version i tested a while back.

Samsung Galaxy S8 Oreo beta 3

[geransmith]

I suspect the main limitation is in those apps. I'm fairly certain bitwarden is just calling the API, if the target app doesn't work with the API, it won't work.

On Fri, Nov 24, 2017, 10:19 AM BigNickBurgess ***@***.***> wrote: Implementation works well in most cases a few issues I've found not sure if these are specific to Bitwarden or not: Google Find Devices (Device manager) doesn't prompt autofill Amazon Shopping app log in doesn't prompt auto fill Twitter and some other apps working fine so great job getting this out. Way better than lastpass buggy separate app version i tested a while back. Samsung Galaxy S8 Oreo beta 3 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#57 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHqNgzuMWVVDE9mgoq5QV-ZEQPm_O5ssks5s5t5cgaJpZM4M6u-a> .

--

…

-Geran

[kspearrin]

Some apps use web views for their login forms. I don't think these work with the Autofill Framework yet. I know Amazon is one I tested.

[hrach]

@kspearrin I mean a situation when the app package name doesn't match the domain, or there is some SSO which I'd like to attach to the app.

Also, I've encountered a bug, when I click an input, it prints the vault is locked. When unlocked, I've returned to the app, but the input shows still the same message that vault is locked.

Third, It also suggest something in my (Nova) launcher search field - is there any way how not to show it here?

[kspearrin]

@hrach

1. You can correct those with https://blog.bitwarden.com/new-feature-equivalent-domains-dd29aa462bb7
2. Can you please let me know what app this is happening in? We saw the same thing in the PayPal app and there wasn't anything we could do there.
3. I can add that app to the exclusion list.

[hrach]

1. Thanks :) Didn't know.
2. Sygic Travel, but I think know the pattern, after unlocking and pressing the back take me back to the app and then it show it's still locked.

[kspearrin]

@hrach If you are unlocking you must select the item to fill from the app UI. If you just press back you will end up with nothing if the app immediately locks back again. That is expected.

[hrach]

If I open the Bitwarden app (after pressing back, leaving my app and launching Bitwarden), it isn't locked. That's the reason why I was confused.

[kspearrin]

Ok, what is your lock option set at?

[hrach]

These two:

• lock after 15 minutes
• unlock after fingerprint

[IOI-655321]

If you go to 'features' you still have to have always scan/scan when password field focused/persist notification selected. Does this affect battery or anything when Oreo autofill is enabled? Can these be disabled when Oreo autofill active or can the accessibility and oreo autofill api be run simultaneously?

Similarly the Tools panel for the accessibility auto fill service should be disabled and maybe a new one added to take you to the Oreo autofill panel.

[kspearrin]

@BigNickBurgess Yes, we still have some work to do on the settings pages. All settings only apply to the accessibility service today. They can be run at the same time if you like.

[anortiz08]

@kspearrin For #3 above, anyway the exclusion list can be managed by the user. Maybe when the auto fill overlay is presented, an option to add to the exclusion list?

[kspearrin]

The overlay UI is not really configurable in that way. If the exclusion list is growing large then it sounds like I have some flaws in my field detection logic that needs to be revisited. Do you have other app examples that are presenting the overlay that should not be?

[pdf]

@kspearrin I think @anortiz08 is trying to solve a problem that would be better served by logic whereby if autofill is triggered, the accessibility overlay should not also pop, when they're both enabled. I suspect this is non-trivial though.

[kspearrin]

@pdf Yea, I am not sure how we could support that.

[pdf]

Off the top of my head, only way I can think would be to add a small delay to the accessibility pop when they're both enabled, set a var against the app id if the autofill is triggered, check that at timeout on the accessibility pop to determine whether to actually pop the accessibility overlay. This would be racey, but probably works most of the time.

Thinking about it further, using the exclusion list (or a separate internal list) to track apps when autofill is successfully triggered would allow preventing future accessibility pops for that app. This could be combined with the above strategy, or could be used to cause the first accessibility overlay to close, though the latter would be a little janky, it would only happen once per app.

[anortiz08]

@kspearrin Thus far I have noticed the auto fill overlay appearing in YouTube search and Nine email client when entering a pin.

[Moxville]

Getting your Android app ready for Autofill
https://android-developers.googleblog.com/2017/11/getting-your-android-app-ready-for.html

[kspearrin]

@Moxville Yes, we already contacted Google through that form... and no reply :-/

[kspearrin]

@anortiz08 For some reason the Youtube Search input field has a "input type" of

Android.Text.InputTypes.ClassText | Android.Text.InputTypes.DatetimeVariationDate | Android.Text.InputTypes.DatetimeVariationTime | Android.Text.InputTypes.TextVariationPassword

TextVariationPassword tells me that it is a password field. Not sure why it is marked this way...

[kspearrin]

@hrach I installed Nova launcher and the Google search field doesn't seem to suggest an autofill it in my tests?

[hrach]

@kspearrin weird. It autosuggests also in search field of Google's contacts app. I have One Plus, so the Android is not "clean"… can I help you somehow?

[kspearrin]

Yes, all of these fields are marked with input type TextVariationPassword for some reason. I have added a check in the next version to filter out any fields with "Search" in them.

[kspearrin]

I just published build 1106 to the beta channel with more fixes and improvements.

[edvorg]

Could you please update here when the app with this feature is in play store?

[IOI-655321]

It was released to the stable branch when he posted yesterday I believe.

If you view the 'autofill' screen in tools the links now go to the autofill API screen as well as the Accessibility version.