Not able to make tests work with AndroidJunitRunner about screenshot-tests-for-android

[tdrhq]

This is correct, we didn't want to implicitly force a dependency on testing-support-lib so we don't provide a runner based off of AndroidJUnitRunner. You can however extend one yourself and call into https://github.com/facebook/screenshot-tests-for-android/blob/master/core/src/main/java/com/facebook/testing/screenshot/ScreenshotRunner.java

(call onCreate() from your custom Runners onCreate, and onDestroy() from finish()).

If you send me a pull request with a simple example of a custom runner using AndroidJUnitRunner I'll be happy to accept! Let me know if this solution works in the meantime.

[naderz]

Thanks for you answer :)

I did that already :

class SSRunner extends AndroidJUnitRunner {

    @Override
    public void onCreate(Bundle arguments) {
        MultiDex.install(getTargetContext());
        ScreenshotRunner.onCreate(this, arguments);
        super.onCreate(arguments);
    }

    @Override
    public void finish(int resultCode, Bundle results) {
        ScreenshotRunner.onDestroy();
        super.finish(resultCode, results);
    }
}

And gradle looks like this :

 apply plugin: 'com.android.application'
 
 android {
     compileSdkVersion 25
     buildToolsVersion "25.0.0"
     defaultConfig {
         multiDexEnabled true
         applicationId "com.example.nadera.myapplication"
         minSdkVersion 15
         targetSdkVersion 25
         versionCode 1
         versionName "1.0"
         testInstrumentationRunner "com.example.nadera.myapplication.SSRunner"
     }
     buildTypes {
         release {
             minifyEnabled false
             proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
         }
     }
 }
 
 dependencies {
     compile fileTree(dir: 'libs', include: ['*.jar'])
     androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
         exclude group: 'com.android.support', module: 'support-annotations'
     })
     compile 'com.android.support:appcompat-v7:25.1.0'
     compile 'com.android.support:design:25.1.0'
     compile 'com.android.support:multidex:1.0.1'
     testCompile 'junit:junit:4.12'
 }
 
 apply plugin: 'com.facebook.testing.screenshot'
 

I seriously cant figure out what is really going wrong

Error im getting is :

$ adb shell pm install -r "/data/local/tmp/com.example.nadera.myapplication.test"
pkg: /data/local/tmp/com.example.nadera.myapplication.test
Success

Running tests

$ adb shell am instrument -w -r -e debug false -e class com.example.nadera.myapplication.LoginTest com.example.nadera.myapplication.test/com.example.nadera.myapplication.SSRunner
Client not ready yet..
Started running tests
Test running failed: Unable to find instrumentation info for: ComponentInfo{com.example.nadera.myapplication.test/com.example.nadera.myapplication.SSRunner}
Empty test suite.

Not that if i remove the imports of your plugin everything runs perfectly with the SSRunner that i created.. If i re-add your plugin it starts failing. @tdrhq

[naderz]

Ill happily do a PR for this when i get it working. @tdrhq

[tdrhq]

You need to also have your custom instrumentation set up in your AndroidManifest

[naderz]

@tdrhq I tried that still not working can you give a sample of the instrumation tag that should be added to the manifest ?

[tdrhq]

<manifest>  <instrumentation
   android:name="com.example.nadera.myapplication.SSRunner"
   android:targetPackage="com.example.nadera.myapplication.test" />
</manifest>

[naderz]

@tdrhq added this to manifest and still not working ... targetPackage is red so its not being recognised by the IDE ...
I tried adding this tag to androidTest manifest and to the main manifest and still no luck.
Something interesting that i found is that in the build package (/Users/nadera/MyApplication/app/build/intermediates/manifest/androidTest/debug) i looked at the manifest being generated and saw this :

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.nadera.myapplication.test" >

    <uses-sdk
        android:minSdkVersion="15"
        android:targetSdkVersion="25" />

    <instrumentation
        android:name="com.facebook.testing.screenshot.ScreenshotTestRunner"
        android:functionalTest="false"
        android:handleProfiling="false"
        android:label="Tests for com.example.nadera.myapplication"
        android:targetPackage="com.example.nadera.myapplication" />

    <application>
        <uses-library android:name="android.test.runner" />
    </application>

</manifest>

Its like its totally ignoring the SSRunner

[tdrhq]

Oh I forgot that if you're using gradle then gradle modifies the AndroidManifest for you, so something like this: https://github.com/facebook/screenshot-tests-for-android/blob/master/core/build.gradle#L45

[naderz]

 apply plugin: 'com.android.application'
 
 android {
     compileSdkVersion 25
     buildToolsVersion "25.0.0"
     defaultConfig {
         multiDexEnabled true
         applicationId "com.example.nadera.myapplication"
         minSdkVersion 15
         targetSdkVersion 25
         versionCode 1
         versionName "1.0"
         testInstrumentationRunner "com.example.nadera.myapplication.SSRunner"
     }
     buildTypes {
         release {
             minifyEnabled false
             proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
         }
     }
 }
 
 dependencies {
     compile fileTree(dir: 'libs', include: ['*.jar'])
     androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
         exclude group: 'com.android.support', module: 'support-annotations'
     })
     compile 'com.android.support:appcompat-v7:25.1.0'
     compile 'com.android.support:design:25.1.0'
     compile 'com.android.support:multidex:1.0.1'
     testCompile 'junit:junit:4.12'
 }
 
 apply plugin: 'com.facebook.testing.screenshot'
 

Already have that there @tdrhq

[naderz]

I think your library is always overriding the instrumentation tag in the manifest.

[naderz]

who modifies customTestRunner = false ? seems its always false
https://github.com/facebook/screenshot-tests-for-android/blob/master/plugin/src/main/groovy/com/facebook/testing/screenshot/build/ScreenshotsPlugin.groovy

[tdrhq]

uggh, thank you for pointing out how ugly this API is.

It looks like https://github.com/facebook/screenshot-tests-for-android/blob/master/plugin/src/main/groovy/com/facebook/testing/screenshot/build/ScreenshotsPlugin.groovy#L104 is setting the test runner unless you set customTestRunner = True manually.

I think in the next release of this library I should make it use testing-support-lib by default and extend AndroidJUnitRunner which would solve a lot of this confusion though pulls in dependencies that people might not want. (Particularly junit)

[tdrhq]

(To be clear, that customTestRunner is a configurable option that you should set if you don't want to use the default runner)

[naderz]

apply plugin: 'com.android.application'
**apply plugin: 'com.facebook.testing.screenshot'

project.screenshots.customTestRunner = true** 

android {
    compileSdkVersion 25
    buildToolsVersion "25.0.0"
    defaultConfig {

        multiDexEnabled true
        applicationId "com.example.nadera.myapplication"
        minSdkVersion 15
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "com.example.nadera.myapplication.SSRunner"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }

}


dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile 'com.android.support:appcompat-v7:25.1.0'
    compile 'com.android.support:design:25.1.0'
    compile 'com.android.support:multidex:1.0.1'
    testCompile 'junit:junit:4.12'
}

This fixed the issue :) thats for your replies

[tdrhq]

So, let me know if you're sending over a pull request with an example :)

[naderz]

@tdrhq #52 :)