Unable to run tests for the project about minkzombiedriver

[aik099]

Can you please:

1. upgrade your node/npm
2. show version of Zombie being used
   ?

[stof]

you need to run the fixtures server:

• vendor/bin/mink-test-server if you want to block a terminal with it (allowing to stop it with Ctrl+C
• vendor/bin/mink-test-server > /dev/null 2>&1 & to run it silently in the background (you then have to find the process id and kill it to stop it)

[berliner]

$ git clone https://github.com/minkphp/MinkZombieDriver.git
$ cd MinkZombieDriver/
$ npm install zombie
npm WARN saveError ENOENT: no such file or directory, open '/private/tmp/mink/MinkZombieDriver/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/private/tmp/mink/MinkZombieDriver/package.json'
npm WARN MinkZombieDriver No description
npm WARN MinkZombieDriver No repository field.
npm WARN MinkZombieDriver No README data
npm WARN MinkZombieDriver No license field.

+ [email protected]
added 98 packages in 9.699s

$ composer install
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 5 installs, 0 updates, 0 removals
  - Installing symfony/phpunit-bridge (v3.3.2): Loading from cache
  - Installing symfony/css-selector (v3.3.2): Loading from cache
  - Installing behat/mink (dev-master 9ea1ceb): Cloning 9ea1cebe3d from cache
  - Installing mink/driver-testsuite (dev-master 1d2e9fc): Cloning 1d2e9fcef1 from cache
  - Installing symfony/process (v3.3.2): Loading from cache
symfony/phpunit-bridge suggests installing symfony/debug (For tracking deprecated interfaces usages at runtime with DebugClassLoader)
symfony/phpunit-bridge suggests installing ext-zip (Zip support is required when using bin/simple-phpunit)
behat/mink suggests installing behat/mink-browserkit-driver (extremely fast headless driver for Symfony\Kernel-based apps (Sf2, Silex))
behat/mink suggests installing behat/mink-goutte-driver (fast headless driver for any app without JS emulation)
behat/mink suggests installing behat/mink-selenium2-driver (slow, but JS-enabled driver for any app (requires Selenium2))
Writing lock file
Generating autoload files

$ node -v
v8.1.0

$ npm -v
5.0.3

(In separate terminal window, to make the server work on my sytem, I had to replace readlink with greadlink)
$ vendor/bin/mink-test-server
PHP 5.6.27 Development Server started at Tue Jun 13 15:22:17 2017
Listening on http://localhost:8002
Document root is /private/tmp/mink/MinkZombieDriver/vendor/mink/driver-testsuite/web-fixtures
Press Ctrl-C to quit.

(In the original terminal window)
$ phpunit --stop-on-failure
PHPUnit 4.5.0 by Sebastian Bergmann and contributors.

Configuration read from /private/tmp/mink/MinkZombieDriver/phpunit.xml.dist

........................E

Time: 152 ms, Memory: 9.25Mb

There was 1 error:

1) Behat\Mink\Tests\Driver\Basic\BasicAuthTest::testSetBasicAuth with data set #0 ('mink-user', 'mink-password', 'is authenticated')
RuntimeException: No connection available. Did you start the server?

/private/tmp/mink/MinkZombieDriver/src/NodeJS/Server.php:441
/private/tmp/mink/MinkZombieDriver/src/NodeJS/Server.php:411
/private/tmp/mink/MinkZombieDriver/src/ZombieDriver.php:197
/private/tmp/mink/MinkZombieDriver/vendor/behat/mink/src/Session.php:159
/private/tmp/mink/MinkZombieDriver/vendor/mink/driver-testsuite/tests/Basic/BasicAuthTest.php:16

FAILURES!
Tests: 25, Assertions: 62, Errors: 1.

Legacy deprecation notices (2)

[stof]

hmm, the driver is not tested with Zombie 5 currently, so it may be broken (I was not aware that they released a version 5). Try with version 4 (use npm install zombie@^4 for the install command).

regarding readlink vs greadlink, this is possible indeed. I actually never tried the server script on Mac OS X (as I don't have such environment). The script was used on Linux for now (I am on Linux, and our CI is as well)

[berliner]

I just tried with zombiejs 4.3.0 with the same result.

I have also tried to follow the steps above on a pretty clean apricity linux, using the same node, npm and phpunit versions, the result is the same.

[aik099]

Actually I've tried to run it myself now (Zombie 4.2.1, Node 5.4.1, Npm 3.3.12) and it failed with same errors as you've seen. :(

[berliner]

@aik099 That's good news to me, so it's not only me ;)

[aik099]

Apparently Zombie did change something again to break everything for us.

[berliner]

Do the test on your Travis CI still work?

[aik099]

I've restarted a couple of jobs from the latest build (1st and last): https://www.travis-ci.org/minkphp/MinkZombieDriver/builds/175044232 . Let's see if they will pass.

[aik099]

It did fail in some places, but not that massively as it did when we've executed it locally:

• Zombie 4.3.0: https://www.travis-ci.org/minkphp/MinkZombieDriver/jobs/175044245
• Zombie 3.1.1: https://www.travis-ci.org/minkphp/MinkZombieDriver/jobs/175044233

[berliner]

@aik099 Will you be working on this? I fear that my knowledge of this project is not sufficient to effectively address these problems.

[aik099]

Unfortunately I'm occupied with other things at the moment.

Here is approach I've used to solve Zombie issues:

1. determine Zombie version for which all tests passed (you can get it from last passed build on Travis CI)
2. try increasing used Zombie version until problem starts to appear
3. see what changes were made to Zombie that might have caused issues with tests
4. if it isn't Zombie bug try adopting driver code to work with older and newer version of Zombie
5. repeat process until all tests pass for latest Zombie 5 version

[berliner]

Ok, after further digging I haven't found a problem with zombie itself, but instead I have found these 2 problems in MinkZombieDriver and Mink:

1. Session start: minkphp/Mink@9ea1ceb introduced lazy starting of sessions when calling the visit method. This makes all the tests in Behat\Mink\Tests\Driver\Basic\BasicAuthTest create errors. If I add the following code to the top of Behat\Mink\Session::setBasicAuth the the tests pass:

        // start session if needed
        if (!$this->isStarted()) {
            $this->start();
        }

2. HTML5 tests: minkphp/driver-testsuite@55ea007 added 2 new tests for HTML5 form elements. These are not yet fully supported by zombiejs and should therefor be added to the exclude list in Behat\Mink\Tests\Driver\ZombieConfig::skipMessage

[aik099]

1 - that is unexpected considering that according to documentation (see https://github.com/minkphp/Mink/blob/master/src/Session.php#L57-L67) the setBasicAuth method should work prior to session being started

2 - sure, please send a PR fixing that

FYI: The support for setting authentication headers is currently limited to these drivers:

• ZombieDriver basicAuth: https://github.com/minkphp/MinkZombieDriver/blob/master/src/ZombieDriver.php#L168
• BrowserKit basicAuth: https://github.com/minkphp/MinkBrowserKitDriver/blob/master/src/BrowserKitDriver.php#L192

The BrowserKit is setting headers, that would be picked up upon session start, but the Zombie apparently wants to start session first and then set authentication to be picked up by next visit call.

[aik099]

1 - since apparently Zombie expects session to be started prior to setting auth info we need to do this on driver level instead of session level.

@stof , thoughts?

[berliner]

@aik099 For 2. Sure, I created #180

[stof]

@aik099 yeah, it would make sense to start the session at the driver level in this case (the starting actually happens at the driver anyway. Starting a session just delegates starting to the driver)

[aik099]

@berliner , does it work for you now, when PR was merged?

[berliner]

@aik099 Yes, the test are passing now:

$ phpunit
PHPUnit 4.5.0 by Sebastian Bergmann and contributors.

Configuration read from /private/tmp/mink/MinkZombieDriver/phpunit.xml.dist

...............................................................  63 / 190 ( 33%)
..........................S...S......SSSSS...............SSSS.S 126 / 190 ( 66%)
S.............S..........................S...............S..SSS 189 / 190 ( 99%)


Time: 15.9 seconds, Memory: 11.75Mb

OK, but incomplete, skipped, or risky tests!
Tests: 187, Assertions: 438, Skipped: 19.

Legacy deprecation notices (2)

[aik099]

Then you can safely close the issue I guess.

[berliner]

Ah, right. Thanks for your support.