Coder Social home page Coder Social logo

robo-paracept's Introduction

robo-paracept

PHP Composer Latest Stable Version Total Downloads License

Robo tasks for Codeception tests parallel execution. Requires Robo Task Runner

Install via Composer

composer require codeception/robo-paracept --dev

Include into your RoboFile

<?php

require_once 'vendor/autoload.php';
require_once 'vendor/codeception/codeception/autoload.php';

class RoboFile extends \Robo\Tasks
{
    use Codeception\Task\Merger\ReportMerger;
    use Codeception\Task\Splitter\TestsSplitterTrait;
}

Idea

Parallel execution of Codeception tests can be implemented in different ways. Depending on a project the actual needs can be different. So we prepared a set of predefined Robo tasks that can be combined and reconfigured to fit your needs.

Tasks

SplitTestsByGroups

Load tests from a folder and distributes them between groups.

$result = $this->taskSplitTestsByGroups(5)
    ->testsFrom('tests/acceptance')
    ->projectRoot('.')
    ->groupsTo('tests/_data/group_')
    ->run();

// task returns a result which contains information about processed data:
// optionally check result data   
if ($result->wasSuccessful()) {
    $groups = $result['groups'];
    $tests = $result['tests'];
    $filenames = $result['files'];
}

This command loads Codeception into memory, loads and parses tests to organize them between group. If you want just split test file and not actual tests (and not load tests into memory) use taskSplitTestFilesByGroups:

SplitTestFilesByGroups

To split tests by suites (files) without loading them into memory use taskSplitTestFilesByGroups method:

$result = $this->taskSplitTestFilesByGroups(5)
   ->testsFrom('tests')
   ->groupsTo('tests/_data/paratest_')
   ->run();

// optionally check result data
if ($result->wasSuccessful()) {
    $filenames = $result['files'];
}   

SplitTestsByTime

Enable extension for collect execution time of you use taskSplitTestsByTime

extensions:
    enabled:
        - Codeception\Task\Extension\TimeReporter

Load tests from a folder and distributes them between groups by execution time.

$result = $this->taskSplitTestsByTime(5)
    ->testsFrom('tests/acceptance')
    ->projectRoot('.')
    ->groupsTo('tests/_data/group_')
    ->run();

// optionally check result data
if ($result->wasSuccessful()) {
    $filenames = $result['files'];
}

this command need run all tests with Codeception\Task\TimeReporter for collect execution time. If you want just split tests between group (and not execute its) you can use SplitTestsByGroups. Please be aware: This task will not consider any 'depends' annotation!

SplitFailedTests

Enable extension for collect failed tests if you use taskSplitFailedTests
The extension saves the report files into \Codeception\Configuration::outputDir()

extensions:
    enabled:
        - Codeception\Task\Extension\FailedTestsReporter

Merge the created report files from the FailedTestsReporter into single file

$this->taskMergeFailedTestsReports()
    ->fromPathWithPattern(\Codeception\Configuration::outputDir(), '/failedTests_\w+\.txt$/')
    ->into(\Codeception\Configuration::outputDir() . 'failedTests.txt') // absolute path with Filename
    ->run();

Load the failed Tests from a reportfile into the groups:

$result = $this
    ->taskSplitFailedTests(5)
    ->setReportPath(\Codeception\Configuration::outputDir() . 'failedTests.txt') // absoulute Path to Reportfile
    ->groupsTo(\Codeception\Configuration::outputDir() . 'group_')
    ->run();

// optionally check result data
if ($result->wasSuccessful()) {
    $filenames = $result['files'];
} 

MergeXmlReports

Mergex several XML reports:

$this->taskMergeXmlReports()
    ->from('tests/result/result1.xml')
    ->from('tests/result/result2.xml')
    ->into('tests/result/merged.xml')
    ->run();

MergeHtmlReports

Mergex several HTML reports:

$this->taskMergeHtmlReports()
    ->from('tests/result/result1.html')
    ->from('tests/result/result2.html')
    ->into('tests/result/merged.html')
    ->run();

Filters

You can use a custom filter to select the necessary tests.

Two filters already included: DefaultFilter, GroupFilter

  • DefaultFilter is enabled by default, takes all tests.
  • GroupFilter (Can only be used by taskSplitTestsByGroups), allows you to filter the loaded tests by the given groups. You have the possibility to declare groups which you want to include or exclude. If you declare foo and bar as included, only tests with this both group annotations will be matched. The same thing is happend when you add excluded groups. If you combine the included and excluded group the only tests which have exactly the correct group annotations for the included items and none of the excluded items.

You can add as many filters as you want. The FIFO (First In - First Out) principle applies. The next filter will only get the result of the filter before.

Usage

For example, you want all tests which have in the doc comment the groups 'foo' AND 'bar' but not 'baz' then you can do it like this:

$filter = new GroupFilter();
$filter
    ->groupIncluded('foo')
    ->groupIncluded('bar')
    ->groupExcluded('baz');

$this->taskSplitTestsByGroups(5)
   ->testsFrom('tests')
   ->groupsTo('tests/_data/paratest_')
   ->addFilter($filter)
   ->run();

Now create your own filter class:

<?php

declare(strict_types=1);

namespace ...;

use Codeception\Task\Filter\DefaultFilter;

class CustomFilter extends DefaultFilter {

}

The TestFileSplitterTask.php pushes an array of SplFileInfo Objects to the filter.
The TestsSplitterTask.php pushes an array of SelfDescribing Objects to the filter.

Configuration

Load Codeception config file to specify the path to Codeception before split* tasks:

\Codeception\Configuration::config('tests/codeception.yml');

Contributing

Thank you for contributing to codeception/robo-paracept!

  1. Fork this project
  2. install all deps
  3. create a branch from master
  4. make your changes
  5. extend or create tests for your changes
  6. run composer test (This will execute lint, codestyle and unit tests sequential)
  7. open a Merge Request

Coding Standard

Please note that this project follows the PSR-12 Coding Standard. You can check your coding style with:

composer codestyle

Unit Tests

All changes which you will done must pass the unit tests. If you change some logic or you add some new methods please be fair and write a test.

composer unit

License MIT

robo-paracept's People

Contributors

arhell avatar ccsuperstar avatar danijelk avatar davertmik avatar dependabot[bot] avatar dhiva avatar dzoeteman avatar exorus avatar frol-kr avatar grahamcampbell avatar grzegorzdrozd avatar hirowatari avatar ironsmile avatar ivan1986 avatar ivanzuev avatar jmrieger avatar maxgorovenko avatar pingers avatar reinholdfuereder avatar taufek avatar tavoniievez avatar vansari avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

robo-paracept's Issues

Is this package outdated?

Hi

Thanks a lot for this package and the work behind them. Is this package actually maintained or not?! There are many outdated dependencies and I would to ask to update the dependencies and the code base...

If you need any support please let us know...

It would be great if we / you can update this package for usage of php 8.0 and Robo 3.0

Greetings from Berlin

Merging reports with replacing tests results

I have report RA with failed test ST and report RB with same name test ST but with success result. If i merge report RA and RB i will get report RC with two results of test ST(one succeed and one failed). Is any suggestion for merge only last result of test ST?

Issue with composer dependencies on current master.

I see following issues when running composer install:

(...)
    Skipped installation of bin robo for package consolidation/robo: file not found in package
(...)
    Skipped installation of bin phpunit for package phpunit/phpunit: file not found in package
(...)
    Skipped installation of bin codecept for package codeception/base: file not found in package
(...)

 [RuntimeException]
  Could not scan for classes inside "C:\tmp\r\vendor/consolidation/robo/scripts/composer/ScriptHandler.php" which does not appear to be a file nor a folder

To fix that one can update dependencies in composer files to:

    "require-dev": {
        "codeception/base": "^2.5",
        "codeception/codeception": "^2.5"
    }

and adding

require_once 'vendor/codeception/codeception/autoload.php';

to bootstrap.php

Issue is that i have no idea if that breaks some older installations of this packages. I have no idea which version of codeception broke that and made it unusable.

SplitTestsByGroupsTasks doesn't split properly when using a dataprovider

SplitTestsByGroupsTasks task doesn't split tests properly if the tests are using a dataprovider. It uses the fullpath from /vendor/phpunit/phpunit/src/Framework/DataProviderTestSuite.php instead of the test that is using the dataprovider.

Expected:

Group_1:
tests/Acceptance/Desktop/TestWithDataProvider.php:DataProviderScenario1
tests/Acceptance/Desktop/TestWithDataProvider.php:DataProviderScenario2
tests/Acceptance/Desktop/TestWithDataProvider.php:DataProviderScenario3
tests/Acceptance/Desktop/TestWithDataProvider.php:DataProviderScenario4
...

Got:

/vendor/phpunit/phpunit/src/Framework/DataProviderTestSuite.php:
/vendor/phpunit/phpunit/src/Framework/DataProviderTestSuite.php:
/vendor/phpunit/phpunit/src/Framework/DataProviderTestSuite.php:
/vendor/phpunit/phpunit/src/Framework/DataProviderTestSuite.php:

Reference to missing class `\Codeception\PHPUnit\Init`

Overview

There's a reference to the \Codeception\PHPUnit\Init class here, but using the latest stable versions of both this package and codeception/codeception results in this class going missing.

The class in question comes from codeception/phpunit-wrapper, which isn't included with, and cannot be installed alongside, Codeception.

The lastest version of codeception/codeception 5.x prevents the installation of codeception/phpunit-wrapper by replaceing the dependency in its composer.json.

So currently it's impossible through latest stable channels to run this code.

Steps to reproduce

  1. Install codeception/codeception:^5.0 and codeception/robo-paracept:^2.0.
  2. Follow the steps (up to and including Setting Up Robo // Step 1: Split Tests) in the Codeception Parallel Execution documentation.
  3. Try to run php vendor/bin/robo parallel:split-tests.

Workarounds

I believe you can create your own version of the missing class and write an empty init() method, because the code in the original PHPUnit wrapper init() method isn't required anyway. As long as it can be autoloaded and run, it should bypass this issue.

Currently I've got the latest stable codeception/codeception, but capped robo-paracept to version 1.2.4, which is the last release without the missing class reference.

References

Previously mentioned in #112

Filter Tests by Groups

As I saw in the comments there was a Desire for improvement.
Expected behaviour:

Use taskSplitTestsByGroups and have the possibility to add an own Filter to filter out the Tests which are in an expected group or not.

I have opened a Pull Request with the following Feature:

The possibility to use your own Filter for taskSplitTestsByGroups or taskSplitTestFilesByGroups.
Filter will be handled like the FIFO principal. (First In First Out).

I have already added a GroupFilter and a DefaultFilter which will be loaded by Default.

Fatal undefined method task()

Tasks currently fatal if you don't allow release candidates in composer. The composer.json in this project says it requires at least robo 0.7, but it actually requires 1.0, which is not stable yet.

Pass an array to the testsFrom() method

In the previous version, we could pass an array of paths to the testsFrom() method
->testsFrom(['tests/bdd', 'tests/integration'])
This is not possible anymore, because testsFrom() only accepts a string, but it is useful when you want to run only some suites.

TaskIO used in tasks, while BaseTask already uses it

The TaskIO trait is already used in Robo's BaseTask, but both are used in the tasks. This creates the following error:

ERROR: Robo\Task\BaseTask and Robo\Common\TaskIO define the same property ($config) in the composition of Codeception\Task\TestsSplitter. This might be incompatible, to improve maintainability consider using accessor methods in traits instead. Class was composed

Query issue when merging HTML files

THe DOM Query seems to be wrong in the countSummary method of MergeHTMLReportsTask class.

Current Query
"//table//tr[contains(@class,'scenarioRow')]"

Fix
"//table/tr[contains(@class,'scenarioRow')]"

Tag 2.1.0 dependencies too strict

The newly released tag 2.1.0 for PHP 8.0, codeception 5 and Symfony components 6.0 is too strict. It requires version 6.0.0 and nothing above it, for example 6.0.11

Is there any reason for that?

Thank you

Merged report with multiple suites

When I run my test runs in parallel, the merged html report does not contain all the suite titles. I think this is because the distribution of tests is done in such a way that each /_data/parallel_* file does not always contain tests from each suite.

Also, the execution time in the final report is not the maximum execution time.

Before v1, I had made a modification, with a report template file which match with our need, but I'm looking forward to your ideas!

addFilter() method doesn't work with taskSplitTestFilesByGroups() method

The use of addFilter() with the split strategy taskSplitTestFilesByGroups() does not work while in the class TestFileSplitterTask, it is well given as an example

<?php
$this->taskSplitTestFilesByGroups(5)
   ->testsFrom('tests/unit/Acme')
   ->codeceptionRoot('projects/tested')
   ->groupsTo('tests/_log/paratest_')
   ->addFilter(new Filter1())
   ->addFilter(new Filter2())
    ->run();

I want to exclude groups (as the notName method has not been taken over I have no other solution) but it does not work

Thanks for your help

Codeception 5.04
robo-paracept 3.0.0

GroupFilter doesn't handle dataprovider tests correctly

(Just sounds similar to #40 "SplitTestsByGroupsTasks doesn't split properly when using a dataprovider" but is different)

/**
 * @group groupToBeFiltered
 */
class MyExampleCest {

  /**
   * @example [1, true]
   * @example [2, true]
   *
   * @param \Codeception\Example $example
   */
  public function testExample(\Codeception\Example $example) {
    Assert::assertTrue($example[1]);
  }

}
$groupFilter = new \Codeception\Task\Filter\GroupFilter();
$groupFilter->groupExcluded('groupToBeFiltered');

$this->taskSplitTestsByGroups(...)
  ->testsFrom(...)
  ->groupsTo(...)
  ->addFilter($groupFilter)
  ->run();

Will include the dataprovider test into the generated split group test files and so the parallel test sets are unequally distributed. (During actual execution these dataprovider tests will be skipped correctly).

The list of gathered/detected groups is wrong/empty.

(@dataProvider based tests have the same problem)

Groups are not same size

Hi,
I am using robo-paracept to run codeception tests in multiple threads (obviously). My problem is that I have about 200 tests which are splitted to 8 groups. For some reason some of the groups can be up to 30 tests long while some groups only have something like 5 or 6 tests. Why is this happening? Am I missing some configuration or is there a bug in the code?

Codeception tests ran with Robo Parallel has errors 0 and timeout 0.000000 in single test but in main raport all tests has errors

What are you trying to achieve?

I trying to run acceptance tests on TeamCity using Robo Parallel.

What do you get instead?

Every test has assertions="0" time="0.000000" in XML report, but in main report there is information:

<testsuite name="acceptance" tests="12" assertions="0" errors="12" failures="0" skipped="0" time="0.000000">

Provide console output if related. Use -vvv mode for more details.

  Starting: /usr/bin/php vendor/bin/robo -vvv parallel:run
07:00:34
  in directory: /opt/teamcity/buildAgent/work/28e4ceac98318065
07:00:34
   [ParallelExec] /opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_1 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_1.xml
07:00:34
   [ParallelExec] /opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_2 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_2.xml
07:00:34
   [ParallelExec] /opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_3 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_3.xml
07:00:34
   [ParallelExec] /opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_4 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_4.xml
07:00:34
   [ParallelExec] /opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_5 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_5.xml
07:00:43
   [ParallelExec]  '/opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_1 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_1.xml' exited with code 255 
07:00:43
  '/opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_2 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_2.xml' exited with code 1 
07:00:43
  '/opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_3 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_3.xml' exited with code 1 
07:00:43
  '/opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_4 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_4.xml' exited with code 1 
07:00:43
  '/opt/teamcity/buildAgent/work/28e4ceac98318065/vendor/bin/codecept run acceptance --group paracept_5 --debug -c /opt/teamcity/buildAgent/work/28e4ceac98318065/codeception.yml --xml /opt/teamcity/buildAgent/work/28e4ceac98318065/tests/_output/result_5.xml' exited with code 1 
07:00:43
    Time 8.171s
07:00:43
   [ParallelExec]  Exit code 255  Time 8.171s
07:00:43
  Process exited with code 255
07:00:43
  Process exited with code 255 (Step: Acceptance tests run parallel (Command Line))
<testsuites>
<testsuite name="acceptance" tests="12" assertions="0" errors="12" failures="0" skipped="0" time="0.000000">
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/ClientsWebsites/ClientsWebsitesCreateCest.php" name="createClientWebsite" class="Tests\Acceptance\ClientsWebsites\ClientsWebsitesCreateCest" feature="create client website" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/ClientsWebsites/ClientsWebsitesDeleteCest.php" name="deleteClientWebsite" class="Tests\Acceptance\Clients\ClientsWebsitesDeleteCest" feature="delete client website" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/ClientsWebsites/ClientsWebsitesSearchCest.php" name="searchUsingClientId" class="Tests\Acceptance\ClientsWebsites\ClientsWebsitesSearchCest" feature="Search using client_id select" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/ClientsWebsites/ClientsWebsitesSearchCest.php" name="searchUsingProductId" class="Tests\Acceptance\ClientsWebsites\ClientsWebsitesSearchCest" feature="Search using product_id select" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/ClientsWebsites/ClientsWebsitesSearchCest.php" name="searchUsingWebsiteUrl" class="Tests\Acceptance\ClientsWebsites\ClientsWebsitesSearchCest" feature="Search using website_url input" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/ClientsWebsites/ClientsWebsitesSearchCest.php" name="searchUsingExpirationDate" class="Tests\Acceptance\ClientsWebsites\ClientsWebsitesSearchCest" feature="Search using website_url input" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/Countries/CountriesCreateCest.php" name="createLanguage" class="Tests\Acceptance\Countries\CountriesCreateCest" feature="create language" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/Countries/CountriesDeleteCest.php" name="deleteCountry" class="Tests\Acceptance\Countries\CountriesDeleteCest" feature="delete country" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/Countries/CountriesEditCest.php" name="editCountry" class="Tests\Acceptance\Countries\CountriesEditCest" feature="edit country" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/Countries/CountriesSearchCest.php" name="seeLanguages" class="Tests\Acceptance\Countries\CountriesSearchCest" feature="See list of countries" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/Countries/CountriesSearchCest.php" name="searchUsingName" class="Tests\Acceptance\Countries\CountriesSearchCest" feature="Search using name input" assertions="0" time="0.000000"/>
<testcase file="/opt/teamcity/buildAgent/work/28e4ceac98318065/tests/acceptance/Countries/CountriesSearchCest.php" name="searchUsingContinent" class="Tests\Acceptance\Countries\CountriesSearchCest" feature="Search using continent_id select" assertions="0" time="0.000000"/>
</testsuite>
</testsuites>

Provide test source code if related

<?php

require_once 'vendor/autoload.php';

use Codeception\Task\MergeReports;
use Codeception\Task\SplitTestsByGroups;
use Robo\Result;
use Robo\Tasks;

class RoboFile extends Tasks
{
    use MergeReports;
    use SplitTestsByGroups;

    public function parallelSplitTests(): void
    {
        $this->taskSplitTestFilesByGroups(5)
            ->projectRoot('.')
            ->testsFrom('tests/acceptance')
            ->groupsTo('tests/_data/paracept_')
            ->run();
    }

    public function parallelRun(): Result
    {
        $parallel = $this->taskParallelExec();
        for ($i = 1; $i <= 5; $i++) {
            $parallel->process(
                $this->taskCodecept()
                    ->suite('acceptance')
                    ->group("paracept_$i")
                    ->debug()
                    ->configFile(__DIR__ . '/codeception.yml')
                    ->xml(__DIR__ . "/tests/_output/result_$i.xml")
            );
        }

        return $parallel->run();
    }

    public function parallelMergeResults(): void
    {
        $merge = $this->taskMergeXmlReports();
        for ($i = 1; $i <= 5; $i++) {
            $merge->from(__DIR__ . "/tests/_output/result_$i.xml");
        }
        $merge->into(__DIR__ . '/tests/_output/result_paracept.xml')->run();
    }

    public function parallelRunAndMergeResults(): Result
    {
        $result = $this->parallelRun();
        $this->parallelMergeResults();

        return $result;
    }
}

Details

  • Codeception version: ^4.1
  • PHP Version: 7.4
  • Operating System: Linux
  • Installation type: Composer
  • List of installed packages (composer show)
aws/aws-sdk-php                     3.134.3            AWS SDK for PHP - Use Amazon Web Services in your PHP project
barryvdh/laravel-ide-helper         v2.6.7             Laravel IDE Helper, generates correct PHPDocs for all Facade classes, to improve auto-completion.
barryvdh/reflection-docblock        v2.0.6            
beberlei/assert                     v3.2.7             Thin assertion library for input validation in business models.
behat/gherkin                       v4.6.2             Gherkin DSL parser for PHP 5.3
brianium/habitat                    v1.0.0             A dependable php environment
brianium/paratest                   4.0.0              Parallel testing for PHP
clue/stream-filter                  v1.4.1             A simple and modern approach to stream filtering in PHP
codeception/codeception             4.1.4              BDD-style testing framework
codeception/lib-asserts             1.11.0             Assertion methods used by Codeception core and Asserts module
codeception/lib-innerbrowser        1.3.1              Parent library for all Codeception framework modules and PhpBrowser
codeception/module-db               1.0.1              WebDriver module for Codeception
codeception/module-laravel5         1.1.0              Codeception module for Laravel 5 framework
codeception/module-webdriver        1.0.7              WebDriver module for Codeception
codeception/phpunit-wrapper         9.0.1              PHPUnit classes used by Codeception
codeception/robo-paracept           0.4.2              Codeception Parallel Execution Tasks via Robo Task Runner
codeception/stub                    3.6.1              Flexible Stub wrapper for PHPUnit's Mock Builder
composer/ca-bundle                  1.2.6              Lets you find a path to the system CA bundle, and includes a fallback to the Mozilla CA bundle.
composer/composer                   1.10.1             Composer helps you declare, manage and install dependencies of PHP projects. It ensures you have the right stack everywhere.
composer/semver                     1.5.1              Semver library that offers utilities, version constraint parsing and validation.
composer/spdx-licenses              1.5.3              SPDX licenses list and validation library.
composer/xdebug-handler             1.4.1              Restarts a process without Xdebug.
consolidation/annotated-command     4.1.0              Initialize Symfony Console commands from annotated command class methods.
consolidation/config                1.2.1              Provide configuration services for a commandline tool.
consolidation/log                   2.0.0              Improved Psr-3 / Psr\Log logger based on Symfony Console components.
consolidation/output-formatters     4.1.0              Format text by applying transformations provided by plug-in formatters.
consolidation/robo                  1.4.12             Modern task runner
consolidation/self-update           1.1.5              Provides a self:update command for Symfony Console applications.
container-interop/container-interop 1.2.0              Promoting the interoperability of container objects (DIC, SL, etc.)
davejamesmiller/laravel-breadcrumbs 5.3.2              A simple Laravel-style way to create breadcrumbs.
dflydev/dot-access-data             v1.1.0             Given a deep data structure, access data by dot notation.
doctrine/cache                      1.10.0             PHP Doctrine Cache library is a popular cache implementation that supports many different drivers such as redis, memcache, apc, mongodb and ...
doctrine/dbal                       v2.10.1            Powerful PHP database abstraction layer (DBAL) with many features for database schema introspection and management.
doctrine/event-manager              1.1.0              The Doctrine Event Manager is a simple PHP event system that was built to be used with the various Doctrine projects.
doctrine/inflector                  1.3.1              Common String Manipulations with regard to casing and singular/plural rules.
doctrine/instantiator               1.3.0              A small, lightweight utility to instantiate objects in PHP without invoking their constructors
doctrine/lexer                      1.2.0              PHP Doctrine Lexer parser library that can be used in Top-Down, Recursive Descent Parsers.
dragonmantank/cron-expression       v2.3.0             CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due
egulias/email-validator             2.1.17             A library for validating emails against several RFCs
facade/ignition-contracts           1.0.0              Solution contracts for Ignition
filp/whoops                         2.7.1              php error handling for cool kids
fzaninotto/faker                    v1.9.1             Faker is a PHP library that generates fake data for you.
grasmash/expander                   1.0.0              Expands internal property references in PHP arrays file.
grasmash/yaml-expander              1.4.0              Expands internal property references in a yaml file.
guzzlehttp/guzzle                   6.5.2              Guzzle is a PHP HTTP client library
guzzlehttp/promises                 v1.3.1             Guzzle promises library
guzzlehttp/psr7                     1.6.1              PSR-7 message implementation that also provides common utility methods
hamcrest/hamcrest-php               v2.0.0             This is the PHP port of Hamcrest Matchers
http-interop/http-factory-guzzle    1.0.0              An HTTP Factory using Guzzle PSR7
jean85/pretty-package-versions      1.2                A wrapper for ocramius/package-versions to get pretty versions strings
justinrainbow/json-schema           5.2.9              A library to validate a json schema.
laravel/framework                   v6.18.3            The Laravel Framework.
laravelcollective/html              v6.1.0             HTML and Form Builders for the Laravel Framework
league/commonmark                   1.3.3              Highly-extensible PHP Markdown parser which fully supports the CommonMark spec and Github-Flavored Markdown (GFM)
league/container                    2.4.1              A fast and intuitive dependency injection container.
league/flysystem                    1.0.66             Filesystem abstraction: Many filesystems, one API.
mockery/mockery                     1.3.1              Mockery is a simple yet flexible PHP mock object framework
monolog/monolog                     2.0.2              Sends your logs to files, sockets, inboxes, databases and various web services
mtdowling/jmespath.php              2.5.0              Declaratively specify how to extract elements from a JSON document
myclabs/deep-copy                   1.9.5              Create deep copies (clones) of your objects
nesbot/carbon                       2.32.2             An API extension for DateTime that supports 281 different languages.
ocramius/package-versions           1.7.0              Composer plugin that provides efficient querying for installed package versions (no runtime IO)
opis/closure                        3.5.1              A library that can be used to serialize closures (anonymous functions) and arbitrary objects.
paragonie/random_compat             v9.99.99           PHP 5.x polyfill for random_bytes() and random_int() from PHP 7
phar-io/manifest                    1.0.3              Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                     2.0.1              Library for handling version information and constraints
php-http/client-common              2.1.0              Common HTTP Client implementations and tools for HTTPlug
php-http/discovery                  1.7.4              Finds installed HTTPlug implementations and PSR-7 message factories
php-http/guzzle6-adapter            v2.0.1             Guzzle 6 HTTP Adapter
php-http/httplug                    2.1.0              HTTPlug, the HTTP client abstraction for PHP
php-http/message                    1.8.0              HTTP Message related tools
php-http/message-factory            v1.0.2             Factory interfaces for PSR-7 HTTP Message
php-http/promise                    v1.0.0             Promise used for asynchronous HTTP requests
php-webdriver/webdriver             1.8.2              A PHP client for Selenium WebDriver. Previously facebook/webdriver.
phpdocumentor/reflection-common     2.0.0              Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock   5.1.0              With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is embedded in a Doc...
phpdocumentor/type-resolver         1.1.0              A PSR-5 based resolver of Class names, Types and Structural Element Names
phpoption/phpoption                 1.7.3              Option Type for PHP
phpspec/prophecy                    v1.10.3            Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage           8.0.1              Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator           3.0.0              FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                 3.0.0              Invoke callables with a timeout
phpunit/php-text-template           2.0.0              Simple template engine.
phpunit/php-timer                   3.0.0              Utility class for timing
phpunit/php-token-stream            4.0.0              Wrapper around PHP's tokenizer extension.
phpunit/phpunit                     9.1.1              The PHP Unit Testing framework.
predis/predis                       v1.1.1             Flexible and feature-complete Redis client for PHP and HHVM
psr/container                       1.0.0              Common Container Interface (PHP FIG PSR-11)
psr/http-client                     1.0.0              Common interface for HTTP clients
psr/http-factory                    1.0.1              Common interfaces for PSR-7 HTTP message factories
psr/http-message                    1.0.1              Common interface for HTTP messages
psr/log                             1.1.3              Common interface for logging libraries
psr/simple-cache                    1.0.1              Common interfaces for simple caching
ralouphie/getallheaders             3.0.3              A polyfill for getallheaders.
ramsey/uuid                         3.9.3              Formerly rhumsaa/uuid. A PHP 5.4+ library for generating RFC 4122 version 1, 3, 4, and 5 universally unique identifiers (UUID).
roave/security-advisories           dev-master 0f73cf4 Prevents installation of composer packages with known security vulnerabilities: no API, simply require it
sebastian/code-unit                 1.0.0              Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup  2.0.0              Looks up which function or method a line of code belongs to
sebastian/comparator                4.0.0              Provides the functionality to compare PHP values for equality
sebastian/diff                      4.0.0              Diff implementation
sebastian/environment               5.0.2              Provides functionality to handle HHVM/PHP environments
sebastian/exporter                  4.0.0              Provides the functionality to export PHP variables for visualization
sebastian/global-state              4.0.0              Snapshotting of global state
sebastian/object-enumerator         4.0.0              Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector          2.0.0              Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context         4.0.0              Provides functionality to recursively process PHP variables
sebastian/resource-operations       3.0.0              Provides a list of PHP built-in functions that operate on resources
sebastian/type                      2.0.0              Collection of value objects that represent the types of the PHP type system
sebastian/version                   3.0.0              Library that helps with managing the version number of Git-hosted PHP projects
seld/jsonlint                       1.7.2              JSON Linter
seld/phar-utils                     1.1.0              PHAR file format utilities, for when PHP phars you up
sentry/sdk                          2.1.0              This is a metapackage shipping sentry/sentry with a recommended http client.
sentry/sentry                       2.3.2              A PHP SDK for Sentry (http://sentry.io)
sentry/sentry-laravel               1.7.1              Laravel SDK for Sentry (https://sentry.io)
swiftmailer/swiftmailer             v6.2.3             Swiftmailer, free feature-rich PHP mailer
symfony/browser-kit                 v5.0.7             Symfony BrowserKit Component
symfony/console                     v4.4.7             Symfony Console Component
symfony/css-selector                v5.0.7             Symfony CssSelector Component
symfony/debug                       v4.4.7             Symfony Debug Component
symfony/dom-crawler                 v5.0.7             Symfony DomCrawler Component
symfony/error-handler               v4.4.7             Symfony ErrorHandler Component
symfony/event-dispatcher            v4.4.7             Symfony EventDispatcher Component
symfony/event-dispatcher-contracts  v1.1.7             Generic abstractions related to dispatching event
symfony/filesystem                  v4.4.7             Symfony Filesystem Component
symfony/finder                      v4.4.7             Symfony Finder Component
symfony/http-foundation             v4.4.7             Symfony HttpFoundation Component
symfony/http-kernel                 v4.4.7             Symfony HttpKernel Component
symfony/mime                        v5.0.7             A library to manipulate MIME messages
symfony/options-resolver            v5.0.7             Symfony OptionsResolver Component
symfony/polyfill-ctype              v1.15.0            Symfony polyfill for ctype functions
symfony/polyfill-iconv              v1.15.0            Symfony polyfill for the Iconv extension
symfony/polyfill-intl-idn           v1.15.0            Symfony polyfill for intl's idn_to_ascii and idn_to_utf8 functions
symfony/polyfill-mbstring           v1.15.0            Symfony polyfill for the Mbstring extension
symfony/polyfill-php72              v1.15.0            Symfony polyfill backporting some PHP 7.2+ features to lower PHP versions
symfony/polyfill-php73              v1.15.0            Symfony polyfill backporting some PHP 7.3+ features to lower PHP versions
symfony/polyfill-uuid               v1.15.0            Symfony polyfill for uuid functions
symfony/process                     v4.4.7             Symfony Process Component
symfony/routing                     v4.4.7             Symfony Routing Component
symfony/service-contracts           v2.0.1             Generic abstractions related to writing services
symfony/translation                 v4.4.7             Symfony Translation Component
symfony/translation-contracts       v2.0.1             Generic abstractions related to translation
symfony/var-dumper                  v4.4.7             Symfony mechanism for exploring and dumping PHP variables
symfony/yaml                        v4.4.7             Symfony Yaml Component
theseer/tokenizer                   1.1.3              A small library for converting tokenized PHP source code into XML and potentially other formats
tijsverkoyen/css-to-inline-styles   2.2.2              CssToInlineStyles is a class that enables you to convert HTML-pages/files into HTML-pages/files with inline styles. This is very useful when...
tomko/common2                       1.3.10             Common2
vlucas/phpdotenv                    v3.6.2             Loads environment variables from `.env` to `getenv()`, `$_ENV` and `$_SERVER` automagically.
webmozart/assert                    1.7.0              Assertions to validate method input/output with nice error messages.
  • Suite configuration:
paths:
  tests: tests
  output: tests/_output
  data: tests/_data
  support: tests/_support
  envs: tests/_envs
actor_suffix: Tester
extensions:
  enabled:
    - Codeception\Extension\RunFailed
groups:
  paracept_*: tests/_data/paracept_*
actor: AcceptanceTester
modules:
    enabled:
        - WebDriver:
           url: 'http://ci-office.statscore.com/'
           window_size: false
           log_js_errors: true
           port: 4444
           browser: chrome
           capabilities:
               chromeOptions:
                  args: ["--window-size=1600,900", "--headless"]
        - \Helper\Acceptance
        - Db
        - DbHelper
        - Laravel5:
            part: ORM
    config:
        Db:
          dsn: 'mysql:host=localhost;dbname=head'
          user: ''
          password: ''
          populate: false
          cleanup: false
extensions:
    enabled:
        - Codeception\Extension\RunProcess:
            0: exec docker run --net=host selenium/standalone-chrome
            sleep: 7 # wait 7 second for processes to boot

Define an interface for merge tasks

It would be nice to have a common interface for the merge tasks. Now if you want to generalise the process of building parallel execution of tasks, you probably end up having your report type as a config, from which task is built.

/**
* @return MergeXmlReportsTask|MergeHTMLReportsTask|\Robo\Collection\CollectionBuilder
*/
public function makeMergeTask()
{
     if ('xml' === $this->reportType) {
        /** @var MergeXmlReportsTask $merge */
        $merge = $this->roboFile->taskMergeHTMLReports();
     } else {
         /** @var MergeHTMLReportsTask $merge */
         $merge = $this->roboFile->taskMergeXmlReports();
     }
     return $merge;
}

instead of

/**
* @return MergeReportsTaskInterface|\Robo\Collection\CollectionBuilder
*/
public function makeMergeTask()
{
     return ('xml' === $this->reportType)
         ? $this->roboFile->taskMergeXmlReports()
         : $this->roboFile->taskMergeHTMLReports()
}

So what do you think about this? Here a pr

Does not work when codeception is already installed

As written in this documentation http://codeception.com/docs/12-ParallelExecution robo-paracebt does not work when codeception is already installed.

Will this issue be solved ?

What is the recommended way to handle this when codeception is already installed?
I tried: Removing codeception from the composer.json and run composer update --> Add robo-paracebt to composer.json --> composer update --> add codeception to composer.json --> run composer update.

I still get the error "Neither composer nor phar installation of Codeception found" when I try to execute a robo task. What could be the problem?

Dependencies between codeception and robo

Hello,

I try to use this solution but there is dependencies issue.

`
composer require "codeception/codeception"
composer require "codeception/robo-paracept":"@dev"
Problem 1
- codeception/robo-paracept dev-master requires codegyre/robo ~0.6.0 -> satisfiable by codegyre/robo[0.6.0].
- Installation request for codeception/robo-paracept @dev -> satisfiable by codeception/robo-paracept[dev-master].
- Conclusion: remove symfony/console v3.0.4
- Conclusion: don't install symfony/console v3.0.4
- codegyre/robo 0.6.0 requires symfony/console ~2.5 -> satisfiable by symfony/console[v2.5.0, v2.5.1, v2.5.10, v2.5.11, v2.5.12, v2.5.2, v2.5.3, v2.5.4, v2.5.5, v2.5.6, v2.5.7, v2.5.8, v2.5.9, v2.6.0, v2.6.1, v2.6.10, v2.6.11, v2.6.12, v2.6.13, v2.6.2, v2.6.3, v2.6.4, v2.6.5, v2.6.6, v2.6.7, v2.6.8, v2.6.9, v2.7.0, v2.7.1, v2.7.10, v2.7.11, v2.7.2, v2.7.3, v2.7.4, v2.7.5, v2.7.6, v2.7.7, v2.7.8, v2.7.9, v2.8.0, v2.8.1, v2.8.2, v2.8.3, v2.8.4].
- Can only install one of: symfony/console[v2.5.0, v3.0.4].
- Can only install one of: symfony/console[v2.5.1, v3.0.4].
- Can only install one of: symfony/console[v2.5.10, v3.0.4].
- Can only install one of: symfony/console[v2.5.11, v3.0.4].
- Can only install one of: symfony/console[v2.5.12, v3.0.4].
- Can only install one of: symfony/console[v2.5.2, v3.0.4].
- Can only install one of: symfony/console[v2.5.3, v3.0.4].
- Can only install one of: symfony/console[v2.5.4, v3.0.4].
- Can only install one of: symfony/console[v2.5.5, v3.0.4].
- Can only install one of: symfony/console[v2.5.6, v3.0.4].
- Can only install one of: symfony/console[v2.5.7, v3.0.4].
- Can only install one of: symfony/console[v2.5.8, v3.0.4].
- Can only install one of: symfony/console[v2.5.9, v3.0.4].
- Can only install one of: symfony/console[v2.6.0, v3.0.4].
- Can only install one of: symfony/console[v2.6.1, v3.0.4].
- Can only install one of: symfony/console[v2.6.10, v3.0.4].
- Can only install one of: symfony/console[v2.6.11, v3.0.4].
- Can only install one of: symfony/console[v2.6.12, v3.0.4].
- Can only install one of: symfony/console[v2.6.13, v3.0.4].
- Can only install one of: symfony/console[v2.6.2, v3.0.4].
- Can only install one of: symfony/console[v2.6.3, v3.0.4].
- Can only install one of: symfony/console[v2.6.4, v3.0.4].
- Can only install one of: symfony/console[v2.6.5, v3.0.4].
- Can only install one of: symfony/console[v2.6.6, v3.0.4].
- Can only install one of: symfony/console[v2.6.7, v3.0.4].
- Can only install one of: symfony/console[v2.6.8, v3.0.4].
- Can only install one of: symfony/console[v2.6.9, v3.0.4].
- Can only install one of: symfony/console[v2.7.0, v3.0.4].
- Can only install one of: symfony/console[v2.7.1, v3.0.4].
- Can only install one of: symfony/console[v2.7.10, v3.0.4].
- Can only install one of: symfony/console[v2.7.11, v3.0.4].
- Can only install one of: symfony/console[v2.7.2, v3.0.4].
- Can only install one of: symfony/console[v2.7.3, v3.0.4].
- Can only install one of: symfony/console[v2.7.4, v3.0.4].
- Can only install one of: symfony/console[v2.7.5, v3.0.4].
- Can only install one of: symfony/console[v2.7.6, v3.0.4].
- Can only install one of: symfony/console[v2.7.7, v3.0.4].
- Can only install one of: symfony/console[v2.7.8, v3.0.4].
- Can only install one of: symfony/console[v2.7.9, v3.0.4].
- Can only install one of: symfony/console[v2.8.0, v3.0.4].
- Can only install one of: symfony/console[v2.8.1, v3.0.4].
- Can only install one of: symfony/console[v2.8.2, v3.0.4].
- Can only install one of: symfony/console[v2.8.3, v3.0.4].
- Can only install one of: symfony/console[v2.8.4, v3.0.4].
- Installation request for symfony/console (locked at v3.0.4) -> satisfiable by symfony/console[v3.0.4].

`
Is it possible to use robo-paracept with codeception ?

Thanks

Execution time on the merged html report is incorrect (0s)

On merged report.html, execution time = 0s
Capture d’écran 2022-11-15 à 14 16 10

On each html report, we have execution time formatted in "minutes:seconds.milliseconds" and not as attended (103.29s) for example (since Codeception 5 I think)
Capture d’écran 2022-11-15 à 14 16 16

Maybe related to $timeTaken = $this->timer->stop()->asString(); in HtmlReporter

public function asString(): string
    {
        $result = '';

        if ($this->hours > 0) {
            $result = sprintf('%02d', $this->hours) . ':';
        }

        $result .= sprintf('%02d', $this->minutes) . ':';
        $result .= sprintf('%02d', $this->seconds);

        if ($this->milliseconds > 0) {
            $result .= '.' . sprintf('%03d', $this->milliseconds);
        }

        return $result;
    }

I have modified countExecutionTime() method, but it is to test that it works, not the solution to apply, I am well aware of it

private function countExecutionTime(DOMDocument $dstFile): void
    {
        $xpathHeadline = "//h1[text() = 'Codeception Results ']";
        $nodeList = (new DOMXPath($dstFile))
            ->query($xpathHeadline);
        if (!$nodeList) {
            throw XPathExpressionException::malformedXPath($xpathHeadline);
        }

        $pregResult = preg_match(
            '#^Codeception Results .* \((?<timesum>\d+\:\d+\.\d+|\d+\.\d+s)\)$#',
            $nodeList[0]->nodeValue,
            $matches
        );

        if (false === $pregResult) {
            throw new RuntimeException('Regexpression is malformed');
        }

        if (0 === $pregResult) {
            return;
        }

        if (str_contains($matches['timesum'], 's')) {
            $matches['timesum'] = str_replace('s', '', $matches['timesum']);
        } else {
            sscanf($matches['timesum'], "%d:%d.%d", $minutes, $seconds, $milliseconds);
            $matches['timesum'] = isset($milliseconds)
                ? ($minutes * 60 + $seconds) . '.' . $milliseconds
                : $minutes * 60 + $seconds;
        }

        if (!$this->maxTime) {
            $this->executionTimeSum += (float)$matches['timesum'];
        } else {
            $this->executionTime[] = (float)$matches['timesum'];
        }
    }

Moreover, in a parallel run context, each run has an execution time.
The final merge of the reports aggregates each of the reports
However, in my opinion, the execution time of the final merge should be the execution time of the longest run and not the sum of the execution times of each run.

Is it possible to stock all reports execution time in array and make finally
$this->executionTimeSum = max($executionTimeArray);

My proposal here #120

Exited with code 1

I'm having an issue with ParallelExec and Codeception exit codes. Running the codeception commands works manually, but when ran through ParallelExec they fail because codeception exits with code 1 even though --no-exit is set. Any help would be much appreciated!

[ParallelExec] ../vendor/bin/codecept run -c ../codeception.yml --group api_1 --no-exit api [ParallelExec] ../vendor/bin/codecept run -c ../codeception.yml --group api_2 --no-exit api [ParallelExec] '../vendor/bin/codecept run -c ../codeception.yml --group api_1 --no-exit api' exited with code 1 [ParallelExec] Exit code 1 Time 13.654s

Full paths generated in group file created by taskSplitTestsByGroups are not recognized

Issue - taskSplitTestsByGroups is generating group files with full paths in them. When I run codecept and specify one of the group files created, it does not recognize any tests listed in those group files, reporting zero acceptance tests.

I'm using the following:

  • PHP 7.2.0RC6
  • Codeception 2.4.1
  • Robo 1.2.3
  • robo-paracept 0.4.0

I've got the following in my RoboFile.php:

class RoboFile extends \Robo\Tasks
{
    use \Codeception\Task\SplitTestsByGroups;

    public function parallelSplitTests() {
        $this->taskSplitTestsByGroups(5)
            ->testsFrom('tests')
            ->projectRoot('.')
            ->groupsTo('tests/_data/paracept_')
            ->run();
    }
}

Running the robo task:

$ ../../vendor/bin/robo parallel:split-tests                                                                                                      10:49:56 AM
 [Codeception\Task\SplitTestsByGroupsTask] Processing 2 tests
 [Codeception\Task\SplitTestsByGroupsTask] Writing tests/_data/paracept_1
 [Codeception\Task\SplitTestsByGroupsTask] Writing tests/_data/paracept_2

Contents of paracept_1:

Users/rleland/web/tests/codeception-new/tests/acceptance/backend/LoginLogoutCest.php:loginFailsWithInvalidCredentials

Running codecept using that group file:

$ ../../vendor/bin/codecept run acceptance -g paracept_1                                                                                          10:54:37 AM
Codeception PHP Testing Framework v2.4.1
Powered by PHPUnit 4.8.28 by Sebastian Bergmann and contributors.
[Groups] paracept_1

Acceptance Tests (0) -----------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------


Time: 92 ms, Memory: 10.00Mb

No tests executed!

I see that taskSplitTestsByGroups calls \Codeception\Test\Descriptor::getTestFullName($test) which yields the full path. Note that if I use taskSplitTestFilesByGroups it generates a group file with a relative path, which works fine:

$ ../../vendor/bin/codecept run acceptance -g paracept_1                                                                                          10:57:31 AM
Codeception PHP Testing Framework v2.4.1
Powered by PHPUnit 4.8.28 by Sebastian Bergmann and contributors.
[Groups] paracept_1

Acceptance Tests (2) -----------------------------------------------------------------------------------------------------------------------------------------
✔ LoginLogoutCest: Login succeeds with valid credentials (5.14s)
✔ LoginLogoutCest: Login fails with invalid credentials (1.46s)
--------------------------------------------------------------------------------------------------------------------------------------------------------------


Time: 7.46 seconds, Memory: 10.00Mb

OK (2 tests, 4 assertions)

Any guidance/input would be great.

If parallel exec fails and no report produced MergeHTMLReportsTask throws an error

Parallel Exec fails with exit code 138 and does not produce a HTML report when merging the reports this is throwing an uncaught exception.

ERROR: Uncaught Error: Call to a member function appendChild() on null in /Users/.../vendor/codeception/robo-paracept/src/MergeReports.php:269
Stack trace:
#0 /Users/.../vendor/codeception/robo-paracept/src/MergeReports.php(211): Codeception\Task\MergeHTMLReportsTask->moveSummaryTable(Object(DOMDocument), NULL)
#1 /Users/.../vendor/consolidation/robo/src/Collection/CollectionBuilder.php(428): Codeception\Task\MergeHTMLReportsTask->run()
#2 /Users/.../vendor/consolidation/robo/src/Collection/CollectionBuilder.php(413): Robo\Collection\CollectionBuilder->runTasks()
#3 /Users/.../Project/Gaming-Codeception/gaming-codeception/src/Robo.php(64): Robo\Collection\CollectionBuilder->run()
#4 /Users/.../Project/Gaming-Codeception/gaming-codeception/src/Robo.php(75): Gaming\Codeception\Robo->parallelMergeResults()
#5 [
in /Users/.../vendor/codeception/robo-paracept/src/MergeReports.php:269

Since PHPUnit 6 Parallel:Split-Tests has stopped working

Splitting tests now fails since the upgrade to PHPUnit6. Running taskSplitTestsByGroups throws the following error:

ERROR: Interface 'PHPUnit_Framework_Test' not found
in /project/vendor/codeception/base/src/Codeception/TestInterface.php:7

Call Stack:
0.0010 356648 1. {main}() /ProjectDir/ui-tests/vendor/consolidation/robo/robo:0
0.0103 949600 2. Robo\Runner->execute() /ProjectDir/ui-tests/vendor/consolidation/robo/robo:20
0.0265 1649712 3. Robo\Runner->run() /ProjectDir/ui-tests/vendor/consolidation/robo/src/Runner.php:111
0.1245 4442688 4. Symfony\Component\Console\Application->run() /ProjectDir/ui-tests/vendor/consolidation/robo/src/Runner.php:157
0.1419 4443520 5. Symfony\Component\Console\Application->doRun() /ProjectDir/ui-tests/vendor/symfony/console/Application.php:130
0.1421 4443520 6. Symfony\Component\Console\Application->doRunCommand() /ProjectDir/ui-tests/vendor/symfony/console/Application.php:223
0.1458 4470032 7. Symfony\Component\Console\Command\Command->run() /ProjectDir/ui-tests/vendor/symfony/console/Application.php:887
0.1476 4482768 8. Consolidation\AnnotatedCommand\AnnotatedCommand->execute() /ProjectDir/ui-tests/vendor/symfony/console/Command/Command.php:264
0.1480 4559112 9. Consolidation\AnnotatedCommand\CommandProcessor->process() /ProjectDir/ui-tests/vendor/consolidation/annotated-command/src/AnnotatedCommand.php:394
0.1480 4559168 10. Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter() /ProjectDir/ui-tests/vendor/consolidation/annotated-command/src/CommandProcessor.php:119
0.1493 4574088 11. Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback() /ProjectDir/ui-tests/vendor/consolidation/annotated-command/src/CommandProcessor.php:153
0.1494 4575160 12. call_user_func_array:{/ProjectDir/ui-tests/vendor/consolidation/annotated-command/src/CommandProcessor.php:207}() /ProjectDir/ui-tests/vendor/consolidation/annotated-command/src/CommandProcessor.php:207
0.1494 4575568 13. Gaming\Codeception\Robo->parallelSplitTests() /ProjectDir/ui-tests/vendor/consolidation/annotated-command/src/CommandProcessor.php:207
0.1496 4576000 14. Robo\Collection\CollectionBuilder->run() /ProjectDir/ui-tests/vendor/gaming/codeception/src/Robo.php:43
0.1500 4583928 15. Robo\Collection\CollectionBuilder->runTasks() /ProjectDir/ui-tests/vendor/consolidation/robo/src/Collection/CollectionBuilder.php:413
0.1500 4583928 16. Codeception\Task\SplitTestsByGroupsTask->run() /ProjectDir/ui-tests/vendor/consolidation/robo/src/Collection/CollectionBuilder.php:428
0.1575 5114592 17. Codeception\Test\Loader->loadTests() /ProjectDir/ui-tests/vendor/codeception/robo-paracept/src/SplitTestsByGroups.php:78
0.1575 5114592 18. Codeception\Test\Loader->loadTest() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Loader.php:123
0.1576 5114616 19. Codeception\Test\Loader->loadTests() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Loader.php:113
0.1699 5414104 20. Codeception\Test\Loader\Cest->loadTests() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Loader.php:135
0.1721 5503680 21. spl_autoload_call() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Loader/Cest.php:95
0.1721 5503808 22. Composer\Autoload\ClassLoader->loadClass() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Loader/Cest.php:95
0.1721 5503968 23. Composer\Autoload\includeFile() /ProjectDir//ui-tests/vendor/composer/ClassLoader.php:322
0.1726 5549408 24. include('/ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Cest.php') /ProjectDir/ui-tests/vendor/composer/ClassLoader.php:440
0.1726 5549408 25. spl_autoload_call() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Cest.php:16
0.1726 5549536 26. Composer\Autoload\ClassLoader->loadClass() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Cest.php:16
0.1727 5549696 27. Composer\Autoload\includeFile() /ProjectDir/ui-tests/vendor/composer/ClassLoader.php:322
0.1731 5567384 28. include('/ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Test.php') /ProjectDir/ui-tests/vendor/composer/ClassLoader.php:440
0.1747 5590664 29. spl_autoload_call() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Test.php:17
0.1747 5590800 30. Composer\Autoload\ClassLoader->loadClass() /ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/Test/Test.php:17
0.1748 5590960 31. Composer\Autoload\includeFile() /ProjectDir/ui-tests/vendor/composer/ClassLoader.php:322
0.1750 5592464 32. include('/ProjectDir/ui-tests/vendor/codeception/base/src/Codeception/TestInterface.php') /ProjectDir/ui-tests/vendor/composer/ClassLoader.php:440

Oddly enough taskSplitTestFilesByGroups works fine.

Setting maximum number of threads for taskParallelExec

Let's assume I want to run 20 tests in total but only 10 at the same time to avoid putting a big load on the application under test.

I would like to configure robo-paracept to run maximum 10 threads (once one thread finishes another from the queue is grabbed and launched) - is it possible?

Error merging HTML reports

I'm getting error when merging html reports:

[ParallelExec] ../vendor/bin/codecept run unit --group paracept_1 --html html/result_1.html [ParallelExec] ../vendor/bin/codecept run unit --group paracept_2 --html html/result_2.html [ParallelExec] ../vendor/bin/codecept run unit --group paracept_3 --html html/result_3.html [ParallelExec] ../vendor/bin/codecept run unit --group paracept_4 --html html/result_4.html [ParallelExec] ../vendor/bin/codecept run unit --group paracept_5 --html html/result_5.html 5/5 [============================] 100% [ParallelExec] 5 processes finished running [ParallelExec] Done in 19.795s [Codeception\Task\MergeHTMLReportsTask] Merging HTML reports into tests/_output/result_paracept.html ERROR: Call to a member function item() on null in ../vendor/codeception/robo-paracept/src/MergeReports.php:319

Using codeception/base 3.1.2

Could be fixed by changing line 316 into:
$nodes = (new \DOMXPath($dstFile))->query("//div[@class='layout']/table/tbody/tr[contains(@class, 'scenarioRow')]");

Occasionally a test process exits with status code 135 because of a SIGBUS

I have a set up which is pretty much the one explained in the Codeception documentation. With robo and robo-paracept for running tests.

While using it one of the processes for running Codeception would fail with an exit code 135. Here is an output from such an incident:

 [ParallelExec] Output for ../vendor/bin/codecept run --skip-group api --group parallel_4 --xml paracept/_results/result_4.xml acceptance:

Codeception PHP Testing Framework v2.2.7
Powered by PHPUnit 4.8.31 by Sebastian Bergmann and contributors.
[Groups] parallel_4 
 [ParallelExec]  Exit code 135  Time 01:54

There is no other output from the process. Now, this error code 135 is very suspicious. So I went ahead and investigated further. In the Symfony/Process one can see that this exit code is only possible in two cases:

  1. The process (codeception in our case) has exited with exit code 135
$this->exitcode = $this->processInformation['exitcode'];
  1. The process has received a signal from the OS with the numerical value of 7:
$this->exitcode = 128 + $this->processInformation['termsig'];

We can rule out the first case since Codeception exits with 0 or 1 and nothing else. So we are left with the hypothesis that a signal stops the process. The signal with numerical value 7 is "SIGBUS". One can verify that by running kill -l which would print all signals and their numerical values.

So why would that happen? Trying to find out what the problem is I found PHP bug 52752. It is still open and impacts all PHP versions up to and including 7.

So, is this our problem? I set up the environment so that a core dump file would be generated for the php process when an abortion signal is received. And sure enough, the next time our tests failed with status code 135, the same back trace as the one in the bug report is found. See the last few lines of it:

#0  0x00007f2097dd86ed in lex_scan ()
#1  0x00007f2097df7062 in zendlex ()
#2  0x00007f2097dd1956 in zendparse ()
#3  0x00007f2097dd6f5b in compile_file ()
#4  0x00007f2097dfd12a in dtrace_compile_file ()
#5  0x00007f2085d7bc9a in phar_compile_file () from /usr/lib64/php/modules/phar.so
#6  0x00007f2097ec08b3 in ZEND_INCLUDE_OR_EVAL_SPEC_CV_HANDLER ()
#7  0x00007f2097e39e68 in execute_ex ()
#8  0x00007f2097dfd1a9 in dtrace_execute_ex ()
#9  0x00007f2097ec27cb in zend_do_fcall_common_helper_SPEC ()
#10 0x00007f2097e39e68 in execute_ex ()
#11 0x00007f2097dfd1a9 in dtrace_execute_ex ()
#12 0x00007f2097ec27cb in zend_do_fcall_common_helper_SPEC ()
#13 0x00007f2097e39e68 in execute_ex ()
#14 0x00007f2097dfd1a9 in dtrace_execute_ex ()
#15 0x00007f2097ec27cb in zend_do_fcall_common_helper_SPEC ()
#16 0x00007f2097e39e68 in execute_ex ()
#17 0x00007f2097dfd1a9 in dtrace_execute_ex ()

Basically, this happens when the PHP interpreter reads a source file while this file is being edited. In the bug report there is a one liner which reproduces the bug.

The following is just a hypothesis

So how do we end up triggering this bug? My suspicion is that the reason is the code which Codeception generates. I am talking about the file support/_generated/AcceptanceTesterActions.php. It seems to be programatically created and then used while running a Codeception Cest test. The guess goes like this: every process invoked by robo-paracept tries to write in the same file and then uses it. Most of the time this is fine because all processes do create functionally the same file but this fails from time to time because of the PHP bug 52752.

Why am I writing here?

As far as I can tell this is not really a problem in robo-paracept. It is just a curious side effect of the way Codeception works grouped with the PHP bug. But one can argue that Codeception alone would never hit this so it is robo-paracept which have to do something about it.

But this is a PHP bug, just wait for it to be fixed. Well, the bug is quite old already and I don't see it being fixed anytime soon. It has been 7 years since it was reported and two (one? PHP 6 got skipped?) major versions of the language have been released since then. In the meantime robo-paracept is bound to fail occasionally.

You should've created this issue in the Codeception project. Maybe I should have. If you think that it should be there, then I would move it.

Proposed solution

I have only limited understanding of Codeception. Mainly, I do understand how it decides its exit code 😄 But if robo-paracept can be made to invoke Codeception in a way that different generated files would be created for every parallel process.

An other way would be to stop using generated files altogether in Codeception. But I have no idea what this entails.

Environment:

OS: CentOS Linux 7.3
Codeception: v2.2.7
robo-paracept: v0.2
robo: v1.0.5
PHP: v5.5.38

Additional data

I can supply you with the mentioned coredump and binary with which it happened if this is of any help. Or I can create a minimal example which hits this bug. But the one in the example should be more than enough. Actually, most of the robo-paracept usage is bound to be susceptible to this issue. I've seen it happen dozens of times a day to once every other day.

Depends annotation.

Hey,

Good day.

Can't figure out how the depends annotation works. Tried:

# Above class
/**
 * @depends Scenarios\Compliance\Toolkit\Category\AdminUserCest
 *
 */
class ModuleOwnerUserCest {

# With use statement
use Scenarios\Compliance\Toolkit\Category\AdminUserCest;
/**
 * @depends AdminUserCest
 *
 */
class ModuleOwnerUserCest {

Even tried the same as above on a function within the class as well. No luck. When I split the tests using taskSplitTestFilesByGroups() it ignores the depends annotation.

Any help would be much appreciated.

Regards.
Jarrett

"Return value must be of type Codeception\Task\Splitter\TestFileSplitterTask" when splitting the tests after upgrading to version 1.0

After updating the robo-paracept version, and making the changes indicated in the readme, I get this error when I run the split command
PHP Fatal error: Uncaught TypeError: RoboFile::taskSplitTestFilesByGroups(): Return value must be of type Codeception\Task\Splitter\TestFileSplitterTask, Robo\Collection\CollectionBuilder returned in /vendor/codeception/robo-paracept/src/Splitter/TestsSplitterTrait.php:26

Set logger

Running latest master of Robo triggers the following error which fails on CI tests.

robo parallel:split

PHP Deprecated: No logger set for Codeception\Task\SplitTestFilesByGroupsTask. Use $this->task(Foo::class) rather than new Foo() in loadTasks to ensure the builder can initialize task the task, or use $this->collectionBuilder()->taskFoo() if creating one task from within another. in vendor/consolidation/robo/src/Common/TaskIO.php on line 43

One quick fix could be to add a $this->setLogger(..) in the run method of split/merge method.
Though I'm not sure this can be done simply from the RoboFile.php?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.