meltwaterarchive / storyplayer

Bring your user and service stories to life through automation

Storyplayer

Bring your user and service stories to life through your tests.

Note

This project is no longer maintained by DataSift.

Installation

Storyplayer should be installed via Composer. To install Storyplayer, create a composer.json file with the following contents:

{
    "require": {
        "datasift/storyplayer": "~2.0"
    }
}

Then, run composer install. Once that's completed, run ./vendor/bin/storyplayer install to install any additional dependencies.

Usage

If your tests need a web browser, make sure you've started browsermob-proxy and selenium:

./vendor/bin/browsermob-proxy.sh start
./vendor/bin/selenium-server.sh start

Basic usage is:

./vendor/bin/storyplayer <story>

where:

• <story> is the path to the PHP file containing your story and its test

Full Documentation

You'll find our docs at http://datasift.github.io/storyplayer. They're currently a work-in-progress.

Community

• Storyplayer-dev group / mailing list

Contributing

Contributions are most welcome.

1. Fork on GitHub
2. Create a feature branch
3. Commit your changes with tests
4. New feature? Send a pull request against the develop branch.
5. Bug fix? Send a pull request against the master branch.

Please don't break backwards-compatibility :)

License

New BSD license. Full details are in the LICENSE.txt file.

storyplayer's Issues

[Feature] Add list-hoststable command

Output the contents of the hoststable. Useful for seeing if any VMs have been left behind that shouldn't have been.

[Feature] Add -e flag for choosing the environment to test against

Feature: stories can set their default test device

Due to bugs in Chromedriver, some of our stories need to be tested using Firefox. It'd be great if we implemented something like:

$story->setDefaultTestDevice('firefox');
$story->addToDeviceBlacklist('chrome');

so that folks don't have to remember which stories don't work with which devices.

usingHttp sends null body when given non-array.

I'm just getting started with StoryPlayer, so please excuse me if I'm missing something.

I'm attempting to post a raw JSON body to my API with usingHttp()->post() (see below). My test API is receiving empty request bodies.

$cartJSON = '{"products":[{"SKU":"ABC123","quantity":1}]}';
$response = $st->usingHttp()->post('http:/store.com/api/carts', $params, $cartJSON);

Looking in UsingHttp::makeHttpRequest(), it appears that if something other than an array is passed as the body, it is omitted altogether from the request.

if (is_array($body)) {
    foreach ($body as $key => $value) {
        $request->addData($key, $value);
    }
}

Is this the intended behaviour? I would expect my JSON body to be sent as-is.

As far as I can tell, this just needs an else branch here, such as:

if (is_array($body)) {
    foreach ($body as $key => $value) {
        $request->addData($key, $value);
    }
}else{
    $request->setPayload($body);
}

Thanks,
Ross

[Feature request] Ability to run storyplayer against a directory of tests

It would be nice to be able to say "run all of the tests in this directory" and get a summary of what passed/failed at the end

SauceLabs config not working or documentation incorrect

On the SauceLabs documentation page (http://datasift.github.io/storyplayer/devices/saucelabs.html) it shows the config required for using saucelabs is:

{
    "environments": {
        "defaults": {
            "saucelabs": {
                "username": "<saucelabs-username>",
                "accesskey": "<saucelabs-accesskey>"
            }
        }
    }
}

However doing this results in exception in DataSift\Stone\ObjectLib\BaseObject for No such property: saucelabs on instance of class.

Tracing it back the storyTeller object inits the device from the $staticConfig and seems to make no attempt to merge in the saucelabs section in. As a workaround this does work:

"devices": {
        "sl_ie9_win7": {
            "saucelabs": {
                "username": "<saucelabs-username>",
                "accesskey": "<saucelabs-accesskey>"
            }
        }
    }

But I am having to repeat the config per device this way.

[Feature request] Remotely use an iOS / Android browser

I don't even know if this kind of wizardry is possible, but in much the same way you can control a desktop Chrome browser, I'd love for Storyplayer to be able to connect to a real iOS device (e.g. iPhone) or Android device, and remotely control the browser, feeding back to Storyplayer.

It would allow me to write stories like this:

1. As a PC user, I want to submit the form so that I can see search results
2. As an iPhone user, I want to submit the form so that I can see search results

I can presently test the first, but not the second story.

Composer compat

Missing composer manifest file

[Feature] deviceSetup() and deviceTeardown() support in story templates

This feature will add two new callbacks to stories and story templates: deviceSetup() and deviceTeardown(). They will be called whenever we start a device (such as a web browser) and whenever we stop the device.

The immediate use case is for setting HTTP Basic Auth credentials. At the moment, this is either done explicitly in the story, or in perPhaseSetup(). Unfortunately, doing it in perPhaseSetup() forces the browser to start, even if the phase itself doesn't need a browser.

It would be much better to provide a deviceSetup() callback for this, so that we only set HTTP Basic Auth if the current phase is trying to do something with the browser.

PHP Segfault

When running a simple test I noticed that PHP was segfaulting. I tracked it down to the following code:

In UsingRuntimeTable::remoteItem:

This segfaults:

$log->addStep("table '{$tableName}' is empty, removing from runtime config", function() use ($tables, $tableName){
      unset($tables->$tableName);
});

This works:

$r = array($tables, $tableName);
$log->addStep("table '{$tableName}' is empty, removing from runtime config", function() use ($r){
      list($tables, $tableName) = $r;
      unset($tables->$tableName);
});

It's not related to $tables and $tableName as the following code fails too:

$a = 1;
$b = 2;
$log->addStep("table '{$tableName}' is empty, removing from runtime config", function() use ($a, $b){
      unset($tables->$tableName);
});

$ php --version
PHP 5.5.12-2ubuntu4.4 (cli) (built: Apr 17 2015 11:47:44) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2014, by Zend Technologies
    with Xdebug v2.2.4, Copyright (c) 2002-2014, by Derick Rethans

[Feature] IFrame support

Storyplayer needs to be able to interact with IFrames in a web browser.

The proposed syntax for this is:

$st->usingBrowser()->switchToIframe($id);

where $id is the value of the iframe's ID attribute. In theory, that should be all that we need to do.

If we want to detect whether a specific iframe is in the DOM or not, currently the only way is to attempt to switch to the iframe, and see if we get an error from Selenium or not. If we don't get an error, we just need to switch back to the default context (by passing null for $id).

Release v1.5.0

Feature: --params-file

This is a new feature to load story parameters specified in a file (the params we can set via the -D switch). This will be a new ParamsFromSwitch in Storyplayer/Cli.

It will support the following formats:

• JSON
• YAML
• ini file

It'll use the right parser by examining the file extension.

[Feature] Let a story say which environments it can be run against

At the moment, there's no official / easy way for a story to say that it can / cannot be run against a specific environment. We need to add this.

Why do we need this? So that we can blacklist stories that aren't safe to run against production!

Proposed syntax:

$story->canRunInAllEnvironments()
         ->except('production');

or

$story->canRunIn('staging')->and('production');

First syntax is blacklist-based, second syntax is whitelist-based. We should support both. If a story uses neither syntax, the default is to support running in all environments.

If a user attempts to run a story against a blacklisted environment, it will be a fatal error.

OS X interface list too limited

The comment here says it all ;)

https://github.com/datasift/storyplayer/blob/develop/src/php/DataSift/Storyplayer/PlayerLib/StoryContext.php#L213-L214

en0 and en1 are generally used for first two physical interfaces. On most Macs that will be built-in WiFi and built in Ethernet port.

So story player works great for me if I am connected to WiFi. But at my workstation when I plug in cinema display with it's fixed Ethernet cable (en4), story player stops working. Actually that's not quite true - it works as long as wifi is turned on (even though I'm actually using ethernet connection). Same would happen with any other network device (Bluetooth is en3, other USB or similar devices would presumably have higher numbers)

The simplest thing that would avoid confusing others in future woul be to bump up the en* ones tried in the line linked above. up to en4 would work for me I'd assume having a cinema display is not that uncommon (perhaps turning of WiFi is...). Not sure how you would decide where to stop though...

Dunno if you want to bump up the numbers you check to catch this case? It's probably rareish but it's pretty confusing when it comes up. Seems less than ideal that changing wifi status when I'm not even using WiFi makes my test framework stop running...

issue while installing storyplayer : storyplayer config file 'storyplayer.json.dist' not found or unreadable

Hi..
When i am trying to run this command :./vendor/bin/storyplayer install

i am getting this error :

*** warning: storyplayer config file 'storyplayer.json.dist' not found or unreadable
PHP Warning: count(): Parameter must be an array or an object that implements Countable in D:\xampp-7.3.5\htdocs\dmsl-api\vendor\datasift\storyplayer\src\php\DataSift\Storyplayer\Injectables\DefaultSystemUnderTestName.php on line 63

Warning: count(): Parameter must be an array or an object that implements Countable in D:\xampp-7.3.5\htdocs\dmsl-api\vendor\datasift\storyplayer\src\php\DataSift\Storyplayer\Injectables\DefaultSystemUnderTestName.php on line 63
PHP Fatal error: Declaration of DataSift\Storyplayer\Cli\ListSystems_Switch::process(Phix_Project\CliEngine $engine, $invokes = 1, $params = Array, $isDefaultParam = false) must be compatible with Phix_Project\CliEngine\CliSwitch::process(Phix_Project\CliEngine $engine, $invokes = 1, $params = Array, $isDefaultParam = false, $additionalContext = NULL) in D:\xampp-7.3.5\htdocs\dmsl-api\vendor\datasift\storyplayer\src\php\DataSift\Storyplayer\Cli\ListSystems\Switch.php on line 104

Fatal error: Declaration of DataSift\Storyplayer\Cli\ListSystems_Switch::process(Phix_Project\CliEngine $engine, $invokes = 1, $params = Array, $isDefaultParam = false) must be compatible with Phix_Project\CliEngine\CliSwitch::process(Phix_Project\CliEngine $engine, $invokes = 1, $params = Array, $isDefaultParam = false, $additionalContext = NULL) in D:\xampp-7.3.5\htdocs\dmsl-api\vendor\datasift\storyplayer\src\php\DataSift\Storyplayer\Cli\ListSystems\Switch.php on line 104

Release 1.3.0

Just a note that keeps the 1.3.0 milestone open until we release the code :)

[Feature] Default environment support

Requested by mheap ...

He'd like Storyplayer to use the machine's hostname as the default test environment, when no environment is specified.

I'm going to implement this by adding a config setting (something like options->hostnameAsDefaultEnvironment).

[Feature] Add 'list-environments' command

New: ability to list the different test environments that are defined in the config files (including all of the additional config files)

Release 1.1.0

A reminder to release 1.1.0 with accumulated bug fixes and new features.

[Feature] Per-device config files

At the moment, if users want to test with alternative browsers, they have to use a lot of command-line options. This greatly reduces the repeatability of the tests, because it's too easy to make mistakes.

We will add per-device config files, patterned after per-environment config files:

• -d switch to select which device's config to load
• a 'devices' top-level section
• the per-device config will be loaded after the environment
• new API to make the per-device config available to the Story
• StoryTeller will use the new per-device section when building up $browserDetails

I wish I'd thought of this for v1.4, as it will make SauceLabs and the experimental RemoteWebDriver a lot easier to reuse!

[Feature] Install Storyplayer via composer

New: Install Storyplayer + it's dependencies via composer

Test Environment Setup / Teardown Script Usage

Hello Stuart

At DataSift, we are evaluating whether to move to a later version of Storyplayer for writing new tests. Would you consider the most recent current develop branch (on Mar 18, 2016) to be stable for production use or is it a work-in-progress?

Currently, we are using the test-environments .json file to specify the target server details. In the latest version, it no longer uses a .json file for defining this configuration. Are there any guides or examples to illustrate how to use the new ...Env.php target files?

Vincent

Support for phpunit 5.x

Hello Stuart,

Is it possible for Storyplayer to be compatible with phpunit 5.x? I would like to use it with php 7 which forces me to switch to phpunit 5.

Best regards,
Adrian

Storyplayer v2 doesn't support HTTPS requests

Any requests using the functions FromHTTP or UsingHTTP don't work when using htttpS addresses.

Always get this response when hitting our API....

{"type":1,"httpVersion":"1.1","statusCode":"400","statusMessage":"Bad Request","body":"\r\n<title>400 The plain HTTP request was sent to HTTPS port</title></head>\r\n<body bgcolor="white">\r\n","chunks":[],"frames":[],"bytesRead":351,"rawResponse":"","responsesCount":0,"errorMsgs":[],"mustClose":false,"combineChunks":true,"headers":{"Server":"nginx","Date":"Mon, 01 Dec 2014 15:31:44 GMT","Content-Type":"text/html","Content-Length":"264","Connection":"close"}}

[Feature] Multi-window support

Storyplayer needs to be able to switch between open browser windows. We can't automate testing Managed Sources until this is done.

The proposed syntax is:

$st->usingBrowser()->switchWindow($titleOrId);

This will POST to /session/:sessionId/window.

Storyplayer install is not handling DNS errors

Additional files will be added to the vendor/ folder
PHP Warning: get_headers(): php_network_getaddresses: getaddrinfo failed: System error in /home/courtney/projects/DataSift/Clobber/vendor/datasift/storyplayer/src/php/DataSift/Storyplayer/Cli/InstallCommand.php on line 111
PHP Warning: get_headers(http://chromedriver.googlecode.com/files/chromedriver_linux64_2.1.zip): failed to open stream: php_network_getaddresses: getaddrinfo failed: System error in /home/courtney/projects/DataSift/Clobber/vendor/datasift/storyplayer/src/php/DataSift/Storyplayer/Cli/InstallCommand.php on line 111
PHP Warning: array_change_key_case() expects parameter 1 to be array, boolean given in /home/courtney/projects/DataSift/Clobber/vendor/datasift/storyplayer/src/php/DataSift/Storyplayer/Cli/InstallCommand.php on line 111
Downloading: http://chromedriver.googlecode.com/files/chromedriver_linux64_2.1.zip (0mb)

Unable to have any .json files in test-environment folder

Hey Stuart,

Here at Datasift we recently experimented with the newer test-environment method of having a unique dsbuild/vagrant/config etc within each test environment folder and came across this blocking issue.

If you have any .json files within a test-environment folder storyplayer they are incorrectly assumed to be storyplayer config files and cause storyplayer to fail because they aren't valid. This is the error that is thrown...

*** error: environment has no 'groups' array

This was caused by an unexpected exception DataSift\Storyplayer\TestEnvironmentsLib\E4xx_TestEnvironmentNeedsGroups

#0 /Users/davidshrive/storyplayer/src/php/DataSift/Storyplayer/ConfigLib/ConfigList.php(195): DataSift\Storyplayer\TestEnvironmentsLib\TestEnvironmentConfig->validateConfig()
#1 /Users/davidshrive/storyplayer/src/php/DataSift/Storyplayer/ConfigLib/ConfigList.php(165): DataSift\Storyplayer\ConfigLib\ConfigList->loadJsonConfig('storyplayer/tes...')
#2 /Users/davidshrive/storyplayer/src/bin/storyplayer(255): DataSift\Storyplayer\ConfigLib\ConfigList->findConfigs()
#3 /Users/davidshrive/storyplayer/src/bin/storyplayer(361): main(Array)
#4 {main}

Here is a link to the file with the issue -> Link

and I've created a branch to demo the issue, just try and run the storyplayer test suite on the following branch...

https://github.com/datasift/storyplayer/tree/bug/unable-to-have-json-in-test-environment

This occurs on develop and 2.3.2.

Thanks

[Feature] Download Storyplayer as a phar

Download Storyplayer + all it's dependencies as a phar file. Any non-php dependencies can be downloaded via a command (e.g. storyplayer install-deps) once command support is added.

Storyplayer binary missing when installing via composer

I noticed that when installing via composer with:

{
    "require": {
        "datasift/storyplayer": "1.3.3"
    }
}

You get an error: "Skipped installation of src/bin/storyplayer for package datasift/storyplayer: file not found in package".

I noticed under the git tags, the src/bin/storyplayer script has been renamed to storyplayer.REMOVED.git-id. What should I do here?

[Feature request] Add support for -v flag

When running Storyplayer from a git checkout, I don't know what version I'm currently running. Being able to run storyplayer -v would be very useful

[Feature] Add Ec2 support

Add the ability to use EC2 as a supported test environment.

Fix: reduce dev mode output

To make the output easier to read, we need to reduce the amount of it.

• No longer display the value when we store/retrieve from the checkpoint
• No longer display HTTP responses
• Make the output more diff friendly by switching to relative timestamps rather than absolute ones (or at least make it an option)

Do we want to write the suppressed data into a machine-readable file via the output plugins?

Please add support for Arch Linux

[alan@iss ~]$ cat /etc/issue
Arch Linux \r (\l)

[alan@iss ~]$ ifconfig
eth0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:30:67:cb:5d:35 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
RX packets 10 bytes 1068 (1.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 10 bytes 1068 (1.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.100 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::6a1c:a2ff:fe00:63ae prefixlen 64 scopeid 0x20
ether 68:1c:a2:00:63:ae txqueuelen 1000 (Ethernet)
RX packets 38632 bytes 25285916 (24.1 MiB)
RX errors 0 dropped 5753 overruns 0 frame 0
TX packets 21961 bytes 4182584 (3.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Thank you!

Update text processing to use PHP in HostLib

Using awk has worked so far, but it's not as portable as it could be. If we move text processing to be performed in PHP then it will work identically on OSX and Linux (and maybe even Windows if we ever get that far)

[Feature] Ability to close a browser window

Once we have multi-window support in Storyplayer, we should (for completeness) add the ability to close the current browser window.

This is done by calling DELETE /session/:sessionId/window.

Fix: simplify the story checkpoint

At the minute, debugging stories is made more complicated because the StoryCheckpoint object contains $st - this makes a var_dump() of the checkpoint too large to read.

Simplify StoryCheckpoint by stripping out its reliance on $st.

Docs: add Contributing instructions

I need to add a page explaining how to contribute to the Storyplayer code and/or docs.

Bad Link to See installing Storyplayer.

Please correct the link to "See installing Storyplayer."

Thanks!

[Feature] Add -L to set min log level