Acceptance Tests / CI about todos

[samhatoum]

@tmeasday let us know if you want to pair on this at all, @sanjo and I would be happy to help

[tmeasday]

@samhatoum that sounds useful; let me have a go at them myself (perhaps sometime this week), and get back to you.

[tmeasday]

OK, I built a first acceptance test, a few questions @samhatoum

1. Is there a way to specify the URL to connect to on the commandline? This line seems bad if i want to run the tests pointing somewhere else: 8e8a712#diff-98f10856893a7c43b98eb1f461e47173R12
2. Is there a way to set the inter-step timeout? (ideally on the commandline). In the past I've tended to need set a small delay between steps to let animations etc play out (see 8e8a712#diff-98f10856893a7c43b98eb1f461e47173R16)
3. It'd be nice to be able to write ES2015. I guess this is a bit out of scope though ;)

[samhatoum]

Is there a way to specify the URL to connect to on the commandline? This line seems bad if i want to run the tests pointing somewhere else: 8e8a712#diff-98f10856893a7c43b98eb1f461e47173R12

I would create a hooks file and use a before/beforeEach hook and have that to go the url of the app. It is very possible when doing acceptance tests to have an app that is distributed over many locations. You might have two apps running in a given setup, like a marketing/registration app running, and the main app. Another use-case is to use smoke-tests, where you might send an email from your app, then actually go and check it arrived in a webmail client.

Another point to bear in mind is resetting data. I tend to clear state and revisit the app between tests to ensure a completely clean slate. I use our cleaner package. It's a good practice to clean then reload the page between specs.

We could implement it that Chimp automatically navigates to the app when a URL is provided on the command line, though it may not always be desirable, especially if no data has been setup yet.

Is there a way to set the inter-step timeout? (ideally on the commandline). In the past I've tended to need set a small delay between steps to let animations etc play out (see 8e8a712#diff-98f10856893a7c43b98eb1f461e47173R16)

I believe --timeoutsImplicitWait is what you want. We already set that to a default of 3000ms. Though what is happening in your case is that the .list-todo is visible. You might want to wait for the list length to change instead using waitUntil. This is a common UI testing gotcha!

It'd be nice to be able to write ES2015. I guess this is a bit out of scope though ;)

Already supported out the box. Just write ES6! You can use Mocha's standard --compilers=es6:babel-core option on the cli too.

Also, we added support for config files recently, so if you want to save writing command-line args all the time, create a chimp.js file locally and put this in it:

module.exports = {
  // - - - - CHIMP - - - -
  watch: false,
  watchTags: '@focus',
  mocha: true,

  // - - - - MOCHA - - - -
  path: './tests',
  // use other compilers
  // compilers: 'es6:babel-core/register',

  // '- - - - DEBUGGING  - - - -
  // log: 'info',
  // debug: false,
  // seleniumDebug: false,
  // webdriverLogLevel: false,
  // debugBrkMocha: 5858,
};

I will soon document all the options, and you can look here for a reference of what is supported.

Let me know if you need anything else.

[samhatoum]

Be sure to be using v0.28.2

[tmeasday]

We could implement it that Chimp automatically navigates to the app when a URL is provided on the command line, though it may not always be desirable, especially if no data has been setup yet.

Fair enough. It might be useful to be able to set something via commandline that you can read in the script however. That would make it simpler to point my test at different deployments of the app.

Already supported out the box. Just write ES6!

I seem to get errors about () => {} (Unexpected token ). With or without --compilers=es6:babel-core

Also, we added support for config files recently

Did you consider using a field in package.json for this? I think quite a few other node tools use this convention for configuration. (adding a chimp.js file is going to mess up my project right?)

[samhatoum]

It might be useful to be able to set something via commandline that you can read in the script however. That would make it simpler to point my test at different deployments of the app.

Would env vars work? like MY_URL=hello chimp, and then use process.env.MY_URL

I seem to get errors about () => {} (Unexpected token ). With or without --compilers=es6:babel-core

I might have spoken too soon! It works for Cucumber out the box, I'll add it to Mocha shortly.

Did you consider using a field in package.json for this? I think quite a few other node tools use this convention for configuration. (adding a chimp.js file is going to mess up my project right?)

Good idea to support config in package.json. You can also put the file under .config/chimp.js so that Meteor ignores it, then pass the file to chimp as the first param, like this:

chimp .config/chimp.json <other options>

@tmeasday I have added ES2015 support in mocha tests in chimp 0.29.0. No options needed.