dannycoates / able Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 4.0 78 KB

A/B testing service

Home Page: http://dannycoates.github.io/able/

JavaScript 100.00%

able's People

Contributors

Stargazers

Watchers

Forkers

jrgm vladikoff pdehaan tda

able's Issues

Bump to latest version of browserify

From https://nodesecurity.io/advisories/117

Switch to official github-url-to-object module

I believe that github-url-to-object added clone_url support in github-modules/github-url-to-object@8bd5878 (published as v2.1.0).

We can probably switch from @vladikoff's fork to the core module.

add a db 'type' parameter for future use

Could default to "local" for now, but as this grows, we'll want to be able to spec other storage options/types

Integrating metric events

For A/B experiments to actually work we need a way to analyze the data, but before we can even do that we need a way to report the data.

Right now all that exists is able.report() which will give you some data for each experiment you're enrolled in; the experiment name and the independentVariables:values that were chosen for each subject (usually only one).

{
  "name":"myExperiment",
  "choices": {
    "1a4b2d54c3e": {
      "buttonText": "why not?"
    }
  }
}

That's useful for knowing how many subjects got each value, but its not enough to do anything. Somehow we need to link choices to events that are relevant to the experiment.

In the most naive way I think it would be nice to define which events my experiment is interested in tracking and then have the choices linked to those events.

Borrowing from Shane's example I'd like to add events

module.exports = {
  name: 'signInButtonTextMatters',
  hypothesis: 'The sign in button text affects signins',
  startDate: '2015-01-01',
  subjectAttributes: ['lang'],
  independentVariables: ['signInButtonText'],
  eligibilityFunction: function (subject) {
    return /en-US/.test(subject.lang);
  },
  groupingFunction: function (subject) {
    return {
      signInButtonText: this.uniformChoice([
        this.defaults.signInButtonText,
        'Come on in'
      ]);
    };
  },
  events: ['signInClicked']
};

So, whenever the signInClicked event fired, in addition to whatever it normally logs the "report" for that experiment would get logged so that we can correlate the choices with the event.

This means the experiment author will need to know what events are available to track, just as they need to know the independentVariables.

Gluing Able to whatever generates and logs these events will be an Issue for another day, but I'm wondering if a simple list of events is enough or do we need something more powerful?

@kparlante @shane-tomlinson

Add Travis CI to this repository

We should add Travis to this repo so run tests

Issue with using branches other than 'master', cannot clone.

Seems like the https_url is not the right one to use here: https://github.com/dannycoates/able/blob/master/project.js#L25

Project.prototype.pull = function (cb) {
  gitUtil.cloneOrFetch(
    this.git.https_url,
    this.root,
    this.git.branch,
    cb
  )
}

{ user: 'mozilla',
  repo: 'fxa-content-experiments',
  branch: 'dev',
  tarball_url: 'https://api.github.com/repos/mozilla/fxa-content-experiments/tarball/dev',
  https_url: 'https://github.com/mozilla/fxa-content-experiments/tree/dev',
  travis_url: 'https://travis-ci.org/mozilla/fxa-content-experiments?branch=dev',
  api_url: 'https://api.github.com/repos/mozilla/fxa-content-experiments' }

git clone -b dev https://github.com/mozilla/fxa-content-experiments/tree/dev /Users/vladikoff/mozilla/fxa-local-dev/fxa-content-server/experiments
fxa-content-server.route.500.ERROR: Error: Command failed: Cloning into '/Users/vladikoff/mozilla/fxa-local-dev/fxa-content-server/experiments'...
fatal: repository 'https://github.com/mozilla/fxa-content-experiments/tree/dev/' not found

It tries to clone https://github.com/mozilla/fxa-content-experiments/tree/dev which is not a repository :(

$ nsp check
(+) 2 vulnerabilities found
┌───────────────┬──────────────────────────────────────────────────────────┐
│               │ Incorrect handling of CORS preflight request headers     │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Name          │ hapi                                                     │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Installed     │ 9.3.1                                                    │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Vulnerable    │ <11.0.0                                                  │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Patched       │ >=11.0.0                                                 │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Path          │ able > hapi                                              │
├───────────────┼──────────────────────────────────────────────────────────┤
│ More Info     │ https://nodesecurity.io/advisories/45                    │
└───────────────┴──────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────┐
│               │ Regular Expression Denial of Service                     │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Name          │ uglify-js                                                │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Installed     │ 2.4.24                                                   │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Vulnerable    │ All                                                      │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Patched       │ None                                                     │
├───────────────┼──────────────────────────────────────────────────────────┤
│ Path          │ able > uglify-js                                         │
├───────────────┼──────────────────────────────────────────────────────────┤
│ More Info     │ https://nodesecurity.io/advisories/48                    │
└───────────────┴──────────────────────────────────────────────────────────┘

I filed the Hapi issue upstream as hapijs/hapi#2896. Not sure if there is anything super-actionable we can do with it currently, unless we migrate to hapi@11 (which I believe requires Node 4+).

Update convict and uglify-js dependencies

Ref: mozilla/fxa-content-server#3693

Currently the package.json file hardcodes to [email protected], which [unfortunately] has a non-kosher version of moment.

We should upgrade to the latest version of convict to appease the NSP gods.

Bump to latest version of glob

Need to bump to latest version of glob once isaacs/node-glob#268 is fixed.