mdsol / kender

Kender is a library of rake tasks for continuous integration.

Kender

Kender provides a consistent framework for continuous integration (CI). The principles of Kender are:

• The definition of what constitutes your CI runs is dependent on your code, so it should be managed alongside that code and not isolated in your CI server's configuration.

• CI runs are executed the same way, wherever they run, whoever runs them. Developers should be able to execute the same, simple command on their local machines to execute a CI run as is used on a dedicated CI server. No more publishing a branch that requires a CI configuration change, unsure of whether the CI server will pass or fail.

• CI runs are executed the same way on every branch. A single CI project configuration should be suitable for every branch. No more mysterious failures on the CI server due to branch changes being incompatible with the current CI project configuration.

The main benefit of Kender is that all test related software (e.g. Cucumber, RSpec, etc.) is run by default.

Usage

Add the following to your Gemfile:

group :development, :test do
  gem 'kender', '~> 0.5'
end

This gem and its dependencies should not be used in production, hence the group above.

Configuring and running

The rake ci task can be run locally in precisely the same way as it should be on a CI server:

rake ci

This executes the three sub-tasks ci:config, ci:run and ci:clean. Each of these can be run in isolation.

Configuration typically requires values to be provided. Using dice_bag, this can be done through environment variables which can be passed in the same command line. For example:

DATABASE_USERNAME=root DATABASE_PASSWORD=password DATABASE_NAME=test rake ci

The ci:config rake task creates a DB and configures the project. It will execute the following tasks if they are defined:

• config:deploy
• db:migrate
• db:create
• db:drop

config:deploy is defined in the DiceBag gem. It will always overwrite the configuration files with the values in the templates.

The db tasks are assumed to work like those found in Rails.

Run the following command to see the new tasks:

[bundle exec] rake -D ci

Running the tests without configuration

To bypass any configuration and clean-up side effects, for example if your application is already configured, execute just the ci:run task.

Non-Rails environments

If you are using these Kender tasks outside of a Rails project, add the following to your Rakefile or wherever your local rake tasks are defined:

require 'kender/tasks'

Configuring what is included in the CI run

Kender will detect the test-related gems in your Gemfile and execute whatever CI-related commands make sense. Just ensure the gem is available in at least your test and development environments.

Currently supported gems are:

Cucumber

The Cucumber features will be run. Normally no command-line parameters or switches are passed, so ensure your default profile is correct for a CI run. If the cucumber command fails, the CI run will fail.

If you want to run scenarios that require a headed web browser, you can tell Kender to use a specific browser as part of the CI run. You can set the environment variable HEADED_BROWSER to the name of the browser you want to run. Verify your project can use the HEADED_BROWSER environment variable.

Cucumber scenarios can be spread across multiple nodes by using using the following gems:

• knapsack
  • The following environment variables must be accessible when using travis-ci:
    • CI_NODE_TOTAL
    • CI_NODE_INDEX
• parallel_tests

RSpec

The RSpec specs will be run. No command-line parameters or switches are passed, so ensure your defaults in .rspec are correct for a CI run. If the rspec command fails, the CI run will fail.

Rspecs can be spread across multiple nodes by using using the following gems:

• knapsack
  • The following environment variables must be accessible when using travis-ci:
    • CI_NODE_TOTAL
    • CI_NODE_INDEX

Jasmine

The Jasmine rake task jasmine:phantom:ci will be run. If the task fails, the CI run will fail.

Additionally, you must have PhantomJS pre-installed on the system doing the CI run.

Brakeman

The Brakeman command is run in quiet mode. If any warnings are generated, the CI run will fail.

Bundler Audit

The Bundler-audit check command is run. If any checks fail, the CI run will fail.

Reek

The Reek command is run in quiet mode. The CI run will not fail, regardless of the output.

Consistency Fail

The Consistency Fail command is run. The CI run will not fail, regardless of the output.

I18n Tasks

The I18n Tasks commands to check for both missing and unused translations are executed. The CI run will not fail, regardless of the output.

Shamus

The Shamus command is run. If the command fails, the CI run will fail.

When Shamus is used, RSpec, Jasmine and Cucumber are not run directly by Kender but delegated to Shamus instead. As you may not want to run Shamus by default in a CI context, you must set the environment variable VALIDATE_PROJECT:

rake VALIDATE_PROJECT=true ci

FactoryGirl Lint

The FactoryGirl lint command is run. If the command fails, the CI run will fail. If there is no factory_girl_lint.rake file in the lib/tasks directory of the project, the command will not be run.

Cypress-Rails

Your application should include the following task rake cypress_extended:enhanced_run which should call rake cypress:run from Cypress-Rails.

namespace :cypress_extended do
  task enhanced_run: [:environment] do
    # Do other stuff

    # Run the Cypress-Rails task
    Rake::Task['cypress:run'].invoke
  end
end

Contributors

• Andrew Smith
• Jordi Carres
• Mathieu Jobin
• Will Duty
• Michael West
• Ezra Jennings

kender's Issues

Cleaning up test.log

Our Jenkins box logs keep accumulating MB of data. Would it make sense to put log clearing in rake ci?

Kender suppressing system call errors

When system call (bundle exec shamus) execution fails, kender is suppressing the reason behind the failure and just saying Run time error with an error code ( https://github.com/mdsol/kender/blob/master/lib/kender/command.rb#l24), instead it would be quite useful if the reason for the failure is printed out. I looked up on the net and found the following a better way to handle system call errors.

begin
  %x[system_call]
rescue Exception => e
  puts e.message
end

support for rails 2

Should we support rails2 projects?
On one hand imedidata is still rails2, on the other hand we have rails4 coming soon...

GitHub commit status should work if Jenkins set to merge feature branch into integration (develop) branch

Currently we create GitHub commit statuses against the current HEAD. In Jenkins you can get it to do a prospective merge to an integration branch (typically develop for us), which moves HEAD, causing the status creation to fail.

One possible solution would be to use the Jenkins provided GIT_COMMIT environment variable and maybe default to HEAD otherwise.

can't take kender out of the development group

since kender is used for automated testing, I was hoping I could keep it in the test section.

but it requires to be in development to be run. or the tasks are not found.

ci:status:pending at least, could not be found.

problem loading libraries with 0.1.2

Checkmate had to switch back to 0.1.1

WebMock was not loaded when using 0.1.2

run coverage tool

It would be cool this gem gives you coverage reports also.
But it seems that simplecov needs to be required in the env.rb or whatever helper file used in tests. How can we do that automatically in this gem?

Add the ability to upload build artifacts after a build

For example: upload generated shamus portal to s3 (using e.g. s3cmd).

kender 0.1.5 can no longer update its status to github

this is different from the previous issue. but I'm getting this error now, trying to run rake ci

Setting mdsol/checkmate commit 795f9afe6b5177f6031be2998007c825f4f5427d status to 'pending' on GitHub
rake aborted!
Real HTTP connections are disabled. Unregistered request: POST https://api.github.com/repos/mdsol/checkmate/statuses/795f9afe6b5177f6031be2998007c825f4f5427d?access_token=f0e51680ba0a472c7eb87178f495507b60ce1439 with body '{
        "state": "pending",
        "target_url": "http://ec2-75-101-178-190.compute-1.amazonaws.com/job/Checkmate%20Feature%20Branches/698/",
        "description": "Continuous integration run 698"
      }' with headers {'Accept'=>'*/*', 'User-Agent'=>'Ruby'}

You can stub this request with the following snippet:

stub_request(:post, "https://api.github.com/repos/mdsol/checkmate/statuses/795f9afe6b5177f6031be2998007c825f4f5427d?access_token=f0e51680ba0a472c7eb87178f495507b60ce1439").
  with(:body => "{\n        \"state\": \"pending\",\n        \"target_url\": \"http://ec2-75-101-178-190.compute-1.amazonaws.com/job/Checkmate%20Feature%20Branches/698/\",\n        \"description\": \"Continuous integration run 698\"\n      }",
       :headers => {'Accept'=>'*/*', 'User-Agent'=>'Ruby'}).
  to_return(:status => 200, :body => "", :headers => {})

============================================================/var/lib/jenkins/.rvm/gems/ruby-1.9.3-p327@checkmate/gems/webmock-1.11.0/lib/webmock/http_lib_adapters/net_http.rb:107:in `request'
/var/lib/jenkins/.rvm/gems/ruby-1.9.3-p327@checkmate/bundler/gems/kender-cf9ffe3c5424/lib/kender/github.rb:47:in `block in update_commit_status'

Make run strategies pluggable

Currently we have Medidata-specific run strategies, like using Shamus, baked into this gem. Kender should support pluggable strategies for the run so that we can factor Medidata-specific part into their own gem and open source the base Kender gem.

Successful runs on an integration branch should prune merged branches

Using GitHub pull requests results in a large number of stale feature branches being left behind. CI runs should prune these merged branches automatically when appropriate.

The link to Shamus in the readme is dead

Hi,

The readme points to https://github.com/mdsol/shamus, but this link 404s. I'm curious to know what this project is about. Did it move somewhere else, or is it abandoned?

Executing 'rake ci' is deleting my database...why?

I noticed that running rake ci is dropping my test database which forces me to run RAILS_ENV=test rake db:create; RAILS_ENV=test rake db:migrate immediately after I run it. Looking through the ci task definition I see that during ci:clean it explictly removes your test database. How about using DatabaseCleaner if it's available instead? Thoughts?

support for metric_fu

to do like what code climate does...

after running specs and scenarios, it would be nice to have metric_fu run on our code.
so we have code smell being raised up and suggestions on how to improve our code.

http://metric-fu.rubyforge.org/
http://ruby.sadi.st/Ruby_Sadist.html

Document that brakeman task can be configured locally with .breakman file

This issue is migrated from and closes mdsol/dice_bag#3.

rake ci to show all pending TODO comments

We, developers, add TODOs to our code constantly. but very rarely look at it.

I remember when using IDE software, one of the feature is to have your list of TODO handy.

CI could report those back to us as well

Description of VALIDATE_PROJECT should be added to the README

Kender 1.5 Has Killed Our CI

Maybe we need to set something up different, but the documentation could be clearer.

Add HipChat notification

To complement GitHub PR status notifications, also support HipChat notifications. There's evidence that people are wasting a lot of time struggling to get the Jenkins plugin to work; we can centralize the solution here.

when running rake ci - option to tell dice_bag not to overwrite my config.

when I need to generate new config, I always do rake config:all

carefully check of diff and accept or not the changes. I'm very cautious about changes that could happen in my repo. so, I am not tracking ghost.

anyway. when running rake ci, I believe everything will get overwritten...

I wish to use rake ci without making any changes to my config, or to only apply it to new files.

is that already possible?

thanks

OpenSource kender

I see kender repo is still private?
is there mdsol specific stuff to clean up?
is it planned to get it opensourced, right?

rake task for security

create a security:check rake task to run brakeman or any other tool in isolation

Check for missing unique constraints

I thought you might be interested in the gem Consistency Fail, which will scan for missing uniqueness constraints in the database. It's got some limitations: it doesn't parse has_one :through correctly, and the output format isn't very useful for writing a migration, but it might be nice to have a test to make sure we're not using the thread-unsafe validates_uniqueness_of or has_one methods without db enforcement.

Add task to generate GitHub auth token

One of the coolest features of Kender is it makes it easy to post build status to GitHub for use in pull requests. Currently we document how to generate the necessary auth token in the README but we could go further and add a rake task that automates this.