alexb52 / retest

A simple CLI to watch file changes and run their matching ruby specs. Works on any ruby projects with no setup.

Retest

Retest is a small command-line tool to help you refactor code by watching a file change and running its matching spec. Designed to be dev-centric and project independent, it can be used on the fly. No Gemfile updates, no commits to a repo or configuration files required to start refactoring. Works with every Ruby projects (at least that is the end goal)

Demo

retest-demo.mp4

Installation

Install it on your machine without adding it on a Gemfile:

$ gem install retest

Usage

Retest is used in your terminal after accessing your ruby project folder.

Help

Find out what retest can do anytime with

$ retest -h

For Refactoring

1. Run a hardcoded command

This the most simple usage of retest: running the same command over and over after each file update.

Example:

$ retest 'bundle exec rspec spec/features/posts_spec.rb'

In this example, the feature spec spec/features/posts_spec.rb will be tested after any ruby file is updated.

2. Run a dynamic command with placeholders

Retest provides few placeholders to help you run a command after every file change. The placeholders can be used on their own or together.

1. <test> placeholder

You can use the placeholder <test> to tell the gem where to put the test file path in your command. When a file is changed, the gem will find its matching test and run the test command with it.

Example:

$ retest 'bin/rails test <test>'

In this example, if app/models/post.rb is changed then retest will run bin/rails test test/models/post_test.rb

2. <changed> placeholder

You can use the placeholder <changed> to tell the gem where to put the changed file path in your command. When a file is changed, the gem will run the command with it.

Example:

$ retest 'rubocop <changed>'

In this example, if app/models/post.rb is changed then retest will run rubocop app/models/post.rb

3. Run a dynamic command with shortcuts

Few shortcut flags exist to avoid writing the full test command.

$ retest --rspec
$ retest --rails
$ retest --rake --all

4. Let retest figure it all out

Let retest find your ruby setup and run the appropriate command using:

$ retest
$ retest --all

Running rules

The gem works as follows:

• When a ruby file is changed, retest will run its matching test.
• When a test file is changed, retest will run the test file.
• When multiple matching test files are found, retest asks you to confirm the file and save the answer.
• When a test file is not found, retest runs the last run command or throw a 404.

Pull request scans

You can diff a branch and test all the relevant test files before pushing your branch and trigger a full CI suite.

$ retest --diff origin/main

In this example, retest lists all the files changed between HEAD and origin/main, finds all the relevant tests and only run those.

Why?

It is advised to be one cmd + z away from green tests when refactoring. This means running tests after every line change. Let Retest rerun your tests after every file change you make.

Retest gem is meant to be simple and follow testing conventions encountered in Ruby projects. Give it a go you can uninstall it easily. If you think the matching pattern could be improved please raise an issue.

For fully fledged solutions, some cli tools already exists: autotest, guard, zentest

Docker

Retest works in Docker too. You can install the gem and launch retest in your container while refactoring.

# Enter your container. Ex:
$ docker-compose run web bash

# Install the gem and run retest in your container shell
$ gem install retest
$ retest 'bundle exec rails test <test>'

Disclaimer

• If an error comes in try using bundle exec like so: $ retest 'bundle exec rake test <test>'
• Aliases saved on ~/.bashrc or ~/.zshrc cannot be run that way with the retest command

Ruby Support

Retest supports ruby 2.5 and above.

Roadmap

• MVP
• When multiple test files are found, ask which file to run and save the answer.
• When a test file is not found run the last command again.
• Run within Docker.
• Handle main Ruby setups
  • Bundler Gem
  • Rails
  • Ad-hoc scripts
  • Hanami
• Handle other languages: Go, Elixir, Node, Python, PHP
  • Go (project started)
• Aliases from oh-my-zsh and bash profiles?

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

To run integration tests on one setup (ex: hanami-app): bin/test/hanami-app

To access an app container (ex: ruby-app): docker-compose -f features/ruby-app/docker-compose.yml run retest sh

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/alexb52/retest.

License

The gem is available as open-source under the terms of the MIT License.

retest's Issues

Printing help in a docker container doesn't work

when using retest --help the usage text is not printed although it prints fins when not in a docker container

Print help option

This gem is fantastic, thank you! ❤️

Hi @AlexB52, I love retest! I'm using it with the following bash alias for Rails and non-Rails projects:

alias rtt="if [ -x bin/rails ]; then retest 'bin/rails test <test>'; else retest 'bundle e rake test TEST=<test>'; fi"

Feel free to close this issue. Just wanted to say thanks!

[Possible Bug] Spec file path not getting cached when using <test> ?

So I have a strange issue. I run retest in the docker container with the command

retest 'spring retest rspec <test>'

Running rspec with spring to speed up test initialization works fine.
Also, doing a save on any "*_spec.rb" file immediatelly reruns the spec. This is also fine.

However, when saving changes on an "*.rb" file, it takes about 5 seconds for the corresponding spec to run each time. Spring seems to work fine (spec starts executing, Spring reports it being preloaded, but only after the initial 5 seconds), so I was thinking - is it possible the file path to the spec file is not cached, and retest needs to search for the file each time we save?

Rails 6.1.3.2
Ruby 3.0.3

[Feature Request] Terminal notification

It would be nice to be notified when fails are passing or failing and the terminal window is not being displayed.

Using = in the command breaks the command recognition

This is really specific problem that shouldn't cause a problem right now.

When calling retest 'rake test =<test>' the output is:

Setup identified: [RAKE]. Using command: 'bundle exec rake test TEST=<test>'
Launching Retest...
Ready to refactor! You can make file changes now

The command is not recognised and the default setting kicks in. Passing a = is what makes the command fail.

Do not track changed files listed in .gitignore

When using sqlite in a project like Hanami. Following the guides break:
https://guides.hanamirb.org/introduction/getting-started/

If a file is changed when running retest 'bundle exec rake' the tests will change the sqlite database file which will trigger a changed file event. The rake task is run again modifying the sqlite file which then triggers another spec run... Making it a infinite loop

We should not listen to changes applied on files listed in .gitignore. This should not happen with a database that is not stored as a file like a postgresql or mysql setup.

Instead of excluding files from .gitignore it seems easier (maybe slower) to include only the files git tracks using git ls-files
Maybe something like tis

files = `git ls-files`.gsub("\n", "|")
listener = Listen.to('.', only: files, relative: true) { |modified, added, removed| # ... }

Or send the git ignored patterns in the exclude option of Listen

Or maybe just add sqlite to the ignored patterns 😂

The bug also happens when using byebug and updating the .byebug_history file

Handle new file changes

Currently retest is not trying to run a spec on a new created file. Whether it is with git or not.

Once a new file is added to the repository and showed as changed, we should add that file to the array of files to search from

[FEATURE request] Listen to file changes and rerun all the spec from branch diffs

none/skip option for which file question

Use Case

Repository files sometimes share the same name and is difficult to assess. For some use cases, ex: updating a factory file, retest tries to find a matching file which is sometimes wrong.

The feature is to add a none options where a user can say don't choose any files here because none of them match the one I just modified.

        We found few tests matching: app/models/valuation/holdings.rb
        [0] - test/models/taxation/holdings_test.rb
        [1] - test/models/schedule/holdings_test.rb
        [2] - test/models/performance/holdings_test.rb
        [3] - test/models/holdings_test.rb
        [4] - test/lib/csv_report/holdings_test.rb
        [5] - none matches, skip this
        # Add this ^

        Which file do you want to use?
        Enter the file number now:

Create an End to End Test Spec

Check for gems in Gemfile.lock instead of Gemfile

In rake projects for a gem for example. Dependencies are defined in both Gemfile AND gemspec. The Gemfile.lock is the only file that bundles list all the gems in one spot.

Seems easier to use this lock file to find the correct ruby setup.

• pundit
• devise
• tty-prompt

Move feature repositories to Ruby 2.7 and update gems support for 2.6 and above

The EOL for 2.6 comes at the end of March.

• We need to update retest gem to support 2.6+
• We need to update the feature repositories to Ruby 2.7. List of repositories:

• ruby-app
• ruby-bare
• git-ruby
• hanami-app
• rails-app
• rspec-rails
• rspec-ruby

I still wonder if we need this. As long as we support 2.4 then we should be good.

Create an app using ruby 3 could be helpful though.

Add integration tests lol

Resources:

• https://bundler.io/guides/creating_gem.html
• https://www.devroom.io/2014/04/15/testing-home-with-cucumber-and-aruba/
• https://gist.github.com/bdunn313/4199906

Can't find a spec of untracked files

Use case

On a rails project, there are two untracked files:

• app/models/competency/assessment/assessment_date_range.rb
• spec/models/competency/assessment/assessment_date_range_spec.rb

Behaviour:
🟢 When making a change on the assessment_date_range_spec.rb, retest runs the tests of that file.
🔴 When making a change on the assessment_date_range.rb, retest doesn't find the matching test file.

Expected behaviour:
When making a change on the assessment_date_range.rb, retest should find assessment_date_range_spec.rb and its tests.

Replace file to changed on terminal output

When using both placeholders we list the changed and test files but reference changed as file. Example

Files Selected:
  - file: test/models/coupon_test.rb
  - test: test/models/coupon_test.rb

It should be displaying:

Files Selected:
  - changed: test/models/coupon_test.rb
  - test: test/models/coupon_test.rb

Handle Hanami Projects Setup

With Hanami projects, your repository setup has a lot file with the same names. For example

• apps/web/controllers/posts/index.rb
• apps/web/controllers/comments/index.rb
• apps/web/controllers/authors/index.rb

More over for a same route you can then have three files

• apps/web/controllers/books/index.rb
• apps/web/views/books/index.rb
• apps/web/templates/books/index.html.erb (no spec?)

With the current pattern, a change on one of their files will match all the respective spec files listed above. We need to reduce the number of test files returned or find the exact test file to run.

Running command against changed file

Hey @AlexB52! Really enjoying retest, really lightweight and simple.

I wanted to ping you to see if you had anything in mind or if this is something you have a workflow for. I'd like to run the test like you normally would but then also run rubocop against the changed file as well as the test file.

Something like this:

retest 'bin/rails test <test> && bundle exec rubocop <test> <changed>'

I noticed you just gsub <test> from the command system command.gsub('<test>', cached_test_file) but I think by the time it gets to the runner retest has already detected which spec|test file to use. The rubocop command will take as many files as you throw at it so they can just be listed one after another too.

Open retest to other watching tool than Listen

I wonder if we can open retest to use different watching system like fswatch

Like an external pipeline
We would still use Listen as the default but then open retest to different adapters

Either by opening arguments to retest main command or create a command used to pipe the test based and return the file test based on the codebase setup.

`retest` not recognising spec files as specs when modified.

Say I have some files with the following paths:

<project root>/app/lib/module/myfile.rb
<project root>/spec/module/myfile_spec.rb

retest will find the spec if I modify myfile.rb but If I modify myfile_spec.rb it reports: "Could not find a file test matching" (the wording of that feels a little off).

The behaviour I'm expecting is that retest realises this is a spec and just runs it directly.

Diff with Rake command is broken

When we're using retest --diff main on a rake setup the command raises because we do not pass the correct argument type.

When fixing this bug we need to add some tests for the rake setup.

Default bundler gem inverts filename tests since 2.3.0 - Dec 2021

Bundler decided to change the most used naming behaviour for Minitest files from "test/**/*_test.rb" to "test/**/test_*.rb"

• https://github.com/rubygems/rubygems/releases/tag/bundler-v2.3.0
• rubygems/rubygems#3893

Retest doesn't take into account this other format "test/**/test_*.rb" and is likely to fail identifying the test file.

This issue is to double-check the behaviour and fix this issue if it happens.

Add Github Action Test Pipeline

retest --ruby option

retest should know about few options.
One option is --ruby which should be a shortcut for ruby <test>

Those two commands should be equivalent

retest 'ruby <test>'
retest --ruby

Run latest command when no test found

Queue most recent run

Sometimes specs get run over and over when the changes are made faster than the spec run.
We should

• Queue a file when tests are already running.
• Replace the file queued with the latest file to run when changes are made during a test run.

Remove .ruby-version file?

As pointed out in #124
The .ruby-version file might be redundant with the minimal version specified in the gemspec.

Should developers be able to use any versions supported by the gem when developing on Retest?

Retest defaults to `test_*.rb` pattern even when `*_test.rb` file exists

Here is the original issue

• When updating the file test_options.rb
• I expect retest to select test_options_test.rb
• and not to return the file test_options.rb

I think this is also related to this issue:

• Given there are three files test_fixtures.rb, fixtures.rb and fixtures_test.rb
• A change in fixtures.rb selects test_fixtures.rb by default
• When it should offer the option to choose between test_fixtures.rb and fixtures_test.rb

When a file is discovered with the pattern test_*.rb we do not continue searching for other file patterns which could potentially be more appropriate.

Ruby project with no Gemfile raises

when making a plain ruby projects with no Gemfile. retest does not work and raises

This seems to happen when retest tries to run bundle exec ruby test/points_test.rb where Gemfile doesn't exist

Test File Selected: test/points_test.rb
Could not locate Gemfile or .bundle/ directory

retest --rspec option

retest should know about few options.
One option is --rspec which should be a shortcut for bundle exec rspec <test>

Those two commands should be equivalent

retest 'bundle exec rspec <test>'
retest --rspec

Allow configuring polling-based listener

Right now, retest does not work in cases where the operating system requires a polling-based file change notification (for example, a Linux vagrant guest).

The listen gen has an option for forcing polling mechanism in the Listen.to method:

force_polling: true     # Force the use of the polling adapter

Perhaps this can be exposed as an option to retest CLI? retest --polling or some other convention? (environment variable?)

I see this is the line that starts the Listen.to loop:

retest/exe/retest

Line 36 in 3bf9fc3

Listen.to('.', only: options.extension, relative: true) do |modified, added, removed|

(example rough fix in this fork)

retest --rails option

retest should know about few options.
One option is --rails which should be a shortcut for bundle exec rails test <test>

Those two commands should be equivalent

retest 'bundle exec rails test <test>'
retest --rails

Improve list of files for pattern matching

Add <changed> placeholder in help command

There is no mention of the new placeholder <changed> in the help, there is no example either.
Let's add that

retest --auto option

retest should know about few options.
One option is --auto which should be a shortcut to infer the type of ruby project used and run the appropriate command

Those two commands should be equivalent

retest 
retest --auto

Used on a rails project, it should run retest 'bundle exec rails test <test>'
Used on a rake like a gem, it should run retest 'bundle exec rake test TEST=<test>'
Used on a ruby project, it should run retest 'ruby <test>
Used on a rspec project but not using rails command, it should run retest 'bundle exec rspec <test>

As an example, aliases are used by people this way. See #15

alias rtt="if [ -x bin/rails ]; then retest 'bin/rails test <test>'; else retest 'bundle e rake test TEST=<test>'; fi"

--debug option

On one of my project, retest seems to run fine once and then doesn't trigger a new build unless I wait a really long time. The process running it seems to take a lot of CPU resources. Never encountered this problem on another repository. The repository is 8 years old.

It would be nice to add a debug/verbose option to print some debug statements and identify where it's blocked.

The next step is to fix the issue with that new tool.

Use bin commands instead of bundle exec

People have raised that using bin/rspec bin/rails being faster than using bundle exec

For me bin/rails is significantly faster than bundle exec rails (0.75s vs 1.25s). That speed makes a big difference when doing TDD.

This issue investigates the feasibility of using bin commands instead of bundle exec that works on most ruby projects.

Find matching test for unmatching commands

When passing a hardcoded command (with not <test> placeholder), retest still tried to find the matching spec before running the command. Finding the test is not required as it is not dependent from it.

This is has been identified when changing a file with multiple test possibilities, the output asks you to choose the file to match even though it doesn't need it.

retest --diff to run all tests in one go instead of per file

At the moment when running retest --diff this will identify each changed file and run the appropriate spec for each of them. Ideally you would like one run with all the spec files together.

Open watch option to other files than ruby extensions.

I'm currently learning Go and would like to retest when coding/learning/testing.

At the moment the software only listens to files with .rb extensions.

Create a --listen flag to list file extensions you want to listen to. Something like

$ retest 'go test party_robot_test.go' --listen '.go'

Cache Test File after first run

Handle Multiple Files

retest --rake option

retest should know about few options.
One option is --rake which should be a shortcut for bundle exec rake test TEST=<test>

Those two commands should be equivalent

retest 'bundle exec rake test TEST=<test>'
retest --rake

Investigate Hanami 2 (alpha version)

Retest should work with Hanami 2. This issues investigates whether the file structure of Hanami 2 is different than Hanami 1

Remove the --auto flag

The auto option is used by default and I don't see a use case where we want to use the flag anymore.

Retest specs related to diffs against a branch

It would be nice to git diff a branch and pipe all the changed files to identify which specs need to be run for a sanity check before pushing to remote and triggering the whole CI pipeline.

retest --all option

retest should know about few options.
One option is --all which should run all the test of the ruby projects (without reference)

Equivalent commands

command	hard coded equivalent
retest --rails --all	retest 'bundle exec rails test'
retest --rake --all	retest 'bundle exec rake test'
retest --rspec --all	retest 'bundle exec rspec'
retest --all	retest --auto --all (which differs from project)

retest is confused if a file is called `rspec.rb`

A change on a file named rspec.rb will try to run the test command with rspec.rb file which doesn't have tests and therefore doesn't output anything even though the change is recognised.

Test File Selected: lib/retest/command/rspec.rb
# nothing happens

Suggestion: Trap `Interrupt` signal (Ctrl+C) and exit gracefully

Right now, when pressing Ctrl+C after running a simple retest, exits with a ruby trace - sometimes a long one.

It would be nice to just see a friendly Goodbye or at least not see the trace.

$ retest
Setup identified: [RSPEC]. Using command: 'bundle exec rspec <test>'
Launching Retest...
^C
/home/vagrant/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/rb-inotify-0.10.1
/lib/rb-inotify/notifier.rb:208:in `directory?': Interrupt
        from /home/vagrant/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:208:in `block in watch'
        from /home/vagrant/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:202:in `each'
        from /home/vagrant/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:202:in `watch'

... snipped ...

(example rough fix in this fork)

Create a Docker image to test different ruby setups

The docker image and repo should have enough to run rails, or rspec or rake.
The docker image should run Ruby 2.4.1