alexb52 / retest Goto Github PK

when using retest --help the usage text is not printed although it prints fins when not in a docker container

On one of my project, retest seems to run fine once and then doesn't trigger a new build unless I wait a really long time. The process running it seems to take a lot of CPU resources. Never encountered this problem on another repository. The repository is 8 years old.

It would be nice to add a debug/verbose option to print some debug statements and identify where it's blocked.

The next step is to fix the issue with that new tool.

When passing a hardcoded command (with not <test> placeholder), retest still tried to find the matching spec before running the command. Finding the test is not required as it is not dependent from it.

This is has been identified when changing a file with multiple test possibilities, the output asks you to choose the file to match even though it doesn't need it.

When we're using retest --diff main on a rake setup the command raises because we do not pass the correct argument type.

When fixing this bug we need to add some tests for the rake setup.

retest should know about few options.
One option is --rake which should be a shortcut for bundle exec rake test TEST=<test>

Those two commands should be equivalent

retest 'bundle exec rake test TEST=<test>'
retest --rake

Say I have some files with the following paths:

<project root>/app/lib/module/myfile.rb
<project root>/spec/module/myfile_spec.rb

retest will find the spec if I modify myfile.rb but If I modify myfile_spec.rb it reports: "Could not find a file test matching" (the wording of that feels a little off).

The behaviour I'm expecting is that retest realises this is a spec and just runs it directly.

Use case

On a rails project, there are two untracked files:

• app/models/competency/assessment/assessment_date_range.rb
• spec/models/competency/assessment/assessment_date_range_spec.rb

Behaviour:
🟢 When making a change on the assessment_date_range_spec.rb, retest runs the tests of that file.
🔴 When making a change on the assessment_date_range.rb, retest doesn't find the matching test file.

Expected behaviour:
When making a change on the assessment_date_range.rb, retest should find assessment_date_range_spec.rb and its tests.

When using sqlite in a project like Hanami. Following the guides break:
https://guides.hanamirb.org/introduction/getting-started/

If a file is changed when running retest 'bundle exec rake' the tests will change the sqlite database file which will trigger a changed file event. The rake task is run again modifying the sqlite file which then triggers another spec run... Making it a infinite loop

We should not listen to changes applied on files listed in .gitignore. This should not happen with a database that is not stored as a file like a postgresql or mysql setup.

Instead of excluding files from .gitignore it seems easier (maybe slower) to include only the files git tracks using git ls-files
Maybe something like tis

files = `git ls-files`.gsub("\n", "|")
listener = Listen.to('.', only: files, relative: true) { |modified, added, removed| # ... }

Or send the git ignored patterns in the exclude option of Listen

Or maybe just add sqlite to the ignored patterns 😂

The bug also happens when using byebug and updating the .byebug_history file

Right now, when pressing Ctrl+C after running a simple retest, exits with a ruby trace - sometimes a long one.

It would be nice to just see a friendly Goodbye or at least not see the trace.

$ retest
Setup identified: [RSPEC]. Using command: 'bundle exec rspec <test>'
Launching Retest...
^C
/home/vagrant/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/rb-inotify-0.10.1
/lib/rb-inotify/notifier.rb:208:in `directory?': Interrupt
        from /home/vagrant/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:208:in `block in watch'
        from /home/vagrant/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:202:in `each'
        from /home/vagrant/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/rb-inotify-0.10.1/lib/rb-inotify/notifier.rb:202:in `watch'

... snipped ...

(example rough fix in this fork)

There is no mention of the new placeholder <changed> in the help, there is no example either.
Let's add that

Currently retest is not trying to run a spec on a new created file. Whether it is with git or not.

Once a new file is added to the repository and showed as changed, we should add that file to the array of files to search from

Hi @AlexB52, I love retest! I'm using it with the following bash alias for Rails and non-Rails projects:

alias rtt="if [ -x bin/rails ]; then retest 'bin/rails test <test>'; else retest 'bundle e rake test TEST=<test>'; fi"

Feel free to close this issue. Just wanted to say thanks!

Bundler decided to change the most used naming behaviour for Minitest files from "test/**/*_test.rb" to "test/**/test_*.rb"

• https://github.com/rubygems/rubygems/releases/tag/bundler-v2.3.0
• rubygems/rubygems#3893

Retest doesn't take into account this other format "test/**/test_*.rb" and is likely to fail identifying the test file.

This issue is to double-check the behaviour and fix this issue if it happens.

In rake projects for a gem for example. Dependencies are defined in both Gemfile AND gemspec. The Gemfile.lock is the only file that bundles list all the gems in one spot.

Seems easier to use this lock file to find the correct ruby setup.

• pundit
• devise
• tty-prompt

when making a plain ruby projects with no Gemfile. retest does not work and raises

This seems to happen when retest tries to run bundle exec ruby test/points_test.rb where Gemfile doesn't exist

Test File Selected: test/points_test.rb
Could not locate Gemfile or .bundle/ directory

It would be nice to be notified when fails are passing or failing and the terminal window is not being displayed.

So I have a strange issue. I run retest in the docker container with the command

retest 'spring retest rspec <test>'

Running rspec with spring to speed up test initialization works fine.
Also, doing a save on any "*_spec.rb" file immediatelly reruns the spec. This is also fine.

However, when saving changes on an "*.rb" file, it takes about 5 seconds for the corresponding spec to run each time. Spring seems to work fine (spec starts executing, Spring reports it being preloaded, but only after the initial 5 seconds), so I was thinking - is it possible the file path to the spec file is not cached, and retest needs to search for the file each time we save?

Rails 6.1.3.2
Ruby 3.0.3

It would be nice to git diff a branch and pipe all the changed files to identify which specs need to be run for a sanity check before pushing to remote and triggering the whole CI pipeline.

retest should know about few options.
One option is --ruby which should be a shortcut for ruby <test>

Those two commands should be equivalent

retest 'ruby <test>'
retest --ruby

This is really specific problem that shouldn't cause a problem right now.

When calling retest 'rake test =<test>' the output is:

Setup identified: [RAKE]. Using command: 'bundle exec rake test TEST=<test>'
Launching Retest...
Ready to refactor! You can make file changes now

The command is not recognised and the default setting kicks in. Passing a = is what makes the command fail.

retest should know about few options.
One option is --rails which should be a shortcut for bundle exec rails test <test>

Those two commands should be equivalent

retest 'bundle exec rails test <test>'
retest --rails

A change on a file named rspec.rb will try to run the test command with rspec.rb file which doesn't have tests and therefore doesn't output anything even though the change is recognised.

Test File Selected: lib/retest/command/rspec.rb
# nothing happens

I wonder if we can open retest to use different watching system like fswatch

Like an external pipeline
We would still use Listen as the default but then open retest to different adapters

Either by opening arguments to retest main command or create a command used to pipe the test based and return the file test based on the codebase setup.

Here is the original issue

• When updating the file test_options.rb
• I expect retest to select test_options_test.rb
• and not to return the file test_options.rb

I think this is also related to this issue:

• Given there are three files test_fixtures.rb, fixtures.rb and fixtures_test.rb
• A change in fixtures.rb selects test_fixtures.rb by default
• When it should offer the option to choose between test_fixtures.rb and fixtures_test.rb

When a file is discovered with the pattern test_*.rb we do not continue searching for other file patterns which could potentially be more appropriate.

I'm currently learning Go and would like to retest when coding/learning/testing.

At the moment the software only listens to files with .rb extensions.

Create a --listen flag to list file extensions you want to listen to. Something like

$ retest 'go test party_robot_test.go' --listen '.go'

With Hanami projects, your repository setup has a lot file with the same names. For example

• apps/web/controllers/posts/index.rb
• apps/web/controllers/comments/index.rb
• apps/web/controllers/authors/index.rb

More over for a same route you can then have three files

• apps/web/controllers/books/index.rb
• apps/web/views/books/index.rb
• apps/web/templates/books/index.html.erb (no spec?)

With the current pattern, a change on one of their files will match all the respective spec files listed above. We need to reduce the number of test files returned or find the exact test file to run.

The auto option is used by default and I don't see a use case where we want to use the flag anymore.

As pointed out in #124
The .ruby-version file might be redundant with the minimal version specified in the gemspec.

Should developers be able to use any versions supported by the gem when developing on Retest?

Sometimes specs get run over and over when the changes are made faster than the spec run.
We should

• Queue a file when tests are already running.
• Replace the file queued with the latest file to run when changes are made during a test run.

retest should know about few options.
One option is --auto which should be a shortcut to infer the type of ruby project used and run the appropriate command

Those two commands should be equivalent

retest 
retest --auto

Used on a rails project, it should run retest 'bundle exec rails test <test>'
Used on a rake like a gem, it should run retest 'bundle exec rake test TEST=<test>'
Used on a ruby project, it should run retest 'ruby <test>
Used on a rspec project but not using rails command, it should run retest 'bundle exec rspec <test>

As an example, aliases are used by people this way. See #15

alias rtt="if [ -x bin/rails ]; then retest 'bin/rails test <test>'; else retest 'bundle e rake test TEST=<test>'; fi"

When using both placeholders we list the changed and test files but reference changed as file. Example

Files Selected:
  - file: test/models/coupon_test.rb
  - test: test/models/coupon_test.rb

It should be displaying:

Files Selected:
  - changed: test/models/coupon_test.rb
  - test: test/models/coupon_test.rb

The EOL for 2.6 comes at the end of March.

• We need to update retest gem to support 2.6+
• We need to update the feature repositories to Ruby 2.7. List of repositories:

• ruby-app
• ruby-bare
• git-ruby
• hanami-app
• rails-app
• rspec-rails
• rspec-ruby

I still wonder if we need this. As long as we support 2.4 then we should be good.

Create an app using ruby 3 could be helpful though.

At the moment when running retest --diff this will identify each changed file and run the appropriate spec for each of them. Ideally you would like one run with all the spec files together.

Retest should work with Hanami 2. This issues investigates whether the file structure of Hanami 2 is different than Hanami 1

Hey @AlexB52! Really enjoying retest, really lightweight and simple.

I wanted to ping you to see if you had anything in mind or if this is something you have a workflow for. I'd like to run the test like you normally would but then also run rubocop against the changed file as well as the test file.

Something like this:

retest 'bin/rails test <test> && bundle exec rubocop <test> <changed>'

I noticed you just gsub <test> from the command system command.gsub('<test>', cached_test_file) but I think by the time it gets to the runner retest has already detected which spec|test file to use. The rubocop command will take as many files as you throw at it so they can just be listed one after another too.

Use Case

Repository files sometimes share the same name and is difficult to assess. For some use cases, ex: updating a factory file, retest tries to find a matching file which is sometimes wrong.

The feature is to add a none options where a user can say don't choose any files here because none of them match the one I just modified.

        We found few tests matching: app/models/valuation/holdings.rb
        [0] - test/models/taxation/holdings_test.rb
        [1] - test/models/schedule/holdings_test.rb
        [2] - test/models/performance/holdings_test.rb
        [3] - test/models/holdings_test.rb
        [4] - test/lib/csv_report/holdings_test.rb
        [5] - none matches, skip this
        # Add this ^

        Which file do you want to use?
        Enter the file number now:

Right now, retest does not work in cases where the operating system requires a polling-based file change notification (for example, a Linux vagrant guest).

The listen gen has an option for forcing polling mechanism in the Listen.to method:

force_polling: true     # Force the use of the polling adapter

Perhaps this can be exposed as an option to retest CLI? retest --polling or some other convention? (environment variable?)

I see this is the line that starts the Listen.to loop:

(example rough fix in this fork)

The docker image and repo should have enough to run rails, or rspec or rake.
The docker image should run Ruby 2.4.1

retest should know about few options.
One option is --rspec which should be a shortcut for bundle exec rspec <test>

Those two commands should be equivalent

retest 'bundle exec rspec <test>'
retest --rspec

People have raised that using bin/rspec bin/rails being faster than using bundle exec

For me bin/rails is significantly faster than bundle exec rails (0.75s vs 1.25s). That speed makes a big difference when doing TDD.

This issue investigates the feasibility of using bin commands instead of bundle exec that works on most ruby projects.

Resources:

• https://bundler.io/guides/creating_gem.html
• https://www.devroom.io/2014/04/15/testing-home-with-cucumber-and-aruba/
• https://gist.github.com/bdunn313/4199906

retest should know about few options.
One option is --all which should run all the test of the ruby projects (without reference)

Equivalent commands