Run tests immediately after launch retest about retest

[AlexB52]

Thanks for reaching out @mpkhr,

retest requires an event to trigger a test run and saving a file is generally what retest listens to.
Currently, we can do an empty save (without any changes) on a file after launch and retest will run its relevant test.
I sometimes press cmd+s on my editor to run the tests before making any changes to make sure the tests pass before refactoring. Would that be good enough?

I'm asking because without a saving event on a file, it would be difficult to know which test we're expecting to run after launching retest. We could run tests after launch successfully if we're using the --all option or if a hardcoded command is passed in. That said, having different behaviour after launch could maybe create some confusion about how the tool is expected to work.

Can you tell me which command you're currently using with retest please?

The example you give shows that no setup has been identified which happens when some options are passed to the command line. Which options do you use?

[mpkhr]

Good points @AlexB52

My favourite mode of retest is running on any file change triggering the tests I specify. For example, if I change a factory (or even an indirect factory behind an association), I want the tests to run immediately, which is not the case when limiting retest to a certain file to listen to. Also I don't want to check the console and see that at some point ago there was a question about choosing what file to run. That being said, the command I showed initially was a simplification of what I really do, thanks for double checking.

Here are some examples of what I use:

• retest --rspec --all
• retest 'rspec spec/models/some_specific_spec.rb -f documentation'
• retest 'rspec spec/system/*'

I think simulating such even would be perfectly fine. If the launch behavior might be unexpected, how about explicitly enabling it with a flag, for example --run-on-launch?

[AlexB52]

I've been thinking about the --run-on-launch option.
I'm not fully convinced it's necessary, but I will leave the issue open and see what it takes to introduce it as an experimental option.

[mpkhr]

I also thought about a bit more and think the following would be good:
The confusing part about retest in my opinion is to understand the difference between WHAT is run and WHEN it's run, in particular for when it's "magically" guessed. So I think generally it would be a good idea to have an option to determine the trigger, like --on:any for any file change and --on:related for any related file changes.

If run with --on:any, an immediate first run shouldn't be a surprise. Also what is to run should be clear, since it's either everything or whatever is given as a path.

This would be a easier to understand way to work with retest I think. Looking forward to your thoughts!

[AlexB52]

@mpkhr

I think you raise a valid issues around the retest experience. This is the problem I understand from your usage of Retest and my experience with the tool.

Locking a test

When using automatic mode from retest, I sometimes want to run tests for a specific file. Some files are more coupled than others and I sometimes need to change another class AND still need to run the previous class I'm refactoring. It often ends up with me:

1. Running the tests for the class I just changed
2. Then switch back to the main class I'm working, save the file and run the tests again.

This is slow and slightly annoying. I think this is what you mean with --on:any and --on:related. It feels like this is related to the previous issue #174: The interactive panel.

One big feature, I would like to add the ability to switch to an interactive panel from retest terminal to change settings on the fly. Things like deleting a wrong cached pair file/test, change the test command, change the options or in this case locking a specific test for every file change.

Am I understanding the problem correctly? Do you think an interactive panel to lock a file would work?

This is not an easy task and I don't know how to do it just yet. For now we have to kill retest and put a hardcoded command.

If this is the correct requirement, I will close this issue and reference it in a new issue specifically about locking a specific file on deman.

[AlexB52]

I think I understand now. Retest focuses on running a test command for every file change, and the suggestion is about running a test on some file changes only. I can see how this would work for “retest --rspec --all” and “retest 'rspec spec/models/some_specific_spec.rb -f documentation’”

The first command would run for every file change regarding the option any/related while the last one would only run for a change on every file for any or only for file resulting in the spec with on related.

This becomes tricky with “retest 'rspec spec/system/*'” Retest works but isn’t great for system and e2e test. The naming conventions are loose for these types of specs. a —on:any option here risks of never getting triggered a command.

Maybe adding —on:demand as well where we can manually trigger a test with an external event other than saving a file could help for this case.

Locking a file helps switch the “what” gets triggered while keeping consistent the “when” it is triggered. It’s highly predictable “file change = run” which is the premise of refactoring in the book 99bottle of OOP from Sandi Metz and why the gem exists in the first place.

Filter triggering events with the :on option is an interesting idea, and I’ll look into it. I wouldn’t necessarily run a file on launch with that on option though. A file save can trigger the first run.

[mpkhr]

Hi Alex,

alright. at this point I also got used to the "save your file" workflow, even though it's slightly annoying. So I kinda switch between running retest and actually triggering rspec as well. Not ideal, but also not a big problem.

About the "on:trigger" conversation. I only use retest with "on:any" file change. I typically scope the tests I want to run (can be a specific one or for example the whole model folder) and I want this to be tested on any file change without getting stopped by retest "on:related" not guessing right (if I remember it didn't recognize changes in factories correctly for example).

To answer your question about the interactive mode: I'd not be interested in it. I initially ran it in the "on:related" mode and it actually asked me for what file I mean, but I didn't notice right away. So I first didn't like I need to decide (since in a future task the scope can change and I might mean something else), and also the piling up of runs that occurred once I made a selection felt wrong to me. Retest to me is something that runs in the background and I often check, however, I might not check it after every single save.

Hope this helps!

[AlexB52]

Thank you @mpkhr and sorry I didn't understand what you meant by on:any and on:related.

I've recently merged the #177 Make a sound when asking for file to run which should notify when a question was asked. Although I understand it's not solving the whole issue as you might want to change the pair file/test mapping as you work.

If you are on MacOS and don't want to check retest terminal window often you can use --notify option which will trigger some sound on start, test failure and test success. A new sound for questions is going to be available in the next release.

I will keep in mind the issues you raised and will try to improve the retest experience. Thanks again

[AlexB52]

Hi @mpkhr

I've released a POC of an interactive panel for Retest. You can install the preview like so: gem install retest --pre
On a hardcoded command (a forced file to test), you can now run your tests by pressing enter straight after launch. You can also pause/unpause running tests after a saved file with p and u respectively.

Here is the current open discussion asking for feedback. I would love your feedback if you have time to check it out.

Here is what you can currently do with Retest interactive panel.

Setup identified: [RSPEC]. Using command: 'bin/rspec <test>'
Launching Retest...
Ready to refactor! You can make file changes now

Type interactive command and press enter
> h

* 'h' or 'help'    -> Prints help
* 'p' or 'pause'   -> Pauses Retest. Tests aren't run until unpaused.
* 'u' or 'unpause' -> Unpauses Retest
* <Enter>          -> Runs last changed triggered command
* 'ra', 'run all'  -> Runs all tests
* 'e' or 'exit'    -> Exits Retest