ParallelComputer has too limited applicability about junit4

[dsaff]

Kristian,

We patched your branch into 4.8b3, ran JUnitCoreConcurrencyTest, and testFixedThreadPool and testClassUnlimited failed on my machine, with 1 and 2 failing tests respectively out of 3000. Is this expected?

[dsaff]

Scratch previous comment: now that we've patched in the fix for issue #16, most of the failures in JUnitCoreConcurrencyTest go away, except that testBothUnlimited is still failing more often than it passes. Ideas?

[krosenvold]

This is probably happening on core2duo; I'll give a try at tracking it down on a c2d box I can use today.

[krosenvold]

testBothUnlimited fail(ed) due to a termination concurrency issue. This is basically junit finishing processing before all the notifiers are done. basically the "finished" method in the RunnerSchedluer needs some kind of implementation.

"ConfigurableParallelComputer" has this bug, and your "ParallelComputer" too. It's easier to reproduce in CPC due to simpler and more efficient thread handling.

I finally managed to nail this bug properly in ConfigurableParallelComputer, and have committed this to github. The essence of the fix, which you will need too, is as follows:

The executor service must both be shutdown and wait for temination of all running threads before you can call it a day. I used code like this:

 fService.shutdown();
    try {
    fService.awaitTermination(10, java.util.concurrent.TimeUnit.SECONDS);
    } catch (InterruptedException e){
        throw new RuntimeException(e);
    }

Conceivably you could keep some counter/semaphore instead of delegating this to the Executorservice, unsure if that's any better.

Whatever solution you come up with needs to close the executor service(s) and wait thread completions before returning to the client. That's why I implemented a close() method on the CPC class. From a "computer" metaphor I find it not totally implausible that it also knows how to wait for all tasks to finish.

Since I have a single exectuor across the entire test-run, I needed to shut it down at the end. If you decide to stick with multiple executor services you will need to shut them down, and you will also need to be sure that all the tasks are finished. Conceivably you could dot this in the RunnerScheduler.finished method, but I am not totally convinced you should be waiting for completion there ;) It may actually be that multi-level thread pools can save you from waiting for shutdown at the lowest level; i must admit that I find the current threading too complex to actually understand for more than a short period of time ;)

[krosenvold]

Also, please accept the JUnitCoreConcurrencyTest as a submission to Junit, you can replace the references to "ConfigurableParallelComputer" to whatever solution you end up with.

[dsaff]

@krosenvold, we've worked through a lot of parallelization issues in the 3 years since we were discussing this bug. Should we close it in favor of more targeted future improvements?

[krosenvold]

I understand that the pace of the junit project is deliberately slow, taking time to think things through. Unfortunately this slowness could also be interpreted as no movement at all.

The overall lack of speed and agility of the junit project had me losing any interest in this project quite some time ago. You also appear unwilling to extend the community of committers, even though there appear to be plenty of well qualified candidates and a large group of reviewers with an obsession for detail.

I work on more than enough OSS projects already, including maintaining maven-surefire which is probably the JUnit client with the largest amount of users. My plate is full and I'm personally not really looking for committership, but I would happily participate if I expected anything to happen within another three years. As things have been up to now it's just a wast of time.

As for this issue, I have no idea. It's been 3 years.

[kcooney]

@krosenvold I wish you had submitted a pull request. For better or for worse, in JUnit issues get resolved by developers proposing solutions via pull requests more than by conversations in bugs. I don't know about the other volunteers that work on JUnit, but I look at pull requests much more often than bugs (especially older bugs). Bugs are often some form of "I have a problem" while pull requests are often"I have a solution". In this case, you had the code for a possible solution in github, and I bet most of the people that review JUnit pull requests in their free time didn't even know it existed.

@dsaff Most of the issues raised in this bug have not been been fixed. I'm not entirely sure that any of the open pull requests would fix them all. I suggest re-opening this.

At my company we did a lot of work on running JUnit3-style tests in parallel, and ran into some of the issues raised in this bug. I personally think ParallelComputer is only part of the solution. It allows you to create a custom class that runs all of your tests in parallel, but that doesn't help you if you are running your tests via an IDE (at my company, we created a ParallelSuite to do this). JUnit also does not allow users to indicate that a particular test method or test class cannot be run in parallel (perhaps because the test references static state, perhaps because the code under test is thread-hostile).

If it would help, I could put you in touch with the people at my company that worked on this.

[krosenvold]

@kcooney I did a few pull requests that were left to rot for quite some time. I spend my energy where I can make a difference. I believe @Tibor17 is actually working on this stuff in pull request #617 I think #617 solves all the problems in this issue, but I find it hard to amass the energy to get involved there when I know nothing is going to happen

[Tibor17]

@krosenvold
It will happen. We should first check the ParallelComputerBuilder method signatures are acceptable, and then we should continue with impl.
Actually, it was hard to understand the decisions in my implementation details by others since they did not develop with me. I will do my best.

[kcooney]

@krosenvold was there a pull request for this issue? If so, could you send me a link? If not, could you give me one or two examples of pull requests from you that you felt were "left to rot"? The search on github isn't that great.

[krosenvold]

No, I dont't think I made one. I hardly remember what i had for dinner the day before yesterday, so I may be wrong ;)