Slower than just running in sequence about karma-parallel

[brandonros]

Compare it to:

Running "karma:source" (karma) task
18 01 2018 23:45:02.811:INFO [karma]: Karma v0.13.22 server started at https://localhost:9876/
18 01 2018 23:45:02.821:INFO [launcher]: Starting browser Chrome
18 01 2018 23:45:06.159:INFO [Chrome 63.0.3239 (Mac OS X 10.12.6)]: Connected on socket 1wIxTkHM_pUkFqOnAAAA with id 57770281
Chrome 63.0.3239 (Mac OS X 10.12.6): Executed 602 of 1075 SUCCESS (0 secs / 21.871 secs)
Chrome 63.0.3239 (Mac OS X 10.12.6): Executed 611 of 1075 SUCCESS (0 secs / 21.973 secs)
Chrome 63.0.3239 (Mac OS X 10.12.6): Executed 844 of 1075 SUCCESS (0 secs / 23.781 secs)
Chrome 63.0.3239 (Mac OS X 10.12.6): Executed 1075 of 1075 SUCCESS (25.253 secs / 24.916 secs)

[joeljeske]

Interesting. Are you using sample test source code that you can post somewhere?

I also would have expected the 1,075 tests (123 matches of describe() across 122 files) to have been divided by 4 and then executed across the browsers.

That is exactly the intention of this project. From the log output, it looks like its running all of the tests, 4 times instead of splitting them up. Is that correct? If so, that would be a bug.

I wonder if there's a cool/fast way to help speed up the loading of 4 browsers. Maybe one browser, then 4 tabs within it?

I have found using ChromeHeadless has a faster startup time, likely to the fact that it does not need to initialize the visual rendering part of the browser. I hadn't thought of attempting multiple tabs... that would be cool. I guess there might be a tradeoff of possible conflicting with other parallel running test suite by manipulating shared storage for the single domain (local-storage, cookies, etc...). Would be a cool feature to toggle though. Do you know of any other karma plugins that uses tabs?

[joeljeske]

Do you have any update on this? I have not been able to reproduce your results. My only thought is that you are using an old version of karma (about 2yrs) that my plugin may not be compatible with. I have not tested that far back. Try upgrading and see if it fixes it

[joeljeske]

Closing due to inactivity.

[intellix]

Came across this and got these results running on a MBP 15" Early 2013 (8 cores supposedly):

1x --------------
real    1m25.659s
user    1m42.960s
sys     0m10.470s

2x --------------
real    1m15.996s
user    2m1.312s
sys     0m13.601s

3x --------------
real    1m11.746s
user    2m23.417s
sys     0m18.218s

4x --------------
real    1m13.970s
user    2m36.800s
sys     0m19.698s

7x (default as cpu-cores -1):

31 05 2018 09:54:53.081:WARN [launcher]: Chrome was not killed in 2000 ms, sending SIGKILL.
...

Output looks like the following. Dunno if it helps:

$ time ng test --watch false

31 05 2018 09:50:12.415:INFO [framework:karma-parallel]: sharding specs across 4 browsers
 10% building modules 1/1 modules 0 active31 05 2018 09:50:24.086:INFO [karma]: Karma v2.0.0 server started at http://0.0.0.0:9876/
31 05 2018 09:50:24.086:INFO [launcher]: Launching browsers ChromeCustom, ChromeCustom, ChromeCustom, ChromeCustom with unlimited concurrency
31 05 2018 09:50:24.092:INFO [launcher]: Starting browser Chrome
31 05 2018 09:50:24.101:INFO [launcher]: Starting browser Chrome
31 05 2018 09:50:24.105:INFO [launcher]: Starting browser Chrome
31 05 2018 09:50:24.110:INFO [launcher]: Starting browser Chrome
31 05 2018 09:50:53.683:INFO [Chrome 66.0.3359 (Mac OS X 10.13.4)]: Connected on socket 9BZR5t24QtttuU6JAAAA with id 28541602
31 05 2018 09:50:53.686:INFO [Chrome 66.0.3359 (Mac OS X 10.13.4)]: Connected on socket 2rQGBDF22x_k02bmAAAB with id 98195268
31 05 2018 09:50:53.688:INFO [Chrome 66.0.3359 (Mac OS X 10.13.4)]: Connected on socket wq0Cgfjrl0OHOBVlAAAC with id 25037401
31 05 2018 09:50:53.702:INFO [Chrome 66.0.3359 (Mac OS X 10.13.4)]: Connected on socket Z7q7CTd-qtBqLDm8AAAD with id 48113952
...
Chrome 66.0.3359 (Mac OS X 10.13.4): Executed 227 of 227 SUCCESS (10.476 secs / 0 secs)
Chrome 66.0.3359 (Mac OS X 10.13.4): Executed 170 of 170 SUCCESS (12.899 secs / 0 secs)
Chrome 66.0.3359 (Mac OS X 10.13.4): Executed 196 of 196 SUCCESS (11.819 secs / 0 secs)
Chrome 66.0.3359 (Mac OS X 10.13.4): Executed 245 of 245 SUCCESS (19.952 secs / 0 secs)
TOTAL: 838 SUCCESS
TOTAL: 838 SUCCESS
TOTAL: 838 SUCCESS

real    1m13.970s
user    2m36.800s
sys     0m19.698s

I'm also not really seeing any benefit. I think about 5-10 seconds were shaved but was kind of expecting at least half the time from x2 :)

227 + 170 + 196 + 245 = 838 so maybe it works but doesn't seem faster with Chrome or ChromeHeadless

[brandonros]

@joeljeske What kind of results do you get?

[brandonros]

@joeljeske Can you give me some insight on how to tell how long each test/test suite takes? That will help me help you.

[perrosen]

I know this issue is closed. But rather than opening another (which isn't really an issue anyways) I add this here instead.

Like @intellix mentioned, I'm seeing very little benefit and in some cases even slower test execution. My project is an Angular CLI project with 1000+ tests.

Was expecting a decent performance boost but maybe the setup of each test suite takes up more time only running on one core.

Was wondering if anyone has worked with a Angular CLI project and managed to work around this issue?