BMI1 better then BMI2 about asmfish

[neles666]

one of short test but have plenty of others

[neles666]

and why is cfish better if it has less kn/s ?

[CounterPly]

Hi Vesely, thank you for the feedback. With regard to your first question, I am not sure why bmi1 is getting better results for you. The only difference between bmi1 and bmi2 is that bmi2 utilizes the very nice PEXT instruction, which would only serve to boost ELO. Other than that, the bmi2 builds contain everything that bmi1 does. My guess would be that your sample size is not large enough to draw a statistically significant conclusion. That said, if this appears to be a reproducible phenomenon by multiple users with bmi2 capable hardware, I'll take a deeper look.

As for your second question, comparing the NPS of Cfish to that of asmFish is comparing apples and oranges. It is important to remember that the Cfish build that you reference contains SF patches from the months of June, July, August, and September, while asmFish does not (it is equivalent to SF from mid-May 2018). Comparing the NPS of these engines makes no sense because the two engines are separated by four months of ELO-boosting patches, simplifications, and parameter tweaks.

As for updates, I haven't stopped working on asmFish. The next patch, which detects upcoming repetitions, is exceptionally difficult because it is taking me a while to set up the framework required to correctly initialize the cuckoo tables. I'll figure it out eventually, but it is difficult to say how long it will take me without help from @tthsqe12. Once this patch is out of the way, asmFish progress should resume.

On a final note, to compensate for this temporary stall in official development, I continue to port patches on my fork of asmFish whenever I have time. For those that may be interested, the branch located here is 4 patches ahead of asmFish/lantonov with additional patches to be added in the next couple of days. (Ignore the "This branch is 216 commits ahead, 5 commits behind" message, as this is just a consequence of various merges with the head branch).

[MichaelB7]

Besides 20 games being nowhere nearly enough to make any statistical assessment as to one version being better than another.

[neles666]

thats right, anyway as i told i did many tests and all of them finished with bmi1 better for example...

[neles666]

You can try some positions and start both engines together, I get different results, as for depth and moves, if they are same, they should produce same moves and ev right?

[CounterPly]

thats right, anyway as i told i did many tests and all of them finished with bmi1 better for example

As previously stated, your screenshot(s) show nowhere near the amount of games required to draw any meaningful conclusion regarding the relative strengths of the engines. That's not my opinion, or @MichaelB7 's opinion, or anyone else's opinion -- it is a statistical fact. Even if you had a sample size of 30,000 high-quality games between a dozen engines, you would still have a rather annoying margin of error associated with your results. This is why the SF testing framework requires so many tests in order for a new patch to successfully pass. A few dozen or a few thousand games on their own hold no statistical value whatsoever.

You can try some positions and start both engines together, I get different results, as for depth and moves, if they are same, they should produce same moves and ev right?

First of all, randomly clicking the "start" button in the Fritz GUI and watching what happens does not even remotely constitute a "fixed-depth" test. Secondly, you are using 2 threads per engine, which will add variance to your evaluations. Similarly, you are 80% full for Hash on both windows, which tells me you are using way too high of a hash setting to be drawing any useful conclusions here. With all of the above factors obfuscating your results, you could start/stop the same engine from the same position and potentially get different evaluations.

As I mentioned before, the PEXT-instruction is the only difference between the two engines. It will allow the bmi2 version to search more nodes per unit time than the bmi1 version. If you restrict the bench depth properly (i.e. via commandline parameters using 1 thread per engine), then you will of course get the same bench for both engines.