Should we rename the repository from `benchmarks` to `idioms`? about benchmarks

[arjenmarkus]

I think that is a goo idea - as Damian mentioned, benchmarks have a long and convoluted, even painful, history. " idioms" is much more neutral and, brings a better message. Op di 29 jun. 2021 om 05:04 schreef Ondřej Čertík ***@***.***

…

: See #10 <#10> for the background discussion. It seems idioms communicates better what we are trying to achieve: - Have mainly idiomatic code (in each language) how to solve a given problem - Document each problem, give a mathematical background and then several versions in each language how to solve it - We'll also have non-idiomatic code that tries to extract the best performance, but the idiomatic code could help compilers to optimize it better - What is "idiomatic" is subjective, and thus we can and should have several different versions - As a user, I would love to browse the approaches how to solve a given problem, even just in Fortran. But also in C++, Julia, Python and other languages. To learn and educate myself. And also to compare how "easy" it is to write something like this myself in a given language. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#22>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN6YR7GY7EDTHB7HH3O7R3TVEZ4TANCNFSM47PFTN7A> .

[arunningcroc]

Yeah, it's probably more neutral this way, and not calling it a benchmark in a sense "absolves us of responsibility" with regard to getting serious about measuring performance, having exactly equivalent algorithms, etc.

[milancurcic]

I like either name. Benchmarks is stronger and more loaded, and has disadvantage that others mentioned already. Idioms is more neutral, but also a weaker, more vague, word. One advantage to "benchmarks" is that it would (I suspect) invite scrutiny from others, which would help us improve.

[milancurcic]

I agree that "idioms" has a broader scope that better fits our goals, and can include benchmarks as well.

[certik]

Thanks @arjenmarkus.

@arunningcroc right. I still want to do the best we can to have as fair benchmarks as we can, but framing it as "idioms" people will hopefully not draw wrong conclusions from it. As an example of this, I am hoping the Julia community will contribute better versions of Julia "benchmarks", and there should be very little friction to include the faster version even if it perhaps does not benchmark exactly the same thing, because in the "idioms" it will just be another data point. It will depend on how we present the results, but we will iterate on that to ensure the overall presentation is more like "idioms" and less like "benchmarks", and allow readers to make their own conclusions.

[certik]

CC @LKedward, @milancurcic, @awvwgk.