Comments (19)
Two of the three slowdowns are fixed, so I'll close this. Thanks for your quick fixes, @ejgallego!
from coq-hott.
Coq 8.19 is slower than Coq 8.18.
Cost of sort polymorphism.
from coq-hott.
Cost of sort polymorphism.
@SkySkimmer Thanks for explaining. Is that expected to be in the 5-8% range? Are there plans to improve this?
from coq-hott.
Regarding the speed of Dune, I've definitely noticed a regression in the past few versions, but I am not certain the cause of it. I haven't been involved with working on Dune as of late, so I am not certain what the problem is.
Regarding how Dune chooses jobs, this is up to the scheduler of Dune and is not something that is easily tweaked externally AFAIK. There is an option to dump a flame graph of the build detailed here: https://dune.readthedocs.io/en/stable/advanced/profiling-dune.html this should allow you to quickly find bottlenecks and see where the build is spending the most time.
cc @ejgallego for the performance regression in Dune.
from coq-hott.
I also tested -j1 builds with make 4.3, and Coq 8.19 is 3.2% slower than Coq 8.18. These timings are more stable, so this is probably a more accurate measure of the difference. My -j8 times above with make show 8.19 7.7% slower, but I think one of those 8.18 runs got especially lucky. Further tests show a difference of more like 5%, still more than the -j1 difference.
from coq-hott.
Thanks for the numbers @jdchristensen , they are very useful!
I'll suggest we handle the two performance problems here as different issues, as they really are. Coq performance regression will likely be improved upstream, but indeed, there is an expected slowdown for HoTT. Here HoTT is a bit unlucky I do believe as it is more impacted than other developments.
Regarding the regression in Dune, that's a bit more worrying and deserve investigation.
@jdchristensen , what are the times for the "zero" build on different dune versions?
[by zero build we mean doing dune build
when the build is already completed, to measure the overhead of Dune itself re-checking everything has been properly built]
from coq-hott.
@ejgallego I timed dune build --cache=disabled --display=quiet
when nothing needs to be built:
dune 3.14.0: 5.9s
dune 3.11.1: 3.3s
make 4.3: 0.6s
(I added make to the list, but I know that it's not a fair comparison, as dune is doing more sophisticated checks, not just checking timestamps.)
I used those dune
flags for the earlier tests as well. --display=quiet
is important when timing.
from coq-hott.
@ejgallego Is it possible to change dune versions in my ocaml switch without having to recompile coq each time? That would make it faster for me to try 3.12 and 3.13, in case that helps. OTOH, you can probably check out the Coq-HoTT library and reproduce these numbers. It has no extra dependencies and builds with "dune build".
from coq-hott.
@SkySkimmer Another comment is that the HoTT library doesn't use Prop
or SProp
, so it's a bit unfortunate that the ability to generalize to those slows down the HoTT library. Is there a way to turn off sort polymorphism, or even to completely disable Prop
and/or SProp
?
from coq-hott.
no.
from coq-hott.
@ejgallego Is it possible to change dune versions in my ocaml switch without having to recompile coq each time? That would make it faster for me to try 3.12 and 3.13, in case that helps. OTOH, you can probably check out the Coq-HoTT library and reproduce these numbers. It has no extra dependencies and builds with "dune build".
Yes, it is possible to use any dune regarless of what you have in your opam switch.
Dune itself is self-contained, so it doesn't depend on what is on the switch, except for reading the list of installed packages.
So for example you can compile dune in ~/external/dune/_build/install
, the version you want, and then just call that dune
instead of the dune
in the path.
Important note tho: due to versioning, it is hard to use an older dune with a switch with the newer dune; while dune works, it will complain about install dune packages being newer than what it knows.
So indeed, it is best to work on a switch that has the lowest dune version you plan to test.
from coq-hott.
@jdchristensen , we have identified two performance regressions, indeed there was one from 3.11 to 3.14 and another one long time ago, when we switched dune to a single coqdep call.
Zero build times for me are now under 0.2 seconds, with the following PR applied:
There is another PR that fixes the regression from 3.11 to 3.14, but actually, that regression is not observable after that one is applied.
from coq-hott.
If you are curious about testing, that can be done with:
$ cd dune_git_working_dir
$ hub checkout https://github.com/ocaml/dune/pull/10116
$ make clean && make release
$ DUNE_BIN=$(pwd)/_build/install/default/bin/dune
$ cd hott_working_dir
$ $DUNE_BIN build # etc...
from coq-hott.
@ejgallego This is wonderful! On my machine, dune now takes 0.16s for a zero build, even faster than make! The -j8 times are also much improved. I've updated the tables:
coq 8.18 / make 4.3: 35.0
coq 8.18 / dune 3.11.1: 36.6
coq 8.18 / dune + PR: 32.3
coq 8.19 / make 4.3: 37.9
coq 8.19 / dune 3.11.1: 38.6
coq 8.19 / dune 3.14.0: 41.9
coq 8.19 / dune + PR: 33.6
Zero build:
dune 3.14.0: 5.9
dune 3.11.1: 3.3
make 4.3: 0.6
dune + PR: 0.16
BTW, my custom build of dune reports its version as 3.12.0-642-gc5fdf4f
. I was expecting to see 3.14 or 3.15, not 3.12. Is that expected?
from coq-hott.
@jdchristensen , that's great! Sorry for that bug, these kind of bugs (missing a memoized table) are actually pretty hard to track / test for; there are not so many parts of dune that do this, as usually things are cached in files (for that Dune provides a much better hardened API)
Indeed, when I decided to start working on Coq + Dune, I had made some benchs, and Dune always outperformed make, so I'm glad to see that's still the case in a bug-free setup.
Again thanks for the very important work of performance testing dune and make.
BTW, my custom build of dune reports its version as 3.12.0-642-gc5fdf4f. I was expecting to see 3.14 or 3.15, not 3.12. Is that expected?
I think that dune will use git describe --tags
to determine the version, so likely the problem is that the tags are not placed in the main
dune branch. If that's the case, that deserves a bug report for Dune, tho I think this may be already be reported.
from coq-hott.
I think that dune will use
git describe --tags
to determine the version, so likely the problem is that the tags are not placed in themain
dune branch. If that's the case, that deserves a bug report for Dune, tho I think this may be already be reported.
@ejgallego Yes, I can confirm that git log
on the main branch shows 3.12.0 as the most recent tag. I couldn't find an issue for this in the dune github issues, but maybe I missed it. Do you want me to open one?
from coq-hott.
Thanks for confirming this, I did open the issue myself, as to provide a bit more info (this is due to a change in Dune's release process)
Here it is ocaml/dune#10141
Thanks again @jdchristensen !
from coq-hott.
IMHO this issue can be closed, dune releases every 6 weeks, so the fix should arrive to the released package pretty soon.
from coq-hott.
Indeed the Coq slowdown won't have an easy fix, but hopefully other improvements in 8.20 will offset this bad case for HoTT.
from coq-hott.
Related Issues (20)
- Use ViCaR to visualize terms in monoidal categories HOT 3
- dune frequently making full builds HOT 9
- Tensor products of modules
- Add break hints to =, ==, $==, $->
- Add `_ $== _ :> _` notation exposing the underlying type of homs
- Homological algebra lemmas HOT 3
- Spectral sequences HOT 5
- Experiment with redefining bifunctors as uncurried functors
- Work out why test/WildCat/Opposite.v is slow HOT 2
- Remove redundant fields from Ring
- Generalize lhs, lhs_V, rhs and rhs_V to all transitive relations HOT 2
- Define matrices as functions HOT 2
- Define matrices as a collection of columns rather than a collection of rows
- General Linear group HOT 1
- Finite fields (Galois fields)
- Prime numbers
- Idempotent ring elements
- grp_pow and related things
- Tensor product of cyclic groups
- use CongruenceQuotient to define coequalizer
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from coq-hott.