Comments (4)
perf 1.2 adds a new "warmup calibration" feature, but it seems unstable. I will probably try to compute manually the number of required warmups on PyPy 5.7 on the speed-python server, and use hardcoded values, instead.
from pyperformance.
I agree the warmup number should be fixed. It will reduce uncertainty about what a benchmark is measuring.
IMO there should be versioning of each benchmark so we can be sure result comparisons are apples-to-apples. On the speed.python.org website it should only show the latest version of each benchmark, which would ensure changing the warmup number would not cause a weird jump in the timeline display
from pyperformance.
I would like to begin testing PyPy3.6 and uploading results to speed.python.org, but there should be agreement that PyPy results are valid and meaningful. Do we require warmup values or just an upper limit for number of loops before we give up on convergance?
Currently there are warmup values for bm_go, bm_hexiom, and bm_tornado_http.
Following the note about warmup, can/should we add values for the other benchmarks? Since the times sometimes do not converge to a single value, we should have a maximum number of loops as well.
from pyperformance.
Something should be done, I don't know what.
from pyperformance.
Related Issues (20)
- Why no releases? HOT 1
- benchmark representing async-heavy, heterogeneous workload HOT 7
- CI fails with 3.12 HOT 2
- django_template benchmark fails on main due to lack of distutils
- Add rinohtype benchmark?
- Tests fail with Python 3.12 HOT 9
- Use more than one core HOT 6
- Impossible to add a benchmark that exercises newer Python features HOT 1
- Add type hints?
- asyncio gather vs TaskGroup in async_tree benchmarks HOT 3
- Need to vendor lib2to3 for CPython 3.13+
- Add benchmark for pip-compile
- Need benchmark using `__getattribute__`
- Editable installation isn't working properly
- Allow compiling dependencies rather than using prebuilts
- Help me to completely disable psutil
- benchmarking: cannot install pypy-devel
- Allow pyperformance to compile CPython with the JIT enabled. HOT 3
- Would it possible to add a .cff file to the repo so that it can be cited more easily?
- django_template and sympy failing because of distils dependency HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyperformance.