Comments (6)
I'd suggest to using fixed core count rather than "auto" in -flto=X
because I've seen a lot of times situation where (gnu) make launches up to N
(in case of make -j N
) jobs with gcc and each (!) gcc spawned up to N
processes. It's confusing me a lot but container runtime confuses whole build process even more when running in a container with limited core count.
upd: you may use -fprofile-dir=path
flag with gcc in order to separate PGO data from build directory.
from pyperformance.
I'd suggest to implement -j threads
rather than running on all available cores. I'm using pyperformance
for gathering profiling data in own weird Python PGO+LTO build.
from pyperformance.
I'm using pyperformance for gathering profiling data in own weird Python PGO+LTO build.
If you don't mind, would you like to introduce your use case?
These days I have an interest in increasing the coverage of profiling for PGO.
from pyperformance.
@corona10 it's pretty weird solution. 😄
In short: build Python with shared library, install "somewhere" pyperformance using "shared" Python, reconfigure Python for static binary and build it - it will run pyperformance while gathering PGO data.
applied patches (related for this case):
build script: debian/rules from package template
upd: benchmark results.
from pyperformance.
upd: benchmark results.
Wow supercool!
I am still conservative with direct supporting profiling workload based on pyperformance suite
but I am open to improving the current PGO and LTO for better performance.
(For example, thinLTO is fast but fullLTO based on GCC is slow if you don't pass the core count or auto flag)
or we can create a new configuration for designating the pre-gen profile directory, which can be used for external profiled data.
from pyperformance.
I'd suggest to using fixed core count rather than "auto" in -flto=X because I've seen a lot of times situation where (gnu) make launches up to N (in case of make -j N) jobs with gcc and each (!) gcc spawned up to N processes
Okay, I agree with you. Let's pile the issue on the CPython and discuss the better way to solve it.
I prefer that we can use seamless ways to support it.
from pyperformance.
Related Issues (20)
- How should we be using dependabot in this repo? HOT 2
- add some more http benchmarks HOT 2
- Have `pyperformance run` run `pyperf system tune` HOT 5
- Benchmark Groups Have Some Minor Problems HOT 3
- speed.python.org has no results since October 17
- Why no releases? HOT 1
- benchmark representing async-heavy, heterogeneous workload HOT 7
- CI fails with 3.12 HOT 2
- django_template benchmark fails on main due to lack of distutils
- Add rinohtype benchmark?
- Tests fail with Python 3.12 HOT 9
- Impossible to add a benchmark that exercises newer Python features HOT 1
- Add type hints?
- asyncio gather vs TaskGroup in async_tree benchmarks HOT 3
- Need to vendor lib2to3 for CPython 3.13+
- Add benchmark for pip-compile
- Need benchmark using `__getattribute__`
- Editable installation isn't working properly
- Allow compiling dependencies rather than using prebuilts
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyperformance.