Comments (3)
The intention was to able to be supply a weight, yes, and perhaps some methods to experimentally determine a good weight on sample workloads.
from rayon.
In response to this comment by @mneumann from #9:
This is sort of roughly what I had in mind, but it'd probably be a stage in the algorithm. Something like foo.into_par_iter().map().weight(22)
, which would indicate that the map
operation takes "22x" the default time. This is trivial to implement, but you'd have to just kind of guess what the right number is (well, experiment). I am also rather concerned about the portability of these constants, both across machines/workloads and across releases of rayon.
One way to address that would be to have (and encourage the use of) an "autotune" option, where you provide a pipeline and a sample, and we vary the weights to find the one that seems to be running best. But that itself will be sensitive to whatever else the machine happens to be doing at the time which you tune, and it assumes you have some convenient time to do that tuning and data to tune with.
Overall, I'd rather find something more automatic -- and just optimizing join
and the overall dispatch is perhaps the best answer I know of there -- but allowing one to manually specify weights might be a good fallback to have on the table.
from rayon.
See https://github.com/nikomatsakis/rayon/pull/12, which adds a weight combinator
from rayon.
Related Issues (20)
- Potential `len` method ambiguity in `src/range.rs` HOT 5
- Potential to fix memory ordering for value HOT 2
- a broadcast that only gets executed on idle threads
- `in_place_scope` documentation is confusing/unclear HOT 14
- use_current_thread and the global thread pool HOT 2
- Why does ThreadPool block until a operation is finished? HOT 1
- A way to hook before/after job execution (and/or a way to see a number of tasks in the processing queue)
- Function dependencies with Rayon without blocking the thread / Rayon synchronization primitives?
- Add synchronization guarantees for `ParallelIterator::for_each*` HOT 1
- Par bridge with optional buffering IO handling HOT 2
- Thread pool without work stealing HOT 6
- Question about return types from .map() HOT 4
- Terribly inefficient design and possible solution HOT 5
- [Discussion] Is it possible to impl `IntoIterator` for `ParallelIterator`? HOT 3
- Yield locks HOT 1
- the method `par_iter` exists for struct `ChunkByMut<'_, BinF32, {[email protected]:207:78}>`, but its trait bounds were not satisfied HOT 5
- Unupdated crossbeam dependency makes MIRI angry HOT 1
- help: How to implement memoized chunks for parallel iterator? HOT 2
- wrong repo, issue deleted HOT 3
- Comparison with Spice HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rayon.