Comments (10)
ok, i finally have a "kind of" reproducible case. in the ocaml toploop, do
#use "topfind";;
#require "lacaml";;
#require "functory";;
Printexc.record_backtrace true;;
open Lacaml.D;;
Functory.Cores.set_number_of_cores 4;;
let a = [2;3;4;3;2;3;3];;
let f size i =
let m = Mat.random size size in
let () = getri m in
let () = Printf.printf "%d" (Mat.dim1 m) in
let () = Unix.sleep 1 in
i + size;;
Functory.Cores.map ~f:(f 12) a;;
this works fine for me, also for much higher matrix sizes. however, when i interrupt the parallel computation with ctrl-c from the toplevel (where i expectedly get some exception backtrace), and then try repeatedly to call the last line, then i get erratic behavior: sometimes the parallel computation finishes, sometimes it doesn't.
i don't know if this is really the same as my original problem since there i do not interrupt the computation. but it might be that some exceptions raised internally and then handled confuse the parallel mapping? (just speculating)
from functory.
i now also tried a version without any lacaml, just Unix.sleep. this does not reproduce the behavior. after the first map after the user interrupt, no additional exceptions happen no subsequent tries.
from functory.
from functory.
from functory.
I could not reproduce your non-termination issue.
sorry, i did not mean it hangs. what i see is: when repeatedly trying Functory.Cores.map ~f:(f 12) a;;
, sometimes the computation finishes with the correct result, sometimes an ECHILD
exception is raised. this erratic behavior does not seem to go away even after repeatedly doing it ten times or more. this only happens if i once use control-c in the very beginning.
from functory.
from functory.
yes indeed, with only Unix.sleep i, too, only get one set of exceptions after the first interrupt, not the much weirder reappearance of exceptions in later repetitions of the parallel call.
from functory.
i found this:
https://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html
which may indicate that use of the Accelerate framework which provides optimized blas and lapack used by lacaml conflicts with the parallelization strategy in functory?
from functory.
ok, i recompiled lacaml with openblas instead of Accelerate. for me this seems to take care of the intermittent crashing of child processes. it seems that really forking + Accelerate on mac is a bad combination. so i think it's enough to fix anything that appears without lacaml, and to tell os x users to beware of Accelerate.
from functory.
from functory.
Related Issues (1)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from functory.