monad-par's People
Contributors
Stargazers
Watchers
Forkers
tpetricek axman6 athas erjiang bgamari mainland scpmw cgaebel travitch knsd samth prabh-t maoe icelandjack mikerainey maxgabriel creichert 23skidoo herberteuler ivanknmk lematt1991 hvr maximilianalgehed erikd treeowl phadej skorikgg prozacchiwawa locallycompact teofilc chrismanning ocharles andreasabel wendaolee marcin-serwinmonad-par's Issues
thread blocked indefinitely in an MVar operation with parMap
Bug was originally reported against crierion. But Till Berger created reduced test case:
import Control.Monad.Par
test :: [Int] -> IO [Int]
test xs = do
let list = runPar $ parMap (\x -> x + 1) xs
putStrLn $ show list
test list
main = do
test [1]
If compiled with -threaded program fails after a few (5-30) iterations with message thread blocked indefinitely in an MVar operation or occasionaly with message Impossible state in globalWorkComplete. If it's compiled without threadng it still fails but it require much more iterations (tens of thousands)
Second listing in "Other Slides" 404s
The second link under "Other Slides" on this page of the documentation 404s.
http://www.cs.indiana.edu/~rrnewton/talks/2011_HaskellSymposium_ParMonad.pdf
Should it just be removed?
Benchmarks in examples/ should include oracle for correctness
At least in SHORTRUN=1 mode.
Presently, [2011.10.20] it runs timings but does not verify the correctness of the answers that come back.
Is it possible to add a MonadFix instance?
The original paper gives an example of a parallel type checker that manually creates IVars rather than using spawn or spawn_. It advertises that this avoids needing to deal with a fixed-point function. Well, tastes vary in all things, and I personally think it would be very nice to have that fixed-point function! Is it feasible to implement a MonadFix instance, or is the CPSish representation (or something else) incompatible with that?
Nested scheduler bug
The following program fails with <<loop>> and blocked indefinitely on MVar exceptions with monad-par-0.3 but not with monad-par-0.1.0.3. It looks like it is using nesting quite heavily.
{-
$ cabal install -O2 monad-par-0.3
$ ghc -O2 -threaded -rtsopts -with-rtsopts -N turbofibpar.hs
$ ./turbofibpar 10000000
2089877
$ ./turbofibpar 10000000
turbofibpar: <<loop>>turbofibpar: turbofibpar: turbofibpar: thread blocked indefinitely in an MVar operation
<<loop>>
<<loop>>
turbofibpar: <<loop>>
$ ghc -V
The Glorious Glasgow Haskell Compilation System, version 7.4.1
-}
import Control.Monad.Par
import System.Environment (getArgs)
import Control.DeepSeq
data M = M !Integer !Integer !Integer !Integer
instance NFData M
instance Num M where
m * n = runPar $ do
m' <- spawn (return m)
n' <- spawn (return n)
m'' <- get m'
n'' <- get n'
return (m'' `mul` n'')
(M a b c d) `mul` (M x y z w) = M
(a * x + b * z) (a * y + b * w)
(c * x + d * z) (c * y + d * w)
fib :: Integer -> Integer
fib n = let M f _ _ _ = M 0 1 1 1 ^ (n + 1) in f
main :: IO ()
main = print . length . show . fib . read . head =<< getArgsOffer a version of put that can run multiple times
There are some pretty reasonable situations where it makes sense to allow multiple threads to race to fill the same IVar. Currently, that programming model is prohibited. The lvish package provides limited support by using == to test for equality on a second put. I think this seems an eminently reasonable thing to do. I also think it would also make sense to offer an "unsafe" version that simply does nothing when the IVar is full. I think implementation probably shouldn't be too terrible; one way would be to change the Put constructor to
| forall a . Put (IVar a) a (Bool -> Trace)This informs the continuation of whether the IVar was already full.
Test suite fails on powerpc
Hi,
when trying to build this on powerpc with GHC-7.10, I observe sporadic test failures. Unfortunately, Cabal is not very chatty, so I cannot say much:
Running debian/hlibrary.setup test --builddir=dist-ghc --show-details=always
Running 1 test suites...
Test suite test-monad-par: RUNNING...
Test suite test-monad-par: FAIL
Test suite logged to: dist-ghc/test/monad-par-0.3.4.7-test-monad-par.log
0 of 1 test suites (0 of 1 test cases) passed.
The fact that there is no output at all might indicate a GHC bug, so I’m pinging @erikd because often architecture-specific bugs are only ever fixed if he is involved...
monad-par-0.3.4.8 test suite does not compile with ghc-8.6.1
I get several errors of the following kind:
tests/ParTests_shared.hs:88:7: error:
• No instance for (Control.Monad.Fail.MonadFail Par)
arising from a do statement
with the failable pattern ‘[a, b, c, d]’
• In a stmt of a 'do' block:
[a, b, c, d] <- sequence [new, new, new, new]
In the second argument of ‘($)’, namely
‘do [a, b, c, d] <- sequence [new, new, ....]
fork
$ do x <- get a
put b (x + 1)
fork
$ do x <- get a
put c (x + 2)
fork
$ do x <- get b
y <- get c
....
....’
In the expression:
runPar
$ do [a, b, c, d] <- sequence [new, new, ....]
fork
$ do x <- get a
put b (x + 1)
fork
$ do x <- get a
put c (x + 2)
fork
$ do x <- get b
y <- get c
....
....
|
88 | [a,b,c,d] <- sequence [new,new,new,new]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
~~~
getEmpty unit test fails in a way that calls into question exception handling
See notes in ParTests.hs.
This is only an issue with the Direct scheduler, and seems to emerge from the attempt to catch errors on worker threads. Trace doesn't bother, and misses any exceptions from child threads, so it avoids this problem.
Turning on WAIT_FOR_WORKERS in Direct does not help the problem. However, it does seem to result in a bunch of downstream failures currently:
justReturn: [Failed]
ERROR: thread blocked indefinitely in an MVar operation
oneIVar: [OK]
forkNFill: [Failed]
ERROR: thread blocked indefinitely in an MVar operation
getEmpty: [Failed]
ERROR: thread blocked indefinitely in an MVar operation
test diamond: [Failed]
ERROR: thread blocked indefinitely in an MVar operation
Exception inside child thread "(worker 0 of originator ThreadId 13)", ThreadId 40: thread blocked indefinitely in an MVar operation
Exception inside child thread "(worker 1 of originator ThreadId 13)", ThreadId 41: thread blocked indefinitely in an MVar operation
Exception inside child thread "(worker 3 of originator ThreadId 13)", ThreadId 43: thread blocked indefinitely in an MVar operation
test pmrr1: [Failed]
ERROR: thread blocked indefinitely in an MVar operation
Replace forkOnIO with forkOn
forkOnIO has been deprecated since GHC 7.2, and removed in GHC HEAD.
I can prepare a patch if you wish, but I'm not sure whether it is considered sufficient simply to replace forkOnIO with forkOn, or if we should put CPP guards to ensure compatibility with old GHCs (<= 7.0.x, I guess).
Offer lazy put and newFull
I understand that these functions are designed to force values to make it easier to reason about what threads computations are performed on. But this is Haskell, and sometimes we want to do something subtle. Making users add datatype wrappers to avoid forcing their values just seems rude. I suggest lazyPut and lazyNewFull, but I don't care terribly much about the names.
TODO: For idling: use a binary-tree, not a global semaphore
One strategy is to organize workers into a tree such that each worker only listens to its parent and only signals its children to switch from active work-stealing to idle standby mode.
This would reduce contention on a global idle list or semaphore.
The Direct module exposes way too much
I think it's pretty weird that Direct.Par derives a MonadReader Sched instance. It's also pretty weird that Direct exposes the Sched type despite not offering any documentation of it or any functions for working with it. I think it probably makes sense to remove that instance and require users to import DirectInternal to get the Sched type.
Can we improve reschedule?
reschedule queue@Sched{ workpool } = do
e <- atomicModifyIORef workpool $ \ts ->
case ts of
[] -> ([], Nothing)
(t:ts') -> (ts', Just t)
--[...]The pairs will actually be realized, because atomicModifyIORef needs them. But I suspect that atomicModifyIORef is actually overkill in this case. In particular, we only deconstruct ts; we don't have to construct any new values. I suspect, therefore, that casMutVar# can do the job more cheaply. Benchmarking will be needed to test this hypothesis.
Why is LiftIO the way it is?
Currently,
data Trace = ...
| forall a. LiftIO (IO a) (a -> Trace)Why not just
| LiftIO (IO Trace)Does that cause some sort of trouble?
Missing `Applicative Par` instance
Control/Monad/Par/Scheds/DirectInternal.hs:44:15:
No instance for (Applicative Par)
arising from the 'deriving' clause of a data type declaration
Possible fix:
use a standalone 'deriving instance' declaration,
so you can specify the instance context yourself
When deriving the instance for (Monad Par)
Support more IO operations for ParIO monad
I noticed that ParIO monad has a pretty restricted API for dealing with IO computations. Basically, it only provides a liftIO function for embedding IO actions inside ParIO. But what about IO things like bracket, mask, etc? Is there any reason why they were not added apart from a lack of working hands?
Trace sheduler works as stack, not a queue.
For example:
...
runPar $ do
...
fork f1 >> fork f2 >> fork f3 >> fork f4 >> fork f5 >> fork f6 >> fork f7 >> fork f8
fork f9 >> fork f10 >> fork f11 >> fork f12 >> fork f13 >> fork f14 >> fork f15 >> fork f16
...
On a quad-core processor, execution of forks will occur in the order of f1,f2,f3,f4,f16,f15,14,..., f5.
Thus, the time to execute f5 will reach when the remaining threads are processed.
This behavior can degrade performance and reduce parallelism due to the long wait for f5 results.
Cabalize benchmark.hs and remove HSH dependency
This has been a big dependency liability and is one of the things causing trouble for full GHC 7.6 compliance.
Support for modern mtl and transformers?
Hi,
monad-par-extras restricts the its mtl and transformers dependencies to version 0.2. Would it be possible to add support for the current versions of these packages?
Take care,
Peter
Add parallelism to Data.Vector while maintaining fusion
Related to Issue #9:
The monad-par package currently provides a "parMap" for everything traversable.
Sadly, this is woefully inefficient for data structures like arrays and Data.Vector, which can directly support a divide-and-conquer rather than element-wise traversal.
This raises the question of how best to add this functionality to vector. A separate vector-parallel package? Would that present any barriers to fusion laws (e.g. parallel maps fuse, and for that matter a serial + parallel map should perhaps fuse too).
ST monad trick
Hi, I was just re-reading your excellent book and came across this warning:
There is nothing in the types to stop you from returning an IVar from runPar and
passing it to another call of runPar. This is a Very Bad Idea; don’t do it. The
implementation of the Par monad assumes that IVars are created and used
within the same runPar, and breaking this assumption could lead to a runtime
error, deadlock, or worse.
The library could prevent you from doing this using qualified types in the same
way that the ST monad prevents you from returning an STRef from runST. This
is planned for a future version.
Just curious, is this still meant to happen at some point?
Test pmrr1 reveals weird (but common) pattern of non-deterministic failure. (Direct)
When running the test suite generated by --enable-tests for the monad-par package, in revision 60aabe6, I see non-determinstic failures on the Direct scheduler.
Specifically, if I try to run this command 10 times:
./dist/build/test-monad-par/test-monad-par -j1
It will fail. However, the weird part is that it depends on running multiple tests together. If I run JUST the pmrr1 test, even 1000 times, I cannot reproduce the error:
./dist/build/test-monad-par/test-monad-par -j1 -t pmrr1
This is exactly the same pattern I've seen in other projects that use test-framework to drive parallel tests, including lattice-par and haskell-lockfree -- the outcome depends on what set of tests is run. I haven't figured out WHY this is yet.
Release 0.3.4?
Can we do a new release? The big thing is that we need to switch to the direct scheduler to avoid the nested scheduler bug (#23). Also, it would be nice to have ParIO, which isn't in any released version yet, but I want to mention it in my book.
Cabalize examples
Chase-Lev queue looks like trouble
Aside from all their other devilish subtleties, it appears to be that Chase-Lev queues don't completely guarantee that left and right pops don't both take the last queue element. If that happens here, I believe two threads could both try to fill the same IVar, in which case the second one will fill the IVar with an exception thunk (and then throw an exception). To fix this, we'd need put_ to take care to leave the IVar value alone when it's full. That's easy enough, but I don't think it's nearly enough: what do we do about the exception? A particularly nasty case involves two IVars, A and B, and two copies of the same thread, P and Q. Suppose P and Q each spawn threads to fill A and B, and those happen to do so in the opposite orders. Then P will die because B is full, and Q will die because A is full. I therefore believe that when Chase-Lev queues are enabled, non-determinism errors should be disabled.
monad-par-0.3.4.9 violated PVP by introducing new features
Non-breaking change. Otherwise, if only new bindings, types, classes, non-orphan instances or modules (but see below) were added to the interface, then A.B MAY remain the same but the new C MUST be greater than the old C. Note that modifying imports or depending on a newer version of another package may cause extra non-orphan instances to be exported and thus force a minor version change.
Between 0.3.4.8 and 0.3.4.9 I see added fixPar combinators, and MonadFix instances.
cc @hvr
I'd recommend blacklisting the release, and re-releasing it as 0.3.5.
Maybe "pval" should go away or be spawnPure??
I worry about the API having too many convenience routines with put vs. put_ and { fork, spawn, spawn_, pval }.
Apply class hierarchy to manage Par implementations and variants
This is a big TODO item. I'm creating a ticket for it just so we have a place to make comments on the progress of this effort.
Currently (985d866) there's a ParClass file that defines a simple class. But we don't force users to go through it. Once we've verified that there is no performance penalty we need to do that.
Further, we need to replace the one class with a hierarchy that models different capabilities. Here are some examples:
- Futures only
- IVars
- Streams
Adam Foltzer (acfoltzer) is working on hashing out this hierarchy and putting it into place.
Add CI
We could really use some help from Travis (or something) here.
Simpler ParFuture
The current ParFuture type class is parameterized with both the monad m and the future. Since the only operation that can be applied to a future is get why not perform that operation inside spawn so that the user doesn't need to do it and can't accidentally put the future twice. This also eliminates the requirement for MultiParamTypeClasses and FunctionalDependencies for this type class:
class Monad m => Spawn m where
-- | Create a potentially-parallel computation, and return a /future/
-- (or /promise/) computation that can be used to query the result of the forked
-- computation.
--
-- > spawn p = do
-- > r <- new
-- > fork (p >>= put r)
-- > return (get r)
--
spawn :: NFData a => m a -> m (m a)
-- | Like 'spawn', but the result is only head-strict, not fully-strict.
spawn_ :: m a -> m (m a)(Note I also use this interface in my threads package.)
Improve exception story
It seems that BlockedIndefinitelyOnMVar tends to leak through when things go wrong. It would be nice to think through where this can happen and install handlers to provide more informative exceptions.
Basic map causes "thread blocked indefinitely in an MVar operation"
I added parallelism at the top level of my program:
import Control.Monad.Par
subsetProd vals flags [h:hs] =
let valpairs = chunksOf 2 vals
flagpairs = chunksOf 2 flags
pairs = zip valpairs flagpairs
mulSwiRed h' [v0@(p, _),v1] [f0,f1]
| (not f0) && (not f1) = one p
| (not f0) = mswitch v1
| (not f1) = mswitch v0
| otherwise = mswitch $ kswitch h' $ mul v0 v1
prods = runPar $ parMap (uncurry $ mulSwiRed h) pairs
in subsetProd prods (map or flagpairs) hs
The length of pairs (what I'm parMapping over) starts at 64 and gets cut in half in each recursive call.
I'm compiling with:
ghc -rtsopts -threaded -optc-O3 -optlo-O3 -O3 -fllvm -funbox-strict-fields -funfolding-use-threshold1000 -funfolding-keeness-factor1000 -Odph -feager-blackholing SubsetProd
though I get the same error with just -O2 -threaded
After 2 minutes or so, I get:
>./SubsetProd ... +RTS -K1000000000 -M4294967295 -N4 -stderr
Exception inside child thread "(worker 2 of originator ThreadId 20)", ThreadId 26: thread blocked indefinitely in an MVar operation
Exception inside child thread "(worker 3 of originator ThreadId 20)", ThreadId 27: thread blocked indefinitely in an MVar operation
Exception inside child thread "(worker 1 of originator ThreadId 20)", ThreadId 25: thread blocked indefinitely in an MVar operation
...
SubsetProd: thread blocked indefinitely in an MVar operation
604,797,224,864 bytes allocated in the heap
12,114,315,456 bytes copied during GC
1,667,041,344 bytes maximum residency (28 sample(s))
336,490,696 bytes maximum slop
4363 MB total memory in use (586 MB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 383952 colls, 163672 par 47.79s 18.57s 0.0000s 0.0074s
Gen 1 28 colls, 25 par 21.35s 12.59s 0.4495s 5.2658s
Parallel GC work balance: 32.09% (serial 0%, perfect 100%)
TASKS: 10 (1 bound, 9 peak workers (9 total), using -N4)
SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled)
INIT time 0.00s ( 0.00s elapsed)
MUT time 323.44s ( 98.10s elapsed)
GC time 69.14s ( 31.15s elapsed)
EXIT time 0.00s ( 0.03s elapsed)
Total time 392.57s (129.28s elapsed)
Alloc rate 1,869,911,961 bytes per MUT second
Productivity 82.4% of total user, 250.2% of total elapsed
gc_alloc_block_sync: 1103901
whitehole_spin: 0
gen[0].sync: 1242
gen[1].sync: 1532431
I'm using GHC 7.6.2 x64 for *nix, and monad-par-0.3.4.2. The only other clue I have that might help someone debug this is that my code uses a lot of memory, hence the RTS options. I'm willing to help debug this/provide more information if someone tells me what they need.
Replace logging and debugging infrastructure with GHC Events
This would be better than printf messages.
Build failures with mtl-2.3
GHC error messages
Control/Monad/Par/Scheds/Direct.hs:158:24: error:
• Variable not in scope:
lift
:: a0 -> ContT () Control.Monad.Par.Scheds.DirectInternal.ROnly a
• Perhaps you meant one of these:
‘RD.lift’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative)
|
158 | unsafeParIO iom = Par (lift$ lift iom)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:158:30: error:
• Variable not in scope: lift :: IO a -> a0
• Perhaps you meant one of these:
‘RD.lift’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative)
|
158 | unsafeParIO iom = Par (lift$ lift iom)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:210:3: error:
Variable not in scope: when :: Bool -> IO () -> IO a38
|
210 | when dbg $ case mb of
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:220:3: error:
Variable not in scope: when :: Bool -> IO () -> IO a41
|
220 | when dbg $ do sn <- makeStableName task
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:234:3: error:
Variable not in scope: when :: Bool -> IO () -> IO ()
|
234 | when (not (Prelude.null idles)) $ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:235:5: error:
Variable not in scope: when :: Bool -> a39 -> IO a40
|
235 | when dbg$ printf "Waking %d idle thread(s).\n" (length idles)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:272:24: error:
Variable not in scope: when :: Bool -> Par () -> Par a19
|
272 | let userComp' = do when dbg$ io$ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:278:31: error:
Variable not in scope: when :: Bool -> IO () -> IO a18
|
278 | io$ do when (dbglvl>=1) $ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:287:28: error:
• Variable not in scope:
liftIO :: IO Bool -> RD.ReaderT Sched IO t0
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
287 | loop n = do flg <- liftIO$ readIORef newFlag
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:288:21: error:
Variable not in scope:
unless :: t0 -> RD.ReaderT Sched IO () -> RD.ReaderT Sched IO ()
|
288 | unless flg $ do
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:289:23: error:
Variable not in scope:
when :: Bool -> a16 -> RD.ReaderT Sched IO a17
|
289 | when dbg $ liftIO$ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:289:34: error:
• Variable not in scope: liftIO :: IO () -> a16
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
289 | when dbg $ liftIO$ do
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:301:5: error:
Variable not in scope: when :: Bool -> IO () -> IO a15
|
301 | when (dbglvl>=1)$ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:350:8: error:
Variable not in scope: when :: Bool -> a35 -> IO a36
|
350 | when (dbglvl>=1)$ printf " [%d %s] runPar called from existing worker thread, new session (%d)....\n" (no sched) (show tid) (sid0 + 1)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:361:21: error:
• Variable not in scope:
forM
:: [(a33, Sched)]
-> ((Int, Sched) -> IO (Maybe (MVar Int))) -> IO [Maybe a34]
• Perhaps you meant one of these:
‘PC.fork’ (imported from Control.Monad.Par.Class),
‘fork’ (line 576)
|
361 | doneFlags <- forM (zip [0..] allscheds) $ \(cpu,sched) -> do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:371:26: error:
Variable not in scope: when :: Bool -> a31 -> IO a32
|
371 | then do when dbg$ printf " [%d %s] Anonymous worker entering scheduling loop.\n" cpu (show tid2)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:373:26: error:
Variable not in scope: when :: Bool -> a29 -> IO a30
|
373 | when dbg$ printf " [%d] Anonymous worker exited scheduling loop. FINISHED.\n" cpu
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:379:26: error:
Variable not in scope: when :: Bool -> a27 -> IO a28
|
379 | when dbg$ do printf " *** Out of entire runContT user computation on main thread %s.\n" (show tid2)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:387:8: error:
Variable not in scope: when :: Bool -> m1 b3 -> IO a26
|
387 | when _WAIT_FOR_WORKERS $ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:388:12: error:
Variable not in scope: when :: Bool -> a24 -> m1 a25
|
388 | when dbg$ printf " *** [%s] Originator thread: waiting for workers to complete." (show tidorig)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:389:12: error:
Variable not in scope:
forM_ :: [a34] -> (MVar a23 -> IO b2) -> m1 b3
|
389 | forM_ (catMaybes doneFlags) $ \ mv -> do
| ^^^^^
Control/Monad/Par/Scheds/Direct.hs:392:14: error:
Variable not in scope: when :: Bool -> a22 -> IO b2
|
392 | when dbg$ printf " * [%s] Worker %s completed\n" (show tidorig) (show n)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:394:8: error:
Variable not in scope: when :: Bool -> a20 -> IO a21
|
394 | when dbg$ do printf " *** [%s] Reading final MVar on originator thread.\n" (show tidorig)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:411:4: error:
Variable not in scope: when :: Bool -> IO () -> IO a42
|
411 | when dbg$ do tid <- myThreadId
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:413:17: error:
• Variable not in scope:
replicateM
:: Int -> IO (SimpleDeque elt0) -> IO [SimpleDeque (Par ())]
• Perhaps you meant ‘replicate’ (imported from Prelude)
|
413 | workpools <- replicateM numCapabilities $ R.newQ
| ^^^^^^^^^^
Control/Monad/Par/Scheds/Direct.hs:414:17: error:
• Variable not in scope:
replicateM
:: Int
-> IO (HotVar (Random.Gen ghc-prim-0.8.0:GHC.Prim.RealWorld))
-> IO [HotVar (Random.Gen ghc-prim-0.8.0:GHC.Prim.RealWorld)]
• Perhaps you meant ‘replicate’ (imported from Prelude)
|
414 | rngs <- replicateM numCapabilities $ Random.create >>= newHotVar
| ^^^^^^^^^^
Control/Monad/Par/Scheds/Direct.hs:592:7: error:
Variable not in scope: when :: Bool -> Par () -> Par ()
|
592 | when dbg$ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:599:7: error:
Variable not in scope: when :: Bool -> Par () -> Par a56
|
599 | when dbg$ io$ printf " [%d] forking task...\n" (no sch)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:612:3: error:
Variable not in scope:
when :: Bool -> a13 -> RD.ReaderT Sched IO a14
|
612 | when dbg$ liftIO$ do tid <- myThreadId
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:612:13: error:
• Variable not in scope: liftIO :: IO () -> a13
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
612 | when dbg$ liftIO$ do tid <- myThreadId
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:617:13: error:
• Variable not in scope:
liftIO
:: IO (Maybe (Par ())) -> RD.ReaderT Sched IO (Maybe (Par a12))
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
617 | mtask <- liftIO$ popWork mysched
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:620:43: error:
• Variable not in scope:
liftIO :: IO [Session] -> RD.ReaderT Sched IO [Session]
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
620 | (Session _ finRef):_ <- liftIO$ readIORef $ sessions mysched
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:621:26: error:
• Variable not in scope:
liftIO :: IO Bool -> RD.ReaderT Sched IO Bool
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
621 | fin <- liftIO$ readIORef finRef
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:623:28: error:
Variable not in scope:
when :: Bool -> a10 -> RD.ReaderT Sched IO a11
|
623 | then do when (dbglvl >= 1) $ liftIO $ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:623:49: error:
• Variable not in scope: liftIO :: IO b1 -> a10
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
623 | then do when (dbglvl >= 1) $ liftIO $ do
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:629:30: error:
Variable not in scope: when :: Bool -> a9 -> IO b1
|
629 | when (not empt) $ do
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:640:22: error:
• Variable not in scope: liftIO :: IO () -> RD.ReaderT Sched IO a8
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
640 | liftIO$ steal mysched
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:644:22: error:
• Variable not in scope: liftIO :: IO () -> RD.ReaderT Sched IO a7
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
644 | liftIO yield
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:648:8: error:
Variable not in scope:
when :: Bool -> m0 b0 -> RD.ReaderT Sched IO a6
|
648 | when dbg $ do sn <- liftIO$ makeStableName task
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:648:28: error:
• Variable not in scope:
liftIO
:: IO (GHC.StableName.StableName (Par a12))
-> m0 (GHC.StableName.StableName a5)
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
648 | when dbg $ do sn <- liftIO$ makeStableName task
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:649:22: error:
• Variable not in scope: liftIO :: a4 -> m0 b0
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
649 | liftIO$ printf " [%d] popped work %d from own queue\n" (no mysched) (hashStableName sn)
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:654:12: error:
Variable not in scope: when :: Bool -> a2 -> RD.ReaderT Sched IO a3
|
654 | when dbg$ liftIO$ printf " + task finished successfully on cpu %d, calling reschedule continuation..\n" (no sch)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:654:22: error:
• Variable not in scope: liftIO :: a1 -> a2
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
654 | when dbg$ liftIO$ printf " + task finished successfully on cpu %d, calling reschedule continuation..\n" (no sch)
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:664:3: error:
Variable not in scope: when :: Bool -> IO () -> IO a43
|
664 | when (dbglvl>=2)$ do tid <- myThreadId
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:681:22: error:
Variable not in scope: when :: Bool -> a54 -> IO a55
|
681 | when dbg$ printf " [%d] | waking up all threads\n" my_no
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:690:26: error:
Variable not in scope: when :: Bool -> a52 -> IO a53
|
690 | when dbg$ printf " [%d] | shutting down\n" my_no
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:693:26: error:
Variable not in scope: when :: Bool -> a50 -> IO a51
|
693 | when dbg$ printf " [%d] | woken up\n" my_no
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:708:10: error:
Variable not in scope: when :: Bool -> a48 -> IO a49
|
708 | when (dbglvl>=2)$ printf " [%d] | trying steal from %d\n" my_no (no schd)
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:716:15: error:
Variable not in scope: when :: Bool -> IO () -> IO a47
|
716 | when dbg$ do sn <- makeStableName task
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:721:20: error:
Variable not in scope:
when :: Bool -> m3 b5 -> RD.ReaderT Sched IO a46
|
721 | when dbg$ do sn <- liftIO$ makeStableName task
| ^^^^
Control/Monad/Par/Scheds/Direct.hs:721:39: error:
• Variable not in scope:
liftIO
:: IO (GHC.StableName.StableName (Par ()))
-> m3 (GHC.StableName.StableName a45)
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
721 | when dbg$ do sn <- liftIO$ makeStableName task
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:722:33: error:
• Variable not in scope: liftIO :: a44 -> m3 b5
• Perhaps you meant one of these:
‘RD.liftIO’ (imported from Control.Monad.Reader),
‘liftA’ (imported from Control.Applicative),
‘liftA2’ (imported from Control.Applicative)
|
722 | liftIO$ printf " [%d] | DONE running stolen work (unit %d) from %d\n" my_no (hashStableName sn) (no schd)
| ^^^^^^
Control/Monad/Par/Scheds/Direct.hs:863:3: error:
Variable not in scope:
forM_ :: [Sched] -> (Sched -> IO b4) -> IO a37
|
863 | forM_ allscheds $ \ Sched{no, workpool} -> do
| ^^^^^
Control/Monad/Par/Scheds/Direct.hs:865:6: error:
Variable not in scope: when :: Bool -> m2 () -> IO b4
|
865 | when (not b) $ do
| ^^^^
This affects v0.3.4.7, v0.3.4.8 and v0.3.5.
As a Hackage trustee I have created revisions for these versions that should prevent users from encountering these errors. See e.g. https://hackage.haskell.org/package/monad-par-0.3.5/revisions/.
thread blocked indefinitely in an MVar operation (HP 2013 ghc 7.6.3 on mac) with current monad par via criterion
This is the issue i reported in Criterion haskell/criterion#28 and mistakenly in an older closed monad-par ticket.
Likely hit via Criterion calling the following function in Statistics that is the only use of Monad par in Criterion or Statistics haskell/criterion#28
Eratic partree benchmark runtime
I had some irregular results while running the benchmark script. The machine has a quad core i7 Nehalem. The benchmark itself is running in a Vmware virtual machine with 8 cores ( 1 per CPU thread ). Host OS is Win7 x64.
The partree benchmark usually takes under one second to run, but in this particular instance took over 40 seconds.
# TestName Variant NumThreads MinTime MedianTime MaxTime
#
# Tue Oct 4 12:25:17 PDT 2011
# Linux ubuntu 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
# Determined machine to have 8 hardware threads.
# The Glorious Glasgow Haskell Compilation System, version 7.0.3
#
# Running each test for 1 trials.
# ... with default compiler options: -O2 -rtsopts
# ... with default runtime options: -qa
# Using the following settings from the benchmarking environment:
# BENCHLIST= THREADSETTINGS=8 TRIALS= SHORTRUN=1 KEEPGOING= GHC= GHC_FLAGS= GHC_RTS=
# *** Config [0 ..], testing with command/args: parfib.exe
parfib.exe 8 0.00 0.00 0.00
# *** Config [1 ..], testing with command/args: parfib.exe
parfib.exe 8 0.00 0.00 0.00
# *** Config [2 ..], testing with command/args: blackscholes.exe
blackscholes.exe 8 0.21 0.21 0.21
# *** Config [3 ..], testing with command/args: cholesky.exe
cholesky.exe 8
# *** Config [4 ..], testing with command/args: nbody.exe
nbody.exe 8 0.03 0.03 0.03
# *** Config [5 ..], testing with command/args: mandel.exe
mandel.exe 8 0.10 0.10 0.10
# *** Config [6 ..], testing with command/args: coins.exe
coins.exe 8 0.94 0.94 0.94
# *** Config [7 ..], testing with command/args: queens.exe
queens.exe 8 0.22 0.22 0.22
# *** Config [8 ..], testing with command/args: partree/partree.exe
partree/partree.exe 8 40.33 40.33 40.33
# *** Config [9 ..], testing with command/args: matmult/matmult.exe
matmult/matmult.exe 8 0.28 0.28 0.28
# *** Config [10 ..], testing with command/args: sumeuler/sumeuler.exe
sumeuler/sumeuler.exe 8 0.93 0.93 0.93
Duplicate documentation in Haddocks
The current Haddocks have duplicate lines for several of the functions. Potentially this is caused by documenting both the original function and the export? I'm not sure yet.
Would you like me to look into cleaning this up?
Example:
put_ :: IVar a -> a -> Par ()like put, but only head-strict rather than fully-strict.
like put, but only head-strict rather than fully-strict.
https://hackage.haskell.org/package/monad-par-0.3.4.7/docs/Control-Monad-Par.html
Add a Par+Log that adds print messages to Par implemenations
For printf-debugging inside Par it would be awfully nice if there were a layer offering print messages on top of any of the Par schedulers. Just like the Par+RNG this would involve a StateT, but in this case it would keep track of logged messages and return them at the end.
It would be deterministic of course. The result is a binary tree of ordered lists of messages. Displaying that effectively would be important.
You would want to see these messages before Par completes. That would require that runPar return both an answer and a (lazy) log of messages. And that in turn would bring the problems of lazy IO....
Add phantom type parameter (like ST) to prevent IVar escape
As described in the paper, currently IVars can be returned from Par computations and then used in other Par computations. This can be prevented in the type system but it hasn't been done yet.
TODO: Use real CAS and real deques
When they are ready we need to switch over to:
https://github.com/rrnewton/haskell-lockfree-queue
Likewise we need to test replacing our atomicModifyIORefs with their CAS-based counterparts.
Investigate Safe Haskell issues wrt MonadIO instances and run interfaces
From a Safe Haskell perspective, the following combination is safe:
instance MonadIO Par
runParIO :: Par a -> IO aHowever, if this instance exists alongside runPar :: Par a -> a, safety is gone.
A brute-force solution might involve a triangular module structure along with generous amounts of newtype deriving:
module SMP.Internal (Par, runPar, runParIO) where
newtype Par a = ...
deriving (MonadIO ...)
runPar = ...
runParIO = ...{-# LANGUAGE Safe #-}
module SMP.IO (Par, runParIO) where
-- use Par and runParIO from SMP.Internal{-# LANGUAGE Safe #-}
module SMP (Par, runPar) where
newtype Par a = Par (SMP.Internal.Par a)
deriving ({- not MonadIO -})
runPar = SMP.Internal.runPar . unPargit tag needed
Could someone tag the 0.3 release please? Thanks!
Harmonize MonadCont for Trace and Direct
Direct.Par has a MonadCont instance, while Trace.Par does not. Is there a reason for this? I think either both should or neither should. If we want to add one for Trace.Par, I believe it would look like this:
instance MonadCont Par where
callCC f = Par $ \c -> case f (\a -> Par $ \_ -> c a) of Par q -> q cI can see both sides:
-
If these schedulers are committed to operating in the future as they do now, then continuations are available so we might as well expose them.
-
If GHC eventually offers more primitive support, the continuations may cease to be available, in which case the instances should go away.
Enable Benchmark.hs to parallelize compile phase.
In particular it should produce all the executables at the outset, using unique names based on each configuration.
This would also enable shipping off a benchmark as a bundle of executables (e.g. Haskell CnC's "run_from_packed") which is especially useful when trying to benchmark, say, machines with old Redhat installs that don't run GHC easily.
Further, the code to traverse configurations should be generalized a bit and should support testing multiple GHC versions in one go.
Explicit export list in Meta.hs
monad-par-0.3.4 (from Hackage) test fails
I've tried this on Ubuntu 12.04 and Mac OS X 10.8.2, using Haskell Platform 2012.4.0.0 (GHC 7.4.2) in both cases. I started with a clean local package database. The test output is below. Please let me know if I can be of any assistance in helping to track this down.
AList HUnit Tests:
fromList1: [OK]
cons X3: [OK]
tail X3: [OK]
len bintree: [OK]
inspect tree1: [OK]
inspect tree2: [OK]
inspect tree3: [OK]
AList QuickCheck Tests :
map: [OK, passed 100 tests]
filter: [OK, passed 100 tests]
tofrom: [OK, passed 100 tests]
tofromB: [OK, passed 100 tests]
balance: [OK, passed 100 tests]
ParTests:
justReturn: [OK]
oneIVar: [OK]
forkNFill: [OK]
Good. Caught exception: <<timeout>>
getEmpty: [OK]
test diamond: [OK]
test pmrr1: [OK]
async test1: [Failed]
ERROR: Bad temporal pattern: ["A","D","B","C","72.2816102527359","E"]
Properties Test Cases Total
Passed 5 13 18
Failed 0 1 1
Total 5 14 19
Spin off AList into a standalone package for public consumption
It would be nice to separate AList out and fill out the instances & combinators
fork is too strict
This simple program does not run in parallel:
import Control.Monad.Par
fib :: Integer -> Integer
fib 0 = 1
fib 1 = 1
fib n = fib (n-1) + fib (n-2)
main =
args <- getArgs
let [n,m] = map read args
print $ runPar $ do
i <- new
j <- new
fork (put i (fib n))
fork (put j (fib m))
a <- get i
b <- get j
return (a+b)It isn't the old problem of fib not allocating: note that I used the Integer version that allocates.
No, the problem is that in the direct scheduler, fork is too strict: the work item is evaluated when put into the work pool (by pushL, which causes the fib call to be evaluated in the parent. It doesn't happen with spawnP, but I think that's only because spawnP isn't inlined enough to expose the strictness.
I've no problem with put being strict, but I thing fork being strict will cause confusion.
Maybe strictly speaking the inputs should be passed via IVars. That's defensible, but I think it would be nice if we didn't have to do that all the time. Making simple examples like the above work properly will avoid confusing users (it confused me, and I supposedly know what I'm doing :-).
Stack overflow in receive daemon when doing parfib_dist of 30 1 1
The title says it all:
[distmeta_M ThreadId 6] [rcvdmn] RUNNING STOLEN PAR WORK
Exception inside child thread "Daemon thread (ReceiveDaemon)": stack overflow
Stack space overflow: current size 8388608 bytes.
Use `+RTS -Ksize -RTS' to increase it.
Exception inside runParDist: thread blocked indefinitely in an MVar operation
parfib_dist.exe: thread blocked indefinitely in an MVar operation
This is as of 2644cde.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.