Maybe "pval" should go away or be spawnPure?? about monad-par

[simonmar]

pval is a bit dodgy too, because it encourages you to share lazy computations between threads. I've never used pval in presentations, I always use (spawn . return). No objection to getting rid of it.

[rrnewton]

Actually, I thought of one small reason not to. It may be a helpful optimization in some cases to implement pvals as (Full x) IVarContents where the 'x' is a lazy computation. That way they would be treated as futures usually are -- gets would never block, they would just do the work.

This trick can be done for spawn too, IF nested runPars work well and are cheap.

[simonmar]

On 13/10/2011 09:55, Ryan Newton wrote:

Actually, I thought of one small reason not to. It may be a helpful optimization in some cases to implement pvals as (Full x) IVarContents where the 'x' is a lazy computation. That way they would be treated as futures usually are -- gets would never block, they would just do the work.

This trick can be done for spawn too, IF nested runPars work well and are cheap.

But wouldn't that remove the parallelism in examples like

x <- pval a
y <- pval b
get x
get y
....

?

Cheers,
Simon

[rrnewton]

Ah, you mean because X is gotten before y? Then assuming child stealing the first thief would have stolen X only to contend with the original thread.

There would still be some parallelism but it would require two steals. The second thief could work on Y while the main thread computes X.

[simonmar]

On 24/10/2011 14:38, Ryan Newton wrote:

Ah, you mean because X is gotten before y? Then assuming child
stealing the first thief would have stolen X only to contend with the
original thread.

There would still be some parallelism but it would require two
steals. The second thief could work on Y while the main thread
computes X.

I'm not sure I understand. You first said:

It may be a helpful optimization in some cases to implement pvals as
(Full x) IVarContents where the 'x' is a lazy computation

which implied to me that pval would not include a fork, and would not
create a stealable work item, but would just make an IVar that you could
block on. In which case, there's no parallelism in the example I gave:

x <- pval a
y <- pval b
get x
get y
....

regardless of how many steals happen.

I must have misunderstood what you meant - could you elaborate?

Cheers,
Simon

[rrnewton]

Oops! Yes, indeed. I'm sorry, my original description was overly abbreviated. The idea would be to make them regular sparks. (Of course, if the main Par scheduler was not then using sparks this opens up the question of how well spark parallelism and thread parallelism cooperate.)

I hope my subsequent comment then makes more sense:

"This trick can be done for spawn too, IF nested runPars work well and are cheap."

That is spawn would use runPar to make the computation pure and then would spark it. That gets into dangerous terrain, however, because it would seem to require returning an IVar from that runPar and using it in the outer runPar!

[simonmar]

On 24/10/2011 15:50, Ryan Newton wrote:

Oops! Yes, indeed. I'm sorry, my original description was overly
abbreviated. The idea would be to make them regular sparks. (Of
course, if the main Par scheduler was not then using sparks this
opens up the question of how well spark parallelism and thread
parallelism cooperate.)

I hope my subsequent comment then makes more sense:

"This trick can be done for spawn too, IF nested runPars work well
and are cheap."

That is spawn would use runPar to make the computation pure and then
would spark it. That gets into dangerous terrain, however, because
it would seem to require returning an IVar from that runPar and using
it in the outer runPar!

Ah, I see. I'm not keen on using a mixture of sparks and explicit
scheduling in Par - at the least, performance problems would be
difficult to debug. I see the great advantage of Par as being that we
can do away with the implicit/hidden/magic nature of sparks and have
full control of the scheduling in our library.

But don't let me stop you from experimenting!

Cheers,
Simon

[rrnewton]

Yes, I feel compelled to try every scheduling strategy and gather data on all of them ;-).

Also, I fear that we will always have to deal with pure code that creates sparks.