microsoftresearch / dryad Goto Github PK

Now that .net core runs on Linux and plinq is included, that means that dryad could be effectively used on a massive compute cluster.

Seems like a very nice opportunity. Can anyone of the tram comment on this ?

https://github.com/dotnet/corefx

I'm getting this error when running the sample on my laptop, after following the instructions here: http://microsoftresearchsvc.github.io/Dryad/?topic=html/e992fd94-c956-481d-82e6-dbdf45daa722.htm

I think I followed the instructions correctly but may have missed something. Any ideas?

Hi
I have a couple of questions:

1. In the README.txt, in "Building Dryad" section, I wasn't sure about how to follow step
   (5) Run Build.bat in the Java directory at the top-level of the repository.
   I couldn't find "build.bat" in the repos.
2. Any instruction on how to run Dryad? I saw there is a Samples directory but have no idea how to run it.

Thanks
Haifeng

Storage keys should be automatically downloaded when accounts are owned by the user's default subscription.

User code errors are transmitted reasonably well through the job logging, but system failures tend to get discarded by the YARN logging. Need a way to selectively copy failure logs into Azure.

Good morning.

I have this graph code in dryad:

GraphBuilder XSet = moduleX^N;
GraphBuilder DSet = moduleD^N;
GraphBuilder MSet = moduleM^(N_4) ;
GraphBuilder SSet = moduleS ^(N_4) ;
GraphBuilder YSet = moduleY^N;
GraphBuilder HSet = moduleH ^ 1 ;
GraphBuilder XInput s = ( u g r i z 1 >= XSet ) || ( n e i g h b o r >= XSet ) ;
GraphBuilder YInput s = u g r i z 2 >= YSet ;
GraphBuilder XToY = XSet >= DSet >> MSet >= SSet ;
f r ( i = 0 ; i < N*4; ++i )

XToY = XToY || ( SSet . Ge tVe r t e x ( i ) >= YSet . Ge tVe r t e x ( i /4) ) ;

GraphBuilder YToH = YSet >= HSet ;
GraphBuilder HOutputs = HSet >= output ;
GraphBuilder final = XInput s || YInput s || XToY || YToH ||HOutputs ;

Also, in the paper http://research.microsoft.com/en-us/projects/dryadlinq/eurosys07.pdf there is the graph in page 4 that corresponds to the code. In all presentations that exist for DryadLinQ, I see this code but none of them says exactly what "module" is exactly and where it is defined. I would like a further explanation on this. Also,

1. I cannot find documentation on dryad Graph API, something that describes explicitly what VP base class is, where to write programs with the above kind of operators and all the commands and operators inside this C++ library to construct a dryad graph. I suppose "module" exists there and there is a further explanation for this datatype.

2. I cannot find a presentation where a dryadLinq code is explicitly mapped to the graph. I want an example in dryadLINQ code that maps every variable or every step of the code to the graph execution, in order to understand completely how dryadLINQ transforms the code into graph.

3. I would like a presentation concerning how to write parallel programs with Dryad but not DryadLINQ

It seems that the AM is failing since the RM restarts it; the second attempt fails immediately if the storage collections have been partially written.

See discussion here: #5 (comment)