Coder Social home page Coder Social logo

mesos-distcc's Issues

Framework holding onto offers and blocking cluster

Resources on the Mesos console show this when running mesos-distcc with "-j8" and 16 CPU cluster (and one other task running).

Total 16 121.4 GB
Used 9 6.0 GB
Offered 7 115.4 GB
Idle 0 0 B

mesos-distcc is using 8 CPU as expected, but holding onto 7 offered CPUs and blocking use of cluster by other users (including other mesos-distcc runs).

One problem is that the declineOffer when tasks have already been started does "return". Any remaining offers in list won't be declined.

Even with that bug fixed, the framework doesn't seem to be declining the offers. My suspicion is that starting the sub-processs in statusUpdate is blocking any communication with Mesos. It is possible that the declineOffer could not be sent. The framework docs mention that Scheduler callbacks should not block.

My guess is that mesos-distcc needs to run the command either in the background and catch signal when it exits. Or run the scheduler and runner in parallel and use multiprocessing.Condition to signal that ready to run command.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.