depsolver concurrency limitations about chef-server

[sdelano]

You've summarized the issue correctly. This bottleneck was introduced when we switched back from the pure erlang implementation of the dependency solver feature to using the Ruby/Gecode-based solver that shipped with Chef 10. The Chef Server keeps a pool of depsolver processes running and waiting for requests. When this pool is exhausted, the API returns 503, for Service Unavailable. It is not recommended that this pool size exceed the number of CPUs that you have on your Chef Server, as this is a compute-constrained resource.

Chef 12 ships with a new version of the pooler library (the library that handles said pooling) that now allows for requests to the pool to queue up when no workers are available. This hasn't yet been exposed in Chef 12 for the dependency solver, but it wouldn't take a lot of work to add that ability.

[sdelano]

Coming back to this issue again, the conclusion still stands that we should add queuing to the depsolver workers. I'll add the accepted-minor tag to this card, indicating that we should do this and that it's a minor issue.

We should also include upgrading to the newest dep_selector gem if we haven't done so already. @danielsdeleo made some great improvements there to optimize the Ruby portion of the code so that it doesn't have to do as much work building up redundant objects.

[heaven]

Hi, why is it a minor issue? We have 25 nodes and they all constantly fail to run chef-client.

[matthiasr]

You can work around it by raising the number of dependency solvers. The
default is very conservative.

On Mon, Jul 11, 2016, 17:24 Alexander S. [email protected] wrote:

Hi, why is it a minor issue? We have 25 nodes and they all constantly fail
to run chef-client.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#36 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAICBonIjLU9HVIuzBwzuZBqnIUDbpylks5qUm4igaJpZM4DEOqD
.

[heaven]

Yeah, thank you, looking into that now.