hderms / xpool

lightweight and flexible process pool

OVERVIEW

Project	XPool
Homepage	https://github.com/robgleeson/xpool
Documentation	http://rubydoc.info/github/robgleeson/xpool/frames
CI
Author	Rob Gleeson

DESCRIPTION

XPool is a lightweight process pool. The pool manages a group of subprocesses that are used when the pool is asked to dispatch a 'unit of work'. A 'unit of work' is defined as any object that implements the run method.

All subprocesses in the pool have their own message queue that the pool places work onto according to a very simple algorithm: the subprocess who has scheduled the least amount of work is the subprocess who will have the unit of work put onto its message queue. The message queue that each subprocess has is also what ensures work can be queued when the pool becomes dry (all subprocesses are busy).

EXAMPLES

The examples don't demonstrate everything that XPool can do. The API docs cover the missing pieces.

1.

A demo of how to schedule a unit of work:

#
# Make sure you define your units of work before
# you create a process pool or you'll get strange
# serialization errors.
#
class Unit
  def run
    sleep 1
  end
end
pool = XPool.new 3
pool.schedule Unit.new
pool.shutdown

2.

A demo of how to run a single unit of work across all subprocesses in the pool:

class Unit
  def run
    puts Process.pid
  end
end
pool = XPool.new 5
pool.broadcast Unit.new
pool.shutdown

3.

A demo of how you can interact with subprocesses through XPool::Process objects:

class Unit
  def run
    sleep 1
  end
end
pool = XPool.new 2
subprocess = pool.schedule Unit.new 
p subprocess.busy? # => true

4.

A demo of how to resize the pool from 5 to 2 subprocesses at runtime:

pool = XPool.new 5
pool.resize! 1..2
pool.shutdown

5.

A demo of how to gracefully shutdown but force a hard shutdown when 3 seconds pass by & all subprocesses have not exited:

class Unit
  def run
    sleep 5
  end
end
pool = XPool.new 5
pool.schedule Unit.new
pool.shutdown 3

DEBUGGING OUTPUT

XPool can print helpful debugging information if you set XPool.debug to true:

XPool.debug = true

Or you can temporarily enable debugging output for the duration of a block:

XPool.debug do 
  pool = XPool.new 5
  pool.shutdown
end

The debugging information you'll see is all about how the pool is operating. It can be interesting to look over even if you're not bug hunting.

SIGUSR1

All XPool managed subprocesses define a signal handler for the SIGUSR1 signal. A unit of work should never define a signal handler for SIGUSR1 because that would overwrite the handler defined by XPool. SIGUSR2 is not caught by XPool and it could be a good second option.

INSTALL

$ gem install xpool

LICENSE

MIT. See LICENSE.txt