oblq / workerful

Minimal Go worker-pool implementation.

Workerful

Workerful is a minimal yet powerful worker-pool implementation with some helper funcs for a simple usage.

Install

$ go get github.com/oblq/workerful

Quickstart

It is possible to pass the config as a yaml file path, the workerful.Config struct or none of the above (the default values will be loaded).
The config struct takes two parameters:

Parameter	Description	Default
queue_size	Determine the size of the queue. Use 0 to have an unbuffered jobQueue channel.	0 (unbuffered)
workers	The number of parallel jobs to be executed. Using 0 will load the default value.	runtime.NumCPU()

Get a new instance (with unbuffered jobQueue):

wp := workerful.New("", &workerful.Config{QueueSize: 0, Workers: 0})

Push a job

You can push a simple func() error or a more complex custom Job (see workerful.Job interface).

Push a simple func:

wp.PushFunc(func() error { 
    println("func executed...")
    return nil
})

// Non blocking.
wp.PushFuncAsync(func() error { 
    println("func executed...")
    return nil
})

Push a custom job if you need to pass parameters or for more complicated stuff:

// CustomJob implement the workerful.Job interface (F())
type CustomJob struct {
    WG  *sync.WaitGroup
    ID  int
}

func (cj CustomJob) F() error {
    time.Sleep(time.Second)
    println("job", cj.ID, "executed...")
    cj.WG.Done()
    return nil
}
    
wp := workerful.New("", nil)

waitGroup := sync.WaitGroup{}
waitGroup.Add(2)

wp.PushJob(CustomJob{&waitGroup, 1})
    
// Non blocking.
wp.PushJobAsync(CustomJob{&waitGroup, 2})

waitGroup.Wait()

println("finished")
wp.Stop()

Get status

doneJobs, failedJobs, inQueueJobs := wp.Status()

Graceful shutdown

wp := workerful.New("./workerful.yml", nil)
wp.Stop()

Author

• Marco Muratori

License

Workerful is available under the MIT license. See the LICENSE file for more information.