olaven / farmhand

Simple and powerful background jobs for Clojure

Farmhand

Farmhand is a Clojure library for queuing jobs to be processed in the background. It is backed by Redis to enable ease of use, flexibility, and great performance.

In addition to the library, there is a web interface available to easily see which jobs are running, view and re-queue failed jobs, and more.

This project is largely inspired by Sidekiq and RQ.

Warning This library is beta software and the API is subject to change.

Table of Contents generated with DocToc

• Installation
• Usage
• Features
• Documentation
• Web Interface
• LICENSE

Installation

Leiningen:

[com.buckryan/farmhand "0.9.1"]

Usage

Before starting, you need to have a Redis server running. Visit redis.io to download. This example assumes Redis is running on localhost.

Farmhand is designed for both ease of use and power. Below is an example showing you the most common usage, but be sure to check out the Wiki to see what else it can do.

;; STEP 1: Require the Farmhand namespace
(require '[farmhand.core :as farmhand])

;; STEP 2: Instantiate a Farmhand server with 4 workers.
(farmhand/start-server {:redis {:host "localhost"}
                        :num-workers 4})

;; STEP 3: Jobs are regular ol' Clojure functions:
(defn my-long-running-function
  [a b]
  (println "starting long-running function")
  (Thread/sleep 20000)
  (println "exiting long-running function")
  {:farmhand/result (* a b)})

;; STEP 4: Queue that job! It will be processed by the running Farmhand server.
(farmhand/enqueue {:fn-var #'my-long-running-function
                   :args [1 2]})

;; STEP 5: Stop the server. This will allow any running jobs to complete.
(farmhand/stop-server)

Features

• Reliable insofar as Redis is. Dequeue operations are performed using a Lua script which Redis guarantees to be atomic.

• Supports reading from multiple queues. See the Queues documentation for details.

• Job processing is fully customizable through middleware support. See the Middleware docs.

• Jobs can be scheduled to run at a later time. The docs are here

• Automatic job retrying. It's as simple as

  (enqueue {:fn-var #'job-function :retry {:strategy :backoff}})

  This causes your job to retry 8 times over about 10 days. The retry mechanism is also fully customizable. Docs.

Documentation

The bulk of the documentation is available in the Wiki.

API docs are available here.

Web Interface

The Farmhand UI project provides a web interface for Farmhand. Hop over to that project to get started. Here's a preview of what it looks like:

LICENSE

Distributed under the Eclipse Public License, the same as Clojure.