pwfoo / serviceq

Super fault-tolerant gateway for HTTP clusters, written in Go. White paper for reference - https://github.com/gptankit/adareqmodel-paper

ServiceQ

ServiceQ is a fault-tolerant gateway for HTTP clusters. It employs probabilistic routing to distribute load during partial cluster shutdown (k/n nodes experiencing downtimes, timeouts, connection loss etc) and buffers requests during total cluster shutdown (n nodes down). The buffered requests are forwarded in FIFO order when the cluster is available next.

Below graph shows the routing probability (P) on a down node (D) in a 8-node cluster with respect to number of requests (r). Notice how quickly the routing probability on D reduces as the requests on D start to fail. Depending on the rate of request, it will only take a few seconds (sometime even milliseconds) to move all requests away from D, thus ensuring more requests are routed to healthier nodes.

Note that, even when requests keep failing on D (however less), ServiceQ retries them on other nodes until they succeed. If they do not succeed on any of the nodes, they are buffered and periodically retried on the cluster (using the same approach above), until they succeed.

Noticeable features

• HTTP Load Balancing
  
• Probabilistic node selection based on error feedback
  
• Failed request buffering and deferred forwarding
  
• Request retries
  
• Concurrent connections limit
  
• Complete TLS/SSL support (automatic and manual)

Here are the steps to run ServiceQ -

Download

Clone the project into any directory in your workspace (say 'serviceq')

$ mkdir serviceq
$ git clone https://github.com/gptankit/serviceq serviceq/

Change into directory serviceq

How to Build

$ make ('make build' will also work)

Optional: make with debug symbols removed (~25% size reduction)

$ make build-nodbg

This will create a Go binary serviceq in the current directory

How to Install

Make sure the current user has root privileges, then -

$ make install

This will create a folder serviceq in /usr/local directory and copy the serviceq binary to /usr/local/serviceq and sq.properties file (serviceq configuration) to /usr/local/serviceq/config.

How to Run

Before running, make sure the mandatory configurations in /usr/local/serviceq/config/sq.properties are set (LISTENER_PORT, PROTO, ENDPOINTS, CONCURRENCY_PEAK) -

#sq.properties

#Port on which serviceq listens on
LISTENER_PORT=5252

#Protocol the endpoints listens on -- 'http' for both http/https
PROTO=http

#Endpoints seperated by comma (,) -- no spaces allowed, can be a combination of http/https
ENDPOINTS=http://my.server1.com:8080,http://my.server2.com:8080,http://my.server3.com:8080

#Concurrency peak defines how many max concurrent connections are allowed to the cluster
CONCURRENCY_PEAK=2048

Also, verify timeout value (default is set to 5s). Low value is preferable as it allows retries to be faster -

#Timeout (s) is added to each outgoing request to endpoints, the existing timeouts are overriden, value of -1 means no timeout
OUTGOING_REQUEST_TIMEOUT=5

By default deferred queue is enabled with all methods and routes allowed. These options can be controlled as -

#Enable deferred queue for requests on final failures (cluster down)
ENABLE_DEFERRED_Q=true

#Request format allows given method/route on deferred queue -- picked up if ENABLE_DEFERRED_Q is true
#DEFERRED_Q_REQUEST_FORMATS=POST /orders,PUT,PATCH,DELETE
#DEFERRED_Q_REQUEST_FORMATS=ALL
DEFERRED_Q_REQUEST_FORMATS=POST,PUT,PATCH,DELETE

After all is set -

$ sudo /usr/local/serviceq/serviceq

Refer wiki for more details: https://github.com/gptankit/serviceq/wiki

Feel free to play around and post feedbacks