jiajunhuang / guard

NOT MAINTAINED! A generic high performance circuit breaker & proxy server written in Go

Guard

guard is a generic high performance circuit breaker & proxy written in Go. It has four major components:

• radix tree & response status ring: which stores registered URLs
• load balancer: which distributes requests(algorithms: randomized distribute, round robin, weighted round robin)
• circuit breaker: which makes sure your backend services will not be broken down by a large quantity of requests
• proxy server: it's based on fasthttp

Workflow

Benchmark

I've made a simple benchmark on my laptop(i5-3210M CPU @ 2.50GHz with 4 cores):

$ wrk --latency -H "Host: www.example.com" -c 2048 -d 30 -t 2 http://127.0.0.1:9999  # Nginx with 4 workers
Running 30s test @ http://127.0.0.1:9999
  2 threads and 2048 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    99.62ms  158.44ms   1.36s    95.41%       
    Req/Sec     6.43k     2.43k   11.36k    53.85%
  Latency Distribution   
     50%   58.34ms                                       
     75%  153.81ms                                                         
     90%  165.62ms                                                                               
     99%  965.10ms                       
  383670 requests in 30.09s, 311.00MB read
  Socket errors: connect 1029, read 0, write 0, timeout 114
Requests/sec:  12749.73                           
Transfer/sec:     10.33MB                         

$ wrk --latency -H "Host: www.example.com" -c 2048 -d 30 -t 2 http://127.0.0.1:23456  # guard with GOMAXPROCS=4
Running 30s test @ http://127.0.0.1:23456
  2 threads and 2048 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    45.78ms   34.36ms   1.12s    89.09%
    Req/Sec    11.35k     1.18k   14.19k    73.20%
  Latency Distribution                                   
     50%   41.87ms     
     75%   56.55ms       
     90%   74.52ms                                       
     99%  111.06ms                                                         
  676392 requests in 30.07s, 532.82MB read
  Socket errors: connect 1029, read 0, write 0, timeout 0
Requests/sec:  22494.06
Transfer/sec:     17.72MB

For now, guard's proxy performance is about 55% of Nginx 1.76x faster than Nginx, and I'm still working on it! Don't worry, it will become better and better!

By the way, thanks the suggestion from @dongzerun, by configure the GOGC in environment, guard's proxy performance is about 70% of Nginx. guard does not allocate much memory now, GOGC does not make a change, but still say thanks to @dongzerun!

TODO

moved to https://github.com/jiajunhuang/guard/projects/1

Set it up

1. build it using go get -u:

$ go get -u github.com/jiajunhuang/guard

2. start it

$ guard

3. now you need to register an application by send a POST request to http://127.0.0.1:12345/app with json like this:

{
    "name": "www.example.com",
    "backends": ["127.0.0.1:80", "127.0.0.1:80", "127.0.0.1:80"],
    "weights": [5, 1, 1],
    "ratio": 0.3,
    "paths": ["/"],
    "methods": ["GET"],
    "fallback_type": "text",
    "fallback_content": "hello world"
}

I'm doing it like this:

$ http POST :12345/app < backends.json 
HTTP/1.1 200 OK
Content-Length: 8
Content-Type: text/plain; charset=utf-8
Date: Sun, 21 Jan 2018 08:51:16 GMT

success!

4. and now, it works! whoops! try it:

$ http :23456 'Host: www.example.com'
HTTP/1.1 200 OK
...

5. by the way, you can inspect your configuration by visit http://127.0.0.1:12345/ :

$ http :12345
HTTP/1.1 200 OK
Content-Length: 235
Content-Type: application/json
Date: Fri, 26 Jan 2018 14:11:02 GMT

{
    "apps": {
        "www.example.com": {
            "backends": [
                "127.0.0.1:80",
                "127.0.0.1:80",
                "127.0.0.1:80"
            ],
            "disable_tsr": false,
            "load_balance_method": "rr",
            "methods": [
                "GET",
                "GET"
            ],
            "name": "www.example.com",
            "paths": [
                "/",
                "/doc"
            ],
            "ratio": 0.3,
            "weights": [
                5,
                1,
                1
            ]
        }
    }
}

Changelogs

• 2018-01-25: rewrite proxy from net/http to fasthttp, it's super fast now!
• 2018-01-24: rewrite radix tree & statistics module, it's lock-free now!
• 2018-01-22: add randomized distribute, naive round robin algorithms
• 2018-01-21: rewrite status statistics module
• 2018-01-20: guard works!
• 2018-01-19: first commit

Thanks

• Golang
• fasthttp
• httprouter

guard's Issues

rewrite Bucket, use radix tree

after #5 , rewrite Bucket from using map to radix tree.

试试调整 GOGC

试试调整 GOGC 再压测？一般转发类的 go 服务都要调高，默认肯定不行

试试 1000 吧，再看看压测结果，并发情况下 QPS 不如 nginx 是不可能的

dynamic config load & save

maybe read configuration from:

• files
• etcd
• Redis
• HTTP Request
• etc...

radix tree, use `[]byte` to save path & indices

convert []byte to string is expensive.

add fallback option

add option to config balance algorithm

#9 we've implemented the random algorithm and naive round robin, so we should provide a way to use them.

graceful restart

it's time to implement this feature.

try to optimize proxy function from `net.http`

from #7 , that's the bottleneck.

add benchmark for breaker.ServeHTTP

• breaker.ServeHTTP
• Proxy

use a coarse implementation

in benchmark, I found that time.Now is really expensive. use a coarse implementation.

refer:

• https://github.com/valyala/fasthttp/blob/master/coarseTime.go

TLS

Have you thought about how to handle TLS / SSL requests?

more load balance algorithms

• random
• naive round robin

rewrite radix tree, drop dependency of httprouter

httprouter is fast, but there are too many functions we don’t need, also a little bit heavy(and slow down the process). so drop it, keep main functionalities only.

Differences with Kong

From your readme i understand that you want to achieve what Kong/nginx currently does. Is there a particular reason?
Thanks

benchmark for guard vs nginx

a better, more detailed benchmark is required.

web ui

maybe use https://github.com/jteeuwen/go-bindata