gnet-io / gnet-benchmarks

Benchmark test of gnet

gnet benchmark tools

Required tools:

• wrk for HTTP
• tcpkali for Echo

Required Go packages:

go get gonum.org/v1/plot/...
go get -u github.com/valyala/fasthttp

And of course Go is required.

Run bench.sh for all benchmarks.

Notes

• The current results were run on both Linux and FreeBSD.
• The servers started in multiple-threaded mode (GOMAXPROCS=Default).
• Network clients connected over Ipv4 localhost.

Like all benchmarks ever made in the history of whatever, YMMV. Please tweak and run in your environment and let me know if you see any glaring issues.

Benchmark Test

On Linux (epoll)

Test Environment

# Machine information
        OS : Ubuntu 20.04/x86_64
       CPU : 8 CPU cores, AMD EPYC 7K62 48-Core Processor
    Memory : 16.0 GiB

# Go version and settings
Go Version : go1.17.2 linux/amd64
GOMAXPROCS : 8

# Network settings
TCP connections : 1000/2000/5000/10000
Packet size     : 512/1024/2048/4096/8192/16384/32768/65536 bytes
Test duration   : 15s

Echo benchmark

On MacOS (kqueue)

Test Environment

# Machine information
        OS : MacOS Big Sur/x86_64
       CPU : 6 CPU cores, Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory : 16.0 GiB

# Go version and settings
Go Version : go1.16.5 darwin/amd64
GOMAXPROCS : 12

# Network settings
TCP connections : 300/400/500/600/700
Packet size     : 512/1024/2048/4096/8192 bytes
Test duration   : 15s

Echo benchmark

gnet-benchmarks's Issues

Fasthttp implementation

As you can see here https://github.com/panjf2000/gnet_benchmarks/blob/master/fasthttp-server/main.go#L24

actually, you writing redundant \r\n. fasthttp already handles this for you. also, in such cases I'd recommend you to use https://godoc.org/github.com/valyala/fasthttp#Request.SetBody or SetBodyString.

finally, https://github.com/panjf2000/gnet_benchmarks/blob/master/fasthttp-server/main.go#L21 you actually don't need to use a goroutine for this log message

evio/gnet http测试中的parser尚不完备，测试数据不适合与标准库、fasthttp对比

evio用于测试的parser不完备，gnet现在使用的 wildcat的parser应该也尚未完备，刚简单扫了几眼，提了3个issue: https://github.com/evanphx/wildcat/issues

因为不完备，测试得到的数据偏高，不能作为符合生产标准的http服务，与标准库和fasthttp对比可能会造成误解

ring buffer在高在线量场景可能导致高内存占用

刚才看到自己仓库中的 Used By 有这个仓库，来回访下，但是没看到我的库相关，所以浏览了下 issue ，看到 3 中关于 ring buffer 的讨论。刚好前阵子有人在我的 issue 中提到能不能支持 ring buffer，应该是看到你们其他各位作者的库有使用 ring buffer 所以也给我提起的，在这里交流下关于 ring buffer 的思考。

对于异步库，减少了大量的协程数，缓解了内存、gc、调度、stw，但是我自己库的早期版本进行http压测时仍然遇到高内存占用，其原因主要是 7层协议codec过程中 half-packet 需要为未解析完整的数据进行缓存，以及魔改标准库 tls 过程中 tls 原有的 bufio 等数据缓存，因为旧的方案每个连接上都挂载有一个或者多个这种 buffer 作为缓存，所以当连接数很高时，这些缓存数据结构的内存占用仍然很高、负担较大，bufio或者ring buffer都存在类似的问题，当遇到单个连接的类似大包体攻击的场景，这种问题会更加明显，大包体的问题，通常需要对单个连接最大包size进行限制，超过了就close，这点我早就做了，所以解决数据缓存占用高的问题，改成了解析到完整的就释放掉，正常的数据通信，绝大多数也就是这种。

举个例子，在线量1000k，qps 50k：

1. 按照每个连接持有cache的方式，则cache需要的内存为 cache size * 1000k
2. 按照每个协议包解析后释放，则每个连接最大持有协议包数量为 qps 个，则cache的内存只需要：cache size * 50k，节约了很多

4层的网络库在结合7层的实际应用并且遇到海量连接数时，仍然会有很多很多细节需要优化，ring buffer可能不是银弹。

是否有关于udp的压测示例啊

目前在Windows用tcpkali压测大概能到30wqps，放在服务大概是能到400多w的qps，这是在100个连接下，tcp建立连接。如果是udp不用建立连接的话，不知道能到什么水准

gnet 的示例源码？

gnet 的示例源码怎么不见了？

go net benchmark use a small buff

这个Buff是不是有点忒小

gnet-benchmarks/echo-net-server/main.go

Line 33 in cda087a

buf [0x4000]byte // 16 KB buffer

关于 gnet 代码的问题

借鉴不加以说明，就是抄袭

前期 gnet 就是从 eviop copy 过来改的，当时文件名称都一样。 之前我也提了 issue ，你说只借鉴了 ringbuffer ，火速关了 issue，说要加上说明。panjf2000/gnet#1

现在说明呢？

---

gev 项目创建日期，远早于 gnet ,从提交记录时间可以看出。如果你从 gev 有“借鉴”之处，还请附上说明。

我用 options 做默认配置，你也用。
https://github.com/panjf2000/gnet/blob/master/options.go
https://github.com/Allenxuxu/gev/blob/master/options.go

我前两天刚给 ringbuffer 加 pool，你今天也加。
Allenxuxu/ringbuffer@1048a07
panjf2000/gnet@998b920

如果去看代码，相似之处非常多。

some issues in this project may cause misleading information

Hi, thanks for providing a benchmark project between gnet and netpoll. But it seems that the project has some critical issues which may cause misunderstanding.

I list the issues as follows, let me know if there is anything I missed.

Incorrect use of netpoll

Forget to release buffer in netpoll-echo-server which cause huge memory usage and make it cannot reuse the memory:

func handler(ctx context.Context, connection netpoll.Connection) error {
	reader := connection.Reader()
	buf, err := reader.Next(reader.Len())
        // You should release reader buffer after flush finished:
        // defer reader.Release()

        _, err = connection.Writer().WriteBinary(buf)
	return connection.Writer().Flush()
}

You will see a huge performance improvement when you fixed it.

Unfair copying

When using gnet, you just assign the input buffer to the output buffer without any copy:

func (es *echoServer) React(frame []byte, c gnet.Conn) (out []byte, action gnet.Action) {
	// Echo synchronously.
	out = frame
}

But for net-echo-server you do too much copy:

inBuf := make([]byte, 4*1024)
outBuf := bytes.NewBuffer(make([]byte, 0, 4*1024))

As to the netpoll, since it assuming that the input and output buffer cannot be the same buffer in production, so it uses different buffers and copies them internally.

I think it should have a declaration of this critical difference if you want to benchmark them all together.

Different design goals

netpoll is designed for RPC use cases so it will create/reuse a goroutine for every handler function which will have some earning in production but cause overhead in the ECHO benchmark.

It's ok if this benchmark project only wants to consider the ECHO/Redis-like use cases, but it should be better to point out this key difference at the beginning.

We know it's hard to do an absolute FAIR benchmark since the design goals of network packages are different, but still want to make the benchmark have the reference value for the people who want to make choice for their own use case. So we also opensourced our own benchmark repo in netpoll-benchmark which more focus on the RPC use case benchmark.

It's glad to see have more benchmark in this field. Also, it welcomes to raise your questions if you think there have any points for improvement in our benchmark.

gnet 性能数据问题

gnet 更新版本

gnet 的版本似乎不是最新的 tag，有空更新下？