dbalabka / aerys-benchmark

Amphp HTTP server (aka Ayres) PHP non-blocking HTTP server benchmark

Links

LinkedIn
Twitter
📜  Publications
📄  CV
🎥  Public presenations
🏡 https://dmitry.balabka.com

aerys-benchmark's Issues

Add benchmark PHP + Dynasm JIT

Message from Dmitry Stogov about new version of PHP that support just-in-time compilation
https://externals.io/thread/268

Need to build from source:
https://github.com/zendtech/php-src/tree/jit-dynasm/ext/opcache/jit

JIT possible configuration:
https://github.com/zendtech/php-src/blob/jit-dynasm/ext/opcache/jit/zend_jit.h#L52

Review how HTTP parsing is affecting performance

There is an assumption that HTTP parsing in Amphp might affect overall server performance.

The assumption is based on experimental results. The variant of pure TCP Amphp server without HTTP parsing performs closer to Swoole and faster than Nodejs.

HTTP parsing implemented with PHP PECL Regexp can be replaced with bind C or Rust library via FFI or Wasmer extension.

Possible fast parsing libraries are:

1. https://github.com/seanmonstar/httparse/
2. https://github.com/h2o/picohttpparser

Libraries benchmark: https://github.com/rust-bakery/parser_benchmarks/blob/master/http/README.md

Remove res.sendDate = false

For NodeJS res.sendDate = false; should be removed, as any HTTP server SHOULD send a valid Date header.

> See #18

          > See #18

😠😠😠😠😠🔥🔥🔥🔥🔥😠😡

Originally posted by @Stupidpeopleicoming in #15 (comment)

Add benchmark for PHP version with JIT

https://github.com/zendtech/php-src/tree/jit-dynasm/ext/opcache/jit
https://externals.io/thread/268

Comments on your conclusions

As was mentioned otherwhere, enabling opcache (via -dopcache.enable_cli=1 -dopcache.optimization_level=0xffffffff) and using PHP 7.1 may improve performance a bit more.

Also, note that the node http module is even simpler than ours; as in, it has no concept of flexible middlewares etc. It ultimately just has to handle even less logic and thus may be faster. A tiny bit of logic, but not totally insignificant as it has to create a bunch of generators for the middleware machinery.
There are also other options to tune for the benchmark, like "deflateEnable" => false: Aerys always buffers by default for up to Options->deflateMinimumLength to check whether it's worth to use gzip compression]. NodeJS does no such thing.

In general, the nodejs server is more lightweight than Aerys; Aerys is packed with a bit more functionality which will be handy in more complex … in real world scenarios.

I have no idea why the standard deviation, especially as it is very low for node... I'd be interested in that too.

P.s.: $resp->addHeader('Content-Length', strlen($data)); is redundant; Aerys will set it for you (chunked encoding if you stream and normal content-length if end() is used without prior stream() call)

Add ext-async into benchmark

Prepare additional benchmark with:

1. https://github.com/concurrent-php/ext-async
2. https://github.com/amphp/amp/tree/ext-async
3. https://github.com/concurrent-php/http

Extension instalation example:
https://github.com/concurrent-php/async-api/blob/master/vagrant-async.sh#L46

Need to warm-up server before benchmark

For example NodeJS JIT have huge performance hit. During first 30 seconds NodeJS usually response with high deviation due to JIT compilation.
To avoid this issue we have to warm-up server during few seconds before benchmark and use --always-opt.

Benchmark statistics check

To avoid unstable benchmark results we must check results distribution.
For example: http://perf.readthedocs.io/en/latest/cli.html#check-cmd

I have assumption that 30 seconds is not enought to get representive sample of requests time perfomance. Futhermore using differente enviroments sufficient bechmark time can vary.

Add other benchmarks

Currently it only benchmarks a plain GET /, would be interesting to see POST requests with multipart and JSON as well as larger responses with gzip / deflate.

Unfair comparison

Your current benchmark uses NodeJS and React with one process, while Aerys spawns as many workers as you have CPU cores available.

Add other servers implementations to benchmark

Swoole is an event-driven, asynchronous & concurrent networking communication engine with higher performance written only in C for PHP. Swoole includes components for different purposes: TCP/UDP Server and Client, Task Worker, Database Connection Pooling, Millisecond Timer, Event, Async IO, Async Http/WebSocket Client, Async Redis Client, Async MySQL Client, and Async DNS Requiring. With these components, Swoole allows you to build many features, which are widely used in the applications about Internet, mobile communication, enterprise software, cloud computing, online game, IoT, IoV, and smart home, etc. For IT development group, it can improve the efficiency, which makes them focus on the innovational products.

http://www.swoole.com/

Workerman An asynchronous event driven PHP framework for easily building fast, scalable network applications. Supports HTTP, Websocket, SSL and other custom protocols. Supports libevent, HHVM, ReactPHP.

https://github.com/walkor/Workerman

Express.js Fast, unopinionated, minimalist web framework for node.

https://github.com/expressjs/express