reactive-relational-database-tests

Some combinations of frameworks are tested below. You can find implementation details by clicking through title. Presentation link is here: https://github.com/farukcankaya/reactive-relational-database-tests/blob/master/reactive-relational-database.pdf

Load test was run with maximum 1000000 users and 100000 hatch rate. Also, minimum user waiting duration was 1000ms and maximum user waiting duration was 2000ms. You can change this configuration in load-test.py file.

Follow instructions below to run load test:

Install Locust

For python2.7: python -m pip install locustio
For python3: python3 -m pip install locustio

Run load test

locust -f load-test.py --host=<host>

Spring Web + JPA (blocking io, thread per request)

Approx. Memory Usage: 500MB
Approx. CPU Usage: 15%
It creates thread per request. Threads will increase up to instant request counts. It does not scale.
Request fail: 0%
Load test host: http://localhost:9494/api/jpa

Webflux + R2dbc

Approx. Memory Usage: 1GB
Approx. CPU Usage: 30%
Approx. Thread count: 30
Request fail: 36%
Load test host: http://localhost:9595/api/r2dbc

R2DBC has not have connection pool in this project. It causes some request fails. After adding R2DBC connection pool, it could be better. R2DBC Pool repository: https://github.com/r2dbc/r2dbc-pool

Webflux + Reactive PG Client

Approx. Memory Usage: 500MB
Approx. CPU Usage: 15%
Approx. Thread count: Fixed 30 threads from pool.
Request fail: 0%
Load test host: http://localhost:9696/api/reactive-pg

Webflux + JDBC and JPA

JDBC

Approx. Memory Usage: 2GB
Approx. CPU Usage: 40%
Approx. Thread count: 230
Request fail: 1%
In /jdbc endpoint, fixed 200 thread pool is used to open JDBC connection to database.
Load test host: http://localhost:9393/api/jdbc

JPA

Approx. Memory Usage: 500MB
Approx. CPU Usage: 15%
Approx. Thread count: 22
Request fail: 0%
In /jpa endpoint, it uses NIO threads to open JDBC connection to database.
Load test host: http://localhost:9393/api/jpa

Webflux + JPA looks better but only select query is tested here. JPA gives better result with using caching, lazy loading properties. However, JPA is not work with webflux properly. JPA uses `ThreadLocal` to store some variables but in reactor threads could be changed. To look in details you can checkout the feature/city branch. JPA throws `org.hibernate.LazyInitializationException` exception when /api/jpa-city endpoint tries to return City with collection of addresses.

Conclusion

All tests run in 8-Core 2,9 GHz Intel Core i7, 16 GB 2133 MHz LPDDR3 machine and VisualVM is used to get information from applications. Results could be change based on physical server configurations. As a result, Webflux + PgClient gives best result compares to other implementations(Spring Web + JPA, Webflux + JDBC and JPA, Webflux + R2dbc) for now(10/05/2019).

farukcankaya / reactive-relational-database-tests Goto Github PK