Add Tornado about web-frameworks

[waghanza]

Thanks for the idea.

A PR will be very ❤️

[bdarnell]

Looks fine to me. You don't want to use tornado.wsgi if you don't have to. The one thing you may want to change (depending on how this server is run and if the other frameworks are run in comparable configurations) is to replace http_server.listen(port) with http_server.bind(port); http_server.start(0) to run in multi-process mode.

[waghanza]

@greed2411

Framework (Middleware)	Requests / s	Latency	99 percentile	Throughput
sanic	4081.00	273718.00	2009469.33	2.40 MB
japronto	47752.67	21337.33	32104.00	19.44 MB
flask	1143.33	86290.00	104374.33	0.88 MB
django	1176.33	243895.00	4071113.33	0.78 MB
tornado	7054.00	137939.67	294950.67	5.21 MB

[waghanza]

@tbrand Can we close since Tornado was added) ?

[greed2411]

Okay, just saw the results, it's at the bottom. And what I gave was just a synchronous version of tornado because there is no db querying or API calls, if that's there as a test, Im sure it would be comparable to Sanic. Could you guys try to include that in a test as well?? Ik the test is just for routing, but still wanted to ask.

[waghanza]

@greed2411 the result will be display on the next release

@tbrand If you don't have time ;-) I can help you releasing this project 😜

[waghanza]

@greed2411 tornado can run on gunicorn
is there any diff between gunicorn and built-in server ?

DISCLAIMER : not a python guy

For example, in ruby world the recommended set-up is to use puma or unicorn (or else) but not the same server as in development mode

@benoitc As the gunicorn author, you'll probably explain this like anyone ❤️

[greed2411]

@waghanza yes it can. If I'm not wrong the in built server is wsgi.

I guess someone could rewrite it for gunicorn. Should I look over it?

[waghanza]

@greed2411 sure, as flask also does ( werkzeug) but not recommended for production

I can make the PR (making uniform python implementations), just want to ask before 😜

[greed2411]

@waghanza if you are familiar with tornado then please go ahead and do the PR. Please do mention me in the PR. I've never tried gunicorn I wanna have a look. I'd be happy to help anyway possible. Also thanks.

[waghanza]

@greed2411 gunicorn is made to be a production-ready app server in python ;-)

[greed2411]

@waghanza Cool. Can't wait to see the results. I just wanna see tornado on top of django nothing else.

[waghanza]

After cloudifying results, I'm sure it will be 😝

I am working on a digitalocean version of the benchmarks

[greed2411]

Also one more request @waghanza could you try it with tornado 5?

[waghanza]

@greed2411 sure. I intend to fix any version (in any languages)

[greed2411]

Can't wait to see the results. :P

[greed2411]

@waghanza last chance for tornado, could you try these, run the results and lemme know?

http://www.tornadoweb.org/en/stable/wsgi.html

I still can't settle with the fact that tornado is slower than flask and django for this task.

Also thanks.

[greed2411]

@waghanza the tornado script is performing actually synchronously, there's nothing async about it because we don't have and external api calls or db calls.

but, if I may, can you ask the core contributor, the one man army, bdarnell in the gitter or any personal means to look over the tornado script. Since you are a maintainer, he could take the time and give suggestions.

I don't have the face to ask him, because of me, the script is sitting at 44th place (last position T_T )
He could look at the table from the vibora PR, and let you know whether wsgi should be used, if yes whether we should use tornado.wsgi inside the script, or whether those routes can be handled asynchronously.

could you ask him, please?

[waghanza]

@greed2411 do not be afraid ... you can do it 😛

@bdarnell could you check our tornado implementation https://github.com/tbrand/which_is_the_fastest/blob/master/python/tornado/server.py. The purpose is to rank frameworks and I'm afraid this implementation is not at tornado's advantage

[waghanza]

@bdarnell thanks for advice.
As we target production-ready app, do you recommand this one, https://github.com/waghanza/which_is_the_fastest/blob/python_standardization/python/tornado/server.py, or the actual is better ?

PS : The goal of this implementation is to run with gunicorn in front

The only goal is to have 3 endpoints :

• GET on /
• POST on /user
• GET on /user/(\d+)

[bdarnell]

I use Tornado without gunicorn in production; I don't know what gunicorn adds here.

[waghanza]

Log uniformization ;-) same start-up script fot tornado / flask ...

We need to implement what is the best practices for tornado folks 😜

@benoitc Do you recommand using gunicorn on top of tornado ?

[greed2411]

The reason we are considering because I still can't believe the fact that flask and Django are out performing tornado's requests/sec, because of gunicorn. Previously before gunicorn, they didn't even make the list.
(Tornado's at the bottom, 44th)

Btw I'm the one who wrote the original script.:relieved:

[waghanza]

@greed2411 I'm sure that tornado will climb after having production-ready results 😝

[waghanza]

@greed2411 you'll be happy 😛 with @bdarnell advices tornado climb on 41th place (and is before django)

40. ....
41. tornado (python)
42. rails (ruby)
43. flask (python)
44. django (python)

[waghanza]

Here is a list of what we SHOULD use (as we are target performance 😜)

• django, gunicorn @see https://docs.djangoproject.com/fr/2.0/howto/deployment/wsgi/
• flask, waitress @see http://flask.pocoo.org/docs/1.0/tutorial/deploy/#deploy-to-production
• japronto, built-in
• sanic, asyncio
• tornado, built-in loop with above advices from @bdarnell

[greed2411]

@waghanza wowowow. Amazing. I never stopped believing in torando, thank the heavens its better than django and flask, btw where can I see the actual results? I wanna see the margin by which we defeated django, also I would like to see the modified tornado script.

[waghanza]

@greed2411 so impatient ... 😜

[greed2411]

Yussssssh. I feel overpowered now. I'll go and rub this onto every friend of mine at University. Thanks. I'm looking forward to see these results on readme soon. Good luck and thanks once again @waghanza. You made my dream come true.

[greed2411]

But wait. Tornado is faster than Sanic????
You sure???? So the only thing on top of tornado is Japronto and Vibora huh?

[waghanza]

@greed2411 I was NOT using https://github.com/Pylons/waitress at full speed

Framework (Middleware)	Requests / s	Latency	99 percentile	Throughput
sanic	4289.00	291132.67	2697882.00	2.57 MB
japronto	53348.00	20165.33	38664.00	21.56 MB
flask	1149.67	85609.33	104615.00	0.85 MB
django	1008.00	307748.00	4666972.33	0.70 MB
tornado	7219.33	133642.33	274002.33	5.49 MB

⚠️ tornado 5 seems to be faster (results are still on local docker)

[greed2411]

Lel. Django slowed down. 😂😂😂.
Nonetheless tornado 5 is awesome.

[waghanza]

@bdarnell I ran some tests to see if gunicorn add some performances, and I've found

Appserver	Requests / s
Tornado	5640.67
Gunicorn	1835.33
Gunicorn (reuse port)	1719.00
Tornado (reuse port)	7051.00

PS : This is mainly for information (mine) and implementation in this project 😜
PSS : Result are fluctuating due do docker but the ratio is an indicator

[benoitc]

what is that test? each worker in gunicorn should run 1 tornado server so i would be quite surprised it would be slower...

…

On Monday, July 2, 2018, Marwan Rabbâa ***@***.***> wrote: @bdarnell <https://github.com/bdarnell> I ran some tests to see if gunicorn add some performances, and I've found Appserver Requests / s Tornado *5640.67* Gunicorn *1835.33* Gunicorn (reuse port) *1719.00* Tornado (reuse port) *7051.00* — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#131 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAA4osAYqL9qbLGcZGnc2WCeTTqCKGGuks5uCn35gaJpZM4SRyqb> .

-- Sent from my Mobile

[waghanza]

Hi @benoitc,
I ran some benchmarks and I have

Framework (Middleware)	Requests / s	Latency	99 percentile	Throughput
tornado (code)	7030.67	136106.00	326479.33	5.18 MB
tornado-reuseport (code)	7933.00	123524.33	316101.33	5.37 MB
gunicorn (code)	1181.00	858316.00	5904862.00	0.93 MB
gunicorn-reuseport (code)	1135.00	774897.67	2993184.33	0.88 MB

Gunicorn run with : gunicorn --bind 0.0.0.0:3000 --workers `nproc` -k tornado server:app

[benoitc]

the question is how you run the benchmark. There should be no reason to have a request slower since a tornado server will run on each worker process with gunicorn. Can you share the code of it?

[waghanza]

sure, it is this repo, we use wrk to request server

⚠️ Still on (local) docker, so result are fluctuating (cloudify this in planed)

[bdarnell]

These new results don't look right to me. I haven't done any performance comparisons myself but I expect Sanic to be faster than Tornado. I also expect that gunicorn -k tornado performs similarly to plain Tornado. Maybe other python configurations also need to be updated to use the right number of processes?

[waghanza]

@bdarnell effectively, I do not understand the diffs between gunicorn and plain tornado

may you try running the code above #131 (comment) ?

[bdarnell]

I don't understand that difference either. But unfortunately good benchmarking is a lot of hard work, and barely have time to benchmark my own software, much less somebody else's.

[waghanza]

@bdarnell sure, that's could also be a argument for this project ^^

trying to compare tools is a burden, but it we can gather community, I am sure it will be helpful 😜

[bdarnell]

Absolutely. It's useful work, I'm just not volunteering to do it :)

[waghanza]

no problem at all, anybody will always be welcome, at least when we have time