Comments (11)
Good question! It needs us investigate deeply. I have a blog that analysis why using hyper based porting is slow. those numbers are coming from such conditions: while tokio
version using the latest master branch of hyper, may
version using the old 0.10.x branch which is not very optimized. so the comparison is not that fairly positive.
I have a minihttp, that should be a fair comparison.
tokio_minihttp
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.98ms 284.06us 12.53ms 98.92%
vs
may_minihttp
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.70ms 812.42us 20.17ms 97.94%
may
is faster! But with bigger jitters.
from may.
👍 looks good. I am planning to convert my existing threaded application to may
coroutine-based. This is a near-real-time application with 99.9% latency below 1ms with max latency ~3ms. The application runs on 16 core machine but actively uses only 9 threads (1 thread per TCP connection + 1 cache updater thread).
The number of TCP connections is fixed to 8. Let me know if you have any suggestions.
from may.
Your application is very special, there are enough resources for every thread and only 8 connections running on 16 cores! I think you can try clone each stream into two part: one for reading and one for writing, so that read and write don't block each other. and spawn coroutines to handle each request. Don't forget to config the io_workers to a reasonable number, so that fully utilize the CPU.
from may.
Is it possible to reduce jitter if we use a work stealing queue per worker thread instead of a global mpmc queue for coroutines?
from may.
@alkis
I don't thinks so. IO related tasks will not schedule from the mpmc channel (used by normal worker thread), they will run directly in the io_workers threads. I think the jitter comes from the fact that coroutine will not suspend if there is already some data on the io, say for the reading, the coroutine would only suspend when the unblocking read returns no data, this would cause some coroutines spin the cpu for a while until it drains out and consumes all of the available data, while other corouines have to wait for the cpu release.
from may.
Is there some documentation on the tradeoffs and/or different designs for the scheduler? More particularly why do we need separate io_workers? How does it compare to N generic workers plus a single epoll_wait()
worker whose only job is to handle the slow path of park/unpark coros that wait for I/O?
from may.
well, at the beginning I do use unified thread to handle all of the events, but it's not as efficient as I expect. the sync primitive events and the timeout events would not go through the io thread, they are scheduled in normal worker thread. if generate event to the epoll system, this would have another system call which is not necessary. running coroutines in normal worker thread for the io event would have a noticeable delay after receiving the io event which generated in io worker thread.
from may.
For the record, I wrote (with GuillaumeGomez) an FTP server with tokio and one with may and the version with may
seems a little bit faster when benchmarking with this tool, so I think may
is fast enough.
from may.
Can you please share your results? Did you measure latency differences?
from may.
I'll write a blog post with Guillaume (hopefully soon) to talk about our experience and show these results.
This benchmark only shows the number of connection that remained at the desired speed, so I guess the latency was mesured somehow, but I only have the number of users.
If you know a better tool to benchmark an FTP server, I'll be happy to try it.
Thanks.
from may.
Hello @Xudong-Huang, thanks for your great work!
I'm very interested in may and would like to know if you have more recent data on tokio vs may?
from may.
Related Issues (20)
- support cancel feature gate
- macro select! : Polling a channel that already contained data may not trigger recv HOT 3
- Question about coroutine HOT 2
- `go!` macro allows its arguments to do `unsafe` operations. HOT 1
- No `peek` implementation for `net::TcpStream` HOT 2
- How can May be used for shared-nothing architecture? HOT 3
- The go! macro is unsound HOT 3
- Unsound problems in `may_queue` crate
- Why is the http example so much slower when simplified?
- Examples don't compile HOT 2
- about performace.md The description is unfair HOT 3
- 建议参考ants,整个协程池
- `may_http`耗时不符合预期 HOT 5
- 建议参考libco,hook系统函数,减少改造成本 HOT 1
- 问下如何正确地hook系统函数?
- support linux io_uring HOT 1
- unify thread and coroutine io handling
- split io read and write event HOT 1
- How does it actually work
- linux: file descriptor leak in epoll code HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from may.