FYI: Another stackfull coroutine lib about asio-grpc

[Tradias]

Hi, thanks for the link, looks neat :).

Do you think it makes sense to collaborate in some way, should asio-grpc maybe provide a backend for your library? Would that even make sense?

[beef9999]

Yes, welcome to integrate our lib into your project.
By the way I saw your performance data, I think the reason c++ grpc-asio slower than go-grpc is that asio is weak in steaming socket mode, compared with Golang. See our tests in https://github.com/alibaba/PhotonLibOS#21-tcp , however, much better in ping-ping mode.

[Tradias]

Mmh, this library does not use asio's socket classes whatsoever. All I/O is handled by gRPC itself (epoll event engine).
The go-grpc server used to be much slower until it started uses a very fast Protobuf implementation: https://github.com/planetscale/vtprotobuf. So I suspect that is the performance bottleneck in the asio-grpc server.

I had a brief look at your library and I think it should be rather simple to bridge from your mutex+condition_variable to asio or libunifex and thereby into asio-grpc.

[beef9999]

Is there any way to replace gRPC's epoll engine and use customized socket?

[beef9999]

And our mutex and condition variable is based on our own event engine

[Tradias]

Not yet, but they are working on it: https://github.com/grpc/grpc/tree/master/include/grpc/event_engine

[beef9999]

So you will create two threads, one for grpc event loop, and the other for your own ?

And with across-thread communication

[Tradias]

gRPC exposes the event loop through CompletionQueue which I wrap and run on one thread. I suspect that a naïve implementation against your library would create another thread for your library. Maybe there is another way, does your library have the concept of an event loop and allows polling it?

[beef9999]

I think that is a problem. Some old codes have their own event loop, and when they decide to use gRPC, they find it unlikely to integrate into gRPC's own loop.

Our library has the polling mechanism, and all the mutex and cv are running within event loop. The mutex and cv can work across mutil-thread, they are equivalant to std::mutex and std::cv, the only difference is that they are designed for coroutines.

[beef9999]

epoll engine can be cascading. I suspect you can use our event engine and then poll the one of gRPC, if it has exposed its fd.

[beef9999]

Cross-thread communication is also feasible. We have a semaphore (based on eventfd), that can do wait / notify.

[Tradias]

Yep gRPC having its own event loop was also the challenge with asio-grpc. With the gRPC event_engine it should be possible to pluck something like asio::io_context in natively, which makes things a little cleaner.

Your code is the best we can do at the moment I believe and it doesn't even require asio-grpc :).

[beef9999]

But cross-thread communication is not effective in terms of performance...

[Tradias]

I am afraid that is the best we can do with gRPC's current API.

Once the event_engine can be customized it will be possible to cq->Next() in the I/O thread without blocking it.

[Tradias]

I started playing with the EventEngine a bit https://github.com/Tradias/asio-grpc/blob/event-engine/test/utils/utils/grpc_context_test.cpp but it is still too experimental. gRPC only uses the engine for some experimental features like load balancing and service discovery which makes it very difficult to test the EventEngine. I have put my efforts on hold for now.

Anyways, I hope you understand why asio-grpc provides an entire asio::execution_context-compatible wrapper for gRPC's CompletionQueue. Simple services/clients can then be written without cross-thread communication and everything else can reach out to e.g. an asio::io_context and use asio's completion token system to bounce between the two contexts/threads.

[Tradias]

Hi, I will close the issue until gRPC has made significant progress on their EventEngine. Feel free to open another issue if you have any more questions or suggestions for asio-grpc.