alibaba / photonlibos

Probably the fastest coroutine lib in the world!

PhotonLibOS

PhotonlibOS.github.io

What's New

• Feb 2024，中文文档在官网上线了
• Since 0.7, Photon will use release branches to enhance the reliability of software delivery. Bugfix will be merged into a stable release at first, then to higher release versions, and finally main.
• Since version 0.6, Photon can run with a userspace TCP/IP stack on top of DPDK. En / 中文.
• How to transform RocksDB from multi-threads to coroutines by only 200 lines of code? En / 中文.

Click to show more history...

• Version 0.5 is released. Except for various performance improvements, including spinlock, context switch, and new run queue for coroutine scheduling, we have re-implemented the HTTP module so that there is no boost dependency anymore.
• Version 0.4 has come, bringing us these three major features:
  1. Support coroutine local variables. Similar to the C++11 thread_local keyword. See doc.
  2. Support running on macOS platform, both Intel x86_64 and Apple M1 included.
  3. Support LLVM Clang/Apple Clang/GCC compilers.
• Photon 0.3 was released on 2 Sep 2022. Except for bug fixes and improvements, a new photon_std namespace is added. Developers can search for std::thread, std::mutex in their own projects, and replace them all into the equivalents of photon_std::<xxx>. It's a quick way to transform thread-based programs to coroutine-based ones.
• Photon 0.2 was released on 28 Jul 2022. This release was mainly focused on network socket, security context and multi-vcpu support. We re-worked the WorkPool so it's more friendly now to write multi-vcpu programs.
• Made the first tag on 27 Jul 2022. Fix the compatibility for ARM CPU. Throughly compared the TCP echo server performance with other libs.

Community

Join Slack: link

Join DingTalk group: 55690000272

photonlibos's Issues

Is there a way to call `photon::thread_create11()` from C by using a void* function pointer?

Hi,

Thanks for the great lib!

I'm writing a C wrapper to use it from C, and I'm getting a compilation error due to passing a void* instead of a C++ function:

photonwrapper.cpp:12:2: error: no matching function for call to 'thread_create11'
        photon::thread_create11(f);
        ^~~~~~~~~~~~~~~~~~~~~~~
/Users/alex/code/3rd/PhotonLibOS/include/photon/thread/thread11.h:69:5: note: candidate template ignored: requirement 'is_function_pointer<void *>::value' was not satisfied [with F = void *, ARGUMENTS = <>]
    thread_create11(F f, ARGUMENTS&&...args) {
    ^
/Users/alex/code/3rd/PhotonLibOS/include/photon/thread/thread11.h:114:5: note: candidate template ignored: requirement 'is_functor<void *&>::value' was not satisfied [with FUNCTOR = void *&, ARGUMENTS = <>]
    thread_create11(FUNCTOR&& f, ARGUMENTS&&...args) {

void photon_thread_create11(void* f) {
	photon::thread_create11(f);
}

According to this, converting a void* to a function pointer directly is not allowed in C++:
https://stackoverflow.com/questions/57435515/how-to-cast-void-pointers-to-function-pointers

So how can we call it from C? I guess a new overloaded function photon::thread_create11() that accepts a void* instead of F can be added?

Thanks.

Creating and joining a number of threads only known at runtime

Hi,

Is there a way to create and then join photon threads dynamically during runtime without specifying the number during build time. I don't see this in the documentation

example with pseudo code that will create multiple threads and wait for all to complete before moving on to the next

num_of_threads = user_input;  // e.g.  4
thread tasks [num_of_threads];
for(int i=0; i < num_of_threads; i++){
    
    auto my_thread = .....  // create new thread
    tasks[i] = my_thread;
}

// Similar to .waitAll() in other languages that waits for all threads to complete
for(int i=0; i < num_of_threads; i++){
    
     thread_join(tasks[i]);   
}

// continue with code after all threads completed.....

More detailed documents?

doc目录下就一张图片

update benchmark result(asyncio)

hello, would you like to update the asyncio benchmark result? I have refactored the code, and the ping-pong performance up 2~3x, thank you!
netcan/asyncio#9

gcc version

Hello, thanks for open source such a great program, I would like to ask which version supports the lowest gcc, because I may use it on an embedded development board

Socket recv_errqueue:deferred copy occurs message

Hi,

I will like to know what this message means: "PhotonLibOS-main/net/basic_socket.cpp:287|recv_errqueue:deferred copy occurs, might harm performance" and the reason it appears.

I am testing tcp_socket_server / tcp_socket_client vs zerocopy_tcp_server/zerocopy_tcp_client with / without TLS

1. I am getting the 'deferred copy occurs' message when using either tcp or zerocopy_tcp.

• One message on the client and one message on the server.
• During the connection there is one stream->send and one stream->write

2. When TLS is used, I get 4 'deferred copy occurs' messages on client and 4 times on server

Also, are there plans to support TLS 1.3?

Thanks.

Do coroutines support local storage (like thread_local)?

IO_Uring send: Broken pipe

Hi guys,

Here is the error:

2023/05/02 15:55:23|ERROR|th=00007F95DBA21B00|/x.cpp:388|sendMessage:SEND FAILURE: errno=32(Broken pipe)

• 2 separates threads trying to send to the same end point
• Both connects, 1st thread sends and receives msgs no probs.
• as soon as 2nd thread sends a msg, 1st thread gets the above msg when trying to send

Macos构建失败

1 warning generated.
/Users/beihaifeiwu/Workspace/study/cpp/PhotonLibOS/net/http/message.cpp:244:19: error: return type of out-of-line definition of 'photon::net::http::Message::body_size' differs from that in the declaration
uint64_t Message::body_size() const {
~~~~~~~~          ^
/Users/beihaifeiwu/Workspace/study/cpp/PhotonLibOS/net/http/message.h:100:12: note: previous declaration is here
    size_t body_size() const;
    ~~~~~~ ^
1 warning generated.
/Users/beihaifeiwu/Workspace/study/cpp/PhotonLibOS/net/http/server.cpp:211:21: error: variable has incomplete type 'struct stat'
        struct stat buf;
                    ^
/Users/beihaifeiwu/Workspace/study/cpp/PhotonLibOS/include/photon/fs/filesystem.h:41:8: note: forward declaration of 'stat'
struct stat;
       ^
1 warning and 1 error generated.
make[2]: *** [CMakeFiles/photon_obj.dir/net/http/message.cpp.o] Error 1
1 warning generated.
1 warning and 2 errors generated.
make[2]: *** [CMakeFiles/photon_obj.dir/thread/thread.cpp.o] Error 1
1 warning and 1 error generated.
make[2]: *** [CMakeFiles/photon_obj.dir/net/http/server.cpp.o] Error 1
1 warning generated.
/Users/beihaifeiwu/Workspace/study/cpp/PhotonLibOS/net/http/client.cpp:31:23: warning: unused variable 'kMinimalStreamLife' [-Wunused-const-variable]
static const uint64_t kMinimalStreamLife = 300UL * 1000 * 1000;
                      ^
2 warnings generated.
2 warnings generated.
make[1]: *** [CMakeFiles/photon_obj.dir/all] Error 2
make: *** [all] Error 2

Unable to compile with IO_URING enabled

Hi,

I am trying to build PhotonLibOS with io_uring.
"option(ENABLE_URING "enable io_uring function" ON)" in CMakeList.txt

I am getting errors during build process as per attached image. Will really appreciate it if I can get some pointers on how to resolve this.

question: how to use multi-core when writing applications with photonlibos

Looks like photonlibos can only use one CPU core. If I want to write a service running on a multi-core server, how can I use all of the cores?

Can you give an example?

compare with async_simple?

https://github.com/alibaba/async_simple

natice stackless coroutine and it's also a alibaba's project

Is there a plan to support RDMA ?

see apache/brpc#1836

comparison with lib_fiber

can you guys add a comparison with lib_fiber? I mean lib_fiber module in https://github.com/acl-dev/acl

and https://github.com/idealvin/cocoyaxi

curl.cpp:198:25: error: 'unique_ptr' is not a member of 'std'

Describe
Hi, I was interested in this project, so I attempted to compile it. However, compilation error arise when I build this repo.
The compilation steps are described in full in the README.md file.
I guess it may be caused by the problem of the compiler version.

/home/yxb/workspace/PhotonLibOS/net/curl.cpp:198:25: error: 'unique_ptr' is not a member of 'std'
  198 | static std::vector<std::unique_ptr<std::mutex>> mutex_buf;
      |                         ^~~~~~~~~~
/home/yxb/workspace/PhotonLibOS/net/curl.cpp:29:1: note: 'std::unique_ptr' is defined in header '<memory>'; did you forget to '#include <memory>'?
   28 | #include <photon/common/timeout.h>
  +++ |+#include <memory>
   29 | #include <photon/common/utility.h>

My version of tools :
g++ (GCC) 11.3.0
cmake version 3.22.3

Solution
Add #include <memory> to net/curl.cpp.

More examples / documentation on Photon Filesystem

Hi,
Besides the example on how to init, open and write to a iFile in /tree/main/examples/simple, will there be more examples/documentation provided Photon filesystem? With info on the type of settings (for tuning), read/write APIs
I am hoping to test writing and reading sequential files using Photon with io_uring IO Direct, and see how much performance benefit it provides over standard linux fio.

photon stack 有支持spdk的blobfs计划吗

Integrate into containerd/overlaybd

https://github.com/containerd/overlaybd

overlaybd will be using Photon's coroutine lib and fs lib.

It's our first user. Cheers...

when to support windows

when to support windows

About tuple_assitance::callable

• 不支持noexcept，&， &&, volatile等

• template <class F>
  struct callable<F&> : public callable<F> {};
  
  template <class F>
  struct callable<F&&> : public callable<F> {};

不如直接std::remove_cvref_t(自己封装一个) ?

• ....
• 如果只是应用于thread11和apply，可以参考一下标准库？clang或者gcc的都行，我认为callable槽点有点多，hmm，不如去掉, C++17以下可以用std::__invoke？

std::invoke
std::apply

build failed

gmake[2]: *** [CMakeFiles/boost-populate.dir/build.make:99: boost-populate-prefix/src/boost-populate-stamp/boost-populate-download] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/boost-populate.dir/all] Error 2 gmake: *** [Makefile:91: all] Error 2 CMake Error at /usr/share/cmake-3.22/Modules/FetchContent.cmake:1087 (message): Build step for boost failed: 2 Call Stack (most recent call first): /usr/share/cmake-3.22/Modules/FetchContent.cmake:1216:EVAL:2 (__FetchContent_directPopulate) /usr/share/cmake-3.22/Modules/FetchContent.cmake:1216 (cmake_language) /usr/share/cmake-3.22/Modules/FetchContent.cmake:1259 (FetchContent_Populate) CMakeLists.txt:97 (FetchContent_MakeAvailable)

与 marius-team/marius#16 看起来差不多

PhotonLib crashes with other thread models

As mentioned twice now, Photon crashes if I add another thread that is not Photon.
Q1. How can I use other thread models with Photon? More specifically OMP or if I can write my own coroutines. Because so far any other thread that is not Photon, makes your platform crash (as mentioned on your github site)

2023/05/09 06:19:19|INFO |th=0000000000000000|abc.cpp:642
handleSignal - gracefully stop Session 
Segmentation fault (core dumped)

As you can see, Photon reports thread as: |th=0000000000000000|

Suggestion: Uniform code style

用clang-format格式化一下？

WorkerPool usage uncertainty (multi vcpus)

I know the documentation will be done after feature work is complete, however I'd like some pointers in the meantime, thank you.

As far as I can see WorkerPool creates a std::thread for each vcpu, each of those threads are then initialized for photon thread usage. Then they go into a main loop where they take tasks off of a shared ring queue and perform them.

On the other hand, in net-perf.cpp:
1- photon thread gets created by the server (accept_loop creates a new photon thread upon accept)
2- initially this photon thread is on the OS thread that initialized the server
3- In the handler, as first thing it gets migrated to one of the work_pool vcpus, thus freeing the main vcpu.

Before I ask my question I want to say that so far this is my understanding, please correct me if I'm wrong. So what I'm having trouble understanding is how the vcpu ends up executing the migrated photon thread. It seems like the ring queue for the work pool should be empty since noone queued anything. So the main_loop function that gets executed by the OS thread yields at ring.recv() however since the default template parameter is std::thread::yield, that just means that OS thread yields back to OS scheduler, right? So photon::thread_yield is not called and I don't understand how the migrated photon thread ends up getting executed.

关注了，学习一下

Phonton Threads NOT working

Here is the output, and below is the code:
main() - IN -

2023/05/08 09:01:51|INFO |th=000055D2356115F0|/1/photon/io/iouring-wrapper.cpp:653|new_iouring:Init event engine: iouring [is_master=1]
2023/05/08 09:01:51|INFO |th=000055D2356115F0|/1/photon/io/iouring-wrapper.cpp:486|check_register_file_support:iouring: register_files is enabled

2023/05/08 09:01:51|INFO |th=00007F67446FAB40|/src/test/test-thread.cpp:27|hk:Thrd: HK
HK: i = 25
HK: i = 26
HK: i = 27
HK: i = 28
HK: i = 29
HK: i = 30
2023/05/08 09:01:57|INFO |th=00007F67446F82C0|/src/test/test-thread.cpp:41|mk:Thrd: MK
MK: j = 15
MK: j = 16
MK: j = 17
MK: j = 18
MK: j = 19
MK: j = 20
2023/05/08 09:02:01|INFO |th=00007F67446F8B00|/src/test/test-thread.cpp:55|sk:Thrd: SK
SK: i = 8
SK: i = 9
SK: i = 10
SK: i = 11
SK: i = 12

main() - OUT
Photon Threads:

• Run sequentially
• "coro" 'async_run()/run()' the same, when asked you said "coroutines" are experimental

Here is the simple code:

void hk()
{
	LOG_INFO("Thrd: HK");
	int i =0;
	while (true) {
		if (i >= 25 ) {
			print("HK: i = {}\n", i);
			if (i >= 30 ) break;
		}
		photon::thread_usleep(200000);
		i++;
	}
}

void mk()
{
	LOG_INFO("Thrd: MK");
	int j =0;
	while (true) {
		if (j >= 15 ) {
			print("MK: j = {}\n", j);
			if (j >= 20 ) break;
		}
		photon::thread_usleep(200000);
		j++;
	}
}

void sk()
{
	LOG_INFO("Thrd: SK");
	int i =0;
	while (true) {
		if (i >= 8 ) {
			print("SK: i = {}\n", i);
			if (i >= 12 ) break;
		}
		photon::thread_usleep(200000);
		i++;
	}
}

int main(int argc, char* argv[])
{
	print(fg("\nmain() - IN -\n");

	// Init IO Engine
	photon::init(photon::INIT_EVENT_IOURING, 0);

	auto hk_th = photon::thread_create11(hk);
	photon::thread_enable_join(hk_th);
	photon::thread_join((photon::join_handle*) hk_th);

	auto mk_th = photon::thread_create11(mk);
	photon::thread_enable_join(mk_th);
	photon::thread_join((photon::join_handle*) mk_th);

	auto sk_th = photon::thread_create11(sk);
	photon::thread_enable_join(sk_th);
	photon::thread_join((photon::join_handle*) sk_th);

	print(fg("\nmain() - OUT -\n");
	return 0;
}

use jemalloc as memory pool

I see that posix_memalign is currently used to allocate memory. Maybe jemalloc would be better?

question: Can it be compiled and run on centos7

hello，
I am very interested in the project，and want to write a live media server based on photonlibos, but most of our server‘s operating system is centso7，Upgrading the OS all is difficult.
Is it possible to compile and run on centos7 by downloading all the dependency librarys

Photon Threads Affinity

Q1. Do you provide a mechanism to set the thread affinity (pin to a cpu core) ? Or is the 'pthread_setaffinity' compatible with Proton threads? (Since ProtonLib crashes every time I use something other than Proton)

Integration with Web Sockets

Hello,

I saw your post on websocketpp where you proposed using PhotonLibOS to make a faster web socket server.

I wanted to encourage you - this would be awesome and very useful.
For sure faster web socket tool in C++ that makes use of io_uring would be amazing and likely break records.

If you integrated websockets into your library or integrated into websocketpp, myself and other C++ coders I know would be very interested. Please let me know if you hear of any developments in this area.

Cheers and thanks for this research & hard work,
Michael

Compare with seastar ?

Is there a comparison test with seastar ?

build test failed on windows wsl ubuntu 22.04

build env:
ubuntu 22.04 in windows wsl
gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0

/home/arthur/work/PhotonLibOS/fs/test/test.cpp: In function ‘void malloc_hook()’:
/home/arthur/work/PhotonLibOS/fs/test/test.cpp:693:5: error: ‘__malloc_hook’ was not declared in this scope; did you mean ‘malloc_hook’?
693 | __malloc_hook = my_malloc;
| ^~~~~~~~~~~~~
| malloc_hook
/home/arthur/work/PhotonLibOS/fs/test/test.cpp:694:5: error: ‘__free_hook’ was not declared in this scope
694 | __free_hook = my_free;
| ^~~~~~~~~~~
/home/arthur/work/PhotonLibOS/fs/test/test.cpp: In function ‘void malloc_unhook()’:
/home/arthur/work/PhotonLibOS/fs/test/test.cpp:698:5: error: ‘__malloc_hook’ was not declared in this scope; did you mean ‘malloc_hook’?
698 | __malloc_hook = old_malloc;
| ^~~~~~~~~~~~~
| malloc_hook
/home/arthur/work/PhotonLibOS/fs/test/test.cpp:699:5: error: ‘__free_hook’ was not declared in this scope
699 | __free_hook = old_free;
| ^~~~~~~~~~~
/home/arthur/work/PhotonLibOS/fs/test/test.cpp: In function ‘void init_hook()’:
/home/arthur/work/PhotonLibOS/fs/test/test.cpp:703:18: error: ‘__malloc_hook’ was not declared in this scope; did you mean ‘malloc_hook’?
703 | old_malloc = __malloc_hook;
| ^~~~~~~~~~~~~
| malloc_hook
/home/arthur/work/PhotonLibOS/fs/test/test.cpp:704:16: error: ‘__free_hook’ was not declared in this scope
704 | old_free = __free_hook;
| ^~~~~~~~~~~
[ 80%] Linking CXX executable ../../../output/client_perf

do you plan to support on windows systems?

do you plan to support windows systems?

Any plans on roadmap to add support for DPDK blobfs/blobstore

Hi PhotonLib Authors,

Just want to commend you guys for adding DPDK support via F-Stack. This does make life much easier for us who want high-performance networking but may not have the know how / time to integrate these libraries.

Will like to know if there are plans to add support for DPDK nvme driver via blobfs/blobstore to read/write append only files?

simple in examples crashed

OS: CentOS Linux release 7.9.2009 (Core)
GCC: 11.2.1 20220127 (Red Hat 11.2.1-9) (GCC)
Curl: 7.85.0

I rebuilt the curl source code and make install because it complained the error as below when I built PhotonLibOS (even libcurl-devel was installed).
‘CURLMOPT_MAX_TOTAL_CONNECTIONS’ was not declared in this scope

Core was generated by `./simple'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000000000040356c in run_socket_server(photon::net::ISocketServer*, photon::fs::IFile*, AlignedAlloc&, photon::std::condition_variable&, photon::std::mutex&, bool&)::{lambda(photon::net::ISocketStream*)#1}::operator()(photon::net::ISocketStream*) const ()
(gdb) bt
#0 0x000000000040356c in run_socket_server(photon::net::ISocketServer*, photon::fs::IFile*, AlignedAlloc&, photon::std::condition_variable&, photon::std::mutex&, bool&)::{lambda(photon::net::ISocketStream*)#1}::operator()(photon::net::ISocketStream*) const ()
#1 0x00007fe497f61681 in photon::net::KernelSocketServer::handler(Delegate<int, photon::net::ISocketStream*>, photon::net::ISocketStream*) ()
from /test/PhotonLibOS-0.3.1/examples/simple/libphoton.so
#2 0x00007fe497f61597 in photon::ThreadContext11<void ()(Delegate<int, photon::net::ISocketStream>, photon::net::ISocketStream*)>::stub(void*) ()
from /test/PhotonLibOS-0.3.1/examples/simple/libphoton.so
#3 0x00007fe497f71cbc in photon::thread_stub() () from /test/PhotonLibOS-0.3.1/examples/simple/libphoton.so
#4 0x0000000000000000 in ?? ()

IOURING and EchoServer fails : free() Invalid Pointer

System: HP, ubuntu 22, kernel 6.2, photon: latest git clone.

Running your sample echo_server.cpp coroutine sample, and exchanging the 'INIT_EVENT_EPOLL' with 'INIT_EVENT_IOURING' fails with the following error:

[INFO] .../io/iouring-wrapper.cpp:486|check_register_file_support:iouring:register_files is enabled
free(): invalid pointer
I need tis quite urgently, I thought this code was production ready, as per your website.

General comments

• Maybe a simple doc with all the 'gotchas' of using iouring (or better, a specific iouring sample, client socket please (not files and not server))
• iouring 'settings' facilieies/methods (sqes, poll, callbacks/eventfd, etc.)
• More importantly: a low latency setting possibilities (setting logging per default hurts latency, not everyone is throughput oriented).

Thank you, I need to get to production.

Obi

Repeated Timer and async_run

Hi guys,
Two, seemingly separated, questions, but related in my use case. The goal is:

• 1. A task sending socket msgs (and doing other things) every 30 seconds (until app stopped)
• 2. Another (independant) task sending socket msgs (to the same server) in parallel
• 3. The two tasks using their own different, ideally, for lowest latency, but to the same remote socket server.

Questions (and I've looked at the few provided examples):

• Q1. How to use the timer as in point 1 above?
• Q2. photon::coro::run() seems to run synchronously and on the same thread. run_async() seem, on paper, the right candidate. However, when I run two async_run(task1), async_run(task2), task1 seems to run, parallel to the main thread, but without returning for task2 to also start in parallel.
• So, the question really is how to run async two "run and forget" tasks in parallel? I'm not too kin on using the 'thread_create11' that you've used in several examples.
• Q3. The provided 'socketpool' example doesn't really, clearly, demonsttrate the use of a socket pool. I guess easier to do on a 'server' perspective. I am currently making a separate socket connection for task and task2. How to use the pool, on a 'client' perspective without having to make separate socket connection for each task client? (since ther are both/all talking to the same server?

Txs.

Making Photon work with GC

Turned out libc functions work great with Photon, it was the Boehm GC (libgc) that was causing segfaults. Making Photon work with GC is the only thing left to be done.

Stackful coroutines have their own stacks with dynamic sizes, such behavior makes it hard for the GC to detect alive objects. Does photon have a way to register coroutine stacks and let the GC know about context switching?

As I understand, Boehm GC has a way to let know about a new stack during context switching, but I didn't find this feature in Photon...

Originally posted by @medvednikov in #148 (comment)

build fs and thread test failed

Ubuntu 22.04
g++ version : 11.20
glibc version : 2.35

fs-test

the reason why fs test build failed : From glibc 2.24 onwards, the __malloc_initialize_hook variable has been removed from the API.It wasted me some time to find the answer.I thought it was caused by something changed by the mimmalloc(micsoft's malloc) I installed yesterday. Myabe we could remind users to check the version of glibc before compiling fs/test.

thread-test

I am not very familiar with c++ templates. I am very interested in your implementation of thread pool and want to learn the following. So please help me solve this problem.
build info :
FAILED: thread/test/CMakeFiles/test-thread.dir/test.cpp.o /usr/bin/c++ -I/home/susun/code/cpp/PhotonLibOS/include -mcrc32 -msse4.2 -O0 -g -fPIE -w -Wall -DGTEST_HAS_PTHREAD=1 -std=c++14 -MD -MT thread/test/CMakeFiles/test-thread.dir/test.cpp.o -MF thread/test/CMakeFiles/test-thread.dir/test.cpp.o.d -o thread/test/CMakeFiles/test-thread.dir/test.cpp.o -c /home/susun/code/cpp/PhotonLibOS/thread/test/test.cpp In file included from /home/susun/code/cpp/PhotonLibOS/thread/test/test.cpp:31: /home/susun/code/cpp/PhotonLibOS/thread/test/../thread11.h: In instantiation of ‘typename std::enable_if<(! std::is_void<decltype (& typename std::decay<_Tp>::type::operator())>::value), photon::thread*>::type photon::thread_create11(FUNCTOR&&, ARGUMENTS&& ...) [with FUNCTOR = thread11_lambda_Test::TestBody()::<lambda(photon::semaphore&)>&; ARGUMENTS = {photon::semaphore&}; typename std::enable_if<(! std::is_void<decltype (& typename std::decay<_Tp>::type::operator())>::value), photon::thread*>::type = photon::thread*; decltype (& typename std::decay<_Tp>::type::operator()) = void (thread11_lambda_Test::TestBody()::<lambda(photon::semaphore&)>::*)(photon::semaphore&) const; typename std::decay<_Tp>::type = std::decay<thread11_lambda_Test::TestBody()::<lambda(photon::semaphore&)>&>::type]’: /home/susun/code/cpp/PhotonLibOS/thread/test/test.cpp:1502:28: required from here /home/susun/code/cpp/PhotonLibOS/thread/test/../thread11.h:156:54: error: call of overloaded ‘thread_create11<thread11_lambda_Test::TestBody()::<lambda(photon::semaphore&)>&, photon::semaphore&>(const uint64_t&, thread11_lambda_Test::TestBody()::<lambda(photon::semaphore&)>&, photon::semaphore&)’ is ambiguous 156 | return thread_create11<FUNCTOR, ARGUMENTS...>( | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ 157 | DEFAULT_STACK_SIZE, std::forward<FUNCTOR>(f), | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 158 | std::forward<ARGUMENTS>(args)...); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

init iouring failed

hello i follow the instractions of Build in the file README.md. But when i run PhotonLibOS/build/output/simple-example，i got this error bellow:
2022/08/02 11:06:40|INFO |th=0000000000641FB0|/home/root/workspace/PhotonLibOS/io/iouring-wrapper.cpp:495|new_iouring:Init event engine: iouring [is_master=1] 2022/08/02 11:06:40|WARN |th=0000000000641FB0|/home/root/workspace/PhotonLibOS/io/iouring-wrapper.cpp:74|init:iouring: current user has no permission to set unlimited RLIMIT_MEMLOCK, change to root? 2022/08/02 11:06:46|ERROR|th=0000000000641FB0|/home/root/workspace/PhotonLibOS/io/iouring-wrapper.cpp:84|init:iouring: failed to init queue errno=38(Function not implemented) 2022/08/02 11:06:46|INFO |th=0000000000641FB0|/home/root/workspace/PhotonLibOS/io/iouring-wrapper.cpp:49|~iouringEngine:Finish event engine: iouring [m_master=1] 2022/08/02 11:06:46|ERROR|th=0000000000641FB0|/home/root/workspace/PhotonLibOS/examples/simple/simple.cpp:46|main:failed to init photon environment

i gdb follow step into here
static inline int ____sys_io_uring_setup(unsigned entries, struct io_uring_params *p) { #ifdef CONFIG_NOLIBC return __arch_impl_io_uring_setup(entries, p); #else int ret; ret = syscall(__NR_io_uring_setup, entries, p); return (ret < 0) ? -errno : ret; #endif }

syscall(__NR_io_uring_setup, entries, p); return -1， errno=38(Function not implemented)

my system is：CentOS Linux release 8.5.2111
linux kernel version is： 4.18.0

question: benchmark details

Hi,

how can we rerun the TCP benchmarks with PhotonLibOS ?
Did you use client and server from https://github.com/alibaba/PhotonLibOS/tree/main/examples/rpc ?

Regarding the results in https://github.com/alibaba/PhotonLibOS#21-tcp
Did you achieve 17.1Gb and 560K qps on a single core? Did I understand correctly?

HTTPServerRequest::Body() does not return the body of a request

    BeastErrorCode get_req(EaseTCPStream &stream) {
        BeastErrorCode ec{};
        BeastBuffer buffer;
        char buf[4096];
        BeastRequestParser rp(req);
        rp.get().body().data = buf;
        rp.get().body().size = sizeof(buf);
        rp.get().body().more = true;
        boost::beast::http::read_header(stream, buffer, rp, ec);
        while (!rp.is_done()) {
            rp.get().body().data = buf;
            rp.get().body().size = sizeof(buf);
            rp.get().body().more = true;
            boost::beast::http::read(stream, buffer, rp, ec);
            if (ec == BeastError::end_of_chunk) ec = {};
            if (ec) break;
        }
        if (!ec) {
            req = rp.release();
            m_origin_host = to_std_sv(req["Host"]);
        }
        return ec;
    }

Obviously Body() returns a pointer to a stack variable which makes no sense

测试例子

能否提供更多的简单示例？

Node.js echo server

I wrote this for you, I had forgot you asked about it.
Here is one which takes the same flags as yours, I did not implement streaming because I do not understand much about streaming.

// flags.ts
export const flags = {
    SHOW_STATISTICS_INTERVAL: Number(process.env.SHOW_STATISTICS_INTERVAL || 1),
    CLIENT_MODE: "ping-pong",
    CLIENT_CONNECTION_NUM: Number(process.env.CLIENT_CONNECTION_NUM || 100),
    IP: process.env.IP || "127.0.0.1",
    PORT: Number(process.env.PORT || 9527),
    BUF_SIZE: Number(process.env.BUF_SIZE || 512),
    VCPU_NUM: Number(process.env.VCPU_NUM || 1), // TODO: Node is single-threaded, so this is not useful
}

You can run it with yarn dev:server and yarn dev:client after doing yarn install for dependencies 👍

• FILE: photon-node-echo-server.zip

Code_-_Insiders_yeqjIOQJUA.mp4

How to implement streaming client that terminates after data transfer completion

Hi,

I am hoping someone could please advise or guide me on how to do this asynchronously with PhotonLibOS:

I have a server (A) application that needs to send one message (e.g. byte array) to a PhotonLibOS TCP server (B) and receive many messages in return. Once B has no more data to send it will send a final message to indicate the transfer has completed.
Then A will then terminate the client connection.

i have tried to do this using example from https://github.com/alibaba/PhotonLibOS/blob/main/examples/perf/net-perf.cpp

The TCP server (B) is working. It can receive message and write many messages back.
The Client (A) can sent a message and receive multiple messages.
However I don't know how to implement the client in such a way that it is just a function that will terminate the connection and return to the calling function.
The example for echo server/client is a client that is long running on a thread and just goes to sleep.

Any help will be much appreciated. Thank you.

Is it possible to use libc functions for networking and IO with Photon coroutines?

Hello,

I've successfully integrated Photon into the V programming language, the coroutines work with simple examples.

However when I try to use existing networking and IO libraries which use standard libc functions, I get undebuggable segfaults.

// Build with
// v -use-coroutines examples/simple_coroutines.v
//

import coroutines
import time
import net.http

fn foo(a int) {
    for {
        println('hello from foo2() a=${a}')
        resp := http.get('https://vlang.io/utc_now') or { panic(err) } // segfaults here
        println('resp=${resp.body}')
        coroutines.sleep(2 * time.second)
    }
}

fn main() {
    go foo(10)
    for {
        println('hello from MAIN')
        coroutines.sleep(1 * time.second)
    }
    println('done')
}

It's easy to use Photon's custom sleep functions, but re-writing the entire standard library to use Photon's net/IO is next to impossible.

Is there a way to use standard libc functions together with Photon coroutines?

协程栈分配优化问题

在thread_create的时候是否可以优化下，先在当前pthread栈里执行，在insert进waitq的时候再分配协程栈用于保存上下文；这样在竞争少的时候可以明显减少系统调用的开销；或者像bthread那样使用ResourcePool，不过这样无法实现变长栈的需求

benchmark

Can you add into the network benchmark also https://github.com/Qihoo360/evpp ?
It's way better than many other libraries used to compare with Photon

benchmark of context switch? (latency)

这个参数对我们的业务比较重要.
谢谢啊!

Purpose of asymmetric_spinLock

After reading the source code, I can't find the calling of background_try_lock. Every time an AtomicRunQ is initialized, foreground_lock will be called once. Meanwhile, foreground_lock won't conflict with each other. Therefore, asymmetric_spinLock doesn't seem to play a role in Photon thread.