heiher / hev-task-system

A simple, lightweight multi-task system (coroutines) for Unix (Linux/BSD/macOS)

HevTaskSystem

HevTaskSystem is a simple, lightweight multi-task system (coroutines) for Unix.

The task system is executed within a Unix process/thread. In task system, you can create many tasks and attach them to the task system. When a task yields or is blocked by I/O, the scheduler will pick a suitable task from running list and switch to it. Memory space, file descriptors, and other resources are shared among all tasks in the task system. Each task has its private, standalone task structure (#HevTask) and stack in heap of the process.

Within a task, you can allocate memory from heap, read and write data to the stack, and perform I/O operations in synchronized mode.

Features

• Simple/lightweight task.
• I/O operations wrapper.
• Inter-task synchronous. (Mutex/Condition)
• Inter-task communications. (Channel)
• Slice-based memory allocator.
• Call on new stack.
• Multi-thread support.
• Multi-platform support. (Linux/BSD/macOS)

How to Build

Unix:

git clone https://gitlab.com/hev/hev-task-system
cd hev-task-system
make

# Link with librt (only for glibc versions before 2.17)
make LDFLAGS=-lrt

# Disable stack overflow detection
make ENABLE_STACK_OVERFLOW_DETECTION=0

# Set stack backend to heap (Recommended for 32-bit)
make CONFIG_STACK_BACKEND=STACK_HEAP

# Disable sliced memory allocator
make ENABLE_MEMALLOC_SLICE=0

# Disable I/O splice by splice syscall (for old Linux kernel)
make ENABLE_IO_SPLICE_SYSCALL=0

# Demos
make apps

# Tests
make tests

Android:

mkdir hev-task-system
cd hev-task-system
git clone https://gitlab.com/hev/hev-task-system jni
ndk-build

Demos

1. simple
2. channel
3. timeout
4. wakeup
5. echo-server
6. call
7. gtk
8. curl

Contributors

• hev - https://hev.cc

License

MIT

hev-task-system's Issues

Memory pool using lru algorithm

May I ask why your memory pool implementation uses the LRU algorithm? Where is the specific role reflected?

error: dereferencing pointer ‘dst.6’ does break strict-aliasing rules

测试用例编译过程中出现告警：

src/lib/utils/hev-compiler.h:41: error: dereferencing pointer ‘dst.6’ does break strict-aliasing rules

GCC版本：4.4.7

此告警的原因在于进行小内存拷贝时，使用了强制类型转换：

static inline void
__copy_once_size (void *dst, const volatile void *src, int size)
{
    switch (size) {
    case sizeof (char):
        *(char *)dst = *(volatile char *)src;
        break;
    case sizeof (short):
        *(short *)dst = *(volatile short *)src;
        break;
    case sizeof (int):
        *(int *)dst = *(volatile int *)src;
        break;
    case sizeof (long long):
        *(long long *)dst = *(volatile long long *)src;
        break;
    default:
        barrier ();
        __builtin_memcpy ((void *)dst, (const void *)src, size);
        barrier ();
    }
}

依照manual的建议，这种拷贝方式并不安全。使用原生memcpy操作则是安全可靠的。

#define __copy_once_size(dst, src, size) memcpy(dst, src, size)

setjmp / longjmp 实现的？

longjmp 只有栈顶向栈底跳才是安全的，而且某些平台下不排除析构对象（例如windows），慎重使用

io-reactor: edge-triggered desn't work for nested epoll

fd-set per task: https://github.com/heiher/hev-task-system/tree/fd-set-dev
Issue: http://lkml.kernel.org/r/[email protected]

task_io_splicer_fini defined twice in hev-task-io.c

task_io_splicer_fini (HevTaskIOSplicer *self) is defined twice in hev-task-io.c

hev-task-system/src/lib/io/basic/hev-task-io.c

Line 245 in c2fcff7

task_io_splicer_fini (HevTaskIOSplicer *self)

hev-task-system/src/lib/io/basic/hev-task-io.c

Line 299 in c2fcff7

task_io_splicer_fini (HevTaskIOSplicer *self)

is that intentional?

io-splicer: splice system calls may block on nanopi-m4

[scp <-> fsh-connect] <-> [fsh-server] <-> [fsh-forward <-> sshd]

scp + fsh-connect: Host A.
fsh-server: Nanopi-M4.
sshd + fsh-forward: Host B.

fsh-server:
forward: scp -> fsh-connect -> socket fd 9 -> pipe wfd 12 -> pipe rfd 11 -> socket fd 10 -> sshd
backward: sshd -> socket fd 10 -> pipe wfd 14 -> pipe rfd 13 -> socket fd 9 -> fsh-forward -> scp

The splice(13, NULL, 9, NULL, 8192, SPLICE_F_MOVE, SPLICE_F_NONBLOCK) may returns false -1 (EAGAIN).

This problem has only been observed on nanopi-m4.

valgrind: Use of uninitialised value of size 8

valgrind执行测试用例test-task时报告异常：

Use of uninitialised value of size 8
   at 0x4016BB: hev_task_executer (in /home04/29593/mnt/hev-task-system-4.6.9/bin/test-task)
   by 0x40229B: hev_task_execute (in /home04/29593/mnt/hev-task-system-4.6.9/bin/test-task)
   by 0x400DAD: hev_task_system_run_new_task (in /home04/29593/mnt/hev-task-system-4.6.9/bin/test-task)
 Uninitialised value was created by a stack allocation
   at 0x40229C: hev_task_execute (in /home04/29593/mnt/hev-task-system-4.6.9/bin/test-task)

检测指令为

valgrind -v --vgdb=full --leak-check=full --track-fds=yes --show-reachable=yes --trace-children=yes --tool=memcheck --num-callers=64 --log-file=valgrind.log --track-origins=yes ./test-task

去除-O3优化之后，valgrind报告未初始化的变量来自于hev-task-execute-x86_64.s的第23行：

movq  (%rsp), %rax

不太熟悉汇编，因此这里不知道如何排查了。

Add counter to Task IO splice

Hello,
This is implementation I did so that counter can be used with task_io_splice call.

master...vavrecan:hev-task-system:master

Usage is simple them,

    size_t received;
    size_t sent;
    hev_task_io_splice_counter (cfd, cfd, fd, fd, 8192, task_io_yielder, self, &sent, &received);

Let me know if you find it useful or if it makes sense to integrate, I can make PR then.
Thank you