Coder Social home page Coder Social logo

jrmadsen / ptl Goto Github PK

View Code? Open in Web Editor NEW
39.0 5.0 13.0 773 KB

Parallel Tasking Library (PTL) - Lightweight C++11 mutilthreading tasking system featuring thread-pool, task-groups, and lock-free task queue

License: MIT License

C 0.44% Shell 1.47% CMake 21.00% C++ 72.73% Python 4.36%

ptl's Introduction

Parallel Tasking Library (PTL)

Lightweight C++11 multithreading tasking system featuring thread-pool, task-groups, and lock-free task queue

Basic Interface

#include "PTL/PTL.hh"

#include <cassert>

inline long
fibonacci(long n)
{
    return (n < 2) ? n : (fibonacci(n - 1) + fibonacci(n - 2));
}

int main()
{
    bool use_tbb     = false;
    auto num_threads = 4;
    auto run_manager = PTL::TaskRunManager(use_tbb);

    run_manager.Initialize(num_threads);

    auto* task_manager = run_manager.GetTaskManager();

    // add a task via the task manager
    auto baz = task_manager->async<long>(fibonacci, 40);

    // functor to combine results
    auto join = [](long& lhs, long rhs) { return lhs += rhs; };

    // create a task group for 10 fibonacci calculations
    PTL::TaskGroup<long> foo(join);
    for(uint64_t i = 0; i < 10; ++i)
        foo.exec(fibonacci, 40);

    // create a task group for 10 fibonacci calculations
    PTL::TaskGroup<void> bar{};

    long ret_bar = 0;
    auto run     = [&ret_bar](long n) { ret_bar += fibonacci(n); };
    for(uint64_t i = 0; i < 10; ++i)
        bar.exec(run, 40);

    auto ret_baz = baz->get();
    auto ret_foo = foo.join();
    bar.join();

    assert(ret_baz * 10 == ret_foo);
    assert(ret_baz * 10 == ret_bar);
    assert(ret_foo == ret_bar);
}

Explicit Thread-Pool

Using PTL::TaskRunManager is not necessary with task-groups. You can create new thread-pools directly and pass them to task-groups:

long example()
{
    // create a new thread-pool explicitly
    PTL::ThreadPool tp(4);

    // combines results
    auto join = [](long& lhs, long rhs) { return lhs += rhs; };

    // specify thread-pool explicitly
    PTL::TaskGroup<long> foo(join, &tp);

    for(int i = 0; i < 10; ++i)
        foo.exec(fibonacci, 40);

    // blocks until tasks in group are completed
    // thread-pool is destroyed after function returns
    return foo.get();
}

ptl's People

Contributors

amadio avatar badshah400 avatar carterbox avatar drbenmorgan avatar jrmadsen avatar loximann avatar slyon avatar stephanlachnit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

ptl's Issues

minimal.cc example fails on ARM

Describe the bug
ptl-minimal crashes with SIGFPE, Arithmetic exception on 32bit ARM. If commit 4e230f6 is reverted, the problem disappears. Especially the following line seems to introduce the Floating point exception:
static intmax_t nincr = std::max<intmax_t>(ncores / ncpus, 1);

See also: https://bugs.debian.org/1001237

(gdb) run
Starting program: /root/ptl-2.3.0/examples/build/minimal/ptl-minimal 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".


          ##############################
          !!! Backtrace is activated !!!
          ##############################

[ptl-minimal]> Number of threads: 1
[New Thread 0xf7c259a0 (LWP 17519)]

Thread 1 "ptl-minimal" received signal SIGFPE, Arithmetic exception.
__libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
47	../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S: No such file or directory.
(gdb) bt
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
#1  0xf7cac3e0 in __pthread_kill_implementation (threadid=4160618512, signo=signo@entry=8, no_tid=no_tid@entry=0)
    at pthread_kill.c:43
#2  0xf7cac424 in __pthread_kill_internal (signo=<optimized out>, threadid=<optimized out>) at pthread_kill.c:80
#3  0xf7c7b690 in __GI_raise (sig=8) at ../sysdeps/posix/raise.c:26
#4  0xf7d8201e in __aeabi_ldiv0 () from /lib/arm-linux-gnueabihf/libgcc_s.so.1
#5  0x00407e5c in main::{lambda(long long)#3}::operator()(long long) const ()
#6  0x0040d49e in long long std::__invoke_impl<long long, main::{lambda(long long)#3}&, long long>(std::__invoke_other, main::{lambda(long long)#3}&, long long&&) ()
#7  0x0040ca6e in std::enable_if<is_invocable_r_v<long long, main::{lambda(long long)#3}&, long long>, long long>::type std::__invoke_r<long long, main::{lambda(long long)#3}&, long long>(main::{lambda(long long)#3}&, long long&&) ()
#8  0x0040baf6 in std::_Function_handler<long long (long long), main::{lambda(long long)#3}>::_M_invoke(std::_Any_data const&, long long&&) ()
#9  0xf7f9f620 in PTL::ThreadPool::set_affinity(long long, std::thread&) const ()
   from /lib/arm-linux-gnueabihf/libptl.so.2
#10 0xf7fa1234 in PTL::ThreadPool::initialize_threadpool(unsigned int) () from /lib/arm-linux-gnueabihf/libptl.so.2
#11 0xf7fa22d8 in PTL::ThreadPool::ThreadPool(PTL::ThreadPool::Config const&) ()
   from /lib/arm-linux-gnueabihf/libptl.so.2
#12 0x00408ada in main ()

To Reproduce
Steps to reproduce the behavior:

  1. apt install libptl-dev (v2.3.0-1)
  2. cd examples && mkdir build && cd build
  3. cmake .. && make
  4. ./minimal/ptl-minimal (Observe crash)

Expected behavior
Test should be run successfully.

Desktop (please complete the following information):

  • OS: Ubuntu Jammy (devel)
  • Version 22.04

Additional context
Does not seem to happen on other architectures but armhf (32bit ARM)

Minimal examples fail on armhf

The minimal examples fail to compile on the armhf architecture. This was found by Debian's CI (tested twice to make sure it's not a random failure).
The bug report is here and the log here.

The interesting part starts in line 1941:

Building CXX object minimal/CMakeFiles/ptl-minimal.dir/minimal.cc.o
In file included from /usr/include/c++/10/map:61,
                 from /usr/include/PTL/Utility.hh:30,
                 from /usr/include/PTL/Globals.hh:36,
                 from /usr/include/PTL/Threading.hh:28,
                 from /usr/include/PTL/AutoLock.hh:246,
                 from /usr/include/PTL/PTL.hh:22,
                 from /tmp/autopkgtest-lxc.czgi6jo2/downtmp/autopkgtest_tmp/minimal/minimal.cc:22:
/usr/include/c++/10/bits/stl_map.h: In member function ‘std::map<_Key, _Tp, _Compare, _Alloc>::mapped_type& std::map<_Key, _Tp, _Compare, _Alloc>::operator[](const key_type&) [with _Key = std::thread::id; _Tp = long long unsigned int; _Compare = std::less<std::thread::id>; _Alloc = std::allocator<std::pair<const std::thread::id, long long unsigned int> >]’:
/usr/include/c++/10/bits/stl_map.h:501:37: note: parameter passing for argument of type ‘std::_Rb_tree<std::thread::id, std::pair<const std::thread::id, long long unsigned int>, std::_Select1st<std::pair<const std::thread::id, long long unsigned int> >, std::less<std::thread::id>, std::allocator<std::pair<const std::thread::id, long long unsigned int> > >::const_iterator’ changed in GCC 7.1
  501 |    __i = _M_t._M_emplace_hint_unique(__i, std::piecewise_construct,
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  502 |          std::tuple<const key_type&>(__k),
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  503 |          std::tuple<>());
      |          ~~~~~~~~~~~~~~~             

I have to be honest, I don't know what is going on here. The examples work fine on other architectures (amd64, arm64, i386, ppc64el), and iirc it worked fine before with PTL 1.0.2.

The CI runs on Debian Sid, which is the rolling development version of Debian. GCC is at version 10.3.0.

Not reproducible

PTL 1.0.0 is currently not reproducible. The diffs are created by the template cmake/Templates/PTLConfig.cmake.in:

set_and_check(@PROJECT_NAME@_INCLUDE_DIR "@PACKAGE_INCLUDE_INSTALL_DIR@")
set_and_check(@PROJECT_NAME@_LIB_DIR "@PACKAGE_LIB_INSTALL_DIR@")

set(CMAKE_MODULE_PATH @PROJECT_SOURCE_DIR@/cmake/Modules ${CMAKE_MODULE_PATH})

include("@PROJECT_BINARY_DIR@/@PROJECT_NAME@-${_COMPONENT}.cmake")

The problem is that the current build path is always inserted into PTLConfig.cmake, even if it is a system install which shouldn't be the case. I'm not fluent enough in CMake to propose a solution though.

Issue with Requires.private in pkgconfig file

Describe the bug
The Requires.private line in the pkgconfig file seems to be erroneous according to the pkg-config specifications. Both Requires and Requires.private should list names of pkg-config modules and not refer to flags directly, whereas -pthread looks more appropriate for Libs.private. Indeed on my system (openSUSE Tumbleweed) there does not seem to be any pkg-config module by the name of -pthread (or even pthread) installed even though PTL can find pthread.h when building and link against it just fine (glibc-devel provides the pthread header).

Please let me know if I am wrong here; if not, here is a straightforward patch which I can make a PR with if you wish:

Index: PTL-2.2.0/cmake/Templates/ptl.pc.in
===================================================================
--- PTL-2.2.0.orig/cmake/Templates/ptl.pc.in
+++ PTL-2.2.0/cmake/Templates/ptl.pc.in
@@ -6,6 +6,6 @@ Name: PTL
 Description: Parallel Tasking Library for C++
 Version: @PTL_VERSION_STRING@
 Requires: @PTL_PC_TBB_REQUIREMENT@
-Requires.private: -pthread
 Libs: -L${libdir} -lptl
+Libs.private: -pthread
 CFlags: -std=c++@CMAKE_CXX_STANDARD@ -pthread -I${includedir}

Many thanks for your library btw.

/usr/include/PTL/Backtrace.hh:511:12: error: ‘strlen’ was not declared in this scope

Describe the bug
When compiling the minimal example, this error appears:

Building CXX object minimal/CMakeFiles/ptl-minimal.dir/minimal.cc.o
In file included from /usr/include/PTL/PTL.hh:23,
                 from /tmp/PTL/examples/minimal/minimal.cc:22:
/usr/include/PTL/Backtrace.hh: In static member function ‘static void PTL::Backtrace::Message(int, siginfo_t*, std::ostream&)’:
/usr/include/PTL/Backtrace.hh:511:12: error: ‘strlen’ was not declared in this scope
  511 |         if(strlen(itr) == 0)
      |            ^~~~~~
/usr/include/PTL/Backtrace.hh:191:1: note: ‘strlen’ is defined in header ‘<cstring>’; did you forget to ‘#include <cstring>’?
  190 | #    include <regex>
  +++ |+#include <cstring>
  191 | #    include <set>

To Reproduce
Compile the minimal examples on Debian Sid, see also https://bugs.debian.org/1016942

Expected behavior
Compilation does not fail

Screenshots

Building CXX object minimal/CMakeFiles/ptl-minimal.dir/minimal.cc.o
In file included from /usr/include/PTL/PTL.hh:23,
                 from /tmp/PTL/examples/minimal/minimal.cc:22:
/usr/include/PTL/Backtrace.hh: In static member function ‘static void PTL::Backtrace::Message(int, siginfo_t*, std::ostream&)’:
/usr/include/PTL/Backtrace.hh:511:12: error: ‘strlen’ was not declared in this scope
  511 |         if(strlen(itr) == 0)
      |            ^~~~~~
/usr/include/PTL/Backtrace.hh:191:1: note: ‘strlen’ is defined in header ‘<cstring>’; did you forget to ‘#include <cstring>’?
  190 | #    include <regex>
  +++ |+#include <cstring>
  191 | #    include <set>

Desktop (please complete the following information):

  • OS: Debian Sid
  • Compiler: GCC12
  • Version: 2.3.3-1

Additional context
This seems like a rather odd error to me, since this did not happen before, and cstring is already included here:

#include <cstring>

Including cstring at the position the compiler suggest fixes the issue indeed, however I don't understand why.

PTL uses deprecated TBB headers

While compiling Geant4 10.7 beta 01, many warnings show up from usage of deprecated headers from TBB:

In file included from geant4.10.07.b01/source/externals/ptl/include/PTL/ThreadData.hh:35,
                 from geant4.10.07.b01/source/externals/ptl/include/PTL/ThreadPool.hh:34,
                 from geant4.10.07.b01/source/externals/ptl/include/PTL/TaskGroup.hh:35,
                 from geant4.10.07.b01/source/externals/ptl/include/PTL/TBBTaskGroup.hh:32,
                 from geant4.10.07.b01/source/tasking/include/G4TBBTaskGroup.hh:42,
                 from geant4.10.07.b01/source/tasking/include/G4TaskRunManager.hh:43,
                 from geant4.10.07.b01/source/tasking/include/G4RunManagerFactory.hh:40,
                 from geant4.10.07.b01/source/tasking/src/G4RunManagerFactory.cc:28:
/usr/include/tbb/task_scheduler_init.h:21:154: note: #pragma message: TBB Warning: tbb/task_scheduler_init.h is deprecated. For details, please see Deprecated Features appendix in the TBB reference manual.
 #pragma message("TBB Warning: tbb/task_scheduler_init.h is deprecated. For details, please see Deprecated Features appendix in the TBB reference manual.")

Git tags for version

I want to package this for Debian. For this workflow, git tags make my life much easier.

However, even though there is a VERSION file which keeps track of a version, there are no version specific git tags. It would be nice if that could be done.

Windows min max macro imported unexpectedly

Describe the bug
Building Geant4 version on windows imports min max macros from windows.h when using 2021.6.0 version of tbb on visual studio 16.11.1, which breaks code in PTL that uses std::min and std::max (e.g. in ThreadPool.hh line 468 in function void ThreadPool::execute_on_all_threads(FuncT&& _func)).

To Reproduce
Description contains reproduction details.

Expected behavior
Normal compilation.

Screenshots
non

Putting this block (or NOMINMAX) somewhere in the code should solve the problem. Still locating the include location.

#ifdef _WIN32
#define WIN32_LEAN_AND_MEAN
#endif

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.