Coder Social home page Coder Social logo

jrmadsen / ptl Goto Github PK

View Code? Open in Web Editor NEW
39.0 5.0 13.0 778 KB

Parallel Tasking Library (PTL) - Lightweight C++11 mutilthreading tasking system featuring thread-pool, task-groups, and lock-free task queue

License: MIT License

C 0.43% Shell 1.46% CMake 20.95% C++ 72.80% Python 4.34%

ptl's Issues

PTL uses deprecated TBB headers

While compiling Geant4 10.7 beta 01, many warnings show up from usage of deprecated headers from TBB:

In file included from geant4.10.07.b01/source/externals/ptl/include/PTL/ThreadData.hh:35,
                 from geant4.10.07.b01/source/externals/ptl/include/PTL/ThreadPool.hh:34,
                 from geant4.10.07.b01/source/externals/ptl/include/PTL/TaskGroup.hh:35,
                 from geant4.10.07.b01/source/externals/ptl/include/PTL/TBBTaskGroup.hh:32,
                 from geant4.10.07.b01/source/tasking/include/G4TBBTaskGroup.hh:42,
                 from geant4.10.07.b01/source/tasking/include/G4TaskRunManager.hh:43,
                 from geant4.10.07.b01/source/tasking/include/G4RunManagerFactory.hh:40,
                 from geant4.10.07.b01/source/tasking/src/G4RunManagerFactory.cc:28:
/usr/include/tbb/task_scheduler_init.h:21:154: note: #pragma message: TBB Warning: tbb/task_scheduler_init.h is deprecated. For details, please see Deprecated Features appendix in the TBB reference manual.
 #pragma message("TBB Warning: tbb/task_scheduler_init.h is deprecated. For details, please see Deprecated Features appendix in the TBB reference manual.")

Minimal examples fail on armhf

The minimal examples fail to compile on the armhf architecture. This was found by Debian's CI (tested twice to make sure it's not a random failure).
The bug report is here and the log here.

The interesting part starts in line 1941:

Building CXX object minimal/CMakeFiles/ptl-minimal.dir/minimal.cc.o
In file included from /usr/include/c++/10/map:61,
                 from /usr/include/PTL/Utility.hh:30,
                 from /usr/include/PTL/Globals.hh:36,
                 from /usr/include/PTL/Threading.hh:28,
                 from /usr/include/PTL/AutoLock.hh:246,
                 from /usr/include/PTL/PTL.hh:22,
                 from /tmp/autopkgtest-lxc.czgi6jo2/downtmp/autopkgtest_tmp/minimal/minimal.cc:22:
/usr/include/c++/10/bits/stl_map.h: In member function ‘std::map<_Key, _Tp, _Compare, _Alloc>::mapped_type& std::map<_Key, _Tp, _Compare, _Alloc>::operator[](const key_type&) [with _Key = std::thread::id; _Tp = long long unsigned int; _Compare = std::less<std::thread::id>; _Alloc = std::allocator<std::pair<const std::thread::id, long long unsigned int> >]’:
/usr/include/c++/10/bits/stl_map.h:501:37: note: parameter passing for argument of type ‘std::_Rb_tree<std::thread::id, std::pair<const std::thread::id, long long unsigned int>, std::_Select1st<std::pair<const std::thread::id, long long unsigned int> >, std::less<std::thread::id>, std::allocator<std::pair<const std::thread::id, long long unsigned int> > >::const_iterator’ changed in GCC 7.1
  501 |    __i = _M_t._M_emplace_hint_unique(__i, std::piecewise_construct,
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  502 |          std::tuple<const key_type&>(__k),
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  503 |          std::tuple<>());
      |          ~~~~~~~~~~~~~~~

I have to be honest, I don't know what is going on here. The examples work fine on other architectures (amd64, arm64, i386, ppc64el), and iirc it worked fine before with PTL 1.0.2.

The CI runs on Debian Sid, which is the rolling development version of Debian. GCC is at version 10.3.0.

Issue with Requires.private in pkgconfig file

Describe the bug
The Requires.private line in the pkgconfig file seems to be erroneous according to the pkg-config specifications. Both Requires and Requires.private should list names of pkg-config modules and not refer to flags directly, whereas -pthread looks more appropriate for Libs.private. Indeed on my system (openSUSE Tumbleweed) there does not seem to be any pkg-config module by the name of -pthread (or even pthread) installed even though PTL can find pthread.h when building and link against it just fine (glibc-devel provides the pthread header).

Please let me know if I am wrong here; if not, here is a straightforward patch which I can make a PR with if you wish:

Index: PTL-2.2.0/cmake/Templates/ptl.pc.in
===================================================================
--- PTL-2.2.0.orig/cmake/Templates/ptl.pc.in
+++ PTL-2.2.0/cmake/Templates/ptl.pc.in
@@ -6,6 +6,6 @@ Name: PTL
 Description: Parallel Tasking Library for C++
 Version: @PTL_VERSION_STRING@
 Requires: @PTL_PC_TBB_REQUIREMENT@
-Requires.private: -pthread
 Libs: -L${libdir} -lptl
+Libs.private: -pthread
 CFlags: -std=c++@CMAKE_CXX_STANDARD@ -pthread -I${includedir}

Many thanks for your library btw.

Not reproducible

PTL 1.0.0 is currently not reproducible. The diffs are created by the template cmake/Templates/PTLConfig.cmake.in:

PTL/cmake/Templates/PTLConfig.cmake.in

Lines 41 to 42 in 6049e69

set_and_check(@PROJECT_NAME@_INCLUDE_DIR "@PACKAGE_INCLUDE_INSTALL_DIR@")
set_and_check(@PROJECT_NAME@_LIB_DIR "@PACKAGE_LIB_INSTALL_DIR@")

PTL/cmake/Templates/PTLConfig.cmake.in

Line 84 in 6049e69

set(CMAKE_MODULE_PATH @PROJECT_SOURCE_DIR@/cmake/Modules ${CMAKE_MODULE_PATH})

PTL/cmake/Templates/PTLConfig.cmake.in

Line 90 in 6049e69

include("@PROJECT_BINARY_DIR@/@PROJECT_NAME@-${_COMPONENT}.cmake")

The problem is that the current build path is always inserted into PTLConfig.cmake, even if it is a system install which shouldn't be the case. I'm not fluent enough in CMake to propose a solution though.

minimal.cc example fails on ARM

Describe the bug
ptl-minimal crashes with SIGFPE, Arithmetic exception on 32bit ARM. If commit 4e230f6 is reverted, the problem disappears. Especially the following line seems to introduce the Floating point exception:
static intmax_t nincr = std::max<intmax_t>(ncores / ncpus, 1);

See also: https://bugs.debian.org/1001237

(gdb) run
Starting program: /root/ptl-2.3.0/examples/build/minimal/ptl-minimal 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".


          ##############################
          !!! Backtrace is activated !!!
          ##############################

[ptl-minimal]> Number of threads: 1
[New Thread 0xf7c259a0 (LWP 17519)]

Thread 1 "ptl-minimal" received signal SIGFPE, Arithmetic exception.
__libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
47	../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S: No such file or directory.
(gdb) bt
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
#1  0xf7cac3e0 in __pthread_kill_implementation (threadid=4160618512, signo=signo@entry=8, no_tid=no_tid@entry=0)
    at pthread_kill.c:43
#2  0xf7cac424 in __pthread_kill_internal (signo=<optimized out>, threadid=<optimized out>) at pthread_kill.c:80
#3  0xf7c7b690 in __GI_raise (sig=8) at ../sysdeps/posix/raise.c:26
#4  0xf7d8201e in __aeabi_ldiv0 () from /lib/arm-linux-gnueabihf/libgcc_s.so.1
#5  0x00407e5c in main::{lambda(long long)#3}::operator()(long long) const ()
#6  0x0040d49e in long long std::__invoke_impl<long long, main::{lambda(long long)#3}&, long long>(std::__invoke_other, main::{lambda(long long)#3}&, long long&&) ()
#7  0x0040ca6e in std::enable_if<is_invocable_r_v<long long, main::{lambda(long long)#3}&, long long>, long long>::type std::__invoke_r<long long, main::{lambda(long long)#3}&, long long>(main::{lambda(long long)#3}&, long long&&) ()
#8  0x0040baf6 in std::_Function_handler<long long (long long), main::{lambda(long long)#3}>::_M_invoke(std::_Any_data const&, long long&&) ()
#9  0xf7f9f620 in PTL::ThreadPool::set_affinity(long long, std::thread&) const ()
   from /lib/arm-linux-gnueabihf/libptl.so.2
#10 0xf7fa1234 in PTL::ThreadPool::initialize_threadpool(unsigned int) () from /lib/arm-linux-gnueabihf/libptl.so.2
#11 0xf7fa22d8 in PTL::ThreadPool::ThreadPool(PTL::ThreadPool::Config const&) ()
   from /lib/arm-linux-gnueabihf/libptl.so.2
#12 0x00408ada in main ()

To Reproduce
Steps to reproduce the behavior:

  1. apt install libptl-dev (v2.3.0-1)
  2. cd examples && mkdir build && cd build
  3. cmake .. && make
  4. ./minimal/ptl-minimal (Observe crash)

Expected behavior
Test should be run successfully.

Desktop (please complete the following information):

  • OS: Ubuntu Jammy (devel)
  • Version 22.04

Additional context
Does not seem to happen on other architectures but armhf (32bit ARM)

/usr/include/PTL/Backtrace.hh:511:12: error: ‘strlen’ was not declared in this scope

Describe the bug
When compiling the minimal example, this error appears:

Building CXX object minimal/CMakeFiles/ptl-minimal.dir/minimal.cc.o
In file included from /usr/include/PTL/PTL.hh:23,
                 from /tmp/PTL/examples/minimal/minimal.cc:22:
/usr/include/PTL/Backtrace.hh: In static member function ‘static void PTL::Backtrace::Message(int, siginfo_t*, std::ostream&)’:
/usr/include/PTL/Backtrace.hh:511:12: error: ‘strlen’ was not declared in this scope
  511 |         if(strlen(itr) == 0)
      |            ^~~~~~
/usr/include/PTL/Backtrace.hh:191:1: note: ‘strlen’ is defined in header ‘<cstring>’; did you forget to ‘#include <cstring>’?
  190 | #    include <regex>
  +++ |+#include <cstring>
  191 | #    include <set>

To Reproduce
Compile the minimal examples on Debian Sid, see also https://bugs.debian.org/1016942

Expected behavior
Compilation does not fail

Screenshots

Building CXX object minimal/CMakeFiles/ptl-minimal.dir/minimal.cc.o
In file included from /usr/include/PTL/PTL.hh:23,
                 from /tmp/PTL/examples/minimal/minimal.cc:22:
/usr/include/PTL/Backtrace.hh: In static member function ‘static void PTL::Backtrace::Message(int, siginfo_t*, std::ostream&)’:
/usr/include/PTL/Backtrace.hh:511:12: error: ‘strlen’ was not declared in this scope
  511 |         if(strlen(itr) == 0)
      |            ^~~~~~
/usr/include/PTL/Backtrace.hh:191:1: note: ‘strlen’ is defined in header ‘<cstring>’; did you forget to ‘#include <cstring>’?
  190 | #    include <regex>
  +++ |+#include <cstring>
  191 | #    include <set>

Desktop (please complete the following information):

  • OS: Debian Sid
  • Compiler: GCC12
  • Version: 2.3.3-1

Additional context
This seems like a rather odd error to me, since this did not happen before, and cstring is already included here:

PTL/source/PTL/Backtrace.hh

Line 72 in 3a258fd

#include <cstring>

Including cstring at the position the compiler suggest fixes the issue indeed, however I don't understand why.

Non-use of perfect forwarding in TaskGroup run/exec?

In preparing some guides for Geant4 developers/users on the use of PTL in that code, I ran into a slightly un-intuitive (but not necessarily wrong...) gotcha in the use of TaskGroup. Take the rather contrived case of:

#include "PTL/AutoLock.hh"
#include "PTL/TaskGroup.hh"
#include "PTL/Threading.hh"
#include "PTL/Types.hh"

#include <iostream>
#include <vector>

class ExpensiveToCopy
{
public:
    ExpensiveToCopy() { log("constructor"); }
    ~ExpensiveToCopy() { log("destructor"); }
    ExpensiveToCopy(const ExpensiveToCopy&) { log("copy-constructor"); }
    ExpensiveToCopy(ExpensiveToCopy&&) { log("move-constructor"); }
    ExpensiveToCopy& operator=(const ExpensiveToCopy&)
    {
        log("copy-assignment");
        return *this;
    }
    ExpensiveToCopy& operator=(ExpensiveToCopy&&)
    {
        log("move-assignment");
        return *this;
    }

    void log(std::string_view msg) const
    {
        std::cout << this << ": " << msg << std::endl;
    }
};

void
pass_by_constref(const ExpensiveToCopy& val)
{
    val.log("pass_by_constref called!");
    return;
}

int
main()
{
    ExpensiveToCopy y;
    PTL::TaskGroup<void> group;
    group.exec(pass_by_constref, y); // passes y by value...
    // this is pass by ref: group.exec(pass_by_constref, std::cref(y));
    group.join();
}

As in the comment, whilst pass_by_constref takes a constant reference, exec doesn't deduce this automatically as the parameter pack is passed by value (and consequently there are several copies/moves before the final call). std::ref/cref or a lambda with capture-by-reference do result in the expected behaviour.

The implementations of exec are obviously not so trivial, so I wondered if this prevented use of perfect forwarding of arguments (Ala make_unique) or whether this could be supported in future developments (I'm happy to have a go at it if so!)?

Windows min max macro imported unexpectedly

Describe the bug
Building Geant4 version on windows imports min max macros from windows.h when using 2021.6.0 version of tbb on visual studio 16.11.1, which breaks code in PTL that uses std::min and std::max (e.g. in ThreadPool.hh line 468 in function void ThreadPool::execute_on_all_threads(FuncT&& _func)).

To Reproduce
Description contains reproduction details.

Expected behavior
Normal compilation.

Screenshots
non

Putting this block (or NOMINMAX) somewhere in the code should solve the problem. Still locating the include location.

#ifdef _WIN32
#define WIN32_LEAN_AND_MEAN
#endif

Git tags for version

I want to package this for Debian. For this workflow, git tags make my life much easier.

However, even though there is a VERSION file which keeps track of a version, there are no version specific git tags. It would be nice if that could be done.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.