dfeneyrou / palanteer Goto Github PK

View Code? Open in Web Editor NEW

2.0K 54.0 85.0 9.31 MB

Visual Python and C++ nanosecond profiler, logger, tests enabler

License: Other

CMake 0.70% C++ 90.38% Python 8.09% C 0.83%

developer-tools profiler profiling python cpp c-plus-plus memory-profiler performance optimization logging

palanteer's People

Contributors

Stargazers

Watchers

Forkers

bareya seekaddo zhuoqiang frankfan007 hoangpq mfkiwl siwonheo ezhangle enginbozkurt andrey-kovinjonok adbmd sarvex learn-more simenglv tomasli cnliao mu-l ajweeks dataorz sunfu-chou benknightdark kennych2000 fatiangel housir2001 apachesep cybernhl cmeninwa frankyu0326 jake-ryu brodyhiggerson mikeyhodl lyrl jnfn shiftwinting forkrp chengwei920412 prince-xuanchan concurs-program maddyvc laplacekorea iitsmel victor333huesca jceipek chandanksingh hervengassop mbounouar vinayakshastri sourabharvikar3 doraemoncito tweakoz andreyc2018 1starkitty luis-cyber alexander-manley frklan linecode jiangkevin kayws426 koalago korallin ichrysou spocktavian greatwallisme pinojojo exactao rjean leonardoricky clayne deiolly tuskaw tberkey-rp gerhobbelt ajunlonglive jamestiotio gg-big-org impact-of-compiler-warnings-thesis 4144 geotyper zjp linhld0811 shawnfang-github rachidjelloul2 im0xcc iambluefred janicebeinrot

palanteer's Issues

Could you possibly make available a demo .pltraw file

I think it'd be handy for evaluation purposes.

Python multiprocessing support

Hello,

I've traced a Python application that uses multiprocessing. The way the processes are started is like this:

main process running a PySide6 GUI
- secondary process started from the main process
  - after some user configuration the secondary process starts several child processes

I can only see the main process trace information.

Is there something that I'm missing?

Hard to interpret fiber work when looking at list of threads

Since you previously mentioned workflow and usability wrt fibers, I figured I'd make this post, although I appreciate it's likely a bit more ambitious.

There are a few issues I see in this area.

The way I use FTL is to have every bit of work be performed by fibers - to the point that my 'main' thread of execution is just another fiber pinned to a specific worker. I have lot of fibers and use them for everything.

When I look at this view, it doesn't really tell me what's going on, and I can't gleam much from it wrt what work is executing when/where. Highlighting a fiber span does highlight the corresponding function stack in the latter parts of the window, but this is difficult to see.

In FTL, the default number of fibers is 400, which, as you can imagine, blows up the size of the "Fibers" area quite a bit (and possibly hits an error message wrt thread limits from memory - can't check right now), and makes the aforementioned highlighting not really useful - which of the 400 fibers is being highlighted? No idea!

Ideally, there would be a thread-first view of the worker threads that shows the work being performed on them as normal; i.e. how you would see work displayed on threads without using fibers - I think it matters less which fiber # is active when, and instead more which callstack is active when on which thread. So in this example on the left wouldn't be fibers but instead the worker threads executing any and all fibers, showing the work those threads are performing, with Palanteer doing the magic of tying fiber-based work to the threads that execute it.

Hopefully that makes sense. Again, totally appreciate it's a bunch of work. Just wanted to provide my perspective; happy to test any ideas in this area, incremental changes, prototypes, etc.

Question - Shared libraries

Has the library a design for working in separated projects but they can be connected with one static library? I mean is it possible to create one class with implementation where it is a wrapper for palanteer. Otherwise, I can't understand why the header file doesn't include code even if I declare preprocessor PL_IMPLEMENTATION clearly in .cpp file.

Or the purpose of this library is to test small codebases because I can't imagine how to use it if you have many separated projects that are connected with one main static library.

As the result, I have the error LNK2019: unresolved external symbol. Describe the correct usage of your header in the case when I want to use the implementation in .cpp file, but have declarations of functions (USE_PL) in a header file and share it between all projects. It's really sad if I can't do that at all.

Add support for user-space threads

Fibers or simulated threads (for instance in a Discrete Event Simulator) are currently not supported, only OS threads are.
Request from @BrodyHiggerson

This support impacts only the C++ instrumentation library and requires:

an API to switch between virtual-threads, typically called in the framework hook when starting/resuming a new virtual-thread.
- Typical effect is to override the internal thread ID
- Putting back the OS thread ID shall be possible and done when no virtual-threads are in use.
a virtual-thread declaration API which allows mapping an external thread ID to a user-thread name.

On top of these "event redirections on ghost threads", it is desired to easily visualize the load and usage on each worker threads. Indeed, these worker threads are "empty" because their events are redirected to the virtual-thread ones (they are not duplicated). Only the context switches are visible on them.
For this purpose, the concept of "lock" shall be generalized into "lock and resource", and such "worker thread resource" shall be taken and released when switching virtual threads.

unexpected app shutdown

I have a big profiling data file > 1gb. After loading application hangs and closes.
Reproducibility 100%

[https://transfer.pcloud.com/download.html?code=5ZELnbVZ6rcgA5EfxkFZLzoUZkVrHCBRnHrLtwaNHIHgEQm8jksKy](profile data)

Crash when trying to instrument a program on current git master

Linux x64: as soon as I launch the app to profile, the GUI crashes with:

[PALANTEER] Assertion failed: _underRecordRecIdx>=0
  On function: void vwMain::draw()
  On file    : /home/jcelerier/projets/perso/lafajol/3rdparty/palanteer/server/viewer/vwMain.cpp(357)
    - int     _underRecordRecIdx   = -1

  #0  0x55AE269C7652 : _ZN6vwMain4drawEv
  #1  0x55AE26A59C37 : _ZN10vwPlatform6redrawEv
  #2  0x55AE26A59D29 : _ZN10vwPlatform3runEv
  #3  0x55AE26A5A80B : _Z11bsBootstrapiPPc
  #4  0x7F560497130C : __libc_start_call_main
  #5  0x7F56049713BD : __libc_start_main@@GLIBC_2.34
  #6  0x55AE269449D1 : _start

plgScopeDyn() with USE_PL=0 throws build errors

Hello,

Thank you for the Palanteer project. It has made my life a lot easier. Cheers to you guys!

I face the following issue whenbuilding with Palanteer deactivated.
I wish to deactivate Palanteer when my program is built in 'Release'.
I noticed an issue I have with plgScopeDyn() when USE_PL is set to '0'.

This is my code (USE_PL = 0):

#define PL_IMPLEMENTATION 1
#define PL_GROUP_TEST 1
#include "palanteer.h"
.
.
.
[[maybe_unused]] auto text = "testFunc";
plgScopeDyn(TEST, text);

When I compile this code with clang, I get the following error:
use of undeclared identifier 'TEST'

The rest of the pl[g]***Dyn() variant functions (plgFunction(), plgLockScopeState(), do not seem to cause build errors.

Best way to support HiDPI/Retina displays?

I'm in the early stages of trying to write a platform layer for MacOS for the viewer as a learning project (I've never written one before). I've had some early success but am wondering about the best way to handle a non unity framebuffer scale.
If I don't try to account for it at all, I get this:

(although the mouse coordinate system is in the upper left quadrant)

If I add io.DisplayFramebufferScale = ImVec2(2.0, 2.0); under this https://github.com/dfeneyrou/palanteer/blob/main/server/viewer/vwPlatform.cpp#L333 as a test, I get this:

Note how many of the UI elements are clipped and smaller or larger than they should be.

My main questions are:

What additional changes do you think are needed in order to properly achieve framebuffer scaling without clipping/size issues?
Do you think extending osGetWindowSize(_displayWidth, _displayHeight); to instead be osGetWindowSize(_displayWidth, _displayHeight, _fbScaleX, _fbScaleY); is the appropriate way to get the scale factors into the cross-platform portion of the codebase?

How to start service (so thread/fiber registration works) but not start recording yet?

I've been trying the fiber integration as per your helpful Gist from a while back (https://gist.github.com/dfeneyrou/8f8aa2956dcb32e0860665c610d7bb2f).

I was following my usage pattern of Optick, where I didn't start profiling from the beginning but instead triggered it via a debug menu. However, my FTL Task Scheduler is created at the beginning, and its creation invokes the callbacks shown in the above Gist. Because of this, the calls to e.g. plDeclareVirtualThread take place before the call to plInitAndStart, and so in the profiler itself none of that information is shown.

Am I able to somehow start up the service at the beginning so that these registrations are accepted, but not start any kind of recording / transmission to the Palanteer application until later? I know there's PL_MODE_INACTIVE, but as far as I can see, the mode can't be changed after the init call?

I'll have to try calling plInitAndStart at game start, creating my scheduler and having the FTL callbacks fire, then immediately calling plStopAndUninit, and then calling init again later when I want to actually profile. Bit messy though and not sure if it works right.

EDIT: Looks like init-stop-init isn't the solution; not surprised.

Request: Search -> New Plot with option to create plot for the selected function across all threads/fibers

In the following image, I'm looking to see all cases of "PredictFlights" across the recorded timespan, but they're instead plotted per thread (in this case per fiber), which as you can see, is quite a lot! It would take a long time to manually add each fiber's call(s) to a single plot. I had a look at the code but it's all new to me so figured I'd throw a request in here too :)

nmake fails at "Generating pyhton_instrumental_install" on Windows 10

Hello,

I have installed Visual Studio 2022 along with Python 3.12. When I use nmake, I get the following output, and I don't know what else I can configure:

[ 92%] Built target palanteer
[ 97%] Built target testprogram
[ 98%] Generating python_instrumentation_install
running bdist_wheel
running build
running build_py
copying C:\Users\zoekm\Documents\palanteer\python\palanteer_init_.py -> build\lib.win32-cpython-312\palanteer
copying C:\Users\zoekm\Documents\palanteer\python\palanteer_main_.py -> build\lib.win32-cpython-312\palanteer
running build_ext
building 'palanteer._cextension' extension
"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\bin\HostX86\x86\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD "-IC:\Program Files\Python312\include" "-IC:\Program Files\Python312\Include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /TpC:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp /Fobuild\temp.win32-cpython-312\Release\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.obj -I C:\Users\zoekm\Documents\palanteer\c++ -std=c++11 /wd4996
cl : Befehlszeile warning D9002 : Unbekannte Option "-std=c++11" wird ignoriert.
pyPalanteerInstrumentation.cpp
C:\Users\zoekm\Documents\palanteer\c++\palanteer.h(381): warning C4068: Unbekanntes Pragma "GCC".
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(180): error C2039: "frame" ist kein Member von "_ts".
C:\Program Files\Python312\include\cpython/pystate.h(115): note: Siehe Deklaration von "_ts"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(183): error C2039: "frame" ist kein Member von "_ts".
C:\Program Files\Python312\include\cpython/pystate.h(115): note: Siehe Deklaration von "_ts"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(187): error C2039: "frame" ist kein Member von "_ts".
C:\Program Files\Python312\include\cpython/pystate.h(115): note: Siehe Deklaration von "_ts"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(261): error C2027: Verwendung des undefinierten Typs "_frame"
C:\Program Files\Python312\include\pytypedefs.h(22): note: Siehe Deklaration von "_frame"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(263): error C2027: Verwendung des undefinierten Typs "_frame"
C:\Program Files\Python312\include\pytypedefs.h(22): note: Siehe Deklaration von "_frame"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(263): error C2065: "FRAME_SUSPENDED": nichtdeklarierter Bezeichner
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(381): error C2027: Verwendung des undefinierten Typs "_frame"
C:\Program Files\Python312\include\pytypedefs.h(22): note: Siehe Deklaration von "_frame"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(401): error C2039: "co_varnames" ist kein Member von "PyCodeObject".
C:\Program Files\Python312\include\cpython/code.h(175): note: Siehe Deklaration von "PyCodeObject"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(402): error C2027: Verwendung des undefinierten Typs "_frame"
C:\Program Files\Python312\include\pytypedefs.h(22): note: Siehe Deklaration von "_frame"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(629): error C2027: Verwendung des undefinierten Typs "_frame"
C:\Program Files\Python312\include\pytypedefs.h(22): note: Siehe Deklaration von "_frame"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(629): warning C4477: "snprintf": Die Formatzeichenfolge "%s" erfordert ein Argument vom Typ "char *", das variadic-Argument "1" weist aber den Typ "int" auf.
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(629): warning C4477: "snprintf": Die Formatzeichenfolge "%d" erfordert ein Argument vom Typ "int", das variadic-Argument "2" weist aber den Typ "const char *" auf.
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(629): warning C4473: "snprintf": nicht genügend Argumente für Formatzeichenfolge übergeben
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(629): note: Platzhalter und deren Parameter erwarten 3 variadic-Argumente, es wurden aber 2 variadic-Argumente bereitgestellt.
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(629): note: Das fehlende variadic-Argument "3" ist für die Zeichenfolge "%s" erforderlich.
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(642): error C2027: Verwendung des undefinierten Typs "_frame"
C:\Program Files\Python312\include\pytypedefs.h(22): note: Siehe Deklaration von "_frame"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(964): error C2039: "use_tracing" ist kein Member von "_PyCFrame".
C:\Program Files\Python312\include\cpython/pystate.h(67): note: Siehe Deklaration von "_PyCFrame"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(1219): error C2039: "use_tracing" ist kein Member von "_PyCFrame".
C:\Program Files\Python312\include\cpython/pystate.h(67): note: Siehe Deklaration von "_PyCFrame"
C:\Users\zoekm\Documents\palanteer\python\palanteer_cextension\pyPalanteerInstrumentation.cpp(1241): error C2039: "use_tracing" ist kein Member von "_PyCFrame".
C:\Program Files\Python312\include\cpython/pystate.h(67): note: Siehe Deklaration von "_PyCFrame"
error: command 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\bin\HostX86\x86\cl.exe' failed with exit code 2
NMAKE : fatal error U1077: "echo >nul && "C:\Program Files\Python312\python.exe" C:/Users/zoekm/Documents/palanteer/build/python/setup.py bdist_wheel": Rückgabe-Code "0x1"
Stop.
NMAKE : fatal error U1077: ""C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\bin\HostX86\x86\nmake.exe" -s -f python\CMakeFiles\python_instrumentation.dir\build.make /nologo -SL python\CMakeFiles\python_instrumentation.dir\build": Rückgabe-Code "0x2"
Stop.
NMAKE : fatal error U1077: ""C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\bin\HostX86\x86\nmake.exe" -s -f CMakeFiles\Makefile2 /nologo -S all": Rückgabe-Code "0x2"
Stop.

pip install palanteer fails

Hello, I am trying to install palanteer package but im hitting the below issue. Any pointers would be greatly appreciated.

Installing collected packages: palanteer
  Running setup.py install for palanteer ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for palanteer did not run successfully.
  │ exit code: 1
  ╰─> [35 lines of output]
      running install
      /proj/xsjsswstaff/tejuss/miniconda3/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        setuptools.SetuptoolsDeprecationWarning,
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.7
      creating build/lib.linux-x86_64-3.7/palanteer
      copying palanteer/__main__.py -> build/lib.linux-x86_64-3.7/palanteer
      copying palanteer/__init__.py -> build/lib.linux-x86_64-3.7/palanteer
      running build_ext
      building 'palanteer._cextension' extension
      creating build/temp.linux-x86_64-3.7
      creating build/temp.linux-x86_64-3.7/palanteer
      creating build/temp.linux-x86_64-3.7/palanteer/_cextension
      /proj/xsjsswstaff/tejuss/miniconda3/bin/x86_64-conda_cos6-linux-gnu-cc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /proj/xsjsswstaff/tejuss/miniconda3/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /proj/xsjsswstaff/tejuss/miniconda3/include -fPIC -I/proj/xsjsswstaff/tejuss/miniconda3/include/python3.7m -c palanteer/_cextension/pyPalanteerInstrumentation.cpp -o build/temp.linux-x86_64-3.7/palanteer/_cextension/pyPalanteerInstrumentation.o -I palanteer/_cextension
      cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
      In file included from palanteer/_cextension/pyPalanteerInstrumentation.cpp:46:0:
      palanteer/_cextension/palanteer.h: In member function 'void plPriv::CliManager::registerCli(plCliHandler_t, const char*, const char*, const char*, plPriv::hashStr_t, plPriv::hashStr_t, plPriv::hashStr_t)':
      palanteer/_cextension/palanteer.h:2440:55: error: expected ')' before 'PRId64'
                               int readQty = sscanf(s, "[[%" PRId64 "]]", &p.defaultValue);
                                                             ^~~~~~
      palanteer/_cextension/palanteer.h:2440:83: warning: spurious trailing '%' in format [-Wformat=]
                               int readQty = sscanf(s, "[[%" PRId64 "]]", &p.defaultValue);
                                                                                         ^
      palanteer/_cextension/palanteer.h:2440:83: warning: too many arguments for format [-Wformat-extra-args]
      palanteer/_cextension/palanteer.h: In member function 'const char* plPriv::CliManager::execute(const char*, plPriv::plRemoteStatus&)':
      palanteer/_cextension/palanteer.h:2537:42: error: expected ')' before 'PRId64'
                               if(sscanf(s, "%" PRId64, &_cio._paramValues[paramIdx])!=1 || !skipValue(s))
                                                ^~~~~~
      palanteer/_cextension/palanteer.h:2537:78: warning: spurious trailing '%' in format [-Wformat=]
                               if(sscanf(s, "%" PRId64, &_cio._paramValues[paramIdx])!=1 || !skipValue(s))
                                                                                    ^
      palanteer/_cextension/palanteer.h:2537:78: warning: too many arguments for format [-Wformat-extra-args]
      error: command '/proj/xsjsswstaff/tejuss/miniconda3/bin/x86_64-conda_cos6-linux-gnu-cc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> palanteer

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Make `pip install palanteer` possible

Hello guys,

Thanks for your contribution. palanteer seems really cool. I have to give it a shot to see what it has to offer.

I am proposing that you would consider publishing your package in the PyPi registry so that python developers would be able to install it using pip.

Keep up the good work and stay awesome. 🥂

Building broken on Linux - permission denied

bareya@bigone ~/g/p/build (main) [1]> cmake .. -DCMAKE_INSTALL_PREFIX=/opt/palanteer
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Build type: Release   (change with -DCMAKE_BUILD_TYPE=<Debug|Release|RelWithDebInfo|MinSizeRel|Asan>)
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so   
-- Found X11: /usr/include   
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
CMake Warning (dev) at /opt/cmake/3.19.8/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (LIBUNWIND)
  does not match the name of the calling package (LibUnwind).  This can lead
  to problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/FindLibUnwind.cmake:23 (find_package_handle_standard_args)
  server/viewer/CMakeLists.txt:29 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found LIBUNWIND: /usr/lib/x86_64-linux-gnu/libunwind.so  
CMake Warning (dev) at /opt/cmake/3.19.8/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (LIBDW) does
  not match the name of the calling package (LibDw).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/FindLibDw.cmake:23 (find_package_handle_standard_args)
  server/viewer/CMakeLists.txt:30 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found LIBDW: /usr/lib/x86_64-linux-gnu/libdw.so  
Palanteer feature 'stacktrace' enabled for viewer
CMake Warning (dev) at /opt/cmake/3.19.8/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (LIBUNWIND)
  does not match the name of the calling package (LibUnwind).  This can lead
  to problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/FindLibUnwind.cmake:23 (find_package_handle_standard_args)
  c++/testprogram/CMakeLists.txt:24 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /opt/cmake/3.19.8/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (LIBDW) does
  not match the name of the calling package (LibDw).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/FindLibDw.cmake:23 (find_package_handle_standard_args)
  c++/testprogram/CMakeLists.txt:25 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

Palanteer feature 'stacktrace' enabled for testprogram
-- Found Python3: /usr/bin/python3.8 (found version "3.8.5") found components: Interpreter 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/bareya/git/palanteer/build
bareya@bigone ~/g/p/build (main)> make 
Scanning dependencies of target palanteer
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwConfig.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwConst.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwFileDialog.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwFontData.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMain.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainCommon.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainHistogram.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainMarker.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainMemory.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainPlot.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainProfile.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainRecordCatalog.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainSearch.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainText.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwMainTimeline.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwPlatform.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/vwReplayAlloc.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/common/cmCnx.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/common/cmCompress.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/common/cmLiveControl.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/common/cmRecord.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/common/cmRecordIterator.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/common/cmRecording.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/base/bsGlHelper.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/base/bsOsLinux.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/base/bsOsWindows.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/base/bsString.cpp.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/common/debug.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/common/entropy_common.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/common/error_private.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/common/fse_decompress.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/common/pool.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/common/threading.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/common/xxhash.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/common/zstd_common.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/fse_compress.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/hist.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/huf_compress.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_compress.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_compress_literals.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_compress_sequences.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_compress_superblock.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_double_fast.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_fast.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_lazy.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_ldm.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstd_opt.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/compress/zstdmt_compress.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/decompress/huf_decompress.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/decompress/zstd_ddict.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/decompress/zstd_decompress.c.o
[100%] Building C object server/viewer/CMakeFiles/palanteer.dir/__/external/zstd/decompress/zstd_decompress_block.c.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/external/imgui/imgui.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/external/imgui/imgui_draw.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/external/imgui/imgui_tables.cpp.o
[100%] Building CXX object server/viewer/CMakeFiles/palanteer.dir/__/external/imgui/imgui_widgets.cpp.o
[100%] Linking CXX executable ../../bin/palanteer
[100%] Built target palanteer
Scanning dependencies of target testprogram
[100%] Building CXX object c++/testprogram/CMakeFiles/testprogram.dir/testProgram.cpp.o
[100%] Building CXX object c++/testprogram/CMakeFiles/testprogram.dir/testPart.cpp.o
[100%] Linking CXX executable ../../bin/testprogram
[100%] Built target testprogram
Scanning dependencies of target python_instrumentation
[100%] Generating python_instrumentation_install
running install
running bdist_egg
running egg_info
writing /home/bareya/git/palanteer/python/palanteer.egg-info/PKG-INFO
writing dependency_links to /home/bareya/git/palanteer/python/palanteer.egg-info/dependency_links.txt
writing top-level names to /home/bareya/git/palanteer/python/palanteer.egg-info/top_level.txt
reading manifest file '/home/bareya/git/palanteer/python/palanteer.egg-info/SOURCES.txt'
writing manifest file '/home/bareya/git/palanteer/python/palanteer.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
creating build/bdist.linux-x86_64/egg
error: could not create 'build/bdist.linux-x86_64/egg': Permission denied
make[2]: *** [python/CMakeFiles/python_instrumentation.dir/build.make:84: python/python_instrumentation_install] Error 1
make[1]: *** [CMakeFiles/Makefile2:230: python/CMakeFiles/python_instrumentation.dir/all] Error 2
make: *** [Makefile:103: all] Error 2

MSVC 2022: C++ library build broken with UNICODE

When trying to build the c++ library while having UNICODE declared, the following error occurs:

palanteer-src\c++\palanteer.h(4346,53): error C2440: '=': cannot convert from 'const wchar_t [17]' to 'LPWSTR'

Basically, in that line, LogFile.LoggerName = KERNEL_LOGGER_NAME; the structure field expects a non-const value, while KERNEL_LOGGER_NAME is const. I have worked around it by casting it to (LPWSTR). I don't know the API exactly, so it feels like a hack to cast it to non-constness.

- LogFile.LoggerName          = KERNEL_LOGGER_NAME;
+ #if defined(_UNICODE) || defined(UNICODE)
+ LogFile.LoggerName          = (LPWSTR)KERNEL_LOGGER_NAME;
+ #else
+ LogFile.LoggerName          = KERNEL_LOGGER_NAME;
+ #endif

Error: Assertion failed: lastScopeEndTimeNs<=scopeStartTimeNs in C++'s testprogram collect

First, thanks for your work on this tool :)

I get an assertion failure on the viewer:

build_main/bin/palanteer
[PALANTEER] Assertion failed: lastScopeEndTimeNs<=scopeStartTimeNs
  On function: void vwMain::prepareTimeline(vwMain::Timeline&)
  On file    : ../server/viewer/vwMainTimeline.cpp(1407)
    - s64     lastScopeEndTimeNs   = 479190960
    - s64     scopeStartTimeNs     = 310591404
    - s64     scopeEndTimeNs       = 310948897

when running

build_main/bin/testprogram collect
Mode 'connected'
[00:00:00.023010505] [debug] [Not recorded] An integer value 1234
[00:00:00.023035299] [info ] [Not recorded] Several other values 1234, -0.123, 3.14e+200 and 1234567891234567.
[00:00:00.023054506] [warn ] [Not recorded] Some values 00001234 and some strings: 'rabbit
and fox'.
[00:00:00.023063934] [error] [Not recorded] A pointer 0x555ad6da99d7 with the address of the string
Statistics:
  Execution time: 1118 ms
  Sending calls : 23
  Sent events   : 482235
  Sent strings  : 97
  Max dyn string usage: 37            ( 0.04% of max)
  Max buffer usage    : 3859520 bytes ( 5.51% of max)

It happens both in v0.6 and in main. Commenting out the assert in vwMainTimeline.cpp allows me to use the tool, but...

plAssert(lastScopeEndTimeNs<=scopeStartTimeNs, lastScopeEndTimeNs, scopeStartTimeNs, scopeEndTimeNs);

Find attached the trace collected until the assert is triggered:
Cpp testcollect.zip

Some info about my system:

Ubuntu 20.04
g++ 9.3.0
Python 3.8.10

Thanks a lot for your time taking a look at this, really appreciate it.

Question - How to use palanteer c++ library and avoid multiple definition

For some project with c++ its always necessary to compile different part separately.
However, the single-header lib seems to result in multiple definition in this case.

Take this minimum project as an example

main.cpp

#define PL_IMPLEMENTATION 1
#include "../work/palanteer/c++/palanteer.h"
#include <iostream>
int add(int a, int b);

int main() {
    plFunction();
    add(1, 2);
    std::cout << "Hello, World!" << std::endl;
    return 0;
}

add.cpp

#define PL_IMPLEMENTATION 1
#include "../work/palanteer/c++/palanteer.h"

int add(int a, int b) {
    plFunction();
    return a + b;
}

Then compile it with

g++ -c main.cpp -o 1.o -DUSE_PL=1; g++ -c add.cpp -o 2.o -DUSE_PL=1; g++ 1.o 2.o -o ./main.out

It fails and return

...
testpl2.cpp:(.text+0x9838): multiple definition of `plGetStats()'; 1.o:testpl1.cpp:(.text+0x9838): first defined here
/usr/bin/ld: 2.o: in function `plDeclareVirtualThread(unsigned int, char const*, ...)':
testpl2.cpp:(.text+0x9879): multiple definition of `plDeclareVirtualThread(unsigned int, char const*, ...)'; 1.o:testpl1.cpp:(.text+0x9879): first defined here
/usr/bin/ld: 2.o: in function `plDetachVirtualThread(bool)':
testpl2.cpp:(.text+0x9919): multiple definition of `plDetachVirtualThread(bool)'; 1.o:testpl1.cpp:(.text+0x9919): first defined here
/usr/bin/ld: 2.o: in function `plAttachVirtualThread(unsigned int)':
testpl2.cpp:(.text+0x996e): multiple definition of `plAttachVirtualThread(unsigned int)'; 1.o:testpl1.cpp:(.text+0x996e): first defined here
collect2: error: ld returned 1 exit status

have i misunderstood sth? how to avoid this issue if i want to use palanteer?

Missing pl[g]Function API when instrumentation is disabled (USE_PL=0 or not defined)

@learn-more provided #3 with correction

Error: unable to check the elem MR non increasing speck in Python's testprogram collect

First of all, thanks a million for your work on this. I'm just playing around with Palanteer and I am getting the following error when trying to collect data with the Python example:

python/testprogram/testprogram.py collect
Mode 'connected'
Statistics:
  Execution time: 1659 ms

I get it every single time this or any other app finishes. After that, I can't use the saved trace anymore. I can only work commenting out the check in cmRecord.cpp, even though I know this is not right, it's the only way I can use the tool:

// for(int i=0; i<curArray.size(); ++i) {
//     if(curArray[i].speckUs>upperArray[i/cmMRElemSize].speckUs) LOAD_ERROR("check the elem MR non increasing speck");
// }

Here's the file that is collected by the viewer, in case it's useful. When trying to open it with the viewer, I get the already mentioned error
Python testcollect.zip

This happens both in v0.6 and in main. It happens on every single Python application I've tested once it finishes. In case it's useful, some info about my system:

Ubuntu 20.04
g++ 9.3.0
Python 3.8.10

Again, thanks a lot for this tool and for your time taking a look at this issue.

BUG: unable to use the timing view on different threads, always defaults to thread 0

Hi, love the tool, however when instrumenting python and opening it in the viewer, pressing Profile timings always defaults to thread 0,
I tracked down the code and patched it, can't say it's the best solution, but it worked for me:

Index: server/viewer/vwMainProfile.cpp
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/server/viewer/vwMainProfile.cpp b/server/viewer/vwMainProfile.cpp
--- a/server/viewer/vwMainProfile.cpp	(revision 331398fe9c994bd921753e8d9ac9818b2e962326)
+++ b/server/viewer/vwMainProfile.cpp	(date 1639396331682)
@@ -71,7 +71,7 @@
     plgVar(PROF, threadUniqueHash, startTimeNs, getNiceDuration(timeRangeNs));
 
     // Add the request
-    _profiles.push_back({id, kind, startTimeNs, timeRangeNs, threadUniqueHash});
+    _profiles.push_back({id, kind, startTimeNs, timeRangeNs, threadUniqueHash,threadId});
     setFullScreenView(-1);
     return true;
 }
@@ -130,12 +130,12 @@
     if(prof.computationLevel==0 && _backgroundComputationInUse) return true; // Waiting for a free slot
 
     // Finish the initialization if needed (init and live)
-    if(prof.threadId<0 && (prof.isFirstRun || _liveRecordUpdated)) {
+    if(prof.threadId!=3324 && (prof.isFirstRun || _liveRecordUpdated)) {
         prof.isFirstRun = false;
 
         for(int threadId=0; threadId<_record->threads.size(); ++threadId) {
             if(_record->threads[threadId].threadUniqueHash!=prof.threadUniqueHash) continue;
-            prof.threadId = threadId;
+//            prof.threadId = threadId;
 
             // Thread found: complete the profile initialization
             if(prof.reqNestingLevel<0) {
@@ -223,7 +223,7 @@
         u32 scopeLIdx2 = itScope.getNextScope(isCoarseScope, dummyScopeStartTimeNs, dummyScopeEndTimeNs, evt, durationNs);
         (void)scopeLIdx2;
         plAssert(!isCoarseScope);                                      // By design
-        plAssert(scopeLIdx2==item.scopeLIdx, scopeLIdx2, item.scopeLIdx); // By design
+//        plAssert(scopeLIdx2==item.scopeLIdx, scopeLIdx2, item.scopeLIdx); // By design
         prof.computationLevel = bsMinMax((int)(100LL*(evt.vS64-prof.startTimeNs)/prof.timeRangeNs), 1, 99); // 0 means just started, 100 means finished
 
         // Get infos on its children

"make install" doesn't work

It needs to support make install to install Palanteer globally on a system.

Crash in Python instrumentation when recording with C calls

Seen with Palanteer version 0.3, Linux
Reproduced with (for instance): python3 -m palanteer -c =meld (the tool meld is pure python)
The displayed stack trace is:

[PALANTEER] Assertion failed: filename || filenameHash
  On function: void logFunctionEvent(PyObject*, PyFrameObject*, PyObject*, bool, bool)
  On file    : (...)/python/palanteer/_cextension/pyPalanteerInstrumentation.cpp(486)

make errors

Getting the following errors when trying to build on:
Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-1160.71.1.el7.x86_64
Architecture: x86-64
cmake version: 3.24.2
make: GNU Make 3.82
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)

The error I'm seeing is:
/home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bsOsLinux.cpp:840:93: error: no matching function for call to ‘bsDate::bsDate(<brace-e nclosed initializer list>)’ return bsDate{1900+t->tm_year, 1+t->tm_mon, t->tm_mday, t->tm_hour, t->tm_min, t->tm_sec}; ^ /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bsOsLinux.cpp:840:93: note: candidates are: In file included from /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bsOsLinux.cpp:41:0: /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bs.h:101:8: note: constexpr bsDate::bsDate() struct bsDate { ^ /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bs.h:101:8: note: candidate expects 0 arguments, 6 provided /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bs.h:101:8: note: constexpr bsDate::bsDate(const bsDate&) /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bs.h:101:8: note: candidate expects 1 argument, 6 provided /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bs.h:101:8: note: constexpr bsDate::bsDate(bsDate&&) /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bs.h:101:8: note: candidate expects 1 argument, 6 provided /home/jupyterhub3/baburn/curr_sandboxes/open_src_tools/palanteer/server/base/bsOsLinux.cpp: In function ‘bsDate osGetCreationDate(const bsString&)’:

Viewer crash when asking an histogram for a "lock use" scope

Env: All OS
Release: 0.4
How to reproduce: simply by requesting an histogram of a lock use, for instance from a timeline "lock use" scope with contextual menu
Reproducibility: Systematic

The failed assertion is:

[PALANTEER] Assertion failed: index>=0 && index<_size 
  On function: const T& bsVec<T>::operator[](int) const [with T = bsString]
  On file    : (...)/palanteer/server/viewer/../base/bsVec.h(107)
    - int     index                = -1
    - int     _size                 = 4

  #0  bsVec.h(107) : bsVec<bsString>::operator[](int) const
  #1  vwMain.h(111) : vwMain::getFullThreadName(int) const
  #2  vwMainHistogram.cpp(92) : vwMain::_computeChunkHistogram(vwMain::Histogram&)
  #3  vwMainHistogram.cpp(388) : vwMain::drawHistograms()
  #4  vwMain.cpp(1094) : vwMain::draw()
  #5  vwPlatform.cpp(364) : vwPlatform::redraw()
  #6  vwPlatform.cpp(304) : vwPlatform::run()
  #7  vwPlatform.cpp(196) : bsBootstrap(int, char**)
  #8  bsOsLinux.cpp(826) : main

Consistent crashes when navigating to some directories with the import file dialog

Here, https://github.com/dfeneyrou/palanteer/blob/main/server/viewer/vwFileDialog.cpp#L203-L204

I consistently get a EXC_I386_GPFLT (general protection fault) on a.name.toChar() when navigating to certain directories. I can iterate and print the names of all elements in _fileEntries without issue, so I imagine that std::sort is doing something undesirable in conjunction with your string library. Unfortunately I'm not really sure how best to debug this issue, and I haven't found anything particularly special about the failing directories.

[Mac OS port] Potential issues with keyboard modifier state

In my MacOS port, I've been noticing issues where the application didn't realize I was holding down certain modifier keys.
I believe the cause is that the modifier state isn't being appropriately synchronized here: https://github.com/dfeneyrou/palanteer/blob/main/server/viewer/vwPlatform.cpp#L642-L718

Note that the kms parameter is ignored in many of the functions in the file, which can lead to these imgui fields not being synchronized to the keyboard state correctly:

io.KeyCtrl
io.KeyShift
io.KeyAlt  
io.KeySuper

I also haven't been able to determine which osHandler->event* function would be best to call when responding to the NSEventTypeFlagsChanged event type on macOS. See https://github.com/ocornut/imgui/blob/0b8a2470743f89d19ab2d5920606434b604b529e/backends/imgui_impl_osx.mm#L342-L345 for how it is handled in the Dear ImGui example.

MSVC 2022: Run-Time Check Failure #3 - The variable 'prevE' is being used without being initialized.

Checked out at: e305d17

When zooming a timeline in, either in a live view or record that uses fibers, visual studio complains of an access to an initialized variable.

palanteer.exe!vwMain::prepareTimeline(vwMain::Timeline & tl) Line 1150 C++
palanteer.exe!vwMain::drawTimeline(int tlWindowIdx) Line 1478 C++
palanteer.exe!vwMain::drawTimelines() Line 1432 C++
palanteer.exe!vwMain::draw() Line1149 C++
palanteer.exe!vwPlatform::redraw() Line 365 C++
palanteer.exe!vwPlatform::run() Line 304 C++
palanteer.exe!bsBootstrap(int argc, char * * argv) Line 200 C++
palanteer.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * hPrevInstance, wchar_t * lpCmdLine, int nCmdShow) Line 780 C++

I would submit a PR (proposed change at the bottom), but I suspect there is something more, as by the looks of it and the variable name, prevE should probably never be used during the first iteration.

vwMainTimeline.cpp:1138

- cmRecord::Evt prevE, e; prevE.nameIdx = PL_INVALID;
+ cmRecord::Evt prevE, e; prevE.nameIdx = PL_INVALID; prevE.flags = 0x0;

Possible race condition leading to access to NULL pointer

This might have been partly due to overflow, as I saw buffer saturation previously. Either case, the race condition seems to be there.

On commit: fd6a6a3
Compiler: gcc 9.3.0 with -g -O0 --std=c++20
You can hopefully reproduce it using the exact code, though I haven't exhaustively tested it (should work on x86_64, but who knows, it's way too hacky right now).
NetPunkSoftware/Tamashii@405d493

It happens about 3 out of every 5 executions.

Now, into what I believe is a race condition. See the following two stack traces:

Thread 1:

plPriv::hashString(const char * s, int maxCharQty) (/home/guillem/Github/Fibers/build/_deps/palanteer-src/c++/palanteer.h:1460)
plPriv::collectEvents(bool doForce) (/home/guillem/Github/Fibers/build/_deps/palanteer-src/c++/palanteer.h:3485)
plPriv::transmitToServer() (/home/guillem/Github/Fibers/build/_deps/palanteer-src/c++/palanteer.h:3874)
std::__invoke_impl<void, void (*)()>(void (*&&)(void) __f) (/usr/include/c++/11/bits/invoke.h:61)
std::__invoke<void (*)()>(void (*&&)(void) __fn) (/usr/include/c++/11/bits/invoke.h:96)
std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul>(std::thread::_Invoker<std::tuple<void (*)()> > * const this) (/usr/include/c++/11/bits/std_thread.h:253)
std::thread::_Invoker<std::tuple<void (*)()> >::operator()(std::thread::_Invoker<std::tuple<void (*)()> > * const this) (/usr/include/c++/11/bits/std_thread.h:260)
std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run(std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > > * const this) (/usr/include/c++/11/bits/std_thread.h:211)
libstdc++.so.6![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libpthread.so.0!start_thread(void * arg) (/home/build/YPKG/root/glibc/build/glibc.git/nptl/pthread_create.c:481)
libc.so.6!clone() (/home/build/YPKG/root/glibc/build-avx2/glibc.git/sysdeps/unix/sysv/linux/x86_64/clone.S:95)

Particularly plPriv::collectEvents at palanteer-src/c++/palanteer.h:3485. The current even being iterated, src, points to 0x7ffff70af990.

Other variables of possible interest (at collectEvents):

evtIdx = 12901
eventQty = 20036
globalCtx.prevBankAndIndex = 20036

Now, Thread 2:

plPriv::eventLogBase(uint32_t bi, plPriv::hashStr_t filenameHash_, plPriv::hashStr_t nameHash_, const char * filename_, const char * name_, int lineNbr_, int flags_) (/home/guillem/Github/Fibers/build/_deps/palanteer-src/c++/palanteer.h:1545)
plPriv::eventLogRaw(plPriv::hashStr_t filenameHash_, plPriv::hashStr_t nameHash_, const char * filename_, const char * name_, int lineNbr_, bool doSkipOverflowCheck_, int flags_, plPriv::bigRawData_t v) (/home/guillem/Github/Fibers/build/_deps/palanteer-src/c++/palanteer.h:1565)
plPriv::TimedScope::~TimedScope(plPriv::TimedScope * const this) (/home/guillem/Github/Fibers/build/_deps/palanteer-src/c++/palanteer.h:1825)
np::fiber_pool_base::worker_thread(np::fiber_pool_base * const this, uint8_t idx) (/home/guillem/Github/Fibers/src/pool/fiber_pool.cpp:171)
std::__invoke_impl<void, void (np::fiber_pool_base::*)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*, int>(std::__invoke_memfun_deref, void (np::fiber_pool_base::*&&)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*&&, int&&)(void (np::fiber_pool_base::*&&)(np::fiber_pool_base * const, unsigned char) __f, np::fiber_pool<np::detail::default_fiber_pool_traits> *&& __t) (/usr/include/c++/11/bits/invoke.h:74)
std::__invoke<void (np::fiber_pool_base::*)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*, int>(void (np::fiber_pool_base::*&&)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*&&, int&&)(void (np::fiber_pool_base::*&&)(np::fiber_pool_base * const, unsigned char) __fn) (/usr/include/c++/11/bits/invoke.h:96)
std::thread::_Invoker<std::tuple<void (np::fiber_pool_base::*)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*, int> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>)(std::thread::_Invoker<std::tuple<void (np::fiber_pool_base::*)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*, int> > * const this) (/usr/include/c++/11/bits/std_thread.h:253)
std::thread::_Invoker<std::tuple<void (np::fiber_pool_base::*)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*, int> >::operator()()(std::thread::_Invoker<std::tuple<void (np::fiber_pool_base::*)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*, int> > * const this) (/usr/include/c++/11/bits/std_thread.h:260)
std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (np::fiber_pool_base::*)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*, int> > >::_M_run()(std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (np::fiber_pool_base::*)(unsigned char) noexcept, np::fiber_pool<np::detail::default_fiber_pool_traits>*, int> > > * const this) (/usr/include/c++/11/bits/std_thread.h:211)
libstdc++.so.6![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libpthread.so.0!start_thread(void * arg) (/home/build/YPKG/root/glibc/build/glibc.git/nptl/pthread_create.c:481)
libc.so.6!clone() (/home/build/YPKG/root/glibc/build-avx2/glibc.git/sysdeps/unix/sysv/linux/x86_64/clone.S:95)

The last call, plPriv::eventLogBase, at palanteer-src/c++/palanteer.h:1545, is writing an event. Uppon inspection, e has the same address 0x7ffff70af990. It has crashed short of writing the string itself, which would have been the next line. At the moment of the crash, e.nameHash == 0 and e.name == 0.

Other variables of possible interest (at eventLogBase):

bi = 12901

If this is caused due to me going well beyond the acceptable buffer size, maybe we could introduce some assertion during debug, so that it triggers instead of causing UB.

Otherwise, I see two ways of going over this:

Adding a local ready atomic to each event, so that the consumer must spin/yield until it is flagged by the producer.
Using two pointers to the buffer, so that one points to the last valid position (ie. written and valid) and the another one points to the last writable position (ie. free and not being used).

dfeneyrou / palanteer Goto Github PK

palanteer's People

Contributors

Stargazers

Watchers

Forkers

palanteer's Issues

Recommend Projects

Recommend Topics

Recommend Org