[BUG] ImportError of GLIBCXX in Catalyst frontend about catalyst

[erick-xanadu]

Thanks for reporting this error @glassnotes! It looks to be related to the new compiler driver integration.

[glassnotes]

Thanks for the quick reply, @erick-xanadu !

Some more info for you: upon further testing, in a script, if I import packages in the following order,

import catalyst
import jax.numpy as jnp

I do not get an error, however if I reverse the two lines, I get the error listed above. I've just updated both jax and jaxlib to be versions 0.4.14.

[maliasadi]

Hi @glassnotes, Thanks for reporting this issue.

This comes from the fact that you have multiple versions of libstdc++ on your machine. The compiler_driver library of Catalyst was apparently built with a different version of this library. You can find it by running ldd /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so. The path to libstdc++.so should be something like /lib/x86_64-linux-gnu/libstdc++.so.6.

The conda version of libstdc++ doesn't include GLIBCXX_3.4.30 while the gcc package (in /lib/libstdc++.so.6) has it! The compiler_driver is linked against the gcc package but it doesn't find the lib when it's called from a conda venv.

To fix this issue, you can replace the conda version of this library by this one,

cp /lib/x86_64-linux-gnu/libstdc++.so.6 /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6

Please let us know if this also fixes the issue or you face any other problems with installing Catalyst.

[dime10]

The question is, how do we prevent this from happening more generally? 🤔

[glassnotes]

Copying the library in does work, but feels a little sketchy 😅

I ended up with multiple versions of the library during the installation process. In particular, despite having clang installed on my machine, I initially received the output

CMake Error at /home/olivia/.conda/envs/catalyst/lib/python3.11/site-packages/cmake/data/share/cmake-3.27/Modules/CMakeTestCXXCompiler.cmake:60 (message):
  The C++ compiler

    "/usr/bin/clang++"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: '/home/olivia/Code/catalyst/runtime/build/CMakeFiles/CMakeScratch/TryCompile-k56B71'
    
    Run Build Command(s): /usr/bin/ninja -v cmTC_54cbb
    [1/2] /usr/bin/clang++    -MD -MT CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -MF CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o.d -o CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -c /home/olivia/Code/catalyst/runtime/build/CMakeFiles/CMakeScratch/TryCompile-k56B71/testCXXCompiler.cxx
    [2/2] : && /usr/bin/clang++   CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -o cmTC_54cbb   && :
    FAILED: cmTC_54cbb 
    : && /usr/bin/clang++   CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -o cmTC_54cbb   && :
    /usr/bin/ld: cannot find -lstdc++: No such file or directory
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    ninja: build stopped: subcommand failed.
    
  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:3 (project)

when trying to run make all in the top-level Catalyst directory. This led me to installing libstdc++-12-dev, which fixed the previous problem, but resulted in the new one in the issue description.

For what it's worth, the "not being able to compile a simple test program" happened on more than one of my machines, and I had installed previous versions by manually adjusting the Makefiles to use g++ instead of clang; but today I thought I'd try and fix things to work out-of-the-box...

[maliasadi]

Thanks for checking! I can feel the pain 😢 we're discussing the issue and should be able to come up with a better solution.

Such feedback helps us gain a deeper understanding of edge cases, and we're committed to working on robust solutions that make the process of installing Catalyst from source even smoother.

[dime10]

Still investigating this, but we might need to recommend users not use conda when building from source.