conflicts among packages when installing a conda environment about ffcv

[GuillaumeLeclerc]

Conda can be really annoying. To be honest we don't really need anything specific from there. The only reason we used it was to avoid having to specify installation that vary from linux distribution to linux distribution but it might be doing more harm than good...

[andrewilyas]

Hi @mbsariyildiz ! Do you mind posting the version of CUDA and the NVIDIA Drivers that you have installed? (e.g., the output of nvidia-smi)?

[mbsariyildiz]

Thanks a lot for your quick response.
I use NVIDIA driver version: 470.57.02 and CUDA Version: 11.4
Just to note, I am able to install PyTorch alone with cudatoolkit=11.3.

[GuillaumeLeclerc]

Any idea which package is causing the conflict? I would try without cupy and maybe without numba first and if it works we see from there.

[mbsariyildiz]

Here is what I tried:

• By removing the python=3.9 constraint, I managed to create an environment peacefully. But then I got opencv-related issues (to be more specific, about ffmpeg decoders). ffmpeg was installed from the pytorch channel, and forcing conda to replace it from the version available in the default conda channel, or the conda-forge channel again resulted in a bunch of conflicts.
• Tried to create a conda environment from a fresh Anaconda installation, but again ended up with the same conflicts.
• Tried to use different GCC versions (some of the issues were GCC-related), again, no success.

After 2 hours of struggle, I gave up.
Interestingly, one of my colleagues managed to install ffcv on the same machine successfully. I simply cloned his conda environment, and installed ffcv on top, which worked.
Couple of things I noticed though, the version of major libraries, like PyTorch, CuPy, Numba was different, and ffmpeg was from the conda-forge channel.

[andrewilyas]

Thanks for letting us know @mbsariyildiz ! What version of conda did they have?

[mbsariyildiz]

Sorry for not being clear @andrewilyas , he had the same conda version 4.10.3.
I find that a bit odd, as we were on the same machine, using the same CUDA & NVIDIA drivers and LD_LIBRARY_PATH PATH variables.

Just to add, upgrading my conda to 4.11.0, or installing the latest ffmpeg via pip also didn't help. I guess I couldn't link the correct ffmpeg lib.

Thanks for your attention @andrewilyas and @GuillaumeLeclerc . I'll use this conda env that I cloned from my colleague for now. Please let me know if I can provide more information for you to diagnose. Otherwise, please feel free to close this issue.

[PeriklisTheodoropoulos]

HI, could you please also mention the steps your colleague took to resolve the ffmpeg conflict? I have been trying to install opencv with python 3.9 and conda version 4.10.3 (I am working with a remote server and so can not downgrade conda) and am running into an incomplete installation (the process terminates midway when ffmpeg is being installed).

[andrewilyas]

Based on the stack trace @mbsariyildiz the problem might be with conda's strict channel priority mode. Try something like this in the created env?

conda config --env --set channel_priority flexible

(Edited!)

[mbsariyildiz]

@andrewilyas updating the channel priority of conda as you said solved the issue! 🎆 Wow, great, thank you very much.

@PeriklisTheodoropoulos, he didn't have any issue with ffmpeg actually. He was able to create a working environment.

[andrewilyas]

Great! Closing this issue and added a note to the README regarding the fix.