Multiple versions of the same package in a cmake directory tree about hunter

[rbsheth]

@drywolf That is correct in your current setup. You'd have to initialize Hunter twice, once for app-a and once for app-b. This is because the ZLIB files installed in the Hunter directory are from the first version seen. Something like

- proj-root-dir/
  - CMakeLists.txt      
      project(my-project)
      add_subdir(app-a)
      add_subdir(app-b)
  - app-a/
    - app-a.cpp
    - CMakeLists.txt
        # initialize hunter-gate etc. here
        # TODO: how get ZLIB 1.2.11-p1 via hunter
        ???

        find_package(ZLIB 1.2.11-p1 EXACT CONFIG REQUIRED)

        add_executable(app-a app-a.cpp)
        target_link_libraries(app-a PRIVATE ZLIB::zlib)
  - app-b/
    - app-b.cpp
    - CMakeLists.txt
        # initialize hunter-gate etc. here
        # TODO: how get ZLIB 1.2.8-hunter via hunter
        ???

        find_package(ZLIB 1.2.8-hunter EXACT CONFIG REQUIRED)

        add_executable(app-b app-b.cpp)
        target_link_libraries(app-b PRIVATE ZLIB::zlib)

[NeroBurner]

on how to get a specific version from hunter see hunter_config
https://docs.hunter.sh/en/latest/reference/user-modules/hunter_config.html

and also the documentation on incjection https://docs.hunter.sh/en/latest/user-guides/hunter-user/git-submodule.html#injection

essentially create a file app-a/cmake/Hunter/config.cmake and app-b/cmake/Hunter/config.cmake with the hunter-config() command. For example in app-a

hunter_config(ZLIB VERSION 1.2.11-p1)

then you have to add the LOCAL keyword to your HunterGate() calls in the app-x/CMakeLists.txt files

HunterGate(
    URL "url-to-hunter-release.tar.gz"
    SHA1 "someSHA1forRelease"
    LOCAL # per default cmake/Hunter/config.cmake
)

[drywolf]

@rbsheth That's what I also tried, but then hunter tells me:
Please set HunterGate *before* 'project' command.

My full test example looks like:

# root CMakeLists.txt
cmake_minimum_required(VERSION 3.2)

project(consume-multiple-pkg-versions)

add_subdirectory(app-a)
add_subdirectory(app-b)

# ./app-a/CMakeLists.txt
set(HUNTER_ZLIB_VERSION 1.2.11-p1)
set(HUNTER_PACKAGES ZLIB)

include(FetchContent)
FetchContent_Declare(SetupHunter GIT_REPOSITORY https://github.com/cpp-pm/gate)
FetchContent_MakeAvailable(SetupHunter)

find_package(ZLIB 1.2.11-p1 EXACT CONFIG REQUIRED)

add_executable(app-a app-a.cpp)
target_link_libraries(app-a PRIVATE ZLIB::zlib)

# ./app-b/CMakeLists.txt
set(HUNTER_ZLIB_VERSION 1.2.8-hunter)
set(HUNTER_PACKAGES ZLIB)

include(FetchContent)
FetchContent_Declare(SetupHunter GIT_REPOSITORY https://github.com/cpp-pm/gate)
FetchContent_MakeAvailable(SetupHunter)

find_package(ZLIB 1.2.8-hunter EXACT CONFIG REQUIRED)

add_executable(app-b app-b.cpp)
target_link_libraries(app-b PRIVATE ZLIB::zlib)

[NeroBurner]

remove the project(consume-multiple-pkg-versions) line from top level CMakeLists.txt

[drywolf]

@NeroBurner That's the "old-school" syntax that I was using before already.
I am now trying out the syntax that is shown at https://github.com/cpp-pm/gate/tree/master#usage
I will try the above example out with the other syntax, although I think the same error as above will also happen there.

EDIT: I just tried, and it gives the same error.

remove the project(consume-multiple-pkg-versions) line from top level CMakeLists.txt

Pretty much leads to the same error:

[hunter ** FATAL ERROR **] Please set HunterGate *before* 'project' command. 
[hunter ** FATAL ERROR **] Detected project: Project

[drywolf]

I was able to work around the error check in hunter-gate by removing it from the HunterGate.cmake and then using the HunterGate() + Hunter/config.cmake approach.

I also had to remove the version specifiers from the calls to find_package() since cmake enforces that those not be free strings, but must only be in the major.minor.patch format.

Then the cmake configure goes through, but both apps end up using the same zlib version anyway:

Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.18362.
[hunter] Calculating Toolchain-SHA1
[hunter] Calculating Config-SHA1
[hunter] HUNTER_ROOT: C:/.hunter
[hunter] [ Hunter-ID: 27387cd | Toolchain-ID: 10d3454 | Config-ID: d23989a ]
[hunter] ZLIB_ROOT: C:/.hunter/_Base/27387cd/10d3454/d23989a/Install (ver.: 1.2.11-p1)
[hunter] HUNTER_ROOT: C:/.hunter
[hunter] [ Hunter-ID: 27387cd | Toolchain-ID: 10d3454 | Config-ID: d23989a ]
[hunter] ZLIB_ROOT: C:/.hunter/_Base/27387cd/10d3454/d23989a/Install (ver.: 1.2.11-p1)

Configuring done
Generating done

If I then add the version requirements back to find_package() then the second project complains about not finding the right version for zlib:

find_package(ZLIB 1.2.11 EXACT CONFIG REQUIRED)
find_package(ZLIB 1.2.8 EXACT CONFIG REQUIRED)

CMake Error at app-b/CMakeLists.txt:18 (find_package):
  Could not find a configuration file for package "ZLIB" that exactly matches
  requested version "1.2.8".

  The following configuration files were considered but not accepted:

    C:/.hunter/_Base/27387cd/10d3454/d23989a/Install/lib/cmake/ZLIB/ZLIBConfig.cmake, version: 1.2.11

[drywolf]

What I find strange is that the Config-IDs of both hunter instances are the same, I would have expected them to be different and maybe that would make all of this work. I'm investigating further.

[drywolf]

As far as I can tell it will not really work to initialize multiple HunterGates() within a CMake project ? Am I right ?
Because even if it would work from the Hunter side of things, then there would still always be a race-condition/conflict between all HunterGates trying to set the same variables for a package, e.g. ZLIB_DIR, ZLIB_ROOT, ZLIB_FOUND, etc.

To avoid this collision hunter/cmake would have to include the version number or some hash within the variable-name to uniquely identify for which package this variable stands:

ZLIB_DIR_1_2_11
ZLIB_ROOT_1_2_11
ZLIB_FOUND_1_2_11

ZLIB_DIR_1_2_8
ZLIB_ROOT_1_2_8
ZLIB_FOUND_1_2_8

and then the CMake find_package() scripts would need to be able to recognize that if find_package() is called with a version of e.g. "1.2.8" then they should check if ZLIB_ROOT_1_2_8 is set instead of the plain ZLIB_ROOT.

Am I missing something here ? Right now using different package versions within a single project() tree really seems pretty much impossible to achieve to me ?!

Thanks

[bkotzz]

Is there anything stopping you from making subdir A and B there own projects? Do they share code (ie. do they both link against the same target or something)?

[drywolf]

Right now I am just investigating what C++ package managers can and can't do, so those examples are purely hypothetical at this point.
But for the above I could say that there would be some host application project which has two shared-lib projects that it depends on, and each of the shared libs wants to link against a different version of some 3rd party dependency. But the two shared libs also want to use some common code which might be in a 4th target which would be a static-lib within the project, therefore it would make good sense to have everything together in a single big project.
This is something which I can see naturally happening and I have had some pretty similar requirements in the past already.

                     HostApp
                        +
                        |
             +----------+---------+
             |                    |
             v                    v
      SharedLib_v2019.1    SharedLib_v2019.6
       +     +                    +      +
       |     |                    |      |
       |     +---------+ +--------+      |
       |               | |               |
       v               v v               v
ExtPkg_v1.0.0    CommonStaticLib      ExtPkg_v2.0.0

(all parts are in-house code while only the ExtPkg are external dependencies)

In the meantime I have also looked at conan and also vcpkg and for the same reasons (mentioned above) they also pretty much have no out-of-the box solution for this problem. I guess the problem was already introduced by CMake resolving external dependencies into a globally scoped cache, without any connection to who requested the dependency in the first place, or which version was requested and/or found.

EDIT PS:
what I find more frustrating about this architecture, is that it does not allow the CMake targets to define their dependency requirements themselves, but always lifts the burden up to the "master-mind" project that has to keep track of what everyone uses and make all dependencies work.
This is usually not an approach that can scale, and that's what troubles me.

[rbsheth]

The project command issue stems from the fact that CMake will add a project command if one doesn't exist already before running the commands in a subdirectory.

In this case, you could achieve what you want by treating lib-a and lib-b as ExternalProjects, with their own Hunter configs and initialization. The binaries should not be shared in that case between the Hunter instances. At the app level, you can link against the targets from the external projects.

Usually when two libraries are sharing the same code, sharing the same library dependency and version is good practice. That's my guess as to why most package managers take this approach.

[drywolf]

Thanks @rbsheth for the insights.

sharing the same library dependency and version is good practice.

Yes that's my biggest takeaway after all the digging that I've done 😁
I'm closing this issue, since there is no further action to be taken at this point.

Thanks everyone for sharing your thoughts.