Private package repositories & general package publishing workflow about hunter

[drywolf]

I have just realized I might have been using the "old" version of hunter-gate up until now with only the "manual" way of setting up and calling HunterGate(...).
Only now I have started using the modern version with all the new syntax for specifying package versions & automatic setup of hunter-gate etc.
This already made many things a lot easier and straightforward. Now im diving deeper into what can be done with the updated hunter-gate.

@maintainers: If possible it would be a good idea to also put a deprecation notice into the README.md at the hunter-packages/gate repository, just like there is for the ruslo/hunter repository to avoid such confusion.

[rbsheth]

@drywolf Thanks for your thoughts and welcome to the project! We are thinking of quite a few improvements to the process to remove administrative overhead and allow more flexible packages, especially around testing (See #9 #28 for example). Let us know if you need help while exploring the project, and let's discuss improvements here on Github and/or on Gitter (https://gitter.im/cpp-pm/community).

[drywolf]

Thanks for the welcoming words @rbsheth
I must say I'm already quite a big fan of hunter. For the most part it is very straightforward to read the sources and even customize them for your own needs, and that without having to learn any new syntax/language since it's all CMake.

But back to the original topic:
I would like to use hunter not only to consume publicly available packages, but also create my own private packages from my own source code and integrate them into my builds just as easily as hunter makes it for external code. My impression is that hunter was originally designed quite tightly linked to the GitHub ecosystem and primarily for open-source usage.
I think to make hunter interesting to a wider range of developers & applicable to different usecase scenarios, the project should strive to not rely only on one particular preconceived ecosystem to work properly. Some parts of hunter are already quite flexible in how and where you can use them, but I think this could be taken much further and I think it would make hunter show up on many more people's radars.

For example, just now I am experimenting with allowing to specify an external "package-index" instead of having the list of packages and versions baked into the package-manager itself (cmake/projects/* and cmake/configs/default.cmake).
This allows me to extend the existing "public" package list with definitions for my own privately hosted packages.

Do you (or anyone else) know if it was a conscious decision to always have the list of packages be tied in an unseparable and therefore non-mutable way with the package manager?
This would be one of the parts that we could rethink / redesign to improve the usability of hunter.

[rbsheth]

My impression is that hunter was originally designed quite tightly linked to the GitHub ecosystem and primarily for open-source usage.

I believe this is true for the original design. However, there have been additions to enable the use of private sources. See, for example:

https://cpp-pm-hunter.readthedocs.io/en/latest/user-guides/cmake-user/protected-sources.html
https://cpp-pm-hunter.readthedocs.io/en/latest/reference/user-modules/hunter_private_data_password.html
https://cpp-pm-hunter.readthedocs.io/en/latest/reference/user-modules/hunter_private_data.html
https://cpp-pm-hunter.readthedocs.io/en/latest/reference/user-modules/hunter_protected_sources.html
https://cpp-pm-hunter.readthedocs.io/en/latest/user-guides/cmake-user/private-data.html
https://cpp-pm-hunter.readthedocs.io/en/latest/faq/how-to-download-private-github-asset.html

For example, just now I am experimenting with allowing to specify an external "package-index" instead of having the list of packages and versions baked into the package-manager itself (cmake/projects/* and cmake/configs/default.cmake).
This allows me to extend the existing "public" package list with definitions for my own privately hosted packages.

You can already do this in a way. For example, I have a custom HunterConfig.cmake file that has some un-released/updated packages:

...
hunter_config(libxml2 VERSION 2.9.7-p0)
hunter_config(ZLIB VERSION 1.2.11-p1)
hunter_config(libigl VERSION 2.1.0-2ecb5fc
              URL "https://github.com/libigl/libigl/archive/2ecb5fc85fc124ba4dd839c6b43a836a0d2a017e.tar.gz"
              SHA1 82d03b660db3e414bf5333efa74fa3513398d42e
              CMAKE_ARGS
                LIBIGL_USE_STATIC_LIBRARY=OFF)
hunter_config(tinyobjloader VERSION 2.0.0-d8e95e4-t1
              URL "https://github.com/syoyo/tinyobjloader/archive/d8e95e4.zip"
              SHA1 72c409fba3ebd61bedc373ff54a0ed141aa6c754)
...

As you can see in this config, I can use an updated version of libigl with the existing project config plus some additional CMAKE_ARGS. I can also use a brand new package that has not made it into the package manager yet, tinyobjloader. I assume once the private passwords setup is done correctly, one could pull private packages not in the default config quite easily. If not, we can extend Hunter to encompass this functionality.

Do you (or anyone else) know if it was a conscious decision to always have the list of packages be tied in an unseparable and therefore non-mutable way with the package manager?
This would be one of the parts that we could rethink / redesign to improve the usability of hunter.

Maybe initially? It isn't something that is rigid and can surely be rethought.

Hope that helps.

[drywolf]

Thanks for sharing your insights @rbsheth
I did not know that hunter_config could not only be used to specify usage of an existing package, but also introduce completely new packages, this is really helpful for some quick local testing etc. 👍

I think what I really would want to achieve though is a centrally managed package-index (same structure as the existing one) ... so that packages can be managed in a central place and updates can automatically be shared among many projects / team-members. So it's basically the same as what hunter already does by default, but extended by an additional (potentially private) list of packages / versions.

PS: I already made a hacky proof-of-concept that allows me to use hunter in exactly such a way as I described, and it took not that much code (I really love how extendable hunter can be).
I guess the question that I would like to address and further discuss by raising this issue are the more long-term strategic design decissions for hunter (especially those in regard to the topic title).