Wrong cache directory about run-vcpkg

[lukka]

Please review the following and let me know if you are on the same page.

TL;DR
There is no reason to cache the binary cache directory content on the GitHub cache service, as it will just increase the restore from cache scenario. Continuing caching the $VCPKG_ROOT directory is the most performant solution.

Details
Please remember that run-vcpkg puts into the GitHub cache service basically all of $VCPKG_ROOT (excluding some directories to dramatically reduce the required space). So when the action installs sqlite3 package and the package has been restored from the GitHub cache, it takes <0.1 second.

OTOH, if run-vcpkg is changed to do as requested here, installing sqlite3 (i.e. vcpkg install sqlite3) with the $VCPKG_ROOT/installed directory empty, it would take >3 seconds. The times looks like spent mostly in detecting the "compiler hash for triplet..." (longer on Windows than on Linux, but in both cases it's >1 second), and expanding the archive and writing the new files onto $VCPKG_ROOT/installed directory.

[ras0219-msft]

In the cache hit scenario, that's the correct TL;DR -- as long as every package is in the installed\ folder, we won't consult the archives. The only way a package would be missing from the installed folder is if the cache didn't hit, in which case the archives would also be missing. There is potentially some improvement available via fallback keys on the archives directory, however I think this is relatively low value.

Instead, we recommend using GitHub Packages[1] to store the binaries via our NuGet backend. This will reuse packages as much as possible and complement the Cache Task to maximize performance and minimize rebuilds.

[1] https://github.com/microsoft/vcpkg/blob/master/docs/users/binarycaching.md#github-packages

[lukka]

@Milerius the run-vcpkg action has been designed for vcpkg "classic", where all the installed ports are installed in $VCPKG_ROOT/installed directory.

The binary caching is a preview feature this action is not currently supporting. Before supporting "modern/future" vcpkg, I'd need to understand in detail how this newly created features (e.g., binary caching and manifest) are working and how to support them.
I'd be glad if the exact and complete specs would be provided in here before I can do it.
If I do it first, I'll share them in here before rushing implementing those, as feedback is always useful.

[strega-nil]

@lukka binary caching is no longer unstable, and it's the recommended way to do caching nowadays; see the docs here. The readthedocs are usually out of date, since someone has to manually run the script to update them.

[lukka]

@strega-nil: good to know! I need time to write a design around it, i.e. how would the flowchart change.

TL;DR: I think it would be helpful to have in vcpkg a command that prints out the result of the vcpkg's cache directory computation logic (i.e. it prints the directory).

Details:
Is it needed to implement an high fidelity replica of the logic described here?
run-vcpkg needs to know whether binary caching is being used or not; and it needs to reliably know what is the directory where the cache needs to be restored into (before running vcpkg), and saved later from. I.e. whatever vcpkg thinks it is the right directory for caching, that is the one run-vcpkg must accurately know as well.

[strega-nil]

@ras0219 should be able to tell you more :)

[lukka]

based on the conclusions, no changes are needed to this action regarding the cached directories.

[Be-ing]

There is no reason to cache the binary cache directory content on the GitHub cache service, as it will just increase the restore from cache scenario. Continuing caching the $VCPKG_ROOT directory is the most performant solution.

Caching VCPKG_ROOT is hardly helpful, at least for manifest mode. All it helps with is avoiding needing to bootstrap the vcpkg executable. Caching VCPKG_ROOT/packages, VCPKG_ROOT/buildtrees, VCPKG_ROOT/downloads does not help; vcpkg still does a full rebuild every time (in manifest mode at least).

So when the action installs sqlite3 package and the package has been restored from the GitHub cache, it takes <0.1 second.
OTOH, if run-vcpkg is changed to do as requested here, installing sqlite3 (i.e. vcpkg install sqlite3) with the $VCPKG_ROOT/installed directory empty, it would take >3 seconds. The times looks like spent mostly in detecting the "compiler hash for triplet..." (longer on Windows than on Linux, but in both cases it's >1 second), and expanding the archive and writing the new files onto $VCPKG_ROOT/installed directory.

For these use cases caching is hardly relevant anyway. For complex applications with large dependencies like FFmpeg (30 minute build), wxWidgets (10 minute build), or Qt5 (>2 hour build!), the cost of computing the hash for vcpkg's package cache key is trivial compared to the build time. Running vcpkg install for this dependency list for Tenacity:

  "dependencies": [
    "libflac",
    "lilv",
    "libmad",
    "libogg",
    "libsndfile",
    "soxr",
    "libvorbis",
    "soundtouch",
    "libsbsms",
    "portaudio",
    "portmidi",
    "portsmf",
    "mp3lame",
    "libogg",
    "libvorbis",
    "zlib",
    "expat",
    "sqlite3",
    "ffmpeg",

    "libpng",
    "tiff",
    "zlib"
  ]

takes 9 seconds, which is very fast from my perspective.

There is potentially some improvement available via fallback keys on the archives directory, however I think this is relatively low value.

The benefits of fallback restore keys are huge if an application has even one dependency that takes a long time to build.

I have fixed these issues in #94.

[lukka]

@Be-ing Thanks for adding this comment! I'd like to put your comments on the right context first, as I think we are almost on the same page, only some clarifications are still needed.

Caching VCPKG_ROOT is hardly helpful, at least for manifest mode. All it helps with is avoiding needing to bootstrap the vcpkg executable. Caching VCPKG_ROOT/packages, VCPKG_ROOT/buildtrees, VCPKG_ROOT/downloads does not help; vcpkg still does a full rebuild every time (in manifest mode at least).

Caching $VCPKG_ROOT means to save its content except for what is not useful. That is, all but packages, buildtrees and downloads. And please note that when the vcpkg.json manifest mode is used (which is the mode the action and the samples are tailored with) a full rebuild does not take place.

I'd be curious to see what is the run-vcpkg run time in cache-hit scenario on the tenacity project (which is the "restore VCPKG_ROOT scenario except what is not needed"), compared to the time spent to leverage vcpkg's binary caching created archives (and restoring those from cache).

Also, I really appreciated your contribution on #94, which is something I was preparing already but I did not publish yet. So you beat me to it!

[Be-ing]

I'd be curious to see what is the run-vcpkg run time in cache-hit scenario on the tenacity project (which is the "restore VCPKG_ROOT scenario except what is not needed"), compared to the time spent to leverage vcpkg's binary caching created archives (and restoring those from cache).

Without #94 every package is fully rebuilt for any cache miss. The action is hardly useful for my purposes like that.

[Be-ing]

I just realized I got confused by the incorrect information in the README:

          # Ensure the vcpkg artifacts are cached, they are generated in the 'CMAKE_BINARY_DIR/vcpkg_installed' directory.
          additionalCachedPaths: ${{ env.buildDir }}/vcpkg_installed

Manifest mode installs to vcpkg_installed in the root of the repository being built (the same directory as the vcpkg.json file). This directory is outside the CMake build directory.

[Be-ing]

Caching the vcpkg_installed directory in manifest mode is indeed slightly faster that caching the user local binary cache.