kevincobain2000 / action-cache-http

Action for caching dependencies on Github Enterprise via HTTP. Useful for self hosted runners

Hi there 👋

Portfolio

• I am the developer of
  • laravel-request-docs
  • email_extractor
  • Featured on Product Hunt - action-coveritup
  • instachart
  • gobrew
  • laravel-erd
  • sentiment_classifier
  • jProcessing
  • laravel-alert-notifications
• I write tech blogs on medium
• Google Scholar

action-cache-http's Issues

wish: support input directory as an alternative to lockfile+install command

A common pattern for web apps is to have a folder of frontend end assets when then get processed. For example, compiling TypeScript files to JavaScript.

It would be nice to cache these built assets as well, even though they don't have a lock file.

Instead of a lockfile, we can calculate a SHAsum of the input directory. This StackOverflow post provides an example syntax:

https://superuser.com/questions/458326/sha1sum-for-a-directory-of-directories

For example. I have a node_modules directory that is also 1 GB in size, but I'm able to find the SHAsum of it about a second:

 find node_modules/ -type f -print0 | xargs -0 sha1sum | awk '{print $1}' | sha1sum | awk '{print $1}'

sha1sum and sha256sum are part of the coreutils module, so it's almost certainly installed if the other dependencies are met.

Monorepo [Turborepo]

Can you provide an example of how to cache subfolder node_modules in Monorepo ?
When we have:

Root
package.json
package-lock.json // (1)
node_modules

  | -- Package1
        package.json
        node_modules // <- how i can cache (this node_modules depends of package-lock.json(1))

  | -- Package2
        package.json
        node_modules // <- how i can cache  (this node_modules depends of package-lock.json(1))

  | -- Other Packages

wish: expose option to disable compression, speed up 3x to 10x

If you only keeping a small number of cached artifacts , the storage space is not a concern, and if you are serving the artifacts to the same host, the transfer speed is not much of a factor either.

But both tar and untar operations are faster with out compression. In my tests with a node_modules dir of 890 MB, I found disabling compression had the following effect:

• Compressing was about 10x faster
• Decompressing was about 3x faster

Since the whole point of a cache is to increase performance, let's make it even faster!

Personally, I would disable compression by default, since anyone using the module has both an interest in increasing some performance as well as some extra storage space for caching.

Dependencies on the host operating system: curl not found

This Github Action is implemented a "JavaScript" action but in fact passes through the work to a shell script. The shell script in turn requires a number of commands to exist on the host besides sh, but the code doesn't check to see if any the commands exist, which can result in less helpful failure modes, like this:

Error: Command failed: /__w/_actions/kevincobain2000/action-cache-http/v2/cache-http.sh
/__w/_actions/kevincobain2000/action-cache-http/v2/cache-http.sh: 33: [: =: unexpected operator
/__w/_actions/kevincobain2000/action-cache-http/v2/cache-http.sh: 33: [: =: unexpected operator

I think that failed because the curl command is required but not found on the host. Another dependency is the openssl command.

Requiring extra dependencies on the host could be avoided if this were implemented as a container action instead of JavaScript action that calls a shell script.

Otherwise, it would be helpful there were at least explicit checks for curl and openssl to generate better failure messages along with docs that they are required.

wish: replace SHA1 with SHA256

SHA1 used here is known to be vulnerable to collision attacks.

While it's not clear if there's a practical way to create an exploit due to this projects use of SHA1, it's generally recommended to be sunsetted in favor of SHA265 or new ( https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html ).

The good news is that openssl sha1 can simply be swapped for opensl sha256. Also, coreutils provides the sha1sum binary as well as sha256sum. So there should be no additional dependencies.

In a simple test, there was a not a significant performance difference using SHA256.

Still, the change might quality as a major version bump because it will cause cache misses during the first run due to changing cache keys.

Issues with nested directories

Hello, trying to get this working with a project that has multip sub-directories with lock files etc. Running into issues finding the lock files with the default setup, but also when using any combination of relative/absolute pathing.

General setup:

      - name: Yarn Install (with cache)
        uses: kevincobain2000/action-cache-http@v2
        with:
          version: 14
          lock_file: yarn.lock
          install_command: cd infra && yarn install
          destination_folder: node_modules
          cache_http_api: "http://172.17.0.1:3000/cache-http"
          http_proxy: ""

Getting the following error:

##[group]Run kevincobain2000/action-cache-http@v2
with:
  version: 14
  lock_file: yarn.lock
  install_command: yarn install
  destination_folder: node_modules
  cache_http_api: http://172.17.0.1:3000/cache-http
env:
  NODE_ENV: test
##[endgroup]
##[command]/usr/bin/docker exec  f502a4b5c0371f46515212c324d00451d643784eaabbddec19b547cd439ac08c sh -c "cat /etc/*release | grep ^ID"
Error: Command failed: /__w/_actions/kevincobain2000/action-cache-http/v2/cache-http.sh
yarn.lock: No such file or directory
tar: node_modules: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
    at ChildProcess.exithandler (child_process.js:295:12)
    at ChildProcess.emit (events.js:210:5)
    at maybeClose (internal/child_process.js:1021:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5) {
 1: 0x9da7c0 node::Abort() [/__e/node12/bin/node]
  killed: false,
  code: 2,
  signal: null,
  cmd: '/__w/_actions/kevincobain2000/action-cache-http/v2/cache-http.sh'
}
 2: 0xa4e219  [/__e/node12/bin/node]
 3: 0xba5d59  [/__e/node12/bin/node]
 4: 0xba7b47 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/__e/node12/bin/node]
 5: 0x13750d9  [/__e/node12/bin/node]

Seems the server is working as it should, and as mentioned I've tried every combination of lock_file imaginable: subdir/yarn.lock - yarn.lock - /full/path/to/subdir/yarn.lock - $(cwd)/yarn.lock - nothing works.

I also tried using the README from the root of the project to see if that would work and while it did find the README, it yarn install'd in the root (I believe) which failed because that's not actually the right place. So the issue does seem to be that subdirectories aren't where the action is running from, despite that being the working-directory for the job.

Any suggestions welcome, this would be very helpful for us. Thanks!

node16 actions are deprecated

This needs to be moved to a newer engine version.

README, releases out of date

The README explicitly says to use @v2, and the Releases area on the side also confirms that v2 is the latest release, but from looking at "Tags", it appears that @v2.3 is.

Attempting to follow the syntax in the README can result in unclear errors like:

/__w/_actions/kevincobain2000/action-cache-http/v2/cache-http.sh: 33: [: =: unexpected operator
 1: 0x9da7c0 node::Abort() [/__e/node12/bin/node]
 2: 0xa4e219  [/__e/node12/bin/node]
/__w/_actions/kevincobain2000/action-cache-http/v2/cache-http.sh: 33: [: =: unexpected operator

My clue something was off was the line numbers in the error didn't match what was on Github. It would be helpful if new releases were tagged as Github release and the README was updated as well.

Thanks!