Namespace cache volumes by module about dagger

[shykes]

Sure, since there's a cache volume type, it makes sense that you can pass it as argument. I've never seen that done so hadn't even thought about it, but it seems reasonable to me that we don't break it. As long as it doesn't break namespacing of modules (which I don't think it would), then I don't see any problem with that.

So these things should all be true:

1. When a cache volume is loaded by key (cacheVolume(key: String!)), the key is always namespaced by module address. There is no way to bypass this.
2. A cache volume ID can be used by any module. But a module cannot guess the ID of another module's cache volume: it needs to be passed explicitly.
3. Ideally, cache volume IDs do NOT become sensitive values to be scrubbed from screenshots and logs (because ie. they are not reusable across sessions)

[sagikazarmark]

What does module mean in this context? Each instance/call of a module gets separate cache volumes?

What if I have multiple calls to a Go module within the same project and I want to share the module/build cache between them?

[shykes]

What does module mean in this context? Each instance/call of a module gets separate cache volumes?

What if I have multiple calls to a Go module within the same project and I want to share the module/build cache between them?

I mean the full canonical address of the module, for example github.com/shykes/daggerverse/hello.

All instances of the same module would share the same volume, as long as they share the same persisted cache volume storage.

[sipsma]

I agree we need to do something like this, but do want to note that the fact that cache volumes can be shared across modules is highly beneficial to performance for many common use cases. E.g. anything that uses Go benefits from sharing a cache volume for downloading deps (and possibly build cache, etc.).

Obviously in the choice between security-by-default and performance, security-by-default should win.

But in past discussions around all this the idea of cache volumes being tied to modules but still allowing modules to pass their own cache volumes around came up and is still worth considering imo. So say you are writing a module that calls to a bunch of other modules that do "go things"; you should be able to define a cache volume and pass those cache volumes to be used by modules you call.

• This of course requires that the modules you are calling accept an optional cache volume to use as an override for their default (private) one, which probably just needs to become a best practice in this scenario.

That seems like one reasonable way of maintaining security by default while still allowing opt-in performance benefits. I'm sure there's other approaches possible too.

[sipsma]

Yeah SGTM, coincidentally everything required to implement enforcement of only using cache volumes you create or are explicitly passed is also what's required to safely pass sockets around (#6747), which I'm working on right now, so should be feasible to implement all this in the very near future.