Comments (4)
So the value to be compared against the limit is "the sum of all file sizes for the tree of inputs to an action"?
from bazel-buildfarm.
I was thinking that we would limit just on the digest's size with values read from configuration.
--- a/src/main/java/build/buildfarm/cas/cfc/CASFileCache.java
+++ b/src/main/java/build/buildfarm/cas/cfc/CASFileCache.java
@@ -790,7 +790,11 @@ public abstract class CASFileCache implements ContentAddressableStorage {
if (digest.getSizeBytes() == 0) {
return new CompleteWrite(0);
}
- if (digest.getSizeBytes() > maxEntrySizeInBytes) {
+ Set<String> allowedActions = Sets.newConcurrentHashSet();
+ allowedActions.add("JoinLayers");
+ allowedActions.add("PackageTar");
+
+ if (digest.getSizeBytes() > maxEntrySizeInBytes && !allowedActions.contains(requestMetadata.getActionMnemonic())) {
throw new EntryLimitException(digest.getSizeBytes(), maxEntrySizeInBytes);
}
try {
from bazel-buildfarm.
"the digest" as a subject for comparison does not narrow the scope of the filter - we already limit by size generally, and ensure that no blobs can exist in a worker's cas above that limit.
Your suggested change limits by hard coded action mnemonics. What does that have anything to do with 'size'?
from bazel-buildfarm.
Your suggested change limits by hard coded action mnemonics. What does that have anything to do with 'size'?
Good question. From my prototype snippet above, I'm proposing allowing some actions to go above the hardcoded size based on configured list of mnemonics.
from bazel-buildfarm.
Related Issues (20)
- Support Multiple Hashing Function
- Remote execution service executing processes locally HOT 4
- ERROR: Failed to query remote execution capabilities: UNAVAILABLE: io exception HOT 4
- Feasibility Analysis of Using Buildfarm for Large-Scale Development HOT 2
- Verbose Logging for Servers and Workers through Helm Chart
- Check logs during remote execution HOT 4
- Incorrect container port for Shard-worker in Helm template
- hardlinks in CAS leads to task failure in some cases HOT 2
- Fetch asset support for credential use
- [Bzlmod] No repository visible as '@maven' from main repository HOT 1
- Querying remote cache failed due to Missing Digest HOT 5
- Helm chart won't deploy workers because {ready,live}ness probes are using the wrong port
- When is the release? HOT 1
- Helm chart deployment storage issues
- FindMissingBlobs histogram has too few buckets
- CGroups v2 HOT 1
- RedisShardSubscription::stop is unsafe
- 无任务时redis CPU占用过高 HOT 2
- Transfer bazel-buildfarm to bazel-contrib HOT 5
- When configuring Bazel Buildfarm, setting actionCacheExpire to be less than casExpire is more reasonable ? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bazel-buildfarm.