Comments (2)
If I understand the question, this is impossible to implement with the current architecture.
Currently, cleanup
deletes all the results whose hashes do not match a current task. If you would like to be more selective, however, you'd need to figure out where the hash is coming from (which is, to a first approximation, impossible) or have more information in the results file (which would add overhead).
from jug.
In #59 at https://github.com/luispedro/jug/pull/59/files#diff-b317a82b72a30159f25c028ad6194678R45 I extend cleanup to be able to remove only failed tasks.
Then on another custom-made user-command I use:
def run(self, store, options, *args, **kwargs):
print("Failed tasks:")
for t in task.alltasks:
if not t.can_load() and t.is_failed():
try:
with open(t.store.getlock(t.hash()).fullname) as fh:
data = [line.rstrip() for line in fh]
except AttributeError:
# Not supported by the target backend
data = ""
print(" ", t.name, t.args, t.kwargs, " - ".join(data))
To list failed tasks.
This could be easily extended to remove that lock instead of reading from it's stored information.
Matching a target would be t.name == options.target
.
Would this strategy be viable?
from jug.
Related Issues (20)
- Invalidate inside shell doesn't invalidate dependent tasks HOT 2
- Invalidate --target is too greedy HOT 4
- Ability to pass argument(s) that don't create new tasks HOT 2
- Always trigger `hooks.exit_checks.exit_env_vars`
- Processes only start working after checking status HOT 7
- Failed to assign tasks correctly HOT 6
- The right way to use Jug with class object HOT 5
- Jug sleep-until exits too early when using barrier
- Error while running jug execute prime.py HOT 1
- NoLoad does not track dependencies
- Calling execute from code HOT 2
- Memory "leak" HOT 2
- Tasklet usage/Tasklet.can_load() HOT 3
- jug shell's invalidate() fails if NoLoad task in DAG
- `pdb` option doesn't seem to work HOT 1
- Jug throwing error, while running on PBS based HPC cluster HOT 3
- add options to webstatus subcommand (port, ipaddress) HOT 1
- IterateTask as a decorator HOT 5
- Allow cleanup of only locks HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jug.