Comments (1)
wdbaruni
commented on June 20, 2024
1
Nice work documenting the issue and the current flow. Few comments about the heartbeats flow:
- A network can have thousands of compute nodes managed by a single or few requester nodes. We shouldn't have the requester node heartbeat to all compute nodes about its active log streams
- An alternative would be to only have the compute nodes heartbeat to the requester using a request/response call with a timeout set to single digit seconds and shorter than the heartbeat window. The heartbeat request will contain the IDs for the logstreams the compute node is serving, and the response from the requester should contain the logstream IDs the requester is no longer interested in.
- Keep in mind that we already have a heartbeat mechanism to track the liveliness of the compute nodes. This is a different heartbeat and should only kick in if the compute node have open logstreams.
from bacalhau.
Related Issues (20)
- Simplify node bootstrapping (fx)
- Move `bacalhau create` to `bacalhau job run`
- Move `bacalhau cancel` to `bacalhau job stop`
- Move `bacalhau list` to `bacalhau job list`
- Move `bacalhau id` to `bacalhau agent node`
- Move `bacalhau describe` to `bacalhau job describe`
- Move `bacalhau validate` to `bacalhau job validate`
- Move `bacalhau logs` to `bacalhau job logs`
- KVMigration Tests Flaky HOT 1
- Implement auto download flags and function on wasm and docker commands HOT 3
- Disable WebUI in production environment
- Add a --queue-timeout flag to docker run
- Optimise locking in producer and consumer based NATS streaming clients
- Re-work documentation around CLI to align with V2 CLI commands
- JACoB Installed
- Update docs after IPFS deprecation
- Update docs after removing libp2p
- Document different timeout types
- Document job queue
- installationID is not persisted
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bacalhau.