Comments (2)
AhmedSoliman
commented on July 18, 2024
1
LogDevice protocol was designed with streaming in mind. That was not something available in thrift/fbthrift when we originally looked into it. The protocol allows the client and servers to exchange a reading sliding window changes to efficiently move the reading cursors remotely. This also allows for back pressure control if the client local buffers are full, server will pause sending records accordingly.
While handcrafted protocol codecs are not the most pleasing to implement, they offer us a great deal of control over memory and CPU efficiency. Both are very important for LogDevice scalability .
from logdevice.
theidexisted
commented on July 18, 2024
Great answer, thanks!
from logdevice.
Related Issues (20)
- Inconsistent column type for node_id in Sequencer and Nodes table HOT 3
- Settings abort-on-failed-check, abort-on-failed-catch and external-loglevel don't correctly subscribe to settings update HOT 1
- Tests failed in master due to some folly singletons being used without calling folly init
- Make DSCP reflection configurable in server settings HOT 1
- ModuleNotFoundError: No Module named 'libfb' HOT 1
- LogDevice build failing on Ubuntu bionic HOT 4
- logdevice/build_tools/fedora.deps file doesn't exist HOT 2
- With latest code, linking integration_test fail with SIGSEGV
- The log range specified must be positive 62-bit with start smaller or equal than end of range, supplied range is 0..200000 HOT 3
- Can not build on Ubuntu 19.10 HOT 3
- Will sequencer become a bottleneck for large size record with high throughput HOT 4
- Replication policy cross multiple cities with one special deferred? HOT 4
- Can not build on Ubuntu 18.04.4 HOT 2
- Conditional add question HOT 2
- Run quickstart through docker image failed HOT 1
- I want to integrate Kubernetes with VirtualLog. HOT 1
- Docker build fbthrift compilation thrift/lib/py3/test/gen-py3/testing/clients.cpp failed
- Is this project no longer maintained? HOT 12
- [email protected]
- LogDevice cluster start error --- NOSEQUENCER and sequencer:[invalid NodeID]
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from logdevice.