Comments (6)
I synced with Shahar and he brought up that there should be an api for continues compression. I worked on this CompressorImpl class in the past, and I think we dont need to break changes. If we change the CompressorImpl class to create context and free the context once we finish full entry compression I believe this should be good. This is like breaking the compression to many frames that will be sent one after the other and with last frame once we finished serializing the entry
from dragonfly.
@kostasrim I suggest using RDB_OPCODE_DF_MASK for that. I't a meta opcode that can say in advance what to expect in keyvalue that is gonna be parsed next. Right now we only have DF_MASK_FLAG_STICKY
bit, so you can extend with compression bits like DF_MASK_COMPRESSED_ZSTD
or DF_MASK_COMPRESSED_LZ4
and then know on replica side to treat the value accordingly.
from dragonfly.
@romange LIT
from dragonfly.
@kostasrim does that mean that, once #3241 is merged, older versions of Dragonfly won't be able to read RDB/DFS saved with new Dragonfly versions?
If so, we need to disable that for at least one version to allow rollbacks IMO
from dragonfly.
@chakaz #3241 won't affect compression, but this PR will. If we extend the protocol as suggested by Roman older versions won't work and will break
from dragonfly.
For any such breaking changes, we should first disable (by default, toggle-able via a flag) the write path, and have the read path support it. It should be like that for at least 1 version, and only then we can enable it by default (or even remove the flag if makes sense)
from dragonfly.
Related Issues (20)
- extend HSETEX with NX option HOT 1
- Enable OpenSSF Scorecard to enhance security practices across the project HOT 2
- BZPop bug HOT 6
- Server crash while running replica takeover and info replication
- get rid of inline tables that use dynamic allocation HOT 1
- Flush important INFO loggings
- Include TLS information into INFO command output HOT 4
- test_cluster_flushall_during_migration deadlock HOT 1
- generic_test.py::test_unknown_dfly_env HOT 1
- teiered storage test failure HOT 2
- failure on test_cluster_fuzzymigration
- failure on test_network_disconnect_during_migration
- Takeover attempt timedout on non responding connections
- Sanitizers crash and namespaces HOT 1
- Failure on test_take_over_seeder HOT 4
- Replace SharedMutex in Journal
- DragonflyDB crashes when running BullMQ tests on a MacOS HOT 5
- untracked memory or a leak HOT 1
- test_multi_db failure
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dragonfly.