Coder Social home page Coder Social logo

Comments (6)

adiholden avatar adiholden commented on July 20, 2024 2

I synced with Shahar and he brought up that there should be an api for continues compression. I worked on this CompressorImpl class in the past, and I think we dont need to break changes. If we change the CompressorImpl class to create context and free the context once we finish full entry compression I believe this should be good. This is like breaking the compression to many frames that will be sent one after the other and with last frame once we finished serializing the entry

from dragonfly.

romange avatar romange commented on July 20, 2024 1

@kostasrim I suggest using RDB_OPCODE_DF_MASK for that. I't a meta opcode that can say in advance what to expect in keyvalue that is gonna be parsed next. Right now we only have DF_MASK_FLAG_STICKY bit, so you can extend with compression bits like DF_MASK_COMPRESSED_ZSTD or DF_MASK_COMPRESSED_LZ4 and then know on replica side to treat the value accordingly.

from dragonfly.

kostasrim avatar kostasrim commented on July 20, 2024

@romange LIT

from dragonfly.

chakaz avatar chakaz commented on July 20, 2024

@kostasrim does that mean that, once #3241 is merged, older versions of Dragonfly won't be able to read RDB/DFS saved with new Dragonfly versions?
If so, we need to disable that for at least one version to allow rollbacks IMO

from dragonfly.

kostasrim avatar kostasrim commented on July 20, 2024

@chakaz #3241 won't affect compression, but this PR will. If we extend the protocol as suggested by Roman older versions won't work and will break

from dragonfly.

chakaz avatar chakaz commented on July 20, 2024

For any such breaking changes, we should first disable (by default, toggle-able via a flag) the write path, and have the read path support it. It should be like that for at least 1 version, and only then we can enable it by default (or even remove the flag if makes sense)

from dragonfly.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.