Coder Social home page Coder Social logo

Comments (3)

pdavidson100 avatar pdavidson100 commented on June 9, 2024

Thanks for your suggestion @srolija! The Mirus Source Connector is primarily intended for high-reliability data replication use cases, so our defaults are selected to minimize the risk of data loss at all costs — even at expense of increased risk of duplicate data. By using the earliest policy we guarantee that all available data is replicated, even in exceptional circumstances. This policy covers us when new topics are added to the topic regex and, importantly, also in rare instances where a Kafka bug causes the current offset to become invalid and the consumer offsets are reinitialized. This is something we have occasionally seen, particularly in earlier Kafka releases and using latest in that situation would certainly lead to data loss.

from mirus.

srolija avatar srolija commented on June 9, 2024

Completely get it.

Would it then make sense just to note the changes from the default consumer group? Asking since we had an issue where it started replicating enormous topic; and based on the docs we didn't understand that it has non-default configuration.

from mirus.

pdavidson100 avatar pdavidson100 commented on June 9, 2024

Yes, that does make sense - we should update the the docs.

from mirus.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.