Coder Social home page Coder Social logo

JdbcItemReader restart behavior about jsr352 HOT 4 CLOSED

 avatar commented on August 16, 2024
JdbcItemReader restart behavior

from jsr352.

Comments (4)

chengfang avatar chengfang commented on August 16, 2024

I think the reason is, in the readItem method: https://github.com/jberet/jsr352/blob/master/jberet-support/src/main/java/org/jberet/support/io/JdbcItemReader.java#L242

we can resultSet.next(), which moves the cursor forward by one, so checkpoint - 1 is to compensate that forward move.

For instance, if we read the first item, process it, but failed in writer, hence rollback. The reader checkpoint info is now 1. When restarting, we set reposition the resultSet to checkpoint - 1, which is 0. Then in readItem method, calling resultSet.next() will position it to the first item, and the subsequent resultSet.getObject() will return the first item in the resultSet.

Have you seen any actual unexpected behavior in restarting in your app? Any reproducible app will be even better for pinpointing the problem.

from jsr352.

 avatar commented on August 16, 2024

I think the point is, that it's not the current row that gets persisted as a checkpoint in your example, because the transaction gets rolled back. So the checkpoint will be the index of the last item that was successfully processed and committed depending on your chunk size.

I experienced unexpected behavior in the following szenario: Suppose you process 20 elements with a chunk size of 10. So for every 10 elements, the current row is used as a checkpoint and the entire chunk is passed to the writer. If processing of element 11 leads to an error, the last checkpoint is position 10. On retry, my app ends up processing element 10 twice, because checkpoint - 1 and resultSet.next() will effectively set the cursor to position 10 instead of 11.

The JpaItemReader has the correct positioning behavior. Therefore I'd recommend to just use checkpoint instead of checkpoint - 1 to reposition the cursor.

from jsr352.

chengfang avatar chengfang commented on August 16, 2024

https://issues.jboss.org/browse/JBERET-364 was created to track this issue and fix.

from jsr352.

chengfang avatar chengfang commented on August 16, 2024

This issue should be fixed with the above commit. Thanks for reporting it. Let's know if anything else.

from jsr352.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.