Switch to Java 21 about sirix

[adarshjhaa100]

Hi @JohannesLichtenberger, would like to help out with this issue if its up for grabs

[adarshjhaa100]

@JohannesLichtenberger Im also working on benchmarking the generational ZGC on my own. Will try to so some additional analysis here once im done with that.

Also, if there isn't any hard time constraint on the Java version upgrade, I can reach out to you with some updates on that by next week as I have a busy schedule this week.

Will keep posting the results of my ZGC benchmark here though.

[JohannesLichtenberger]

You'll need this file: https://www.rakuten-drive.com/transfer/download/96EB5CQP (downloadable until Thursday)

[adarshjhaa100]

Sorry for the delayed response. Due to some health issues. I've been away from any sorts of work since past couple of weeks. Logged in today only.

Having said that, i've started working on the analysis with generational ZGC and version upgrade. If it's required I can continue working on this.

[JohannesLichtenberger]

You can check my branch. I think there's only a Gradle issue left...

[adarshjhaa100]

Yea better than before :-). It's the onset of winter when pollution suddenly shoots up.

[JohannesLichtenberger]

@adarshjhaa100 I've had some time after work today, so I managed to finish the Java update :) but of course still looking forward to your input and maybe contributions (would be more than awesome) 👍

[JohannesLichtenberger]

Thanks, go ahead :)

[Adaminite]

Hi there. Is this issue still being worked on?

[adarshjhaa100]

@Adaminite Has been assigned to me. Im working on this.

[JohannesLichtenberger]

@adarshjhaa100 and @Adaminite, I guess I couldn't wait and wanted to check locally, especially the generational ZGC. At least I've locally switched to Java 21 and replaced @NotNull with @NonNull annotations...

[JohannesLichtenberger]

@adarshjhaa100 and @Adaminite, it would be more than excellent if you could work on other (more interesting?) issues :)

[adarshjhaa100]

@JohannesLichtenberger Sure no problem.

I was doing some analysis on new features added to Java 21 and there's some stuff that could be used to enhance the project. The work and time required for this would be significant, and you would be able to do justice to that. Unassigning myself.

Would be interested in tracking this tread though and see what you guys could come up with.

[JohannesLichtenberger]

@adarshjhaa100, seems I've still some stuff to do, but have to go to bed now: https://github.com/sirixdb/sirix/tree/update-to-java21

[JohannesLichtenberger]

So, if you want to work on this... :-) I got to sirix-core at least to work with Java 21 and a specific test I wanted to check

[JohannesLichtenberger]

Current issue is that Kotlin does not have a Java 21 target

[adarshjhaa100]

Yep, official docs suggest support till 20, Iast kotlin release is 1.19.10 that came out last month. Got to track the jetbrains release schedule and roadmap.

[JohannesLichtenberger]

@adarshjhaa100 I wanted to check generational ZGC, but apparently at least with preconfigured buffer sizes maybe it seems to run out of foreign memory, without it's no problem...

[JohannesLichtenberger]

Well, simply seems that direct ByteBuffers are not cleaned up and probably not subject for cleanup somehow.

[adarshjhaa100]

Did you try adding both these flags: -XX:+UseZGC -XX:+ZGenerational

[JohannesLichtenberger]

Yes

[adarshjhaa100]

One question remains, which objects is it identifying "young" or "old". And what's the time in which "young" objects need to be collected.

Btw, is there any reference to ByteBuffer which is preventing it from getting cleaned up.

Do you have any profiling data available for this? These flags can help: -Xlog:gc* and -verbose:gc

[JohannesLichtenberger]

@adarshjhaa100 yes, I know the JVM flags, but not at home today. Maybe I can check tomorrow, but it's in the trx commit in NodePageTrx where the leaf pages are serialized via a parallel stream to temp byte arrays (but the issue is with the byte instances from Chronicle Bytes).

[JohannesLichtenberger]

sirix/bundles/sirix-core/src/main/java/io/sirix/access/trx/page/NodePageTrx.java

Line 444 in dca0033

final Bytes<ByteBuffer> bytes = Bytes.elasticByteBuffer(60_000);

[JohannesLichtenberger]

@ adarshjhaa100 I've attached the log, but I suppose it's not helping that much.

zgc-generational-insert.log

Caused by: java.lang.OutOfMemoryError: Cannot reserve 4096 bytes of direct buffer memory (allocated: 8589931942, limit: 8589934592)
	at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:127)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:360)

Also, I'm currently not having much time to work on the issue, so anyone can grab it (maybe using my branch as a base).

[JohannesLichtenberger]

@adarshjhaa100 any news already? :-)

[adarshjhaa100]

@JohannesLichtenberger been busy with work. Will share the progress by this weekend

[JohannesLichtenberger]

@adarshjhaa100 I guess you have had no time!?

[adarshjhaa100]

@JohannesLichtenberger apologies for the delay. I've been busy with a production issue in my org since last couple of weeks.

Hope you understand.

[JohannesLichtenberger]

Sure

[JohannesLichtenberger]

That said, I'm still looking forward to your insights, once you have some time and if you like to spend it on this 😉

[adarshjhaa100]

Sure, I've taken off this Friday will take a look for sure. The issue's with the generational ZGC not working properly in this case, right?

[JohannesLichtenberger]

Yes, resulting in OutOfMemoryErrors eventually, plus the switch to Java 21 in the first place (you can have a look into the branch).

[JohannesLichtenberger]

Guess you haven't had time!?

[JohannesLichtenberger]

Hope you're doing fine again :-)