Release on hackage? about codec-jvm

[rahulmutt]

It's still pretty unstable since we're trying out new things that come up when implementing Eta features. We would ideally like to make it more complete, document it, and release it once it's done.

If you need it urgently for a project you're working on, I'd be happy to help you set it up with a git dependency.

[minad]

Hi Rahul,

thanks for the status update! It wouldn't be an issue to include codec-vm via git for now. Generally, I was looking around for solutions to bytecode generation. As far as I understood, you used hs-java instead of codec-jvm before, idris-java just generates java code and idris-jvm uses an external java assembler. Since you know the inner workings of haskell/jvm very well, I am pretty sure that codec-jvm would be the preferred library these days - but then, it wasn't on hackage ;)

Compared to java code generation the advantage (besides not relying on javac) is that you can use invokedynamic directly, but that doesn't seems to be supported by codec-jvm yet? You are probably planning to implement this and use it in eta? One of the major downsides of the jvm for functional compilers seems to me that you don't get first class functions. As long as you don't defunctionalize, you have to generate all these classes for lambdas or stg closures. A while ago I made an experiment where I generated java code with many lambdas and equivalent code using anonymous classes. The size of the jar with lambdas was way smaller. I think I've heard that scala also suffers from this. Did you try invokdynamic in eta too or is it not a problem for you to generate large jar files? Or maybe you work around it in another way?

Daniel

[rahulmutt]

Hi Daniel,

Great questions! I can see you've thought about bytecode generation quite a bit :)

Yeah I used hs-java in the beginning. It uses uses knot-tying semantics in its implementation which in practice means it's a pain to debug "missing constants from constant pool" errors. Then @aloiscochard brought up his codec-jvm library and I just loved the simplicity of the design and I've been using it since.

I mean codec-jvm works pretty well. I just haven't kept up with proper versioning and proper API structuring. Once things settle down from the Eta side, I'll sit down and refactor the code. Moreover, if you need to use invokedynamic for your project, I'd be happy to guide you through the implementation. It shouldn't be too difficult to do. Eta has been giving little time for me to work on other things :)

Invokedynamic I've been avoiding primarily for backwards compatibility, but I do intend to work on an indy codgen in the future once all the major features have been implemented. And yes, as of now Eta generates monster jar files (a simple hello world program can be ~20MB if uberjared with the runtime, ghc-prim, integer, and base), but it turns out a lot of the classes are never used and running it through proguard reduced the size by almost 60%. It would've reduced even more if I had implemented lazy initialisation of StgClosures.

Also indy is still very new, and takes up a long time to warmup, and it's not always getting the speedups that they hoped for and probably by Java 9 and beyond it'll be in a state that can add lots of benefit.

Yes, Scala does indeed suffer from the same problem, but I recently was looking at bytecode generated by Scala 2.11 and I noticed use of invokedynamic. I think they're generating lambdas now?

So far generating large jar files hasn't been a huge problem. We'll find ways to address it when the time comes. I did find though, that not using uberjars ends up being pretty fast. Like deploying the dependencies as individual jars and setting the classpath manually (currently epm will generate the classpath for you).