Schema evolution handling about pickling

[crumbpicker]

+1
Maybe Option could be an option ?
I hope there will be support for this.

[guersam]

Probably already checked, but FYI just in case: Schema evolution in Avro, Protocol Buffers and Thrift

[phaller]

At the moment it is necessary to create custom picklers to deal with schema evolution (for example, a custom pickler could unpickle older versions of a message, but create instances of newer versions).

However, it is possible to create new pickle formats which are more evolution-friendly, for example, by including something akin to Protobuf's field tags (the blog post linked above explains well how these tags help with schema evolution). Adding an evolution-friendly variant of the binary pickle format would be very useful, indeed.

[guersam]

Thanks for your advice, @phaller. At least, default parameter support will provide some basic compatibility out of the box, but it seems difficult without additional annotation until SI-5920 is resolved.

I decided to go with Protobuf for now but will keep eyes on Pickler looking for what I can contribute.

[adamdecaf]

Since 0.10.0 came out I was looking at pickling for use to persist messages and was thinking about how schema evolutions would happen. Sadly this test I wrote didn't pass, but I'm mentioning it here to see if now that some of those PR's in the SI have been merged if it's likely for pickling to have support for handling default methods. It looks like it'll be fixed in scalac in 2.11.6?

https://github.com/adamdecaf/pickling-migration-test/blob/master/src/main/scala/Add.scala

Deleting fields works though!

[havocp]

In https://github.com/sbt/serialization we support Option fields that can be missing.

[pjan]

It would be totally awesome if schema evolution came about. It currently is the thing keeping us back from using pickling. @adamdecaf's outlined usecases pretty much sum up the hoped-for functionality for now.

protobuf/avro/thrift was mentioned before, but it is just not cutting it in a pure scala-based environment due to the lack of support for e.g. polymorphism, drastically cutting down the flexibility that scala offers.

So, very keen to get an update on this 😄