Comments (3)
Yes, as you've discovered, the Arrow.jl implementation isn't currently very useful for "serialization" of generic objects, but focused on the limited set of types it was originally designed for. I've looked into this and the DataType cases and they're non-trivial with how the code is currently structured, but it should be much nicer if people want to "hook in" to using Arrow.jl for generic serialization. The biggest thing is making the ArrowTypes module much cleaner in terms of the contracts for the various types: what's required and then making sure hte implementation only relies on those contracts. Then people can overload the necessary methods and everything will just work, similar to the StructTypes.jl system. let me look at the NamedTuple issue and see if it's lower hanging fruit
from arrow-julia.
Ha, yeah - I was actually surprised that Set
s were serializable in the first place 😁 Thanks for checking it out.
I'd consider it a win if the error could be moved to serialization time (and try to prevent non-deserializable content from being serialized in the first place as a general rule)
from arrow-julia.
I'm going to add support for this in #156
from arrow-julia.
Related Issues (20)
- `Arrow.write` performance on large DataFrame HOT 3
- Bus errors when writing `DataFrame` HOT 8
- Arrow stream writer and reader implementation questions
- [feature request] support run-end encoded layout
- Custom type cannot round trip (Colors.jl) HOT 1
- colmetadata does not read custom metadata with multiple writes
- `getindex` broken with `SVector{3, UInt}` in the presence of missing data HOT 2
- Removing .arrow files without closing Julia seems impossible in Windows HOT 18
- support Dates.CompoundPeriod in deserialization?
- copy does not copy to standard Julia Types HOT 5
- Unexpected allocations HOT 2
- Type instability in getcolumn
- Cannot append DictEncode columns to Stream
- Arrow-over-HTTP client and server examples in Julia
- Deeply nested structs cause long compilation times HOT 9
- `snappy_jll v1.2.0` lead to Arrow_jll failed to build HOT 4
- Deserialization as Vector{SubArray} breaks `push!` on DataFrame HOT 7
- Add support for FileIO HOT 2
- interoperability with round-tripping through data format broken (x-issue) HOT 1
- Arrow.jl fails to precompile with error "Magic file identifier "TZjf" not found." HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from arrow-julia.