Comments (8)
This is pretty inefficient. I'd like to see how other msgpack libraries encode structs.
from rust-msgpack.
I could drop the assertion that field-names match which would eliminate the comparison (since the Rust decoder already knows this information.)
Though obviously there'd still be the penalty encoding the field name to make a valid msgpack map.
What if we just dropped the _emit_map_len()
and _read_map_len()
from emit_struct() and read_struct()
and encoded structs as an array of fields instead?
The msgpack array type will still have a length; so we can still perform the sanity check on the number of struct fields during deserialization.
from rust-msgpack.
Array of fields would work too, and I am wondering why I didn't do that! Can you try to encode structs as an array? I think the reason why I was using _emit_map_len() were some legacy msgpack files I had to use. Ideally msgpack would support "struct" natively.
from rust-msgpack.
Check out this branch.
It just is a drop in replacement of _emit_array_len()
and _read_vec_len()
; and it passes all the current tests.
It should perform identically to the map version; the major win here is that third-party decoders will read our structs as an array of fields, rather than a corrupted map.
Also I just wanted to add that serialize::json
does actually encode structs as maps (e.g JSON objects) so using the map type is not without precedent. Though I agree that encoding the field-names is somewhat contrary to the goal of msgpack
-- which is meant to be a compact serialization format.
My main concern is that encoding structs as arrays would be a "breaking change." In the sense that this new decoder will not be able to read Rust structs emitted by the previous version of the encoder. This would obviously be problematic for anyone that has rust-msgpack
data sitting around.
I'm toying around with the idea of using Cargo feature flags so that you can optionally compile msgpack with either the old map-based behavior, or this new array-based behavior.
We would just annotate the read_struct/emit_struct
functions with attributes such as #[cfg(feature="struct-array")]
or #[cfg(feature="struct-map")]
and the user would declare which one they want to use in their own Cargo.toml
.
from rust-msgpack.
Hi, I've created msgpack encoder/decoder for my studying.
https://github.com/rail44/msgpack-rs/
I made Struct to be encoded to Map. It may contain useful code for rust-msgpack implementing, FYI.
from rust-msgpack.
The idea behind msgpack IMHO is that it needs no schema files; if you don't include key values then you relay on schema (rust structure definition) and this breaks this design. Also it is important to keep compatibility with different languages and Ruby won't parse current structure output. JSON serialised stuctures apparently encodes key values.
Also I don's see any other methods of creating msgpack message with map of mixed value types other than using structs.
So please encode key values as well.
There are different formats that are focused on efficiency by use of schema files.
from rust-msgpack.
@jpastuszek @drbawb: I think the problem with this is that we still can't read back structs from other languages, as they might (and very probably will) encode the key/value pairs in a different order than we are trying to read them. Actually, IIRC, the msgpack spec does not say anything about structs in general only about maps. That is, directly mapping a (Rust) struct to a hashmap is a difficult thing inter-language-wise, or at least doing so without buffering the input.
Can we get this to work:
Rust::msgpack_encode(struct) -> Ruby::msgpack_decode(str).msgpack_encode -> Rust::msgpack_decode(str) == struct
I can only think of an array of tuples [(fieldname1, value1), ...]. This would have the advantage that we could also optionally remove the fieldname and it'd still work.
from rust-msgpack.
Recently I have been thinking that the problem is in deserialisation. What your are really telling the library when you try to deserialise a struct is set of expectations on content of the message:
- expect it to be a map (for inter-compatibility with other languages)
- expect that it contains the same keys as fields names (some convention based transformation may be useful here)
- expect that corresponding values can be safely casted to struct data types (integers to i64 or i32 etc.)
Now if there are any extra fields in the map they should be ignored. If one of struct data types is Option and filed in the map is missing you get None value.
Serialisation then is rather straight forward.
I am aware that this cannot be implemented with current API though...
from rust-msgpack.
Related Issues (20)
- Add enum support. HOT 3
- Out of touch with master. HOT 4
- Build failing on serialise::Decoder trait HOT 2
- Travis builds failing since bbd93a7 HOT 1
- Support rust enums HOT 1
- Doesn't compile with the latest rust HOT 1
- Fails to pack/unpack
- task '<main>' failed at 'called `Option::unwrap()` on a `None` value' HOT 5
- Nullary option variants corrupt the next element in the decoder. HOT 2
- Publish to crates.io HOT 1
- Value is not Decodable after the move to associated types HOT 4
- port to rust 1.0.0 alpha
- Clone Trait
- Build is broken against nightly
- msgpack-rpc HOT 5
- Does not compile with rustc 1.7.0 HOT 3
- Won't Compile with rustc 1.15.0-nightly (8f02c429a 2016-12-15)
- It doesn't compile with byteorder v1.2.1 HOT 1
- Mark this library as deprecated
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rust-msgpack.