dtolnay / erased-serde Goto Github PK
View Code? Open in Web Editor NEWType-erased Serialize, Serializer and Deserializer traits
License: Apache License 2.0
Type-erased Serialize, Serializer and Deserializer traits
License: Apache License 2.0
use serde::Serialize;
#[derive(Serialize)]
struct Node {
k: Option<Box<dyn erased_serde::Serialize>>,
}
fn main() {
std::thread::Builder::new()
.stack_size(2 * 1024 * 1024)
.name("test".to_string())
.spawn(move || {
let mut node = None::<Box<dyn erased_serde::Serialize>>;
for _ in 0..100 {
node = Some(Box::new(Node { k: node }));
}
let json = serde_json::to_string(&node).unwrap();
println!("{json}");
})
.unwrap()
.join()
.unwrap();
}
Moved from @rasky's question in #25 (comment):
I already have an object in memory of the correct type, behind a trait object, and I would like to invoke deserialize_in_place on it, using some type erasure magic to make it work through the trait object. Is that possible? Any hint on how to implement it?
I am trying to port code from serde 0.8 to 1.0 with erased_serde 0.3.1 but I am getting stuck on failing to declare a Result with the Ok type because erased_serde::ser is private. Should erased_serde::ser::Ok be made public? Or am I doing something else wrong (that the compiler will tell me about if I got past the module privacy error)?
The code that I have working for serde 0.8 is this
use erased_serde::Serialize;
impl serde::Serialize for Event {
fn serialize<S>(&self, serializer: &mut S) -> Result<(), S::Error>
where S: serde::Serializer
{
self.erased_serialize(serializer).map_err(|err| {
serde::ser::Error::custom(err.to_string()
})
}
}
And the code that is failing to compile with serde 1.0.15 and erased serde 0.3.1 is this
use erased_serde::{Serialize, Serializer, Error};
impl serde::Serialize for Event {
fn serialize<S>(&self, serializer: S) -> Result<erased_serde::ser::Ok, erased_serde::Error>
where S: Serializer
{
self.erased_serialize(&mut serializer).map_err(|err| {
serde::ser::Error::custom(err.to_string())
})
}
}
Event is a trait that extends erased_serde::Serialize
(The one in the title of the GitHub web interface)
Should be: https://docs.rs/erased-serde
If I understand the documentation correctly, this library only works with serde libraries that have a Serializer
object that implements the serde::ser::Serializer
trait.
However in bincode
, serder_yaml
and avro_rs
there are no such objects. Can this library be used with them?
I'm trying to do the same as in issue #5 just with the current version of the crate.
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_json;
extern crate erased_serde;
trait Element: erased_serde::Serialize + Send + Sync {
/* ... */
}
impl serde::Serialize for Element {
fn serialize<S>(&self, serializer: &mut S) -> Result<(), S::Error>
where S: serde::Serializer
{
self.erased_serialize(serializer).map_err(|err| {
serde::ser::Error::custom(err.to_string())
})
}
}
But I assume due to changes in serde::Serialize I now run into:
error[E0053]: method `serialize` has an incompatible type for trait
|
13 | / fn serialize<S>(&self, serializer: &mut S) -> Result<(), S::Error>
14 | | where S: serde::Serializer
15 | | {
16 | | self.erased_serialize(serializer).map_err(|err| {
17 | | serde::ser::Error::custom(err.to_string())
18 | | })
19 | | }
| |_____^ expected type parameter, found &mut S
I managed to get this far, but I am unable get passed the associated type problem. Do you possibly have any idea how to make this work?
#[macro_use]
extern crate serde_derive;
extern crate erased_serde;
extern crate serde;
extern crate serde_json;
trait Element: erased_serde::Serialize + Send + Sync {
/* ... */
}
impl serde::Serialize for Element {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
self.erased_serialize(&mut erased_serde::Serializer::erase(serializer))
}
}
|
13 | fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
| ----------------------- expected `std::result::Result<<S as serde::Serializer>::Ok, <S as serde::Serializer>::Error>` because of return type
...
18 | / self.erased_serialize(&mut erased_serde::Serializer::erase(serializer))
19 | | .and_then(|x| Ok(()))
| |_________________________________^ expected associated type, found ()
|
= note: expected type `std::result::Result<<S as serde::Serializer>::Ok, <S as serde::Serializer>::Error>`
found type `std::result::Result<(), erased_serde::Error>`
error: aborting due to previous error
I'm trying to implement a kind of user-extensible deserialization that allows you to "plug in" extra deserialize implementations. The basic idea is to have a Seed
struct containing a HashMap<String, fn (&Seed, &mut erased_serde::Deserializer) -> erased_serde::Error>
and use serde::DeserializeSeed
to deserialize stuff. This is working pretty well, but there's an annoying issue with the error handling.
Whenever I call one of the deserialize implementations I can get an erased_serde::Error
, but to bubble it up I need to convert it to an error for the deserializer I am using (serde_json
at the moment, but I would like to remain generic), by using serde::de::Error::custom
. These calls can be fairly recursive, so I might end up going back-and-forth between the two error types a few times. This is giving me error messages that look like this:
Error("trailing comma at line 32 column 41 at line 32 column 41 at line 33 column 37 at line 34 column 33 at line 35 column 29 at line 36 column 25 at line 36 column 25 at line 36 column 25 at line 36 column 25", line: 36, column: 25)
The ideal solution for me would be the ability to cast erased_serde::Error
back to the original error type, by representing it as a Box<serde::de::Error>
. I'm not sure whether my use-case is typical for this library so possibly this makes no sense in the general case, but I am interested to know what you think.
Hi
So I have a trait defined like
pub trait Element: Send + Sync
and it in included in another type like
#[derive(Serialize, Deserialize)]
pub struct Pipeline {
name: String,
elements: Vec<Box<Element>>,
}
I am a little confused with the examples how I could get serde with erased-serde to serialise this.
Would I add the trait to Element and implement for all the concrete types of element ? (There may eventually be hundreds of concrete types)
Sorry if it is not meant to be used in this way.
Thanks
Hi! ๐
I was wondering if you had any thoughts on stabilizing erased-serde
. It seems like it's been doing its thing for a while. Are there any blockers or open questions to consider?
I'm guessing at least trait stability is less of a concern for erased-serde::{Serialize, Deserialize}
since you can always treat it like one of those trait objects like a regular serde::{Serialize, Deserialize}
, but having a stable erased-serde
would be great for those cases where it does appear publicly.
This requires figuring out what that even means. What can it return?
I have a use case where a few types have codec-specific serialization and deserialization behaviour (JSON encodes bytes as maps, CBOR should encode a custom type with a specific tag, etc).
I can't erase
the generic Serializer
I get in the Serialize
implementations for these types b/c I cannot require that S::Ok: 'static
,
but perhaps within a new provided functionlike erased_serde::serialize
I can:
1. provide a closure that takes an erased_serde::Serializer
2. do some stuff (say, wrap it in a type like MySerializer<S> { _s: PhantomData<S>, ser: Box<dyn erased_serde::Serializer> }
, and call methods on it that call the underlying erased Serializer, performing the codec-specific behaviour I require)
3. then the caller of the closure can Ok::take
the S::Ok
out for me (since that method is not public)
Not sure if providing this kind of API could/would relax the 'static
requirements since we're take
ing from erased_serde::Ok
directly, but I thought I'd ask.
edited: I don't think what I suggested would do what I need it to do, but the problem I have stated above still remains. I can change the title of this issue if there's something this library can do to help or close it if its irrelevant.
Also, if I'm misunderstanding something about my use case or Rust mechanics, please let me know. Thanks!
Hello,
With the risk of being annoying since this was discussed before, I am currently trying to resolve this exact issue here #5, however I am stuck at the following step:
error[E0277]: the trait bound `S: erased_serde::Serializer` is not satisfied
--> src\task.rs:102:31
|
102 | self.erased_serialize(&mut serializer)
| ^^^^^^^^^^^^^^^ the trait `erased_serde::Serializer` is not implemented for `S`
|
= help: consider adding a `where S: erased_serde::Serializer` bound
= note: required for the cast to the object type `erased_serde::Serializer`
This comes from something similar to the example on the above-mentioned issue:
// Task is just a random trait I made.
impl serde::Serialize for Task {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where S: serde::Serializer
{
self.erased_serialize(&mut serializer)
}
}
Following the suggestion in the error and requiring where S: serde::Serializer + serde::Deserializer
doesn't work either.
What I am basically trying to do is save a vector of Box using the following:
#[derive(Default, Serialize, Deserialize)]
pub struct WorkbenchComponent {
tasks: Vec<Box<Task>>,
...
}
All Tasks structs that implement the Task trait, also implement Serialize in a similar way to this:
impl Serialize for CraftingTask {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where S: Serializer
{
let mut state = serializer.serialize_struct("CraftingTask", 1)?;
state.serialize_field("id", &self.id)?;
state.end()
}
}
I also tried #derive[Serialize]
on the CraftingTask but it doesn't help either.
There is also no clear guide on converting the error types to Serializer::Ok / Error. The map Example in the issue I linked above is not valid anymore (The return type is not Result<(), Error>, but Result<S::Ok, S::Error>
error[E0308]: mismatched types
--> src\task.rs:102:9
|
99 | fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
| ----------------------- expected `std::result::Result<<S as serde::Serializer>::Ok, <S as serde::Serializer>::Error>` because of return type
...
102 | self.erased_serialize(&mut serializer)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected associated type, found struct `erased_serde::ser::Ok`
|
= note: expected type `std::result::Result<<S as serde::Serializer>::Ok, <S as serde::Serializer>::Error>`
found type `std::result::Result<erased_serde::ser::Ok, erased_serde::Error>`
If I manage to solve this, I am happy to contribute a guide in the readme for this use case, but I don't really get how I should fix it right now.
One approach would be just manually implementing Workbench serialize (for the holder of Vec, then manually calling some simpler task_serialize function there.
Something like this:
impl serde::Serialize for WorkbenchComponent {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where S: serde::Serializer
{
for task in self.tasks.iter() {
task.special_serialize(&mut serializer);
}
[...]
}
}
In that method I can implement whatever I need for the serialization routine possibly.
Long story.
Currently we store function pointers in custom vtable.
We want to remove dependency on ptr_metadata
feature to make starlark-rust work in stable.
One possibility is to store functions instead of dyn metadata in vtable. I. e. for Display
, store
fn(*const (), &mut Formatter<'_>) -> fmt::Result
instead of
DynMetadata<dyn Display>
(D41063479).
However, similar transformation seems to be impossible for erased_serde::Serialize
.
We could store function:
fn(*const (), v: &mut dyn erased_serde::Serializer) -> Result<(), erased_serde::Error>
instead of
DynMetadata<dyn erased_serde::Serialize>
however, it is not possible to call this function. There are two options, neither work:
First, erased_serde::Serialize
cannot be implemented directly using stored function because the trait is sealed (Ok
type is private).
Second, this function cannot be called directly, because there's no public API to construct erased_serde::Serializer
from serde::Serializer
.
The latter approach seems to be more appropriate.
Hence feature request: create a public function like:
pub fn into_erased_serializer<S: serde::Serializer>(serializer: S) -> impl erased_serde::Serializer { ... }
I receive the following error while using the bevy_spicy_networking crate any time I try to send a message with a string in it. I'm not sure why it happens. It seems as if it shouldn't, since it's from String to String.
thread 'tokio-runtime-worker' panicked at 'invalid cast: alloc::string::String to alloc::string::String',
[...]/.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/any.rs:129:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
backtrace:
stack backtrace:
0: rust_begin_unwind
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/panicking.rs:498:5
1: core::panicking::panic_fmt
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/core/src/panicking.rs:107:14
2: erased_serde::any::Any::invalid_cast_to
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/any.rs:129:9
3: erased_serde::any::Any::take
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/any.rs:104:13
4: erased_serde::de::Out::take
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/de.rs:254:9
5: core::ops::function::FnOnce::call_once
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/core/src/ops/function.rs:227:5
6: core::result::Result<T,E>::map
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/core/src/result.rs:767:25
7: <&mut dyn erased_serde::de::MapAccess as serde::de::MapAccess>::next_value_seed
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/de.rs:1037:9
8: serde::de::MapAccess::next_value
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.130/src/de/mod.rs:1846:9
9: <warfare::networking::messages::_::<impl serde::de::Deserialize for warfare::networking::messages::ChatMessage>::deserialize::__Visitor as serde::de::Visitor>::visit_map
at ./src/networking/messages.rs:12:21
10: <erased_serde::de::erase::Visitor<T> as erased_serde::de::Visitor>::erased_visit_map
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/de.rs:583:9
11: <&mut dyn erased_serde::de::Visitor as serde::de::Visitor>::visit_map
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/de.rs:994:9
12: <typetag::internally::MapWithStringKeys<A> as serde::de::Deserializer>::deserialize_struct
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/typetag-0.1.8/src/internally.rs:430:9
13: <erased_serde::de::erase::Deserializer<T> as erased_serde::de::Deserializer>::erased_deserialize_struct
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/de.rs:459:9
14: <&mut dyn erased_serde::de::Deserializer as serde::de::Deserializer>::deserialize_struct
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/de.rs:798:17
15: warfare::networking::messages::_::<impl serde::de::Deserialize for warfare::networking::messages::ChatMessage>::deserialize
at ./src/networking/messages.rs:12:21
16: erased_serde::de::deserialize
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/erased-serde-0.3.16/src/de.rs:39:5
17: warfare::networking::messages::_::__init::__INVENTORY::{{closure}}
at ./src/networking/messages.rs:17:1
18: core::ops::function::FnOnce::call_once
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/core/src/ops/function.rs:227:5
19: <typetag::de::FnApply<T> as serde::de::DeserializeSeed>::deserialize
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/typetag-0.1.8/src/de.rs:55:9
20: <typetag::internally::TaggedVisitor<T> as serde::de::Visitor>::visit_map
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/typetag-0.1.8/src/internally.rs:90:32
21: serde_cbor::de::Deserializer<R>::parse_map::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:474:25
22: serde_cbor::de::Deserializer<R>::recursion_checked
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:433:17
23: serde_cbor::de::Deserializer<R>::parse_map
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:473:9
24: serde_cbor::de::Deserializer<R>::parse_value
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:704:28
25: <&mut serde_cbor::de::Deserializer<R> as serde::de::Deserializer>::deserialize_any
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:788:9
26: <&mut serde_cbor::de::Deserializer<R> as serde::de::Deserializer>::deserialize_map
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.130/src/macros.rs:134:13
27: typetag::internally::deserialize
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/typetag-0.1.8/src/internally.rs:46:5
28: bevy_spicy_networking::network_message::_NetworkMessage_registry::<impl serde::de::Deserialize for alloc::boxed::Box<dyn bevy_spicy_networking::network_message::NetworkMessage>>::deserialize
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_spicy_networking-0.5.2/src/network_message.rs:3:1
29: <core::marker::PhantomData<T> as serde::de::DeserializeSeed>::deserialize
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.130/src/de/mod.rs:785:9
30: <serde_cbor::de::MapAccess<R> as serde::de::MapAccess>::next_value_seed
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:1015:9
31: serde::de::MapAccess::next_value
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.130/src/de/mod.rs:1846:9
32: <bevy_spicy_networking::_::<impl serde::de::Deserialize for bevy_spicy_networking::NetworkPacket>::deserialize::__Visitor as serde::de::Visitor>::visit_map
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_spicy_networking-0.5.2/src/lib.rs:207:21
33: serde_cbor::de::Deserializer<R>::parse_map::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:474:25
34: serde_cbor::de::Deserializer<R>::recursion_checked
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:433:17
35: serde_cbor::de::Deserializer<R>::parse_map
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:473:9
36: serde_cbor::de::Deserializer<R>::parse_value
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:704:28
37: <&mut serde_cbor::de::Deserializer<R> as serde::de::Deserializer>::deserialize_any
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:788:9
38: <&mut serde_cbor::de::Deserializer<R> as serde::de::Deserializer>::deserialize_struct
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.130/src/macros.rs:134:13
39: bevy_spicy_networking::_::<impl serde::de::Deserialize for bevy_spicy_networking::NetworkPacket>::deserialize
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_spicy_networking-0.5.2/src/lib.rs:207:21
40: serde_cbor::de::from_slice
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/serde_cbor-0.11.2/src/de.rs:52:17
41: bevy_spicy_networking::server::handle_new_incoming_connections::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/bevy_spicy_networking-0.5.2/src/server.rs:286:67
42: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/core/src/future/mod.rs:80:19
43: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:161:17
44: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/loom/std/unsafe_cell.rs:14:9
45: tokio::runtime::task::core::CoreStage<T>::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:151:13
46: tokio::runtime::task::harness::poll_future::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:461:19
47: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/core/src/panic/unwind_safe.rs:271:9
48: std::panicking::try::do_call
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/panicking.rs:406:40
49: __rust_try
50: std::panicking::try
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/panicking.rs:370:19
51: std::panic::catch_unwind
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/panic.rs:133:14
52: tokio::runtime::task::harness::poll_future
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:449:18
53: tokio::runtime::task::harness::Harness<T,S>::poll_inner
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:98:27
54: tokio::runtime::task::harness::Harness<T,S>::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:53:15
55: tokio::runtime::task::raw::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:113:5
56: tokio::runtime::task::raw::RawTask::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:70:18
57: tokio::runtime::task::LocalNotified<S>::run
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/mod.rs:343:9
58: tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:420:13
59: tokio::coop::with_budget::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:102:9
60: std::thread::local::LocalKey<T>::try_with
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/thread/local.rs:399:16
61: std::thread::local::LocalKey<T>::with
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/thread/local.rs:375:9
62: tokio::coop::with_budget
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:95:5
63: tokio::coop::budget
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:72:5
64: tokio::runtime::thread_pool::worker::Context::run_task
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:419:9
65: tokio::runtime::thread_pool::worker::Context::run
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:386:24
66: tokio::runtime::thread_pool::worker::run::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:371:17
67: tokio::macros::scoped_tls::ScopedKey<T>::set
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/macros/scoped_tls.rs:61:9
68: tokio::runtime::thread_pool::worker::run
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:368:5
69: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:347:45
70: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/task.rs:42:21
71: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:161:17
72: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/loom/std/unsafe_cell.rs:14:9
73: tokio::runtime::task::core::CoreStage<T>::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:151:13
74: tokio::runtime::task::harness::poll_future::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:461:19
75: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/core/src/panic/unwind_safe.rs:271:9
76: std::panicking::try::do_call
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/panicking.rs:406:40
77: __rust_try
78: std::panicking::try
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/panicking.rs:370:19
79: std::panic::catch_unwind
at /rustc/65c55bf931a55e6b1e5ed14ad8623814a7386424/library/std/src/panic.rs:133:14
80: tokio::runtime::task::harness::poll_future
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:449:18
81: tokio::runtime::task::harness::Harness<T,S>::poll_inner
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:98:27
82: tokio::runtime::task::harness::Harness<T,S>::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:53:15
83: tokio::runtime::task::raw::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:113:5
84: tokio::runtime::task::raw::RawTask::poll
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:70:18
85: tokio::runtime::task::UnownedTask<S>::run
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/mod.rs:379:9
86: tokio::runtime::blocking::pool::Inner::run
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/pool.rs:264:17
87: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
at .../.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/pool.rs:244:17
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
This will fail for me, do I need to manually implement something or is there something missing yet?
`#[derive(Serialize, Deserialize)]
struct Foo{
bar: A
}
#[derive(Serialize, Deserialize)]
struct A(HashMap<String, Box>);
#[derive(Serialize, Deserialize)]
struct B{
field: u32
}
pub trait Bar: erased_serde::Serialize + Any{
}
erased_serde::serialize_trait_object!(Bar);
impl Bar for B{
}`
tl; dr;
I'm confused by the &mut in erased_serde::deserialize.
May one use the deserializer afterwards?
Here is some code which panics:
#[derive(Debug, serde::Serialize, serde::Deserialize)]
struct D1 {}
#[derive(Debug, serde::Serialize, serde::Deserialize)]
struct D2 {}
trait T {}
impl T for D1 {}
impl T for D2 {}
use erased_serde::Deserializer;
fn main() {
let d1 = D1 {};
dbg!(&d1);
let serialized = serde_json::to_string(&d1).unwrap();
let mut deserializer = serde_json::Deserializer::from_str(&serialized);
let mut deserializer = Deserializer::erase(&mut deserializer);
dbg!(&erased_serde::deserialize::<D1>(&mut deserializer));
dbg!(&erased_serde::deserialize::<D2>(&mut deserializer));
dbg!(&erased_serde::deserialize::<D1>(&mut deserializer));
dbg!(&serialized);
}
Hi,
this is my first github issue, and this is more of a question.
here is some (unrelated) motivation.
I'm trying to write a plugin-system, and at the moment I'm checking if erased-serde can help.
To this end I'm runtime-loading some shared libraries, and from each I get a type implementing a shared trait.
Now I want to add a function to each library, which tries to deserialize the serialized trait into the libraries type.
Then, in the main executable, I wanted to iterate over my library-functions, feeding the same deseralizer into them.
Hence the example above.
Thank you,
Michael
I can't implement Serialize trait for my type because of return value type contain erased_serde::ser::Ok and ser module is private.
Currently type-erased serialization treats SerializeStruct::skip_field and SerializeStructVariant::skip_field as a no-op.
These APIs were added to serde in serde-rs/serde#1022 which was later than the creation of erased-serde 0.3.0.
How to implement erased_serde::Serialize
manually for custom type?
The types Ok
and Error
are private and this prevents to implement erased_serde::Serialize
for any type or trait outside the library. But if we can't implement serde::Serialize
we can't have erased_serde::Serialize
for this type.
E.g. for some trait TheTrait
we need type-erased serialize because if we implement serde::Serialize
we'll have The trait TheTrait cannot be made into an object
, but we can't implement erased_serde::Serialize
manually because Ok
and Error
of Result
are private.
Please provide any solution or make Ok
and Error
types public.
Do you think this crate has any change of being included in serde proper?
The serde_cbor crate is retired. pyfisch/cbor@347a3f0
Compiling erased-serde v0.3.3
error: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:651: method `is_human_readable` is not a member of trait `serde::Deserializer`
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:658: in this macro invocation
error: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:651: not a member of trait `serde::Deserializer`
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:659: in this macro invocation
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:660: in this macro invocation
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:661: in this macro invocation
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:662: in this macro invocation
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:663: in this macro invocation
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:664: in this macro invocation
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:665: in this macro invocation
error: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\ser.rs:471: method `is_human_readable` is not a member of trait `serde::Serializer`
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\ser.rs:478: in this macro invocation
error: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\ser.rs:471: not a member of trait `serde::Serializer`
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\ser.rs:479: in this macro invocation
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\ser.rs:480: in this macro invocation
help: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\ser.rs:481: in this macro invocation
error: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\de.rs:388: no method named `is_human_readable` found for type `&T` in the current scope
error: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\erased-serde-0.3.3\src\ser.rs:322: no method named `is_human_readable` found for type `&T` in the current scope
error: Could not compile `erased-serde`.
I'm using
serde = "1.0"
serde_derive = "1.0"
erased-serde = "0.3"
(Btw, I'm on nightly 03-06)
Any idea why it's not working? :)
As you can see from the code below, I only have access to the type that needs to be serialized when I create a new instance of new, so I need to store the serialize_fn:: inside the struct to call it later on.
struct Foo
{
serialize_fn: fn(value: *mut u8) -> Result<erased_serde::Ok, erased_serde::Error> //I can't do this because Ok is private
}
impl Foo
{
fn new<T: Serialize>() -> Self
{
fn serializer_fn<T: Serialize>(value: *mut u8, serializer: &mut dyn erased_serde::Serializer)
{
let value = value.cast::<T>();
let value = unsafe { &*value };
let result = value.serialize(serializer);
}
Self {
serialize_fn: serializer_fn::<T>
}
}
pub fn serialize<S: Serializer>(&self, serializer: S, value: *mut u8) -> Option<Result<S::Ok, S::Error>>
where <S as serde::Serializer>::Ok: 'static
{
let mut serializer = <dyn erased_serde::Serializer>::erase(serializer);
let result = (serialize_fn)(value, serializer);
//Here I need to map result to the Result<S::Ok, S::Error> (which is basically what the erased_serde::serialize function does under the hood)
//But I can't do it because erased_serde::ser::Ok (and also erased_serde::any::Any) are private
/*
This is the code that would work
unsafe {
result
.unsafe_map(Ok::take)
.map_err(unerase)
}
*/
}
}
The problem is that I can't do that because erased_serde::Ok (as you can read from the code) is private in the erased-serde crate.
Can you please make it public (since it's the struct is obtainable in public API as you can see with my code, but it becomes unusable since it's private)?
Thanks :D
Would be very nice to use latest available version.
I want to create a HashMap<&'static str, Box<dyn Serialize>>
and be able to clone it. It seems like this might be possible via https://github.com/dtolnay/dyn-clone.
This crate is completely broken.
Consider the following code:
let options = bincode::DefaultOptions::new()
.with_fixint_encoding()
.allow_trailing_bytes();
let mut v: Vec<u8> = Vec::new();
let serializer = bincode::Serializer::new(v, options);
let serializer = erased_serde::Serializer::erase(serializer);
This results in 2 errors:
error[E0277]: the trait bound `bincode::Serializer<Vec<u8>, WithOtherTrailing<WithOtherIntEncoding<DefaultOptions, FixintEncoding>, AllowTrailing>>: erased_serde::private::serde::Serializer` is not satisfied
error[E0782]: trait objects must include the `dyn` keyword
The first error is very confusing and points to a module named private
which does not exist in the docs.
I'm sorry if this may be a duplicate, but I've gone down a bit of a rabbit hole trying to get this to work.
I have a trait Subscriber
which I need to use as a trait object (Box<dyn Subscriber>
), but I also require that it implements Serialize + DeserializeOwned
from serde.
pub trait Subscriber {
fn enabled(&self) -> bool;
}
I've tried adding where Self: erased_serde::Serialize
, though it seems like this crate doesn't provide a Deserialize
trait?
So I attempted to implement it manually:
impl<'de> Deserialize<'de> for Box<dyn Subscriber> {
// ...
}
Though I'm not sure what I could put for the implementation.
After struggling, I came across the typetag
crate, but sadly my target is wasm and I am having issues with ctor
: mmastrac/rust-ctor#14
use erased_serde::serialize_trait_object;
use serde::Serialize;
pub trait MyTrait {}
serialize_trait_object!(MyTrait);
impl MyTrait for i32 {}
fn main() {
let trait_object: &dyn MyTrait = &0i32;
let _ = Serialize::serialize(trait_object, serde_json::value::Serializer);
}
$ cargo run
thread 'main' has overflowed its stack
fatal runtime error: stack overflow
Aborted (core dumped)
The serialize_trait_object!
call is expanding to something like:
impl<'erased> ::serde::Serialize for dyn MyTrait + 'erased {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: ::serde::Serializer,
{
::erased_serde::serialize(self, serializer)
}
}
which calls https://docs.rs/erased-serde/0.3.18/erased_serde/fn.serialize.html. The first argument of that function call is constrained by a trait bound T: erased_serde::Serialize
. The serialize_trait_object!
macro is counting on dyn MyTrait
to have a compiler-generated impl erased_serde::Serialize for dyn MyTrait
because it's expecting trait MyTrait: erased_serde::Serialize
at the trait definition.
However, in the case that the caller forgot their erased_serde::Serialize supertrait, the impl being selected instead is erased-serde's impl<T> erased_serde::Serialize for T where T: ?Sized + serde::Serialize
, which involves calling the serde Serialize impl. But the serde Serialize impl in this case is what's shown above, impl<'erased> serde::Serialize for dyn MyTrait + 'erased
, which involves calling the erased-serde impl. Thus cycle and stack overflow.
The fact that there are 2 potentially applicable impls of erased_serde::Serialize
for dyn MyTrait
(one compiler-generated and one handwritten blanket impl in the erased-serde crate) is related to the soundness issue rust-lang/rust#57893 but is not unsound in our case because erased_serde::Serialize has no associated types.
The fix will be for serialize_trait_object!
to verify that the given trait has the required erased_serde::Serialize
supertrait, for example by doing:
fn require<T: ?Sized + erased_serde::Serialize>() {}
fn check<T: ?Sized + MyTrait>() { require::<T>() }
Hi! I've been using erased-serde
as the basis for some experiments for structured logging in log
. It would be great if we could support object-safe serde
in some no_std
environments.
I hacked up a quick proof-of-concept that relies on at least alloc
first, just to see what it might look like.
I also came across the intermediary trait solution which offered support with no extra assumptions. I haven't given it a lot of thought yet, but I guess the ship has sailed on that exact proposal now that we have a stable release of serde
?
Would you be interested in supporting an alloc
feature in erased-serde
like serde
does? Or would you perhaps rather punt the issue and let the portability lint + unified std
deal with portability?
It seems the deserializer gets confused about types of enums. This code demonstrates the problem. Normal deserialization (in the normal
test) works well, and deserializing some special types (like IpAddr
) also works. But when I try to deserialize an enum from my code, it fails with the error Error { msg: "invalid type: enum, expected enum X" }
extern crate serde;
#[macro_use]
extern crate serde_derive;
extern crate serde_json;
extern crate erased_serde;
#[derive(Debug, Eq, PartialEq, Serialize, Deserialize)]
enum X {
A,
B,
}
#[cfg(test)]
mod tests {
use std::net::IpAddr;
use serde::Deserialize;
use super::*;
#[test]
fn normal() {
let mut deserializer = serde_json::Deserializer::from_str("\"A\"");
assert_eq!(X::A, X::deserialize(&mut deserializer).unwrap());
}
#[test]
fn erased() {
let mut deserializer = serde_json::Deserializer::from_str("\"A\"");
let mut erased = erased_serde::Deserializer::erase(&mut deserializer);
assert_eq!(X::A, erased_serde::deserialize::<X>(&mut erased).unwrap());
}
#[test]
fn works() {
let mut deserializer = serde_json::Deserializer::from_str("\"192.0.2.1\"");
let mut erased = erased_serde::Deserializer::erase(&mut deserializer);
let expected = "192.0.2.1".parse::<IpAddr>().unwrap();
assert_eq!(expected, erased_serde::deserialize::<IpAddr>(&mut erased).unwrap());
}
}
#[derive(Clone)]
pub enum Codecs {
BinCodec(BinCodec),
JsonCodec(JsonCodec),
/// custom Serializer,Deserializer use erased_serde(https://github.com/dtolnay/erased-serde)
Custom((Arc<fn(arg: &mut[u8]) -> Box<dyn erased_serde::Serializer>>, Arc<fn(arg: &[u8]) -> Box<dyn erased_serde::Deserializer>>)),
}
impl Codecs{
pub fn custom(s:fn(arg: &mut [u8]) -> Box<dyn erased_serde::Serializer>,d:fn(arg: &[u8]) -> Box<dyn erased_serde::Deserializer>)->Self{
Self::Custom((Arc::new(s),Arc::new(d)))
}
}
0 | Box::new(<dyn erased_serde::Serializer>::erase(&mut serde_json::Serializer::new(arg)))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= note: expected `Box<(dyn erased_serde::Serializer + 'static)>`
found `Box<dyn erased_serde::Serializer>`
Hello,
Is it possible to have the serialized information be written to a string? I have been playing around with the library using a Cursor<Vec> instead of io::stdout() instead but I couldn't get it to compile (I am fairly new to rust).
If so could you please provide an example?
Thanks,
Tom
Hello!
I have just made a fork of this project, the sole difference being that it makes public the erased version of the serde helper traits (specifically, I needed MapAccess). I can walk you through an API where that is useful if that helps, but it would be nice if these were made public so I didn't have to maintain a fork (even if it's a pretty silly fork).
A feature flag wouldn't be something I'd mind ofc either!
I try it:
pub trait IMessage<'a>:Serialize + Deserialize<'a>{
}
serialize_trait_object!(<'a> IMessage<'a>);
but I get error messgae: IMessage cannot be made into an object
Use case is I have something like this:
async fn foo<A: Serialize, B: DeserializeOwned>(args:A) -> B { ... }
and I want to store it as a callback to be called later, i.e. something that looks like:
struct Callbacks {
pub foo: Box<dyn Fn(Box<dyn erased_serde::Serialize>) -> Pin<Box<dyn Future<Output=Box<dyn erased_serde::DeserializeOwned>>>>>
}
However, erased_serde doesn't have DeserializeOwned.
Is it possible to add this?
I also brought this up in the Rust forum, in case the surrounding context is helpful in some way: https://users.rust-lang.org/t/how-to-store-callbacks-with-generics/78063/12?u=dakom
Hi, I have an application with a map of trait objects and was wondering how you'd use this crate to deserialize them. Here is a simplified problem.
#[macro_use]
extern crate serde_derive;
#[macro_use]
extern crate erased_serde;
extern crate serde;
extern crate serde_json;
use std::fmt::Debug;
#[derive(Serialize, Deserialize, Debug)]
struct A;
impl Foo for A{}
#[derive(Serialize, Deserialize, Debug)]
struct B;
impl Foo for B{}
trait Foo: erased_serde::Serialize + Debug {}
serialize_trait_object!(Foo);
fn main() {
let v: Vec<Box<dyn Foo>> = vec![Box::new(A), Box::new(B), Box::new(A)];
let s = serde_json::to_string(&v).unwrap();
let deser = &mut serde_json::de::Deserializer::from_str(&s);
let mut json = erased_serde::Deserializer::erase(deser);
let w: Vec<Box<dyn Foo>> = erased_serde::deserialize(&mut json).unwrap();
println!("v: {:?}", v);
println!("w: {:?}", w);
}
If I replace w: Vec<Box<dyn Foo>>
with w: Vec<Box<A>>
it compiles and runs but is obviously wrong. Trying to add serde::Deserialize
to Foo
leads to lifetimes hell and the inability to use trait objects because
error[E0038]: the trait
Foo
cannot be made into an object
--> src/main.rs:20:1
|
20 | serialize_trait_object!(<'d>Foo<'d>);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the traitFoo
cannot be made into an object
|
= note: the trait cannot require thatSelf : Sized
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with > -Z external-macro-backtrace for more info)
How would you recommend I implement saving for a container of (among other things) boxed traits?
It would be super-handy if you could publish latest commit as a new version on crates.io.
Thanks in advance!
The documentation at https://docs.rs/erased-serde is not useful. Let's take the examples from the readme and include them in crate-level documentation.
The code you wrote in #33 (comment) is very generic, why not include it in this crate? So that users no longer copy this code to each of their own crates, and erased-serde
can also provide traits that are consistent with serde
.
Looks like 0b99efe
added the fn __check_erased_serialize_supertrait<__T, $($generics)*>()
so that if generics contains a lifetime generic, __T
would be declared before the lifetime, so that it would no longer compile.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.