nox / serde_urlencoded Goto Github PK
View Code? Open in Web Editor NEWx-www-form-urlencoded meets Serde
License: Apache License 2.0
x-www-form-urlencoded meets Serde
License: Apache License 2.0
binary data would be represented in rust as Vec<u8>
or &[u8]
, but this cannot be serialized (with error Custom("unsupported value")
). Related to #46, but i do think this is in scope of this library as it does not include any nested values or sequences but a single binary string
example:
binary hash (displayed in hex): 7736da1861d2d31ac2abc41ccde47379c7a678d2
in url encoding: %77%36%da%18%61%d2%d3%1a%c2%ab%c4%1c%cd%e4%73%79%c7%a6%78%d2
I was attempting to serialize the following struct to send in query params with reqwest:
#[derive(Serialize, Debug)]
struct Params {
pub urls: Vec<String>,
pub return_id: String,
}
And it returned this error:
ReqwestError(
Inner {
kind: UrlEncoded(
Custom(
"unsupported value"
)
),
url: None
}
)
The use case is when an api allows a list of query params formatted like urls[]=...&urls[]=...
. I was able to work around it by using a top level list of two-tuples instead of the struct, but support for serializing this would be nice.
This is not very ergonomic, because an empty string is exactly what a browser will send when an input field is left empty by the user. This means that the only way to have an optional number field is therefore to have client side Javascript to remove optional fields that stay empty.
Right now it's two hops to see API documentation -- the link to crates.io and click Documentation. It would be more convenient to expose that directly from the readme.
thread 'test::asd' panicked at 'called `Result::unwrap()` on an `Err` value: Custom("unsupported value")', asd/src/qwe.rs:32:57
This doesn't say on which value/field the error happened. Could we maybe get the info about the location of the error?
#[macro_use]
extern crate serde_derive;
extern crate serde_urlencoded;
#[derive(Deserialize, Serialize, Debug)]
struct S {
s: Option<String>,
}
fn main() {
println!("{:?}", serde_urlencoded::from_str::<S>("s=abc"));
}
The URL crate is a dependency of this crate, and it has its own dependencies which are unrelated to this crate, from a quick look. Maybe we should consider moving the actually required code to this crate, and drop the dependency?
https://docs.rs/serde_urlencoded/0.5.3/serde_urlencoded/ needs a more helpful landing page.
Consider
#[derive(Serialize)]
struct Foo {
field: Vec<String>,
}
or
let foo = [
("key", vec!["hello", "world"]),
]
or the HashMap
-equivalent of the above. These all currently throw "unsupported value" errors. It would be extremely nice to have this supported, and serialized into key[0]=hello&key[1]=world
.
This would be appreciated for cargo update -Z minimal-versions
builds: rust-lang/cargo#5657. The serde version dependency in a crate is invalid. All 0.7.1 did change is correcting the required version of serde
dependency, so there shouldn't be any concerns about breaking compatibility, and if somebody already has this crate in Cargo.lock it will continue to work.
Currently serde_urlencoded
only supports top-level serialization. An attempt to serialize a new type wrapping a float will fail with "top-level serializer supports only maps and structs".
In some argument signing cases, we want to know the exact serialization result for a particular field serde_urlencoded
has done. Maybe we can have some util to expose these serializers?
It works when all the fields in the enum are String, but failed in other cases.
use serde::Deserialize; // 1.0.104
use serde_urlencoded; // 0.6.1
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum Q1 {
A { foo: String },
B { bar: String },
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum Q2 {
A { foo: String },
B { bar: i32 },
}
fn main() {
let q1 = "bar=123";
let q1s = serde_urlencoded::from_str::<Q1>(q1).unwrap();
println!("q1: {:#?}", q1s);
let q2 = "bar=123";
let q2s = serde_urlencoded::from_str::<Q2>(q2).unwrap();
println!("q2: {:#?}", q2s);
}
Hi,
I was playing around with adding support of nested structs. I hacked together a basic implementation building off the existing code which supports structs like this:
struct Foo { bar: Bar, baz: Baz }
struct Bar { x: u8, y: String }
struct Baz { thing: String, other: u8 }
let foo = Foo {
bar: Bar { x: 10, y: "Ten".to_owned() },
baz: Baz { thing: "Thing".to_owned(), other: 12 }
};
Then
serde_urlencoded::to_string(&foo)
Outputs:
bar[x]=10&bar[y]=Ten&baz[thing]=Thing&baz[other]=12
(although with brackets urlencoded as %5B
, %5D
).
And similarly, that string (with or without using %5B
/%5D
serializes back to the expected struct.
Is this something which people want to see added to this crate? If so, I can clean up the code and submit a PR. Probably would be hidden behind a feature flag, since it is almost certainly going to be slower than the existing method and not formally supported.
Fresh serde_urlencoded v0.7.0
warning: use of deprecated associated function `std::error::Error::description`: use the Display impl or to_string()
--> C:\Users\dangu\.cargo\registry\src\github.com-1ecc6299db9ec823\serde_urlencoded-0.7.0\src\ser\mod.rs:80:37
|
80 | Error::Utf8(ref err) => error::Error::description(err),
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(deprecated)]` on by default
warning: 1 warning emitted
Hi all, I looked a little bit into the docs so I'm not sure if this behavior is currently supported, please correct me if so.
Basically what I'm looking to do is serialize a nested struct
into a query string, i.e. where the value is like json
data essentially.
In my particular use case, here are the API docs which I'm currently referencing:
https://support.monday.com/hc/en-us/articles/4406042650002-Audit-Log-API
As you can see from the docs, the query param fields like start_time
and end_time
would need to be json encoded into the query string, so for a specific example they had:
?page=1&per_page=20&filters={"start_time":"2021-08-24T17:03:00Z"}
To achieve this, I tried to use a nested struct which was defined as below:
/// Get Audit Logs API - max. results per page (default)
const MAX_PER_PAGE: u16 = 1000;
/// Query Parameters to be used in the request for the *Audit Log API*.
#[derive(Debug, Serialize, Deserialize)]
pub struct AuditLogParams<'a> {
pub page: u16,
pub per_page: u16,
#[serde(borrow)]
// Optional
pub filters: Option<FilterParams<'a>>,
}
#[derive(Default, Debug, Serialize, Deserialize)]
pub struct FilterParams<'a> {
#[serde(skip_serializing_if = "str::is_empty")]
pub start_time: &'a str,
#[serde(skip_serializing_if = "str::is_empty")]
pub end_time: &'a str,
}
impl<'a> Default for AuditLogParams<'a> {
fn default() -> Self {
Self {
page: 1,
per_page: MAX_PER_PAGE,
filters: None,
}
}
}
impl<'a> From<FilterParams<'a>> for AuditLogParams<'a> {
fn from(filters: FilterParams<'a>) -> Self {
Self {
filters: Some(filters),
..Default::default()
}
}
}
However, when I serialize a nested struct data with the filters
field correctly populated, and using below syntax with the reqwest
library:
let filters = FilterParams {
start_time: "2021-08-24T17:03:00Z",
..Default::default()
};
let params = AuditLogParams::from(filters);
let r = self.client.get(GET_LOGS_URL).query(¶ms).send().await?;
I am receiving an builder error: unsupported value
error at runtime, from the console logs at least. So I was curious if there was a best practice or recommended approach to building a query parameters string in the above format.
#91 added serializing, but not deserializing.
There has been some discussion in the servo/rust-url library about serialization of arbitrary byte arrays, but it was rejected from their main Serializer. Currently, this library doesn't support serialization of arbitrary byte arrays either due to the above choice. However, this support could be added as rust-url does supply convenience functions to serialize byte arrays.
I was wondering if support for this could be considered. My specific use case is writing a torrent client, which requires the underlying bytes of a SHA1 hash to be urlencoded as a query parameter.
project_id=5
Does not match any variant of untagged enum
#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(untagged)]
pub enum PipelineRefFilter {
ProjectDefault { project_id: i64 },
ExperimentDefault { experiment_id: i64 },
Measurement { plate_id: i64, meas_idx: i32 },
}
Edit: Sorry, just saw the other issue.
I want to just urlencode a String, not a key value pair, but got the error:
SerdeUrlencodedSer(Custom("top-level serializer supports only maps and structs"))
Is it a good idea to support raw string urlencode and urldecode?
In case of our application we need to pass bounded ranges in query string of our list requests
?time=[1551067000, 155106800)
struct Qs {
time: (Bound<DateTime<Utc>>, Bound<DateTime<Utc>),
}
I can also imagine situations where passing tuples and lists would be useful
?colors=[red, green, blue]
struct Qs {
colors: Vec<String>,
}
?color=[100, 100, 100]
struct Qs {
color: Color,
}
struct Color {
red: i8,
green: i8,
blue: i8,
}
There is no RFC for such a string representation, only practical sense. Will you accept the format?
The following example features an enum parameter. Such an Enum would constrain acceptable values passed for a parameter within the query string and so would be a valuable addition to serde_urlencoded. Currently, however, serde_urlencoded raises the following error: Err
value: Error { err: "invalid type: string
"brown", expected enum HairColor" }',
I've also tried declaring a serde tag attribute on the enum, #[serde(untagged)], but that didn't help.
Am I doing this wrong?
extern crate serde_urlencoded;
#[macro_use] extern crate serde_derive;
extern crate serde;
#[derive(Deserialize)]
enum HairColor {
brown,
black,
}
#[derive(Deserialize)]
struct MyParams {
name: String,
age: u32,
hair_color: HairColor
}
fn main() {
let params = "name=Darin&age=38&hair_color=brown";
let decoded = serde_urlencoded::from_str::<MyParams>(¶ms).unwrap();
}
Is there an example of how to pass a chrono::NaiveDateTime
in a querystring?
Here's a sample of what i'm struggling with
use serde_urlencoded; // 0.6.1
use chrono::{NaiveDate, NaiveDateTime}; // 0.4.13
fn main() {
// โ
Good
let qs = "foo=2015-09-05%2011%3A01%3A01";
let res = serde_urlencoded::from_str::<Vec<(String, String)>>(qs);
println!("{:?}", res);
// โ
Good
let qs = "foo=2020-01-01";
let res = serde_urlencoded::from_str::<Vec<(String, NaiveDate)>>(qs);
println!("{:?}", res);
// โ Bad
let qs = "foo=2015-09-05%2011%3A01%3A01";
let res = serde_urlencoded::from_str::<Vec<(String, NaiveDateTime)>>(qs);
println!("{:?}", res);
}
// Ok([("foo", "2015-09-05 11:01:01")])
// Ok([("foo", 2020-01-01)])
// Err(Error { err: "input contains invalid characters" })
tokio-postgres
SQL query for date filtering, where the column is timestamp
Something like (which might be absurd, but I'm curious if it's possible)
http://localhost:3000/users/a5f5d36a-6677-41c2-85b8-7578b4d98972/attempts?limit=15&offset=15&start=2020-08-09%2003%3A10%3A07%2E813517&end=2020-08-09%2003%3A10%3A07%2E813517
The "Getting help" section of the readme points to irc.mozilla.org, which no longer exists.
I expected #[serde(flatten)]
does not change the behaviour of internal struct, but it changes.
#[derive(Deserialize, Serialize, PartialEq, Debug)]
struct Paginate {
limit: u64,
offset: u64,
}
#[derive(Deserialize, Serialize, PartialEq, Debug)]
struct Query {
#[serde(flatten)]
paginate: Paginate
}
fn main() {
let query = "limit=10&offset=0";
println!("{:?}", from_str::<Paginate>(query));
// -> Ok(Paginate { limit: 10, offset: 0 })
println!("{:?}", from_str::<Query>(query));
// -> Err(Error { err: "invalid type: string \"10\", expected u64" })
println!("{:?}", to_string(Paginate {limit: 10, offset:0}));
// Ok("limit=10&offset=0")
println!("{:?}", to_string(Query {paginate: Paginate {limit: 10, offset:0}}));
// Ok("limit=10&offset=0")
}
I think this is a bug.
The stdx
crate depends on reqwest which depends on serde_urlencoded. We would like to use serde 0.9.0 ๐
Serde just released v1.0. Publishing an update of this library will enable dependent crates such as reqwest
to make the update as well.
Just so project using both serde_json
and serde_urlencoded
contain "only" two float formatting implementations rather than three (std, dtoa, ryu). serde_json
switched in 2018.
Currently, attempting to serialize a unit struct returns an error: "top-level serializer supports only maps and structs".
Since serialization of Option::None
is allowed, I would expect the same behavior for unit structs, ie returning empty pairs.
Would that change be acceptable ?
I need to serialize using serde_urlencoded::to_string()
a Vec<i64>
into name[]=val1&name[]=val2&...
.
It does not seems possible serde_urlencoded
... or am I missing something?
This issue was previously reported here : #14.
I still see this problem. I added a test function to test_deserialize.rs
. Perhaps i'm writing this incorrectly, see the function below, followed by the result of running the test.
#[test]
fn deserialize_newtype() {
#[derive(Serialize, Deserialize, Debug, PartialEq, Eq)]
struct TestNewType(u32);
let result = TestNewType(42);
assert_eq!(serde_urlencoded::from_str::<TestNewType>("testnewtype=42"), Ok(result));
}
The result is:
running 7 tests
test deserialize_bytes ... ok
test deserialize_option ... ok
test deserialize_reader ... ok
test deserialize_str ... ok
test deserialize_unit ... ok
test deserialize_unit_enum ... ok
test deserialize_newtype ... FAILED
failures:
---- deserialize_newtype stdout ----
thread 'deserialize_newtype' panicked at 'assertion failed: `(left == right)`
left: `Err(Error { err: "invalid type: map, expected tuple struct TestNewType" })`,
right: `Ok(TestNewType(42))`', tests/test_deserialize.rs:70:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.
failures:
deserialize_newtype
New type structs don't seem to work as intended.
#[derive(Debug, Deserialize, Serialize)]
pub struct RequestToken(String);
#[derive(Deserialize)]
pub struct OauthRequest {
#[serde(rename = "code")]
pub request_token: RequestToken,
}
Given some_data
has code=1d96c242-0e47-212b-a80e-560216
let request: OauthRequest = serde_urlencoded::from_reader(some_data)?;
fails with Error { err: "invalid type: string \"1d96c242-0e47-212b-a80e-560216\", expected tuple struct RequestToken" }
.
I'm using serde 0.9.7, serde_derive 0.9.7, serde_urlencoded 0.4.2 on Rust 1.15.1.
Hi! Thanks for this library! I am pleasantly surprised by everything it lets me do.
However, I have found a hitch. When deserializing an enum
, it seems that the type information is somehow lost, and everything falls back to being String
s. See this test:
#[test]
fn deserialize_i32_in_struct_in_enum() {
#[derive(Deserialize, Debug, PartialEq, Eq)]
struct ItemStruct {
field: i32
}
#[derive(Deserialize, Debug, PartialEq, Eq)]
#[serde(tag = "tag")]
enum Test {
Item(ItemStruct)
};
let result = Test::Item(ItemStruct{ field: 42 });
assert_eq!(serde_urlencoded::from_str("tag=Item&field=42"), Ok(result));
}
It currently fails with the run-time error invalid type: string "42", expected i32
:
---- deserialize_i32_in_struct_in_enum stdout ----
thread 'deserialize_i32_in_struct_in_enum' panicked at 'assertion failed: `(left == right)` (left: `Err(Error { err: "invalid type: string \"42\", expected i32" })`, right: `Ok(Item(ItemStruct { field: 42 }))`)', tests/test_deserialize.rs:79
note: Run with `RUST_BACKTRACE=1` for a backtrace.
I have committed this test along with a similar succeeding one with String
to my fork. (See also the entire patch)
I stumbled upon the same problem as #52, I wanted to deserialize a struct with a Vec
in it. I found it somewhat surprising that this is not possible, since the compiler accepted my code with a Vec
in a struct, and wish I had learned earlier that this is a design decision:
This is a crate for deserializing from query strings as specified by the URL Standard, nothing more, nothing less.
On deserializing query strings, whatwg says "Let output
be [a] list of name-value tuples where both name and value hold a string." Clearly, rust structs are not lists of tuples.
So what kind of structs can be serialized to?
I am using this via the Reqwests crate.
I have an endpoint which requires that I encode some flags as a key only with no value.
I can't figure out if there is a way to do this from a struct.
struct Query {
key_only: bool,
}
If key_only
is true I need the Query URL to be ?key_only
but right now I can only get key_only=true
.
I tried using a combination of skip_if
and unit types but unit types can't be serialized with this yet.
Is there currently a way to do this?
I would say this is likely related to #43
I'm currently working on a little addition to reqwest
and I've noticed this issue. For our usage case, it's possible to only want to serialize a simple tuple composed of a key and a value. I used the target provided the url
crate, via url::query_pairs_mut
for the serializer's target.
extern crate serde_urlencoded;
use serde_urlencoded::to_string;
fn main() {
println!("{:?}", to_string(("foo", "qux"))); // unsupported pair
println!("{:?}", to_string(&("foo", "qux"))); // unsupported pair
println!("{:?}", to_string([("foo", "qux")])); // ok
}
Output:
Err(Custom("unsupported pair"))
Err(Custom("unsupported pair"))
Ok("foo=qux")
Hi,
thanks for this crate. It's very helpful already. ๐
One thing I need the most right now are Option
types.
Here's some test code:
#![feature(plugin, custom_derive)]
#![plugin(serde_macros)]
extern crate serde;
extern crate serde_urlencoded;
#[derive(Debug, Serialize, Deserialize)]
struct Parameters {
channel: String,
last: Option<f64>,
ts: i64,
}
fn main() {
let params = Parameters {
channel: "mychannel".to_string(),
last: Some(1.234),
ts: 23,
};
println!("{:?}", serde_urlencoded::to_string(¶ms));
}
I've seen that the implementation should be at this position and I would like to offer my help.
What I would do is match on Option
and if it's Some(value)
I would self.append_pair(value)
.
On the other hand I'm not sure what I would do in case of None
. Maybe I should simply return?
Hello everyone,
Could we have a feature where from_str
returns values with no parsing ?
use std::collections::HashMap;
use serde_urlencoded::from_str; // 0.7.1
let query_string = "id=6%206&abc=123&def=2%252";
let query_string = query_string.replace("%", "%25");
let map = from_str::<HashMap<String, String>>(&query_string).unwrap();
println!("{:?}", map);
Which yield
{"id": "6%206", "def": "2%252", "abc": "123"}
I'm using the above code for now to escape '%' character
Sorry if issue isn't the best suited place.
Thanks
Edit: added output
Lots of them, to see whether what the documentation claims is actually true.
I'm trying to deserialize a struct that contains a &'a str
:
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_urlencoded; // 0.5.4;
#[derive(Deserialize)]
struct MyParams<'a> {
key: &'a str
}
fn main() -> Result<(), Box<std::error::Error>> {
println!("{:?}", "aaa");
let r: MyParams = serde_urlencoded::from_str("key=bb").unwrap();
println!("{:?}", r.key);
Ok(())
}
But the unwrap
fails.
Anyway if I convert the MyParams.key
to String
, the code works perfectly.
(serde_json works fine with &str).
Can I try to improve this lib?
Thanks
NVM.
It seems to me like the support for deserializing &str should probably be removed (always failing with a panic). Otherwise, whether it works or not is dependent on whether the value to be deserialized contains any special characters, which seems like a footgun. I'm not sure if this is a problem with the changes in #54, but it might be related. If someone wants to avoid this issue, they can put a Cow<'a, str>
in their types instead.
i got error, when try deserialize struct with another struct inside
#[macro_use] extern crate serde_derive;
extern crate serde_json;
extern crate serde_urlencoded;
#[derive(Serialize, Deserialize, Debug)]
pub struct Data {
inner: String,
inner_d: AnotherData,
}
#[derive(Serialize, Deserialize, Debug)]
pub struct AnotherData {
name: String,
phone: String,
}
fn main() {
let data = serde_urlencoded::from_bytes::<Data>(
b"inner=Helloe&inner_d[name]=hello&inner_d[phone]=1231313123"
);
println!("{:?}", data);
}
It would be nice to have support for arrays since url
crate already supports this.
Both cases will result in:
[
("key", "one"),
("key", "two"),
]
key=one&key=two
key[]=one&key[]=two
use url; // 2.1.1
fn main() {
let query = "key=one&key=two";
let parsed = url::form_urlencoded::parse(query.as_bytes());
let values = parsed.collect::<Vec<_>>();
dbg!(values);
}
use url; // 2.1.1
fn main() {
let query = "key[]=one&key[]=two";
let parsed = url::form_urlencoded::parse(query.as_bytes());
let values = parsed.collect::<Vec<_>>();
dbg!(values);
}
I ran into an issue with a parser that doesn't replace +
with spaces, it might be a good idea to have a to_string
function that accepts additional arguments regarding url encoding (like a config struct).
Thanks!
Provide support for unit type serialization.
It may looks like it doesn't make sense to serialize ()
. Unfortunately that kind of serialization may be necessary for generic structures where one of generic argument may be ()
.
Parsing (1-element) tuple structs into their inner type fail.
#[derive(Deserialize, Serialize, PartialEq, Debug)]
struct UserId(u32);
#[derive(Deserialize, Serialize, PartialEq, Debug)]
struct Query {
user_id: UserId,
}
fn main() {
let query = "user_id=1";
println!("{:?}", from_str::<Query>(query));
// Err(Error { err: "invalid type: string \"1\", expected tuple struct UserId" })
println!("{:?}", to_string(Query{user_id: UserId(1)}));
// Ok("user_id=1")
}
I hope it could do. FYI, serde_json can do it.
Hello!
I just started trying out warp, which deserializes querystrings via serde_urlencoded.
I have a use case for nested querystrings, and I've been experimenting using serde_qs.
Is there any interest in supporting nested querystrings in serde_urlencoded, or is that beyond the scope of this project?
Like serde_json, for example.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.