Parsing (1-element) tuple structs into their inner type fail.

#[derive(Deserialize, Serialize, PartialEq, Debug)]
struct UserId(u32);

#[derive(Deserialize, Serialize, PartialEq, Debug)]
struct Query {
    user_id: UserId,
}

fn main() {
    let query = "user_id=1";
    
    println!("{:?}", from_str::<Query>(query));
      // Err(Error { err: "invalid type: string \"1\", expected tuple struct UserId" })
    println!("{:?}", to_string(Query{user_id: UserId(1)}));
      // Ok("user_id=1")
}

I hope it could do. FYI, serde_json can do it.

There has been some discussion in the servo/rust-url library about serialization of arbitrary byte arrays, but it was rejected from their main Serializer. Currently, this library doesn't support serialization of arbitrary byte arrays either due to the above choice. However, this support could be added as rust-url does supply convenience functions to serialize byte arrays.

I was wondering if support for this could be considered. My specific use case is writing a torrent client, which requires the underlying bytes of a SHA1 hash to be urlencoded as a query parameter.

Lots of them, to see whether what the documentation claims is actually true.

Consider

#[derive(Serialize)]
struct Foo {
    field: Vec<String>,
}

or

let foo = [
    ("key", vec!["hello", "world"]),
]

or the HashMap-equivalent of the above. These all currently throw "unsupported value" errors. It would be extremely nice to have this supported, and serialized into key[0]=hello&key[1]=world.

Hi all, I looked a little bit into the docs so I'm not sure if this behavior is currently supported, please correct me if so.

Basically what I'm looking to do is serialize a nested struct into a query string, i.e. where the value is like json data essentially.

In my particular use case, here are the API docs which I'm currently referencing:
https://support.monday.com/hc/en-us/articles/4406042650002-Audit-Log-API

As you can see from the docs, the query param fields like start_time and end_time would need to be json encoded into the query string, so for a specific example they had:

?page=1&per_page=20&filters={"start_time":"2021-08-24T17:03:00Z"}

To achieve this, I tried to use a nested struct which was defined as below:

/// Get Audit Logs API - max. results per page (default)
const MAX_PER_PAGE: u16 = 1000;

/// Query Parameters to be used in the request for the *Audit Log API*.
#[derive(Debug, Serialize, Deserialize)]
pub struct AuditLogParams<'a> {
    pub page: u16,
    pub per_page: u16,
    #[serde(borrow)]
    // Optional
    pub filters: Option<FilterParams<'a>>,
}

#[derive(Default, Debug, Serialize, Deserialize)]
pub struct FilterParams<'a> {
    #[serde(skip_serializing_if = "str::is_empty")]
    pub start_time: &'a str,
    #[serde(skip_serializing_if = "str::is_empty")]
    pub end_time: &'a str,
}

impl<'a> Default for AuditLogParams<'a> {
    fn default() -> Self {
        Self {
            page: 1,
            per_page: MAX_PER_PAGE,
            filters: None,
        }
    }
}

impl<'a> From<FilterParams<'a>> for AuditLogParams<'a> {
    fn from(filters: FilterParams<'a>) -> Self {
        Self {
            filters: Some(filters),
            ..Default::default()
        }
    }
}

However, when I serialize a nested struct data with the filters field correctly populated, and using below syntax with the reqwest library:

let filters = FilterParams {
    start_time: "2021-08-24T17:03:00Z",
    ..Default::default()
};
let params = AuditLogParams::from(filters);

let r = self.client.get(GET_LOGS_URL).query(&params).send().await?;

I am receiving an builder error: unsupported value error at runtime, from the console logs at least. So I was curious if there was a best practice or recommended approach to building a query parameters string in the above format.

Hi,

I was playing around with adding support of nested structs. I hacked together a basic implementation building off the existing code which supports structs like this:

struct Foo { bar: Bar, baz: Baz }
struct Bar { x: u8, y: String }
struct Baz { thing: String, other: u8 }

let foo = Foo {
  bar: Bar { x: 10, y: "Ten".to_owned()  },
  baz: Baz { thing: "Thing".to_owned(), other: 12  }
};

Then

serde_urlencoded::to_string(&foo)

Outputs:

bar[x]=10&bar[y]=Ten&baz[thing]=Thing&baz[other]=12

(although with brackets urlencoded as %5B, %5D).

And similarly, that string (with or without using %5B/%5D serializes back to the expected struct.

Is this something which people want to see added to this crate? If so, I can clean up the code and submit a PR. Probably would be hidden behind a feature flag, since it is almost certainly going to be slower than the existing method and not formally supported.

The stdx crate depends on reqwest which depends on serde_urlencoded. We would like to use serde 0.9.0 🙏

Proposal

It would be nice to have support for arrays since url crate already supports this.

Cases

Both cases will result in:

[
    ("key", "one"),
    ("key", "two"),
]

• key=one&key=two
• key[]=one&key[]=two

Examples

use url; // 2.1.1

fn main() {
    let query = "key=one&key=two";
    let parsed = url::form_urlencoded::parse(query.as_bytes());
    
    let values = parsed.collect::<Vec<_>>();
    dbg!(values);
}

use url; // 2.1.1

fn main() {
    let query = "key[]=one&key[]=two";
    let parsed = url::form_urlencoded::parse(query.as_bytes());
    
    let values = parsed.collect::<Vec<_>>();
    dbg!(values);
}

Just so project using both serde_json and serde_urlencoded contain "only" two float formatting implementations rather than three (std, dtoa, ryu). serde_json switched in 2018.

The "Getting help" section of the readme points to irc.mozilla.org, which no longer exists.

This is not very ergonomic, because an empty string is exactly what a browser will send when an input field is left empty by the user. This means that the only way to have an optional number field is therefore to have client side Javascript to remove optional fields that stay empty.

I'm trying to deserialize a struct that contains a &'a str:

#[macro_use]
extern crate serde_derive;

extern crate serde;
extern crate serde_urlencoded; // 0.5.4;

#[derive(Deserialize)]
struct MyParams<'a> {
    key: &'a str
}

fn main() -> Result<(), Box<std::error::Error>> {
    println!("{:?}", "aaa");
    let r: MyParams = serde_urlencoded::from_str("key=bb").unwrap();
    println!("{:?}", r.key);
    Ok(())
}

But the unwrap fails.
Anyway if I convert the MyParams.key to String, the code works perfectly.

(serde_json works fine with &str).

Can I try to improve this lib?

Thanks

I was attempting to serialize the following struct to send in query params with reqwest:

#[derive(Serialize, Debug)]
struct Params {
    pub urls: Vec<String>,
    pub return_id: String,
}

And it returned this error:

    ReqwestError(
        Inner {
            kind: UrlEncoded(
                Custom(
                    "unsupported value"
                )
            ),
            url: None
        }
    )

The use case is when an api allows a list of query params formatted like urls[]=...&urls[]=.... I was able to work around it by using a top level list of two-tuples instead of the struct, but support for serializing this would be nice.

Currently, attempting to serialize a unit struct returns an error: "top-level serializer supports only maps and structs".

Since serialization of Option::None is allowed, I would expect the same behavior for unit structs, ie returning empty pairs.

Would that change be acceptable ?

Description

Is there an example of how to pass a chrono::NaiveDateTime in a querystring?

Here's a sample of what i'm struggling with

Playground

use serde_urlencoded; // 0.6.1
use chrono::{NaiveDate, NaiveDateTime}; // 0.4.13

fn main() {
    // ✅ Good
    let qs = "foo=2015-09-05%2011%3A01%3A01";
    let res = serde_urlencoded::from_str::<Vec<(String, String)>>(qs);
    println!("{:?}", res);

    // ✅ Good
    let qs = "foo=2020-01-01";
    let res = serde_urlencoded::from_str::<Vec<(String, NaiveDate)>>(qs);
    println!("{:?}", res);
    
    // ❌ Bad
    let qs = "foo=2015-09-05%2011%3A01%3A01";
    let res = serde_urlencoded::from_str::<Vec<(String, NaiveDateTime)>>(qs);
    println!("{:?}", res);
}

// Ok([("foo", "2015-09-05 11:01:01")])
// Ok([("foo", 2020-01-01)])
// Err(Error { err: "input contains invalid characters" })

End goal

• Specify start/end dates in a querystring
• Parse and deserialize them
• Use them in a tokio-postgres SQL query for date filtering, where the column is timestamp

Something like (which might be absurd, but I'm curious if it's possible)
http://localhost:3000/users/a5f5d36a-6677-41c2-85b8-7578b4d98972/attempts?limit=15&offset=15&start=2020-08-09%2003%3A10%3A07%2E813517&end=2020-08-09%2003%3A10%3A07%2E813517

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=395269729d2b25cfb7648717dbb25299

project_id=5

Does not match any variant of untagged enum

#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(untagged)]
pub enum PipelineRefFilter {
    ProjectDefault { project_id: i64 },
    ExperimentDefault { experiment_id: i64 },
    Measurement { plate_id: i64, meas_idx: i32 },
}

Edit: Sorry, just saw the other issue.

The following example features an enum parameter. Such an Enum would constrain acceptable values passed for a parameter within the query string and so would be a valuable addition to serde_urlencoded. Currently, however, serde_urlencoded raises the following error: Err value: Error { err: "invalid type: string
"brown", expected enum HairColor" }',

I've also tried declaring a serde tag attribute on the enum, #[serde(untagged)], but that didn't help.

Am I doing this wrong?

extern crate serde_urlencoded;
#[macro_use] extern crate serde_derive;
extern crate serde;


#[derive(Deserialize)]
enum HairColor {
    brown,
    black,
}


#[derive(Deserialize)]
struct MyParams {
    name: String,
    age: u32,
    hair_color: HairColor
}


fn main() {
    let params = "name=Darin&age=38&hair_color=brown";
    let decoded = serde_urlencoded::from_str::<MyParams>(&params).unwrap();
}

NVM.

This would be appreciated for cargo update -Z minimal-versions builds: rust-lang/cargo#5657. The serde version dependency in a crate is invalid. All 0.7.1 did change is correcting the required version of serde dependency, so there shouldn't be any concerns about breaking compatibility, and if somebody already has this crate in Cargo.lock it will continue to work.

I stumbled upon the same problem as #52, I wanted to deserialize a struct with a Vec in it. I found it somewhat surprising that this is not possible, since the compiler accepted my code with a Vec in a struct, and wish I had learned earlier that this is a design decision:

This is a crate for deserializing from query strings as specified by the URL Standard, nothing more, nothing less.

On deserializing query strings, whatwg says "Let output be [a] list of name-value tuples where both name and value hold a string." Clearly, rust structs are not lists of tuples.

So what kind of structs can be serialized to?

The URL crate is a dependency of this crate, and it has its own dependencies which are unrelated to this crate, from a quick look. Maybe we should consider moving the actually required code to this crate, and drop the dependency?

This issue was previously reported here : #14.

I still see this problem. I added a test function to test_deserialize.rs. Perhaps i'm writing this incorrectly, see the function below, followed by the result of running the test.

#[test]
fn deserialize_newtype() {
    #[derive(Serialize, Deserialize, Debug, PartialEq, Eq)]
    struct TestNewType(u32);
    let result = TestNewType(42);

    assert_eq!(serde_urlencoded::from_str::<TestNewType>("testnewtype=42"), Ok(result));
}

The result is:

running 7 tests
test deserialize_bytes ... ok
test deserialize_option ... ok
test deserialize_reader ... ok
test deserialize_str ... ok
test deserialize_unit ... ok
test deserialize_unit_enum ... ok
test deserialize_newtype ... FAILED

failures:

---- deserialize_newtype stdout ----
thread 'deserialize_newtype' panicked at 'assertion failed: `(left == right)`
  left: `Err(Error { err: "invalid type: map, expected tuple struct TestNewType" })`,
 right: `Ok(TestNewType(42))`', tests/test_deserialize.rs:70:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.


failures:
    deserialize_newtype

I expected #[serde(flatten)] does not change the behaviour of internal struct, but it changes.

#[derive(Deserialize, Serialize, PartialEq, Debug)]
struct Paginate {
    limit: u64,
    offset: u64,
}

#[derive(Deserialize, Serialize, PartialEq, Debug)]
struct Query {
    #[serde(flatten)]
    paginate: Paginate
}

fn main() {
    let query = "limit=10&offset=0";
    
    println!("{:?}", from_str::<Paginate>(query));
      // -> Ok(Paginate { limit: 10, offset: 0 })
   println!("{:?}", from_str::<Query>(query));
      // -> Err(Error { err: "invalid type: string \"10\", expected u64" })
    println!("{:?}", to_string(Paginate {limit: 10, offset:0}));
      // Ok("limit=10&offset=0")
    println!("{:?}", to_string(Query {paginate: Paginate {limit: 10, offset:0}}));
      // Ok("limit=10&offset=0")
}

I think this is a bug.

Currently serde_urlencoded only supports top-level serialization. An attempt to serialize a new type wrapping a float will fail with "top-level serializer supports only maps and structs".

In some argument signing cases, we want to know the exact serialization result for a particular field serde_urlencoded has done. Maybe we can have some util to expose these serializers?

It works when all the fields in the enum are String, but failed in other cases.

use serde::Deserialize; // 1.0.104
use serde_urlencoded; // 0.6.1

#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum Q1 {
    A { foo: String },
    B { bar: String },
}

#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum Q2 {
    A { foo: String },
    B { bar: i32 },
}

fn main() {
    let q1 = "bar=123";
    let q1s = serde_urlencoded::from_str::<Q1>(q1).unwrap();
    println!("q1: {:#?}", q1s);

    let q2 = "bar=123";
    let q2s = serde_urlencoded::from_str::<Q2>(q2).unwrap();
    println!("q2: {:#?}", q2s);
}

playground

In case of our application we need to pass bounded ranges in query string of our list requests

?time=[1551067000, 155106800)

struct Qs {
  time: (Bound<DateTime<Utc>>, Bound<DateTime<Utc>),
}

I can also imagine situations where passing tuples and lists would be useful

?colors=[red, green, blue]

struct Qs {
  colors: Vec<String>,
}

?color=[100, 100, 100]

struct Qs {
  color: Color,
}
struct Color {
  red: i8,
  green: i8,
  blue: i8,
}

There is no RFC for such a string representation, only practical sense. Will you accept the format?

I need to serialize using serde_urlencoded::to_string() a Vec<i64> into name[]=val1&name[]=val2&....

It does not seems possible serde_urlencoded... or am I missing something?

I want to just urlencode a String, not a key value pair, but got the error:

 SerdeUrlencodedSer(Custom("top-level serializer supports only maps and structs"))

Is it a good idea to support raw string urlencode and urldecode?

New type structs don't seem to work as intended.

#[derive(Debug, Deserialize, Serialize)]
pub struct RequestToken(String);

#[derive(Deserialize)]
pub struct OauthRequest {
    #[serde(rename = "code")]
    pub request_token: RequestToken,
}

Given some_data has code=1d96c242-0e47-212b-a80e-560216

let request: OauthRequest = serde_urlencoded::from_reader(some_data)?;

fails with Error { err: "invalid type: string \"1d96c242-0e47-212b-a80e-560216\", expected tuple struct RequestToken" }.

I'm using serde 0.9.7, serde_derive 0.9.7, serde_urlencoded 0.4.2 on Rust 1.15.1.

In particular because of #74, #79.

Serde just released v1.0. Publishing an update of this library will enable dependent crates such as reqwest to make the update as well.

Provide support for unit type serialization.

It may looks like it doesn't make sense to serialize (). Unfortunately that kind of serialization may be necessary for generic structures where one of generic argument may be ().

Hello everyone,

Could we have a feature where from_str returns values with no parsing ?

use std::collections::HashMap;
use serde_urlencoded::from_str; // 0.7.1

let query_string = "id=6%206&abc=123&def=2%252";
let query_string = query_string.replace("%", "%25");

let map = from_str::<HashMap<String, String>>(&query_string).unwrap();

println!("{:?}", map);

Which yield

{"id": "6%206", "def": "2%252", "abc": "123"}

I'm using the above code for now to escape '%' character

Sorry if issue isn't the best suited place.

Thanks

Edit: added output

I'm currently working on a little addition to reqwest and I've noticed this issue. For our usage case, it's possible to only want to serialize a simple tuple composed of a key and a value. I used the target provided the url crate, via url::query_pairs_mut for the serializer's target.

extern crate serde_urlencoded;

use serde_urlencoded::to_string;

fn main() {
    println!("{:?}", to_string(("foo", "qux"))); // unsupported pair
    println!("{:?}", to_string(&("foo", "qux"))); // unsupported pair
    println!("{:?}", to_string([("foo", "qux")])); // ok
}

Output:

Err(Custom("unsupported pair"))
Err(Custom("unsupported pair"))
Ok("foo=qux")

Like serde_json, for example.

i got error, when try deserialize struct with another struct inside

#[macro_use] extern crate serde_derive;
extern crate serde_json;
extern crate serde_urlencoded;

#[derive(Serialize, Deserialize, Debug)]
pub struct Data {
    inner: String,
    inner_d: AnotherData,
}

#[derive(Serialize, Deserialize, Debug)]
pub struct AnotherData {
    name: String,
    phone: String,
}

fn main() {

let data = serde_urlencoded::from_bytes::<Data>(
    b"inner=Helloe&inner_d[name]=hello&inner_d[phone]=1231313123"
);
println!("{:?}", data);

}

I ran into an issue with a parser that doesn't replace + with spaces, it might be a good idea to have a to_string function that accepts additional arguments regarding url encoding (like a config struct).

Thanks!

I am using this via the Reqwests crate.

I have an endpoint which requires that I encode some flags as a key only with no value.
I can't figure out if there is a way to do this from a struct.

struct Query {
    key_only: bool,
}

If key_only is true I need the Query URL to be ?key_only but right now I can only get key_only=true.

I tried using a combination of skip_if and unit types but unit types can't be serialized with this yet.

Is there currently a way to do this?
I would say this is likely related to #43

Hi! Thanks for this library! I am pleasantly surprised by everything it lets me do.

However, I have found a hitch. When deserializing an enum, it seems that the type information is somehow lost, and everything falls back to being Strings. See this test:

#[test]
fn deserialize_i32_in_struct_in_enum() {
    #[derive(Deserialize, Debug, PartialEq, Eq)]
    struct ItemStruct {
        field: i32
    }

    #[derive(Deserialize, Debug, PartialEq, Eq)]
    #[serde(tag = "tag")]
    enum Test {
        Item(ItemStruct)
    };

    let result = Test::Item(ItemStruct{ field: 42 });

    assert_eq!(serde_urlencoded::from_str("tag=Item&field=42"), Ok(result));
}

It currently fails with the run-time error invalid type: string "42", expected i32:

---- deserialize_i32_in_struct_in_enum stdout ----
        thread 'deserialize_i32_in_struct_in_enum' panicked at 'assertion failed: `(left == right)` (left: `Err(Error { err: "invalid type: string \"42\", expected i32" })`, right: `Ok(Item(ItemStruct { field: 42 }))`)', tests/test_deserialize.rs:79
note: Run with `RUST_BACKTRACE=1` for a backtrace.

I have committed this test along with a similar succeeding one with String to my fork. (See also the entire patch)

Hello!

I just started trying out warp, which deserializes querystrings via serde_urlencoded.

I have a use case for nested querystrings, and I've been experimenting using serde_qs.

Is there any interest in supporting nested querystrings in serde_urlencoded, or is that beyond the scope of this project?

seanmonstar/warp#22

Hi,

thanks for this crate. It's very helpful already. 😃
One thing I need the most right now are Option types.

Here's some test code:

#![feature(plugin, custom_derive)]
#![plugin(serde_macros)]

extern crate serde;
extern crate serde_urlencoded;

#[derive(Debug, Serialize, Deserialize)]
struct Parameters {
    channel: String,
    last: Option<f64>,
    ts: i64,
}

fn main() {

    let params = Parameters {
        channel: "mychannel".to_string(),
        last: Some(1.234),
        ts: 23,
    };
    println!("{:?}", serde_urlencoded::to_string(&params));
}

I've seen that the implementation should be at this position and I would like to offer my help.

What I would do is match on Option and if it's Some(value) I would self.append_pair(value).
On the other hand I'm not sure what I would do in case of None. Maybe I should simply return?

Right now it's two hops to see API documentation -- the link to crates.io and click Documentation. It would be more convenient to expose that directly from the readme.

It seems to me like the support for deserializing &str should probably be removed (always failing with a panic). Otherwise, whether it works or not is dependent on whether the value to be deserialized contains any special characters, which seems like a footgun. I'm not sure if this is a problem with the changes in #54, but it might be related. If someone wants to avoid this issue, they can put a Cow<'a, str> in their types instead.

thread 'test::asd' panicked at 'called `Result::unwrap()` on an `Err` value: Custom("unsupported value")', asd/src/qwe.rs:32:57

This doesn't say on which value/field the error happened. Could we maybe get the info about the location of the error?

binary data would be represented in rust as Vec<u8> or &[u8], but this cannot be serialized (with error Custom("unsupported value")). Related to #46, but i do think this is in scope of this library as it does not include any nested values or sequences but a single binary string

example:
binary hash (displayed in hex): 7736da1861d2d31ac2abc41ccde47379c7a678d2
in url encoding: %77%36%da%18%61%d2%d3%1a%c2%ab%c4%1c%cd%e4%73%79%c7%a6%78%d2

#91 added serializing, but not deserializing.

Hi @nox ,

Is it possible to release new version of the crate to crates.io?

I would like to release few private crates to crates.io but they are depending on #69

Please let me know if I can help with it somehow.

Have a great week,
MaxV

https://docs.rs/serde_urlencoded/0.5.3/serde_urlencoded/ needs a more helpful landing page.