xuanwo / serde-env

env support for serde

serde-env  

Deserialize env into structs via serde

Quick Start

use serde::Deserialize;
use serde_env::from_env;

#[derive(Debug, Deserialize)]
struct Cargo {
    home: String,
}

#[derive(Debug, Deserialize)]
struct Test {
    home: String,
    cargo: Cargo,
}

fn main() {
    let t: Test = from_env().expect("deserialize from env");

    assert!(!t.home.is_empty());
    assert!(!t.cargo.home.is_empty());
    println!("{:?}", t)
}

Contributing

Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.

Getting help

Submit issues for bug report or asking questions in discussion.

Acknowledgment

This project is highly inspired by envy

License

^{Licensed under Apache License, Version 2.0.}

serde-env's Issues

The v1.0 release of serde-env

Serde-env has been used in production for a while, and I believe it's ready for wider adoption. It's time for us to clean up some existing obstacles in this project, establish the scope, and deliver on our promises.

Do you have any ideas?

cc all our contributors: @Pscheidl, @jayvdb, @kgtkr, @Tony-Samuels, @acanthite1855, @wyatt-herkamp

Add benchmarks for node operations

Although the performance is not the top thing for serde-env, we need to know how slow we are.

Slow as xx ms is acceptable, but slow as xx s is not allowed.

For the worst case:

:) true | xargs --show-limits
Your environment variables take up 5378 bytes
POSIX upper limit on argument length (this system): 2089726
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2084348
Size of command buffer we are actually using: 131072
Maximum parallelism (--max-procs must be no greater): 2147483647

We need to make sure serde-env works in that case.

Add docs for developer on how to debug serde-env

Serde is complex, and serde-env is challenging to develop for this reason. Let's add documentation so our developers can identify the issues while needed.

Can't deserialize value if itself is a prefix like `CARGO` and `CARGO_HOME`

Given envs like:

CARGO=1
CARGO_HOME=abc

We can't deserialize the value from CARGO.

Flatten not works correctly

Reproduce code:

#[derive(Deserialize, Debug, PartialEq)]
pub struct TestFlatten {
    #[serde(flatten)]
    inner: TestFlattenInner,
}

#[derive(Deserialize, Debug, PartialEq)]
pub struct TestFlattenInner {
    port: u32,
}

#[test]
fn test_from_env_flatten() {
    let _ = env_logger::try_init();

    temp_env::with_vars(vec![("port", Some("123"))], || {
        let t: TestFlatten = from_env().expect("must success");
        assert_eq!(t.inner.port, 123)
    })
}

Missing field when no default specified for field

Hey, first of all, thanks for the library. It's great especially compared to envy since it supports nested structs.

I am having a weird issue when defaults are involved. I worked around it for now, but I am still confused about why it is happening, and I would love to contribute a fix if you can give me some context in the code and point me to it.

Code

use color_eyre::Result;
use serde::Deserialize;
use serde_default::DefaultFromSerde;
use serde_env::from_env;

fn default_sampler_arg() -> f64 {
    1.0
}

#[derive(Debug, Clone, Deserialize, DefaultFromSerde)]
pub struct OTELTracesConfig {
    #[serde(default = "default_sampler_arg")]
    pub sampler_arg: f64,
}

#[derive(Debug, Clone, Deserialize, DefaultFromSerde)]
pub struct OTELConfig {
    pub traces: OTELTracesConfig,
}

#[derive(Debug, Clone, Deserialize, DefaultFromSerde)]
pub struct Config {
    pub otel: OTELConfig,
}

fn main() -> Result<()> {
    println!("default: {:#?}", Config::default());
    println!("from_env: {:#?}", from_env::<Config>()?);

    Ok(())
}

Expected

with no env vars

default: Config {
    otel: OTELConfig {
        traces: OTELTracesConfig {
            sampler_arg: 1.0,
        },
    },
}
from_env: Config {
    otel: OTELConfig {
        traces: OTELTracesConfig {
            sampler_arg: 1.0,
        },
    },
}

with env var OTEL_TRACES_SAMPLER_ARG=0.5

default: Config {
    otel: OTELConfig {
        traces: OTELTracesConfig {
            sampler_arg: 1.0,
        },
    },
}
from_env: Config {
    otel: OTELConfig {
        traces: OTELTracesConfig {
            sampler_arg: 0.5,
        },
    },
}

Actual

with no env vars

default: Config {
    otel: OTELConfig {
        traces: OTELTracesConfig {
            sampler_arg: 1.0,
        },
    },
}
Error: missing field `otel`

Location:
    src/main.rs:28:33

with env var OTEL_TRACES_SAMPLER_ARG=0.5 working correctly

Workaround

It works if I add #[serde(default)] to otel in Config.

#[derive(Debug, Clone, Deserialize, DefaultFromSerde)]
pub struct Config {
    #[serde(default)]
    pub otel: OTELConfig,
}

There are 2 issues regarding this though:

• Why do I need to add #[serde(default)]? Default is implemented for OTELConfig and implementing Default for Config should automatically pick it up.
• If I need to add the above, why do I not need to add it for traces in OTELConfig? Why is it only required to do so for Config struct?

---

Because of the 2 issues I mentioned above, my instinct says that this is an issue with the deserialization in this library. Please let me know how I can help to fix this.

Thanks again.

Optional structs with an Option value always deserialize to None

Example code:

#![allow(dead_code)]

#[derive(Debug, serde::Deserialize)]
struct Inner {
    val: Option<u32>,
}

#[derive(Debug, serde::Deserialize)]
struct Outer {
    inner: Inner,
}

#[derive(Debug, serde::Deserialize)]
struct OuterOpt {
    inner: Option<Inner>,
}

fn main() {
    println!("{:?}", serde_env::from_env::<Outer>());
    println!("{:?}", serde_env::from_env::<OuterOpt>());
}

Example output:

$ INNER_VAL=3 cargo run -q
Ok(Outer { inner: Inner { val: Some(3) } })
Ok(OuterOpt { inner: None })

As both have the same form and accept the same values, the two Inner should have the same value.

Failure to parse enum variant error message doesnt say which field failed

This probably occurs for all struct members, but we have seen it for a struct member which is a simple enum. If the env var contains a value which isnt in the enum, the error message is just

Error(no variant found)

It would be useful to know which envvar wasnt able to be parsed.

Allow deserialize as map

from_iter

The tests use https://crates.io/crates/temp-env , which is nice, but I would prefer to be using something like envy.from_iter https://github.com/softprops/envy/blob/e22035c3b91f999c325a44c0c77f0e6e05750da3/src/lib.rs#L330

For context, I am interested in a from_iter for my tests in a crate which uses serde-env

Simple enum keys for HashMaps

The first of the tests below passes, however it requires an unusual lower casing of the enum variants otherwise serde fails.

The second of these fails with

must succeed: Error(unknown variant `option1_inner2`, expected `Option1` or `Option2`)

Tests

    #[test]
    fn inner_mapping_with_enum_keys() {
        // TODO: the requirement to use a lowercase serde alias is not ideal/unexpected,
        // it matches uppercase environment variable names.
        #[derive(Debug, Deserialize, Eq, Hash, PartialEq)]
        enum MappingKey {
            #[serde(alias = "option1")]
            Option1,
            #[serde(alias = "option2")]
            Option2,
        }
        #[derive(Debug, Deserialize, Eq, PartialEq)]
        struct Mapping {
            val: HashMap<MappingKey, String>,
        }

        let env = vec![("VAL_OPTION1", "FOO"), ("VAL_OPTION2", "BAR")];
        let t: Mapping = from_iter(env).expect("must succeed");
        assert_eq!(
            t,
            Mapping {
                val: HashMap::from_iter(vec![
                    (MappingKey::Option1, "FOO".to_string()),
                    (MappingKey::Option2, "BAR".to_string())
                ])
            }
        );
    }

    #[test]
    fn double_inner_mapping_with_enum_keys() {
        #[derive(Debug, Deserialize, Eq, Hash, PartialEq)]
        enum MappingKey {
            #[serde(alias = "option1")]
            Option1,
            #[serde(alias = "option2")]
            Option2,
        }

        #[derive(Debug, Deserialize, Eq, Hash, PartialEq)]
        enum MappingKey2 {
            #[serde(alias = "inner1")]
            Inner1,
            #[serde(alias = "inner2")]
            Inner2,
        }

        #[derive(Debug, Deserialize, Eq, PartialEq)]
        struct Mapping {
            val: HashMap<MappingKey, HashMap<MappingKey2, String>>,
        }

        let env = vec![("VAL_OPTION1_INNER2", "FOO"), ("VAL_OPTION2_INNER1", "BAR")];
        let t: Mapping = from_iter(env).expect("must succeed");
        assert_eq!(
            t,
            Mapping {
                val: HashMap::from_iter(vec![
                    (
                        MappingKey::Option1,
                        HashMap::from_iter(vec![(MappingKey2::Inner2, "FOO".to_string())])
                    ),
                    (
                        MappingKey::Option2,
                        HashMap::from_iter(vec![(MappingKey2::Inner1, "BAR".to_string())])
                    ),
                ])
            }
        );
    }

Add bind support

Related to Xuanwo/serfig#11

Can we bind some value to env directly?