Coder Social home page Coder Social logo

valico's Introduction

What is Valico?

Build Status

Valico is a validation and coercion tool for JSON objects, written in Rust. It designed to be a support library for the various REST-like frameworks or other tools that need to validate and coerce JSON input from outside world.

Valico has two features:

  • DSL — a set of simple validators and coercers inspired by [Grape]. It has built-in support for common coercers, validators and can return detailed error messages if something goes wrong.
  • JSON Schema — An implementation of JSON Schema, based on IETF's draft v7.

References:

# Cargo.toml
valico = "2"

API docs

JSON Schema

It passes the entire JSON-Schema-Test-Suite except for remoteRefs and maxLength/minLength when using unicode surrogate pairs. It also can validate your schema and give you an explanation about what is wrong in it.

Example

extern crate serde_json;
extern crate valico;

use serde_json::Value;
use valico::json_schema;
use std::fs::File;

fn main() {
    let json_schema: Value = serde_json::from_reader(File::open("tests/schema/schema.json").unwrap()).unwrap();

    let mut scope = json_schema::Scope::new();
    let schema = scope.compile_and_return(json_v4_schema.clone(), false).unwrap();

    println!("Is valid: {}", schema.validate(&json_v4_schema).is_valid());
}

JSON Schema builder

Valico goes with valico::json_schema::schema(|scheme| { /* .. */ }) -> json::Json function that allows to use simple DSL to generate your schemes. It allows you not to use strings and raw JSON manipulation. It also prevent some kinds of spelling and type errors.

builder::schema(|s| {
    s.properties(|properties| {
        properties.insert("prop1", |prop1| {
            prop1.maximum(10f64, false);
        });
    });
    s.pattern_properties(|properties| {
        properties.insert("prop.*", |prop| {
            prop.maximum(1000f64, false);
        });
    });
    s.additional_properties_schema(|additional| {
        additional.maximum(5f64, false)
    });
})

TODO more docs about JSON Schema here

DSL

Basic Usage

All Valico stuff is making by Builder instance. Below is a simple example showing how one can create and setup Builder:

let params = Builder::build(|params| {
	params.req_nested("user", Builder::list(), |params| {
		params.req_typed("name", json_dsl::string());
		params.req_typed("friend_ids", json_dsl::array_of(json_dsl::u64()))
	});
});

Later params instance can be used to process one or more JSON objects with it's process method with signature fn process(&self, tree: &mut JsonObject) -> ValicoResult<()>.

Note that Valico will mutate borrowed JSON value if some coercion is needed.

Example:

extern crate valico;
extern crate serde_json;

use valico::json_dsl;
use serde_json::{from_str, to_string_pretty};

fn main() {

    let params = json_dsl::Builder::build(|params| {
        params.req_nested("user", json_dsl::array(), |params| {
            params.req_typed("name", json_dsl::string());
            params.req_typed("friend_ids", json_dsl::array_of(json_dsl::u64()))
        });
    });

    let mut obj = from_str(r#"{"user": {"name": "Frodo", "friend_ids": ["1223"]}}"#).unwrap();

    let state = params.process(&mut obj, &None);
    if state.is_valid() {
        println!("Result object is {}", to_string_pretty(&obj).unwrap());
    } else {
        panic!("Errors during process: {:?}", state);
    }

}

Also you can look to the specs for more details and examples.

Validation and coercion

You can define validations and coercion options for your parameters using a Builder::build block. Parameters can be optional and required. Requires parameters must be always present. Optional parameters can be omitted.

When parameter is present in JSON all validation and coercions will be applied and error fired if something goes wrong.

Builder

This functions are available in Builder to define parameters:

// Parameter is required, no coercion
fn req_defined(&mut self, name: &str);

// Parameter is required, with coercion
fn req_typed(&mut self, name: &str, coercer: Box<Coercer>);

// Parameter is required, with coercion and nested checks
fn req_nested(&mut self, name: &str, coercer: Box<Coercer>, nest_def: |&mut Builder|);

// Parameter is required, setup with Param DSL
fn req(&mut self, name: &str, param_builder: |&mut Param|);

// Parameter is optional, no coercion
fn opt_defined(&mut self, name: &str);

// Parameter is optional, with coercion
fn opt_typed(&mut self, name: &str, coercer: Box<Coercer>);

// Parameter is optional, with coercion and nested checks
fn opt_nested(&mut self, name: &str, coercer: Box<Coercer>, nest_def: |&mut Builder|);

// Parameter is required, setup with Param DSL
fn opt(&mut self, name: &str, param_builder: |&mut Param|);

Built-in Coercers

Available list of coercers:

  • json_dsl::i64()
  • json_dsl::u64()
  • json_dsl::f64()
  • json_dsl::string()
  • json_dsl::boolean()
  • json_dsl::null()
  • json_dsl::array()
  • json_dsl::array_of()
  • json_dsl::encoded_array() — use it for string-encoded arrays e.g. "red,green,blue" -> ["red", "green", "blue"]
  • json_dsl::encoded_array_of() — use it for string-encoded arrays of some type e.g. "1,2,3" -> [1, 2, 3]
  • json_dsl::object()

Example of usage:

let params = Builder::build(|params| {
    params.req_typed("id", json_dsl::u64());
    params.req_typed("name", json_dsl::string());
    params.opt_typed("is_active", json_dsl::boolean());
    params.opt_typed("tags", json_dsl::array_of(json_dsl::strings()));
});

Nested processing

You can specify rules to nesting processing for lists and objects:

let params = Builder::build(|params| {
    params.req_nested("user", json_dsl::object(), |params| {
        params.req_typed("name", json_dsl::string());
        params.opt_typed("is_active", json_dsl::boolean());
        params.opt_typed("tags", json_dsl::array_of(json_dsl::strings()));
    });
});

let params = Builder::build(|params| {
    params.req_nested("users", Builder::list(), |params| {
        params.req_typed("name", json_dsl::string());
        params.opt_typed("is_active", json_dsl::boolean());
        params.opt_typed("tags", json_dsl::array_of(json_dsl::strings()));
    });
});

Nesting level is not limited in Valico.

Validate with JSON Schema

DSL allows to use JSON Schema validations to validate objects at the Builder level and the Param level:

let params = json_dsl::Builder::build(|params| {
    params.req("a", |a| {
        a.schema(|schema| {
            schema.integer();
            schema.maximum(10f64, false);
        })
    });
});

Note that JSON Schema validates object AFTER coerce pass:

let mut params = json_dsl::Builder::build(|params| {
    params.req("a", |a| {
        a.coerce(json_dsl::u64());
        a.schema(|schema| {
            schema.maximum(10f64, false);
        })
    });
});

Don't forget to create a json_schema::Scope BEFORE processing:

let mut scope = json_schema::Scope::new();
params.build_schemes(&mut scope).unwrap();

Parameters DSL

You can use DSL block to setup parameters with more flexible way:

let params = Builder::build(|params| {
    params.req("user", |user| {
        user.desc("Parameter is used to create new user");
        user.coerce(json_dsl::object());

        // this allows null to be a valid value
        user.allow_null();

        user.nest(|params| {
            params.req_typed("name", json_dsl::string());
            params.opt("kind", |kind| {
                kind.coerce(json_dsl::string());

                // optional parameters can have default values
                kind.default("simeple_user".to_string())
            });
        });
    });
});

Parameter validations

DSL supports several parameter validations. They considered outdated and likely to be removed in the future in favour of JSON Schema validation.

allow_values

Parameters can be restricted to a specific set of values with allow_values:

let params = Builder::build(|params| {
    params.req("kind", |kind| {
        kind.coerce(json_dsl::string());
        kind.allow_values(&["circle".to_string(), "square".to_string()]);
    })
})
reject_values

Some values can be rejected with reject_values:

let params = Builder::build(|params| {
    params.req("user_role", |kind| {
        kind.coerce(json_dsl::string());
        kind.reject_values(&["admin".to_string(), "manager".to_string()]);
    })
})
regex

String values can be tested with Regex:

let params = Builder::build(|params| {
    params.req("nickname", |a| {
        a.coerce(json_dsl::string());

        // force all nicknames to start with "Amazing"
        a.regex(regex!("^Amazing"));
    })
});
validate_with

Sometimes it's usefull to use some custom function as validator:

let params = Builder::build(|params| {
    params.req("pushkin_birthday", |a| {
        a.coerce(json_dsl::u64());

        fn guess(val: &Json) -> Result<(), String> {
            if *val == 1799u.to_json() {
                Ok(())
            } else {
                Err("No!".to_string())
            }
        }

        a.validate_with(guess);
    });
});
validate

One can use custom validator. Docs in Progress.

Builder validations

Some validators can be specified in Builder DSL block to validate a set of parameters.

mutually_exclusive

Parameters can be defined as mutually_exclusive, ensuring that they aren't present at the same time in a request.

let params = Builder::build(|params| {
    params.opt_defined("vodka");
    params.opt_defined("beer");

    params.mutually_exclusive(&["vodka", "beer"]);
});

Multiple sets can be defined:

let params = Builder::build(|params| {
    params.opt_defined("vodka");
    params.opt_defined("beer");
    params.mutually_exclusive(&["vodka", "beer"]);

    params.opt_defined("lard");
    params.opt_defined("jamon");
    params.mutually_exclusive(&["lard", "jamon"]);
});

Warning: Never define mutually exclusive sets with any required params. Two mutually exclusive required params will mean params are never valid. One required param mutually exclusive with an optional param will mean the latter is never valid.

exactly_one_of

Parameters can be defined as 'exactly_one_of', ensuring that exactly one parameter gets selected.

let params = Builder::build(|params| {
    params.opt_defined("vodka");
    params.opt_defined("beer");
    params.exactly_one_of(["vodka", "beer"]);
});
at_least_one_of

Parameters can be defined as 'at_least_one_of', ensuring that at least one parameter gets selected.

let params = Builder::build(|params| {
    params.opt_defined("vodka");
    params.opt_defined("beer");
    params.opt_defined("wine");
    params.exactly_one_of(["vodka", "beer", "wine"]);
});
validate_with

Sometimes it's usefull to use some custom function as validator:

let params = Builder::build(|params| {
    params.req_defined("monster_name");

    fn validate_params(_: &JsonObject) -> Result<(),String> {
        Err("YOU SHALL NOT PASS".to_string())
    }

    params.validate_with(validate_params);
});
validate

One can use custom validator. Docs in Progress.

Building for other targets

Web Assembly

For WebAssembly, enable the js feature:

# Cargo.toml
valico = { version = "2", features = ["js"] }

valico's People

Contributors

agavrilov avatar alpire avatar andor44 avatar azime avatar bcheidemann avatar bridadan avatar chpio avatar colindean avatar daddinuz avatar dlecan avatar dylan-dpc avatar elliottslaughter avatar erichdongubler avatar erickt avatar fourkbomb avatar konradgraefe avatar legokichi avatar nagisa avatar ramonsnir avatar readmecritic avatar rkuhn avatar rushmorem avatar s-panferov avatar sbihel avatar swarkentin avatar willemneal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

valico's Issues

Load from JSON

Hi

Not really sure,...but I am not really seeing how this could be used to store a schema as Json somewhere and reload it for validation. It is nice that a schema can be built in code, but usually I would want to store that schema in a file or a database and load it for validation. Is there a way to achieve that?oO

question about DSL

I'm sorry, I'm struggling with the DSL. In an example, there's params.req_typed("name", json_dsl::string()); OK, a required string called "name". But how do you add other string validations like maxLength?

Adding custom validations

Is there a way to add custom validations? I've seen a way in the DSL, but I don't use the DSL. I'm thinking of adding a custom property to the JSON object like unique which goes to the database and checks for uniqueness.

valico does not compile with beta2 of rust

which is caused by :

  • jsonway not compiling (I've already issue a PR for this one)
  • use of regex macros (this one is easy to fix, there's only one reference, but I dunno what it implies in term of performance)
  • use of phf_macros (a little more use than regex , but same things, i see how to fix it , but I dunno the implication in term of perf)
  • use of mopafy
  • use of some unstable feature of collections crate

Here's what I got so far in term of diff : (as I'm more than new to Rust i put it here for reference)

for regex

--- a/src/json_schema/keywords/format.rs
+++ b/src/json_schema/keywords/format.rs
@@ -7,13 +7,17 @@ use super::super::validators;

 pub type FormatBuilders = collections::HashMap<String, Box<super::Keyword + Send + Sync>>;

-static DATE_TIME_REGEX: regex::Regex = regex!(r"^(?i)(\d\d\d\d)-(\d\d)-(\d\d)T(\d\d):(\d\d):(\d\d)(\.\d+)??([+-]\d\d\d\d|[A-Z]+)$");
-
 fn default_formats() -> FormatBuilders  {
     let mut map: FormatBuilders = collections::HashMap::new();

-    let date_time_builder = Box::new(|_def: &json::Json, _ctx: &schema::WalkContext| {
-        Ok(Some(Box::new(validators::Pattern{ regex: DATE_TIME_REGEX.clone() }) as validators::BoxedValidator))
+    let date_time_regex: regex::Regex = match regex::Regex::new(r"^(?i)(\d\d\d\d)-(\d\d)-(\d\d)T(\d\d):(\d\d):(\d\d)(\.\d+)??([+-]\d\d\d\d|[A-Z]+)$") {
+        Ok(re) => re,
+        Err(err) => panic!("{}", err),
+    };
+
+
+    let date_time_builder = Box::new(move |_def: &json::Json, _ctx: &schema::WalkContext| {
+        Ok(Some(Box::new(validators::Pattern{ regex: date_time_regex.clone() }) as validators::BoxedValidator))

for phf_macros

diff --git a/src/json_schema/schema.rs b/src/json_schema/schema.rs
index a17c2f8..06902c7 100644
--- a/src/json_schema/schema.rs
+++ b/src/json_schema/schema.rs
@@ -1,8 +1,8 @@
 use url;
 use std::collections;
 use rustc_serialize::json::{self};
-use phf;
 use std::ops;
+use std::collections::BTreeSet;

 use super::helpers;
 use super::scope;
@@ -78,35 +78,35 @@ pub struct Schema {
     scopes: collections::HashMap<String, Vec<String>>
 }

-static PROPERTY_KEYS: phf::Set<&'static str> = phf_set! {
+static PROPERTY_KEYS: [&'static str; 2] = [
     "properties",
-    "patternProperties",
-};
+    "patternProperties"
+];

-static NON_SCHEMA_KEYS: phf::Set<&'static str> = phf_set! {
+static NON_SCHEMA_KEYS: [&'static str; 7] = [
     "properties",
     "patternProperties",
     "dependencies",
     "definitions",
     "anyOf",
     "allOf",
-    "oneOf",
-};
+    "oneOf"
+];

-static FINAL_KEYS: phf::Set<&'static str> = phf_set! {
+static FINAL_KEYS: [&'static str; 3]  = [
     "enum",
     "required",
     "type"
-};
+];

-const ALLOW_NON_CONSUMED_KEYS: phf::Set<&'static str> = phf_set! {
+const ALLOW_NON_CONSUMED_KEYS: [&'static str; 6] = [
     "definitions",
     "$schema",
     "id",
     "default",
     "description",
-    "format",
-};
+    "format"
+];

 pub struct CompilationSettings<'a> {
     pub keywords: &'a keywords::KeywordMap,
@@ -142,9 +142,12 @@ impl Schema {

             let mut scopes = collections::HashMap::new();

+            let final_keys : BTreeSet<&str> = FINAL_KEYS.iter().cloned().collect();
+            let non_shema_keys : BTreeSet<&str> = NON_SCHEMA_KEYS.iter().cloned().collect();
+
             for (key, value) in obj.iter() {
                 if !value.is_object() && !value.is_array() { continue; }
-                if FINAL_KEYS.contains(&key[..]) { continue; }
+                if final_keys.contains(&key[..]) { continue; }

                 let mut context = WalkContext {
                     url: &id,
@@ -156,7 +159,7 @@ impl Schema {
                     value.clone(),
                     &mut context,
                     &settings,
-                    !NON_SCHEMA_KEYS.contains(&key[..])
+                    !non_shema_keys.contains(&key[..])
                 ));

                 tree.insert(helpers::encode(key), scheme);
@@ -214,8 +217,9 @@ impl Schema {
         }

         if settings.ban_unknown_keywords && not_consumed.len() > 0 {
+            let allow_non_consumed_keys: BTreeSet<&'static str> = ALLOW_NON_CONSUMED_KEYS.iter().cloned().collect();
             for key in not_consumed.iter() {
-                if !ALLOW_NON_CONSUMED_KEYS.contains(&key[..]) {
+                if !allow_non_consumed_keys.contains(&key[..]) {
                     return Err(SchemaError::UnknownKey(key.to_string()))
                 }
             }
@@ -237,18 +241,24 @@ impl Schema {
         let tree = {
             let mut tree = collections::BTreeMap::new();

+
             if def.is_object() {
+                let property_keys : BTreeSet<&'static str> = PROPERTY_KEYS.iter().cloned().collect();
+
                 let obj = def.as_object().unwrap();
                 let parent_key = &context.fragment[context.fragment.len() - 1];

+
+                let final_keys : BTreeSet<&str> = FINAL_KEYS.iter().cloned().collect();
+                let non_shema_keys : BTreeSet<&str> = NON_SCHEMA_KEYS.iter().cloned().collect();
                 for (key, value) in obj.iter() {
                     if !value.is_object() && !value.is_array() { continue; }
-                    if !PROPERTY_KEYS.contains(&parent_key[..]) && FINAL_KEYS.contains(&key[..]) { continue; }
+                    if !property_keys.contains(&parent_key[..]) && final_keys.contains(&key[..]) { continue; }

                     let mut current_fragment = context.fragment.clone();
                     current_fragment.push(key.clone());

-                    let is_schema = PROPERTY_KEYS.contains(&parent_key[..]) || !NON_SCHEMA_KEYS.contains(&key[..]);
+                    let is_schema = property_keys.contains(&parent_key[..]) || !non_shema_keys.contains(&key[..]);

                     let mut context = WalkContext {
                         url: id.as_ref().unwrap_or(context.url),
@@ -362,4 +372,4 @@ pub fn compile(def: json::Json, external_id: Option<url::Url>, settings: Compila
 #[test]
 fn schema_doesnt_compile_not_object() {
     assert!(Schema::compile(json::Json::Boolean(true), None, CompilationSettings::new(&keywords::default(), true)).is_err());
-}
\ No newline at end of file
+}

cannot resolve generic type name e.g. "$ref": "#/definitions/Foo<Bar>"

code example

let json_v4_schema: ::serde_json::Value = serde_json::from_reader(::std::fs::File::open("/path/to/schema.json").unwrap()).unwrap();
let mut scope = ::valico::json_schema::Scope::new();
let id = scope.compile(json_v4_schema, false).unwrap();
let schema = scope.resolve(&id).unwrap();
let schema = schema.resolve_fragment(&format!("/definitions/{}", $name)).unwrap();
let schema = ::valico::json_schema::schema::ScopedSchema::new(&scope, &schema);
let o = serde_json::to_value(&foo).unwrap();
let o = schema.validate(&o);
json example

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "definitions": {
        "Response": {
            "anyOf": [
                {
                    "properties": {
                        "type": {
                            "enum": [
                                "error"
                            ],
                            "type": "string"
                        },
                        "value": {
                            "$ref": "#/definitions/ErrorResponse<\"NotFoundError\">"
                        }
                    },
                    "required": [
                        "type",
                        "value"
                    ],
                    "type": "object"
                },
                {
                    "properties": {
                        "type": {
                            "enum": [
                                "ok"
                            ],
                            "type": "string"
                        },
                        "value": {
                            "type": "string"
                        }
                    },
                    "required": [
                        "type",
                        "value"
                    ],
                    "type": "object"
                }
            ]
        },
        "ErrorResponse<\"NotFoundError\">": {
            "properties": {
                "errorMessage": {
                    "type": "string"
                },
                "errorType": {
                    "enum": [
                        "NotFoundError"
                    ],
                    "type": "string"
                }
            },
            "required": [
                "errorMessage",
                "errorType"
            ],
            "type": "object"
        }
    }
}

Separate JSON schema library?

Would it be at all possible to separate the JSON schema portion of this crate out into its own crate that's focused only on JSON schema validation?

Backslashes in pattern values are not working, leads to unrecognized escape sequence errors

Example valid JSON schema pattern value:

"type": "string",
"pattern": "\\w+:(\\/?\\/?)[^\\s]+",

However on validation:

The value of pattern MUST be a valid RegExp, but Syntax(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
regex parse error:
    \\w+:(\\/?\\/?)[^\\s]+\n
         ^^
error: unrecognized escape sequence
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~)

Note the double backslashes in the JSON schema, as backslashes must be escaped to be valid JS strings.

Add API documentation

Hello! I was hoping to use Valico to generate Rust source code with types defined in JSON Schema files, but I'm having trouble understanding the details of how to use the library for this purpose. It would be really helpful if API documentation was added for the various types and functions Valico provides. In many cases it's not clear what a function does, how it differs from a similarly named function, or the meaning/purpose of a function's arguments. Thanks for your work on this project—it may prove very useful for me!

draft-06 / draft-07 support?

Hi from the JSON Schema spec project! Are you planning to implement any of the newer drafts in this project? We are reorganizing the lists on json-schema.org to highlight projects supporting (or working on supporting) draft-06 or later.

See http://json-schema.org/specification.html for info on the latest published drafts.

Release new version

Since this issue is now fixed, could a new version be released to crates.io so downstream crates can take advantage of WASM support?

Make Schema::validate_in_scope public

I need to keep many json schemas and validate them against one json document, so I need to iteratre througn collection of schemas. But it isn't possible as Schema does'n expose the way to apply it against given json, that functionality is in ScopedSchema which keeps references to Scope and Schema and calls Schema::validate_in_scope with reference to scope. I want to be able to create one Scope and many Schema objects and call Schema::validate_in_scope while iterating througn that collection of Schemes. Or have some way to iterate schemas in scope without keepeing their ID's and calling resolve repeatedly. I may create one Scope per Schema but it will allocate many of HashMaps that are all identical across that scopes

add facility for supplying schema defaults during validation

JSON schema defines a default property that is only vaguely specified:

7.3. "default"

There are no restrictions placed on the value of this keyword.

This keyword can be used to supply a default JSON value associated
with a particular schema. It is RECOMMENDED that a default value be
valid against the associated schema.

At Actyx we are using JSON schema for settings management, allowing an app author to specify not only the validation rules but also supply default values. Take as an example a language setting that defaults to English:

{
  "type": "string",
  "enum": [ "en", "de", "cz", "fr" ],
  "default": "en"
}

Defaults are propagated upwards in the tree, allowing the end-user to leave out entire subtrees if they have suitable defaults for all their members.

Due to the complex reference rules of JSON schema and the interplay with keywords like anyOf, it makes most sense to perform the extraction and propagation of default values in the compilation phase, and to perform the injection of default values during the validation phase. This approach is drafted in Actyx#1 (to be repointed towards this repository if you think it makes sense to do so).

Design considerations were

  • to change as little as possible for the case where this new feature is not used (i.e. introducing a feature switch on the Scope)
  • to copy as little as possible of the JSON instance in case default values need to be injected
  • to keep the API compatible with version 3.2.0 (hence a possible replacement value with defaults injected is returned as part of the ValidationState
  • to allow the whole instance to have a default (i.e. the Schema itself holds an optional default value that can be validated and used in the absence of an instance)

We have done some performance measurements regarding the impact of this new feature when it is switched off; this has been done by taking one of our schemas (ca. 18kB) and a matching instance (ca. 4.5kB). Based on a few thousand calls, compilation time decreased very slightly (about 5%) due to more efficient handling of the default attribute (compilation does not descend into it anymore to allow it to contain arbitrary values), while validation time increased slightly (about 10%), presumably due to more branches in the code.

We propose to include this feature into Valico for mutual benefit: we use this library in our core libraries and will actively maintain the parts that we rely on. What do you think?

Publish a new version?

I'm trying to use the latest master but cargo.io won't let me publish my crate with a reference to git hash.

Using serde instead of rustc-serialize

It seems like serde (and likewise serde_json) is becoming the canonical way to serialize/deserialize JSON in rust. Even the rustc-serialize docs say:

While this library is the standard way of working with JSON in Rust, there is a next-generation library called Serde that's in the works (it's faster, overcomes some design limitations of rustc-serialize and has more features). You might consider using it when starting a new project or evaluating Rust JSON performance.

It would be helpful to everyone who is taking this recommendation and switching to serde if this library could either support both serde and rustc-serialize, or publish a version with only serde support.

Remove uritemplate as dependency

Valico 3.5 has added a dependency to rust-uritemplate.

This project seems abandoned and it brings in multiple old version of libraries. With the result that now projects including valico need to install a ton of new libraries, for instance an 0.x version of regex while the whole ecosystem is on 1.x already, etc.
There is a PR to update the project but does not seem that there is a maintainer for it anymore.

Send is an unsafe trait

When trying to compile the Basic Usage example of Rustless https://github.com/rustless/rustless I get these compile errors, due to the recent change rust-lang/rust@f436f9c making Send an unsafe trait:

/Users/jacob/.cargo/registry/src/github.com-1ecc6299db9ec823/valico-0.1.3/src/builder.rs:30:12: 30:16 error: Send is an unsafe trait and it should be implemented explicitly
/Users/jacob/.cargo/registry/src/github.com-1ecc6299db9ec823/valico-0.1.3/src/builder.rs:30 #[deriving(Send)]
                                                                                                       ^~~~
/Users/jacob/.cargo/registry/src/github.com-1ecc6299db9ec823/valico-0.1.3/src/param.rs:20:12: 20:16 error: Send is an unsafe trait and it should be implemented explicitly
/Users/jacob/.cargo/registry/src/github.com-1ecc6299db9ec823/valico-0.1.3/src/param.rs:20 #[deriving(Send)]
                                                                                                     ^~~~

add support for V4 boolean exclusiveMaximum and exclusiveMinimum

I found when using a JSON schema with boolean exclusiveMinimum and exclusiveMaximum (based on JSON schema draft 4, see numbers) valico will panic when attempting to compile the schema.

Raising this issue to see if support can be added for the draft 4, i.e. boolean, version of these parameters.

Here is MVCE:

use serde_json::from_str;
use valico::json_schema;

fn main() {
    let j_schema = from_str(r#"
    {
        "type": "object",
        "properties": {
            "amount": {
                "type": "number",
                "minimum": 0,
                "exclusiveMinimum": true,
                "maximum": 100
            }
        },
        "required": ["amount"]
    }
    "#).unwrap();

    let mut scope = json_schema::Scope::new();
    scope.compile_and_return(j_schema, true).unwrap();
}

The resulting panic, after cargo run:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Malformed { path: "properties/amount", detail: "the `minimum/maximum/exclusiveMinimum/exclusiveMaximum` value must be a number" }', src/main.rs:21:5

Object validation with schema

Ok, I managed to load a schema from a JSON file. First I was under the impression that I could use that 'schema' to validate an object. But it seems that this is not what the schema is for. As far as I can see now, one can only validate that the schema itself is correct. Is that right?

For object validation one would need to use the DSL? If that is true: how can I load a Builder from an already existing schema?

Tag version 2.4.0?

I noticed that v2.4.0 was released, but wasn't tagged. Could it be? Thanks so much!

Invalid references are ignored

use serde_json::json;
use serde_json::value::Value;
use valico::json_schema::Scope;

fn main() {
    let schema = json!({
        "properties": {
            "something": {
                "$ref": "nowhere_to_be_found.json#/definitions/some_type"
            }
        }
    });
    let mut scope = Scope::new();
    // Expecting a panic below since the reference cannot be resolved.
    let validator = scope.compile_and_return(schema, true).unwrap();

    // Anything is valid beyond this point.
    let validate = |file: &Value| {
        let result = validator.validate(file);
        println!("Valid? {}", result.is_valid());
    };

    validate(&json!({
        "something": "a random value"
    }));
    // Prints out "Valid? true".

    validate(&json!({
        "something": [ 1, 2 ,3 ]
    }));
    // Prints out "Valid? true".
}

I'd expect some error somewhere if a reference cannot be resolved. Is an empty schema being substituted instead?

valico and threads

I've got something like this - trying to combine valico with iron:

`
lazy_static!{
static ref classify_candidates_schema: ScopedSchema<'static> = read_json_schema("/schemas/schema.json").unwrap();
}

pub fn read_json_schema(path: &str) -> io::Result {
let mut buffer = String::new();

    let file = File::open(
    &Path::new(
        &format!(
            "{}{}",
            &Path::new(&env::current_exe().unwrap())
            .parent()
            .unwrap()
            .to_str()
            .unwrap(),
            path
        )
    )
).ok()
.unwrap();
file.read_to_string(&mut buffer).unwrap();

let json_schema: serde_json::value::Value = serde_json::from_str(&buffer).unwrap();

let mut scope = json_schema::Scope::new();
let schema = scope.compile_and_return(json_schema.clone(), true).ok().unwrap();
Ok(schema)

}
which doesn't compile due to:
error[E0277]: the trait bound std::rc::Rc<valico::json_schema::keywords::KeywordConsumer>: core::marker::Sync is not satisfied
--> <lazy_static macros>:2:32
|
2 | use std :: sync :: ONCE_INIT ; static mut $ NAME : $ crate :: lazy :: Lazy < $
| ^
<lazy_static macros>:21:1: 21:40 note: in this expansion of __lazy_static_create! (defined in <lazy_static macros>)
<lazy_static macros>:4:1: 5:75 note: in this expansion of lazy_static! (defined in <lazy_static macros>)
src/commands/webserver/request_handlers.rs:25:1: 27:2 note: in this expansion of lazy_static! (defined in <lazy_static macros>)
|
= note: std::rc::Rc<valico::json_schema::keywords::KeywordConsumer> cannot be shared between threads safely
= note: required because of the requirements on the impl of core::marker::Sync for std::collections::hash::table::RawTable<&'static str, std::rc::Rc<valico::json_schema::keywords::KeywordConsumer>>
= note: required because it appears within the type std::collections::HashMap<&'static str, std::rc::Rc<valico::json_schema::keywords::KeywordConsumer>>
= note: required because it appears within the type valico::json_schema::Scope
= note: required because it appears within the type &'static valico::json_schema::Scope
= note: required because it appears within the type valico::json_schema::schema::ScopedSchema<'static>
= note: required by lazy_static::lazy::Lazy

error: aborting due to previous error

`

Could it be fixed?

scoped schema only works in simple cases

Version 3.6:
It seems creating and using schema = ScopedSchema::new(&scope, &payload_schema); only works for simple schemas, but using the schema fails for anyOf/oneOf if there is no correct scope ...

case B leads to:

res = ValidationState { errors: [AnyOf { path: "", states: [] }], missing: [Url { scheme: "json-schema", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("d6e5c927-fa12-4bb2-9bb8-75066df83e7d")), port: None, path: "", query: None, fragment: Some("/anyOf/0") }, Url { scheme: "json-schema", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("d6e5c927-fa12-4bb2-9bb8-75066df83e7d")), port: None, path: "", query: None, fragment: Some("/anyOf/1") }], replacement: None }

Should this work or is using the compile_and_return path mandatory ?

Thanks in advance, Bert

`
#[test]
fn aaa_try_schema() {
use serde_json::json;
use valico::json_schema::schema::;
use valico::json_schema::
;
use valico::*;

let schema_json = json!({
    "anyOf": [
        { "type": "object", "properties": { "mode": { "type": "string" } }, "required": [ "mode" ] },
        {
            "oneOf": [
                { "type": "object", "properties": { "a": { "type": "string" } }, "required": [ "a" ] },
                { "type": "object", "properties": { "b": { "type": "string" } }, "required": [ "b" ] }
            ]
        }
    ]
});

let mut scope = scope::Scope::new();

// **case A - ok**
let schema = scope.compile_and_return(schema_json, true).unwrap();

// **case B - fails**
// let payload_schema = schema::compile(
//     schema_json,
//     None,
//     schema::CompilationSettings::new(&keywords::default(), true),
// )
// .unwrap();
// let schema = ScopedSchema::new(&scope, &payload_schema);

let data = json!({"mode":"aaa","b":"bbb"});
let result = schema.validate(&data);
println!("res = {:?}", schema.validate(&data));
assert!(result.is_strictly_valid());

let data = json!({"mode":"aaa"});
let result = schema.validate(&data);
println!("res = {:?}", schema.validate(&data));
assert!(result.is_strictly_valid());

let data = json!({"a":"xxx"});
let result = schema.validate(&data);
println!("res = {:?}", schema.validate(&data));
assert!(result.is_strictly_valid());

let data = json!({"a":"xxx", "b":"zz"});
let result = schema.validate(&data);
println!("res = {:?}", schema.validate(&data));
assert!(!result.is_strictly_valid());

}
`

Example needs to be updated

The first example validates the schema against itself - which works fine because what it's validating is the meta schema. However, most users won't be doing that. Instead I think the example should be using a different test schema and validating an object that should conform to that schema.

Sub-schema resolution

Hi, I'm evaluating using valico for a project I'm on or writing something in house. Is there any way in this library to select a schema for a sub property? For example in the following schema:

{
  "type": "object",
  "properties": {
    "hello": {
      "type": "object",
      "properties": {
        "world": {
          "type": "string"
        }
      }
    }
  }
}

I could call a method like resolve("/hello/world") which would return the schema corresponding to:

{
  "type": "string"
}

Test failure building on HEAD of master

$ cargo test
…
…
---- schema::test_suite stdout ----
	thread 'schema::test_suite' panicked at 'called `Option::unwrap()` on a `None` value', src/libcore/option.rs:335:21
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Seen on 1.24.1 and 1.25.0 stable:

$ rustc --version
rustc 1.24.1 (d3ae9a9e0 2018-02-27)
$ rustup update
$ rustc --version
rustc 1.25.0 (84203cac6 2018-03-25)

Serialize trait on ValidationState does not produce the expected output

Hi,

I'm hoping to return the ValidationState struct from a Rust Node API using serde_wasm_bindgen. However, the struct does not serialize how I would expect it to.

println!("state = {state:#?}");
let json = serde_json::to_string(&state).unwrap();
println!("state (json) = {}", json);
let json = serde_json::to_string(&state.errors.get(0).unwrap()).unwrap();
println!("error (json) = {}", json);
let json = serde_json::to_string(&state.errors.get(0).unwrap().downcast::<OneOf>()).unwrap();
println!("error_downcast (json) = {}", json);

The above code produces the following output (formatted):

state = ValidationState {
  errors: [
      OneOf {
          path: "/jobs/example_job",
          states: [
              ValidationState {
                  errors: [
                      Required {
                          path: "/jobs/example_job/runs-on",
                      },
                  ],
                  missing: [],
                  replacement: None,
                  evaluated: {
                      "/jobs/example_job/steps/0",
                      "/jobs/example_job/steps/0/name",
                      "/jobs/example_job/steps",
                      "/jobs/example_job",
                      "/jobs/example_job/steps/0/with/submodules",
                      "/jobs/example_job/steps/0/uses",
                      "/jobs/example_job/steps/0/with",
                  },
              },
              ValidationState {
                  errors: [
                      Properties {
                          path: "/jobs/example_job",
                          detail: "Additional property 'steps' is not allowed",
                      },
                      Required {
                          path: "/jobs/example_job/uses",
                      },
                  ],
                  missing: [],
                  replacement: None,
                  evaluated: {
                      "/jobs/example_job",
                  },
              },
          ],
      },
  ],
  missing: [],
  replacement: None,
  evaluated: {
      "/on/pull_request/branches",
      "/on/pull_request",
      "/on/pull_request/branches/0",
      "/name",
      "/on",
      "",
      "/jobs",
  },
}

state (json) = {
  "errors": [
    {
      "code": "one_of",
      "path": "/jobs/example_job",
      "title": "OneOf conditions are not met"
    }
  ],
  "missing": []
}

error (json) = {
  "code": "one_of",
  "path": "/jobs/example_job",
  "title": "OneOf conditions are not met"
}

error_downcast (json) = {
  "code": "one_of",
  "path": "/jobs/example_job",
  "states": [
    {
      "errors": [
        {
          "code": "required",
          "path": "/jobs/example_job/runs-on",
          "title": "This property is required"
        }
      ],
      "missing": []
    },
    {
      "errors": [
        {
          "code": "properties",
          "detail": "Additional property 'steps' is not allowed",
          "path": "/jobs/example_job",
          "title": "Property conditions are not met"
        },
        {
          "code": "required",
          "path": "/jobs/example_job/uses",
          "title": "This property is required"
        }
      ],
      "missing": []
    }
  ],
  "title": "OneOf conditions are not met"
}

Note that the "states" field is missing from the serialized error unless it is downcast.

Is this the expected/intended behaviour?

Publish a new version?

Hello! I'm trying to clean up some of my dependencies, and I saw that the currently published valico is pulling in lazy_static 0.2.11, that is no longer used in the master branch. Could a new version be published without this dependency? I'd be happy to packport the patch if you aren't ready to release your other changes.

3.2.0 is not sem-ver compatible with 3.1.0

Yesterday valico released 3.2.0 with updated dependencies. One of the dependencies updated is url which went from 1 to 2.

Sadly, this seems to have been a breaking change in the public API as url::Url is exposed in the pubic API such as the Schema::resolve method.

And so code like the following will fail to compile after a cargo update:

extern crate url; // 1.* series
extern crate valico; // 3.* series

Scope::resolve(unimplemented!(), url::Url::parse("hello").unwrap());

// error[E0308]: mismatched types
//    --> src/test.rs
//     |
// 403 |             .resolve(&msg_schema_url)
//     |                      ^^^^^^^^^^^^^^^ expected struct `url::Url`, found a different struct `url::Url`
//     |
//     = note: expected reference `&url::Url` (struct `url::Url`)
//                found reference `&url::Url` (struct `url::Url`)
//     = note: perhaps two different versions of crate `url` are being used?

We should yank 3.2.0 and consider releasing 4.0.

Stream processing?

Hi, this is actually a question

Is it possible to process large payloads with Valico without having the entire payload in memory at any point in time?

If I understand the implementation correctly it would require being able to read a serde Value lazily from a stream but I don't know if that can be done without wrapping around another struct

If this is not currently supported would it be an acceptable feature-request that Valico adds support for validation of streams into a Vec of errors some error disregarding the content of the payload at all?

JSON Schema example no longer works

The example at https://github.com/rustless/valico#json-schema has a few issues:

  • Uses std::old_io::fs which has been deprecated (rust-lang/rust#22873)
  • Uses serialize::json instead of serde_json::json.

When I try to run something very similar to it I get the following errors:

error[E0308]: mismatched types
  --> main.rs:82:49
   |
82 |     let event_schema = scope.compile_and_return(event_schema_json.clone(), true).ok().unwrap();
   |                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `rustless::Value`, found enum `serde_json::Value`
   |
   = note: expected type `rustless::Value`
              found type `serde_json::Value`

error[E0308]: mismatched types
  --> main.rs:83:52
   |
83 |     println!("Is valid: {}", event_schema.validate(&event_schema_json).is_valid());
   |                                                    ^^^^^^^^^^^^^^^^^^ expected enum `rustless::Value`, found enum `serde_json::Value`
   |
   = note: expected type `&rustless::Value`
              found type `&serde_json::Value`

I've tried to figure out why this happens by looking at scope.compile_and_return but as far as I can gather it's using serde_json::Value and not rustless::Value (which I can't find the definition for).

I'm unable to get this working in anyway, so assistance/clarification very welcome.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.