Including (parts of) another YAML file about yamldotnet

[EdwardCooke]

You would use a custom IYamlTypeConverter. I'll leave it up to you on reading the file. Doing it this way won't care about type of the scalar, number, string, whatever. Up to you however you want to parse your !include scalar.

using YamlDotNet.Core;
using YamlDotNet.Core.Events;
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.NamingConventions;

var deserializer = new DeserializerBuilder()
    .WithTagMapping("!include", typeof(IncludedObject))
    .WithTypeConverter(new TestTypeConverter())
    .WithNamingConvention(CamelCaseNamingConvention.Instance)
    .Build();

var yaml = @"
location: http://some.example/api
auth: !include secret.yaml#auth
";

var o = deserializer.Deserialize<Outer>(yaml);
Console.WriteLine(o.Location);
Console.WriteLine(o.Auth);
Console.WriteLine(o.Auth?.Username);
Console.WriteLine(o.Auth?.Password);

class Outer
{
    public string Location { get; set; }
    public Auth Auth { get; set; }
}

class TestTypeConverter : IYamlTypeConverter
{
    public bool Accepts(Type type) => type == typeof(IncludedObject);

    public object? ReadYaml(IParser parser, Type type)
    {
        var isValid = parser.TryConsume<Scalar>(out var scalar);

        if (!isValid || scalar is null)
        {
            throw new Exception("Not a valid exception");
        }

        var split = scalar.Value.Split('#');
        var deserializer = new DeserializerBuilder().Build();
        if (split.Length != 2)
        {
            throw new Exception($"Invalid format, missing type and/or filename, { scalar.Value} ");
        }

        if (split[1] == "auth")
        {
            var yaml = @"
user: testusername
pass: testpassword
";
            var result = deserializer.Deserialize<Auth>(yaml);
            return result;
        }
        else
        {
            throw new Exception($"Unknown type: {split[1]}");
        }

        throw new Exception("Unexpected failure");
    }

    public void WriteYaml(IEmitter emitter, object? value, Type type)
    {
        throw new NotImplementedException();
    }
}

class IncludedObject
{

}

class Auth
{
    [YamlMember(Alias = "user")]
    public string Username { get; set; }

    [YamlMember(Alias = "pass")]
    public string Password { get; set; }
}

Results in

http://some.example/api
Auth
testusername
testpassword

[EdwardCooke]

Did the above answer your question?

[mikhail-barg]

@EdwardCooke Hi! sorry for a late answer, I've missed notifications (

Getting back to your solution, what bugs me, is that line:

var result = deserializer.Deserialize<Auth>(yaml);

How do I know that it's the Auth object expected at the place of !include? If I get your solution right, it requires knowing expected type upfront, and therefore I would not be able to make both

      location: http://some.example/api
      auth:
        user: !include secret.yaml#auth.user
        pass: !include secret.yaml#auth.pass

and

      location: http://some.example/api
      auth: !include secret.yaml#auth

to work?

[mikhail-barg]

It seems that a solution would be to provide an expected type via the type argument to ReadYaml call (or a new argument), so it would be possible to do

var result = deserializer.Deserialize(type, yaml);

see also this comment: #368 (comment)

[EdwardCooke]

If you don’t know the type you could just call deserialize with the yaml itself, no type. It’ll then return an object. Could be any number of types. An array, a string, int, short, etc. as well as a dictionary of object, object. Usually the key will be a primitive unless you have some crazy yaml where you make the key another mapping. This is not tested since I’m typing this on my phone but something like this may work.

var path = split[1].Split('.');
object last = deserializer.Deserialize(yaml);
foreach (var segment in path)
{
if (last is Dictionary<object, object> dict && dict.ContainsKey(segment))
{
last = dict[segment];
}
else
{
A throw new Exception(“key not found”);
}
}
return last;

[mikhail-barg]

@EdwardCooke thankjs for the answer! Still it seems that it does not solve the problem, if I get your solution right.

The problem is that the object model of the original document (I'm not sure I'm using correct terms here, so please bear with me) already expects a specific type, and if a type converter returns another tyle (something generic like string, object, or a dictionary like you proposed) there would just be a conversion error, correct?

For example, here's my DOM:

public sealed class Doc
{
   public string location { get; set; }
   public Auth auth { get; set; }
}

public sealed class Auth
{
   public string user { get; set; }
   public string pass { get; set; }
}

and for this yaml:

      location: http://some.example/api
      auth: !include secret.yaml#auth

returning a Dictionary for !include secret.yaml#auth would not work, because it's the Auth type that is expected here.

And currently there's no way to know that it should be an Auth type at the time IYamlTypeConverter.ReadYaml is executing (

[EdwardCooke]

Ok. So it would be a combination of the first 2. The if statement where it checks for split[1] would determine what to do. You may even be able to do this and not even worry about what’s passed in after the # sign.

deserializer.Deserialize(yaml, type);

yaml is the included file contents.

It’s the same as what you had in one of your comments just swap the arguments.

[EdwardCooke]

Just came up with a way to do what you are asking without needing to know the type before hand. It uses a custom nodedeserializer.

using YamlDotNet.Core;
using YamlDotNet.Core.Events;
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.NamingConventions;
using YamlDotNet.Serialization.NodeTypeResolvers;

var deserializer = new DeserializerBuilder()
    .WithNodeDeserializer(new IncludeDeserializer(), syntax => syntax.OnTop())
    .WithoutNodeTypeResolver<PreventUnknownTagsNodeTypeResolver>()
    .WithNamingConvention(CamelCaseNamingConvention.Instance)
    .Build();

var yaml = @"
location: http://some.example/api
auth: !include secret.yaml#auth
";

var o = deserializer.Deserialize<Outer>(yaml);
Console.WriteLine(o.Location);
Console.WriteLine(o.Auth);
Console.WriteLine(o.Auth?.Username);
Console.WriteLine(o.Auth?.Password);

class Outer
{
    public string Location { get; set; }
    public Auth Auth { get; set; }
}

class IncludeDeserializer : INodeDeserializer
{
    public bool Deserialize(IParser reader, Type expectedType, Func<IParser, Type, object?> nestedObjectDeserializer, out object? value)
    {
        if (reader.Accept<Scalar>(out var scalar) && scalar.Tag == "!include")
        {
            var filename = scalar.Value.Split('#')[0];
            // Do your file check logic here
            if (filename == "secret.yaml")
            {
                reader.Consume<Scalar>();
                // read your yaml file
                var yaml = @"
user: testusername
pass: testpassword
";
                var deserializer = new DeserializerBuilder().Build();
                // deserialize to the object
                value = deserializer.Deserialize(yaml, expectedType);
                return true;
            }
        }
        value = null;
        return false;
    }
}

class Auth
{
    [YamlMember(Alias = "user")]
    public string Username { get; set; }

    [YamlMember(Alias = "pass")]
    public string Password { get; set; }
}

Results in

http://some.example/api
Auth
testusername
testpassword

[EdwardCooke]

If you want to make it pass correct validation (a comment I saw in aaubry's suggestion), use a custom type resolver instead of removing the PreventUnknownTagsNodeTypeResolver

var deserializer = new DeserializerBuilder()
    .WithNodeDeserializer(new IncludeDeserializer(), syntax => syntax.OnTop())
    .WithNodeTypeResolver(new IncludeNodeTypeResolver(), syntax => syntax.OnTop())
    .WithNamingConvention(CamelCaseNamingConvention.Instance)
    .Build();

class IncludeNodeTypeResolver : INodeTypeResolver
{
    public bool Resolve(NodeEvent? nodeEvent, ref Type currentType)
    {
        if (nodeEvent?.Tag == "!include")
        {
            return true;
        }
        return false;
    }
}

[mikhail-barg]

Thanks for the answer and samples! I am unable to check those right now, so I'll return with results a bit later..
Thanks again for your support and patience!