Validation like Either but with an accumulating Applicative about fsharpplus

[gusty]

@oscarvarto It should be straight-forward to implement. We basically need a DU which can act as a Functor and Applicative (you can have a look here for the required signatures by clicking each abstraction):

open FSharpPlus
open FSharpPlus.Operators

type Validation<'a,'b> =
    | Success of 'a
    | Failure of 'b

    with 
        // as Functor
        static member Map (x, f) = 
            match x with
            | Failure e -> Failure e
            | Success a -> Success (f a)

        // as Applicative
        static member Return x = Success x
        static member inline (<*>) (f, x) =
            match (f, x) with
            | Failure e1, Failure e2 -> Failure (e1 ++ e2)
            | Failure e1, Success _  -> Failure e1
            | Success _ , Failure e2 -> Failure e2
            | Success f , Success x  -> Success (f x)


let a : Validation<int,string list> = Success ((+) 1) <*> Success 7      // Success 8
let b : Validation<int,string list> = Failure ["f1"]  <*> Success 7      // Failure ["f1"]
let c : Validation<int,string list> = Success ((+) 1) <*> Failure ["f2"] // Failure ["f2"]
let d : Validation<int,string list> = Failure ["f1"]  <*> Failure ["f2"] // Failure ["f1"; "f2"]

You can use it with NonEmptyList since it's a Semigroup:

let e : Validation<int,string NonEmptyList> = Failure (NonEmptyList.singleton "f1")  <*> Success 7     
let f : Validation<int,string NonEmptyList> = Success ((+) 1) <*> Failure (NonEmptyList.singleton "f1")
let g : Validation<int,string NonEmptyList> = Failure (NonEmptyList.singleton "f1")  <*> Failure (NonEmptyList.singleton "f2")

If you think it worths adding it to this library we can discuss it, agree on the names or just submit a PR and we discuss it there.

Let me know if this is ok or if you need more information.

[gusty]

@oscarvarto @wallymathieu I don't know about the other libraries, but I also had the sensation that sometimes things move very slow in most fsprojects, I try to do my best maintaining this library and the only reason things are moving slow here is because sometimes it's basically only myself trying to do a library which is clearly too big for one person.

But when I see PRs here I try to respond as quick as possible because I understand the frustration on the other side having worked hard and then get blocked waiting for someone to respond, and also because this library needs more collaboration, not just code, but also participating in discussions.

Regarding Chessie, I had a quick look but it seems to be basically the Either monad which is also part of this library, I should have a deeper look but if you think there is some functionality there that can also be interesting for this library please let me know and I'll be happy to include it.

[vasily-kirichenko]

@gusty yes, but you have to write conversion for all DU cases. What’s more, my approach is too subtle and "magic". I don’t use it anymore, explicitness has won one more game :)

[gusty]

I was digging into some really old (but good) stuff and I found an old gist from @mausch demonstrating Validations with an older version of this library.

I updated the gist to the latest version of F#+

[gusty]

traverse (first result) seems to be the shortest generic expression.

Note that type inference is unable to solve the container of the error part. A type annotation would solve the issue:

let x: Validation<_ list,_> = traverse (first result) lst

[gusty]

@oscarvarto Also let me know if you're able to use this code in Xamarin.Android without issues, @wallymathieu did a big effort to multi-target all platforms.

[gusty]

I'm using Visual Studio 2017 on a PC, this is what I get in intellisense:

Which makes sense. I don't have Ionide, but I think in my mac I saw that 'c only constraint.

Having said that, if I were to implement that Validation type either in F#+ or as a separated library, I would use this definition:

static member inline (<*>) (f, x:Validation<'T,'Semigroup>) =

which makes things easier when using intellisense.

[oscarvarto]

@gusty I have used most recent version (to this moment) of FSharpPlus:
https://www.nuget.org/packages/FSharpPlus/1.0.0-CI00091

on a Xamarin.Android project.
I had no trouble adding that using Nuget on Visual Studio 2017.

• The project can compile the snippet above:
  #51 (comment)
• Type inference is working.

@wallymathieu Thanks for your work to support .NET Standard.

Now I can start using FSharpPlus with Xamarin.Android!

[gusty]

Looks like a nice implementation, but it worth noting that it seems to be an implementation more at the application level, it will address problems in situations where you have to deal with different error types.

Making this decision in a base library could be tricky.
A while ago I though what was the best way to design a base library with "safe" functions (namely total functions). If you ask around many people will say, use option, but if you want to give some additional information about what went wrong you can use a DU, a string or exceptions, as he already explained in that post.

Now when designing a base library that decision will impact the way the function will perform. For example it's cheaper to construct a string than an exception with all the call stack. Still much cheaper is a None. The thing is sometimes you can end up reinventing the exceptions but there are situations when you just need a signal that something did not succeed with a message as part of your logic, not really being an exceptional case (I had situations like that).

This is related to that interesting article about the 8 ways to handle errors in Haskell.

So I came to the conclusion that the best way would be to delegate that decision to the user of the library, so maybe we can use double barreled CPS exceptions, actually I created an issue regarding this when thinking about how to improve the design of the numeric functions, back in FsControl.

The idea is that you provide a specific continuation for the error and from that implementation you can create different versions by partially applying the function with the error constructor.

Having said that, we can certainly include the RResult implementation in F#+ if the author agrees. Or include it in the tests to see how it integrates, actually I think it's likely that it will integrate already since it defines static members.

[wallymathieu]

I will see if I can modify the sample a bit.

[gusty]

BTW regarding the problem with different error types, here's the Rust approach, explained by @vasily-kirichenko which could be easily encoded here using something like an error-convertible class.

[wallymathieu]

F#+ feels more like a f# distribution or language extention than just a lib. There is so much there. I was looking at in order to remove copy paste computation expression builders.

[gusty]

Looks like a valid scenario.
But I think there might be another way to get it, I mean with another abstraction, maybe a bifunctor or something like that. Have you tried that way?

[wallymathieu]

I'm sure FSharpx.Extras get some attention. But as with most open source it might require some patience and some elbow grease 😉

Do you want something like Result<'a, 'error list> ?

[oscarvarto]

@gusty I'll try as soon as possible. I really appreciate that you have taken some time to write down how could I use the abstractions FSharpPlus already has for this (since I am not a FP nor a F# expert).

I think validation of Business Rules is so important in everyday programming, and a principled solution with Applicative Functor, NonEmptyList, is very welcome.

[wallymathieu]

Have you looked at Chessie @oscarvarto ? Sample usage: simple validation chessie.

[oscarvarto]

@wallymathieu Thank you very much for pointing me to Chessie. I briefly took a loot at it and I liked that it was basically 1 source file.

But please, take a look at this:
https://github.com/fsprojects/Chessie/pulls

Pull requests have been there for months. I've seen this for several fsprojects. I understand that this is Open Source, that maintenance is actual and very important work (unpaid most of the time?). People invest time to update libraries, send pull requests, and sometimes wait there ignored a lot of time...

However, it does not look like having good maintenance.

Unfortunately I see this in most fsproject libraries @dsyme, @forki, @matthid

[wallymathieu]

Well, when was the last closed pull request? On some major repositories you have pull requests and issues that are dangling, while some issues get handled fairly quickly.

[wallymathieu]

It's either + a list of messages for Result.Error or Choice2Of2

[gusty]

@wallymathieu So you mean that it's rather the applicative validation discussed above?

[wallymathieu]

Well, it might not be exactly what he is looking for. I've done a fork where I've tried out how it would look with Result: https://github.com/wallymathieu/Chessie/tree/result_from_fsharp_core

[wallymathieu]

and what we see is that Chessie Result becomes:
type RopResult<'TSuccess, 'TMessage> = FSharpCore.Result<'TSuccess * 'TMessage list , 'TMessage list>

[gusty]

Weird, I should have a deeper look. There is a common monad for error handling and there is another abstraction used for validation (the one discussed here) which is not a monad, just applicative. Chessie seems to be a mix.

[oscarvarto]

@gusty Which development environment do you use for FSharpPlus?

I am trying with VSCode Stable + latest Ionide and I see my laptop takes a while to load FSharpPlus project and provide some feedback on the types.

I also see that type inference is somewhat limited. For example:

static member inline (<*>) (f, x) =
            match (f, x) with
            | Failure e1, Failure e2 -> Failure (e1 ++ e2)
            | Failure e1, Success _  -> Failure e1
            | Success _, Failure e2  -> Failure e2
            | Success f, Success x   -> Success (f x)

Ionide reports var e1: ^c , but from the usage of ++ I understand that it cannot be any ^c, but a monoid. So, I think current tooling does not help by providing enough/correct information about the types.

I guess you are pushing the F# compiler by inlining definitions (because of lack of support of Higher Kinded types on .NET). Thanks for your great efforts to encode this abstractions on spite of this.

How do you proceed to get correct feedback from the compiler on your (very generic) code?
I guess that you must be typechecking stuff in your head a lot :), but some help from the compiler and editors is always welcome.
Do you use the command line and fsi? Have you found a way to work with Visual Studio 2017/2015?

Thanks for your help. F# newbie here.

[oscarvarto]

Hello again @gusty.

I have been trying to use FSharpPlus on Xamarin.Android without success. I first tried to add the library as a Nuget Package for my Android project, but I get the following error message:

Could not install package 'FSharpPlus 1.0.0-CI00089'. You are trying to install this package into a project that targets 'MonoAndroid,Version=v8.0', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.

I also tried to add all the source code manually to the project, but the project is not compiling. I upgraded the Nuget packages to latest versions (for example FSharp.Core 4.2.3), and keep getting a lot of compile errors (for example, I get errors about basic stuff that should be OK: not recognizing Some, Choice1of2, etc.)

I must be doing several mistakes...

I am reporting this because at least I want you to know that I am trying...

[gusty]

Hi @oscarvarto.

I have to do a new release. The last nuget is before @wallymathieu work on netstandard.

[wallymathieu]

@gusty should we break out the BitConverter into a separate lib (in order to have something that we can depend)?

[gusty]

Yes. I don’t see other alternative at least until F# supports the unsafe keyword.

If you want to submit a PR for that please go ahead.

[wallymathieu]

This looks a bit interesting: https://gist.github.com/mrange/1d2f3a26ca039588726fd3bd43cc8df3

[wallymathieu]

Thoughts @gusty @oscarvarto ?

[wallymathieu]

Well, some of it is quite untyped. I was hoping that some of the compositions could be tightened. For instance by disallowing 'exn' in the mix and see what kind of ideas can be reused from the code.

[gusty]

What do you mean? Something like a DU of string or described obj?
If so it would be something like the Error typeclass in haskell, where you can define as many types you want to model exceptions as long as they support the method errorMsg str.

Please correct me if I misunderstood it.

[wallymathieu]

After looking closer at the code, it looks to be essentially Result<'Ok, RBad>, since it's not accumulating the results.

[wallymathieu]

I don't get why the person defines rbind, rgood, rbad?

[wallymathieu]

Aa, sorry, he is using the same type for both validation and errors.

[gusty]

Actually I was also confused exactly with the same stuff.
I think he uses validation via <|> which in F#+ is the Append of the Alternative class (which is more or less the same as MonadPlus) which reminds me that I never decided to make Choice (now Result) an instance of that class. Mainly because the problem is how to define an empty error state. But it could be initially defined as an Alternative with only Append, something like a semigoup. Which makes sense for validations.

[wallymathieu]

So the validation type would be something like:

type Validations<'T,'Validation> = { Value:'T; ValidationErrors:'Validation list } 

since validations are not OR. Probably why the Result type in Chessie looks so weird. Perhaps a clearer Chessie type definition would be

type ChessieResult<'T,'Validation,'Error>=Result<Validations<'T,'Validation>, 'Error>

?

[gusty]

Ah ok, now I understand a bit more what Chessie provides.
If so, it should be doable with F#+ by defining the type as you stated.
Maybe a Chessie sample fragment could be illustrative by trying it to translate to F#+ code.

[vasily-kirichenko]

The Rust community found manual error converting too tedious and has invented https://github.com/rust-lang-nursery/error-chain and, more recently, https://github.com/withoutboats/failure. However, both creates use macros heavily, so it would be unrealistic to port them to F#.

[gusty]

@vasily-kirichenko but the F# solution you posted, doesn't solve manually converting?

[gusty]

@wallymathieu It seems to me that another way to represent Chessie type is: Option<'T> * List<'Validation>

[gusty]

Good point: magic vs explicitness. I couldn't agree more, although I use a lot of overloads and SRTPs I try to avoid magic ones, specially on builders. As a result in F#+ there is not a single overload in the builders.

However in projects like Chessie you see that kind of magic conversions everywhere, a la C#. See my comment at the end of this answer

[wallymathieu]

I've started to try to port validations library from Haskell, I'm having some trouble with trying to understand what's going on with the prisms though (probably due to my inexperience with Haskell).
fsharp-validations. I've some trouble with to grokk how the lens (#) coerce operator works.

[gusty]

I just had a quick look and I see that you're using let functions instead of static member.
That won't work.
Could it be an issue? Or was it WIP?

[gusty]

Regarding your question about coerce operators, I remember having encountered coercions when porting pieces of the lens library.

I'm not a Haskell expert, but if you can point me to some Haskell code I can try to give you an explanation.

[gusty]

BTW, talking about lenses in F#+
Since I ported the lenses I was unhappy with the set function since it shadows the F# set function (converts a seq<_> to a Set<_>) and since then I was thinking what alternative name we can use.

I'm about to change it to burn since it seems to be more inline with the analogy of writing through a lens. What do you think?

[wallymathieu]

I'm unfortunately not familiar with lenses to be able to prefer one word over another. I agree that set is problematic. My immediate reaction to the word burn is that you want to destroy something.

The code I'm having trouble with is the following:
https://github.com/qfpl/validation/blob/master/src/Data/Validation.hs#L328-L337

Right now, the code in the above repo only contains some code that looked easy to convert. However the library uses a prism to allow you to use both either and AccValidation (from what I've understood).

[gusty]

The fragment you highlighted seems to be part of a typeclass called Validate (defined about 30 lines of code before that fragment) which is not part of FSharpPlus.

We can try to encode it with the technique used in this library, but I would do it as the last part of the translation process. It doesn't seems to be critical for the functionality provided by Validation.

I would say leave it for the moment and concentrate on the rest of the code. Try to go as far as you can, without headaches, then commit your work and I can review it.

[wallymathieu]

Sounds good 👍

[wallymathieu]

I've updated the code so that it passes all the tests (without any prism logic though)

[wallymathieu]

@oscarvarto what are your thoughts on the port?

[wallymathieu]

Ou, I see that you are referencing NonEmptyList in the comment above. I did not notice that on my first read of the comment.

[wallymathieu]

I guess I have not realized some of the nice things of f#+ 👍

[gusty]

I frequently have the feeling that the readme and the docs fail to explain shortly what are the distinctive features of the lib.

[wallymathieu]

I've started to add some tests of it: https://github.com/wallymathieu/fsharp-validations/pull/4/files
There are some rules related to different aspects that does not hold

[wallymathieu]

I've released it to NuGet so that you ( @oscarvarto ) can try it out before folding it into FSharpPlus https://github.com/wallymathieu/FSharpPlus.Validations

[wallymathieu]

I've added @gusty to the project since it's prefixed by FSharpPlus

[DunetsNM]

what I guess many people may want to achieve is to map Result<'a, 'e> list to Result<'a list, 'e list> that contains either list of all errors (if there's any error) or list of all Ok values (if no errors at all).

Here's how it can be done, not sure if there's a simpler way

/// Maps list of Result<'a, 'e> to Result<'a list, 'e list> which
    /// either has all error values (if any error) or all Ok values (if no error)
    let ofList resultList =
        let wrapErrorInList =
            Result.mapError (fun e -> [e])
        resultList
        |> List.map (wrapErrorInList >> Validation.ofResult)
        |> List.sequence
        |> Validation.toResult

[gusty]

Well, actually that's yet another implementation of the applicative for result.
That one requires a semigroup for both generic parameters in order to accumulate both right and wrong values, so the implementation you did can be generalized to work not just with lists but with any semigroup by just using the generic versions of map, sequence and result instead of (fun e -> [e]).

I've never seen a real life scenario when this comes at handy, though I'm sure there might exists.

The most common scenario for validation is this implementation, where you care only about the (on and only) right value if everything succeeds or the list of errors, which is typically used for form validation.

[DunetsNM]

My case was to map list of raw strings from request to list of tiny types values (each validated independently by the same regex) or get all invalid strings.
I might have invented a new bicycle when there is already more idiomatic way available (most likely so because I'm rather new to the concept of applicatives), or maybe need to reconsider wider design.

[DunetsNM]

I haven't but I will give it a try out of curiosity, thanks for suggestion

[NickDarvey]

@DunetsNM did you end up finding an idiomatic way to do this?
I am trying to do something similar (Validation<'a, 'b> list -> Validation<'a list, 'b list>) but is there a name for such a function? It seems like a variation on sequence.

@gusty, I tried what (I think) you were suggesting and ended up with:

  let ofList (vs : Validation<'a, 'b> list) : Validation<'a list, 'b list> =
    vs |> map (bimap result id) |> sequence

but can't lose the type annotation else it thinks vs is an obj and gives up. What would a generalized version of this function look like? And what would you call such a function?

(FWIW, in my scenario the user creates n objects. If all objects are valid I want to get all the objects, if any one of them is invalid, I want to get all the errors. Not too dissimilar from @DunetsNM's example.

(Let me know if I ought to open a new ticket.)

[gusty]

Yes, since it seems to be a common scenario it might be better to open another issue.

I ll try to answer it later when in front of the computer.

[DunetsNM]

@NickDarvey I'm late to the party but seems like @gusty already nailed it. And no, I haven't come up with anything else or better on my own.