lue-bird / elm-review-documentation-code-snippet

verify code examples by generating tests

🔧 Review.Documentation.CodeSnippet.check checks your small code examples in the readme, module headers and declaration comments for valid syntax, matching types and correctness.

To check this, it generates tests from these code snippets (If you know elm-verify-examples, you also know how this works. This rule has only a few extras like checking for types or actually getting errors for invalid syntax.)

module Dict.Extra exposing (keySet)

{-| `Dict.keys` but returning a `Set` instead of a `List`.

    import Dict
    import Set

    Dict.fromList [ ( 0, "A" ), ( 1, "B" ), ( 2, "C" ) ]
        |> Dict.Extra.keySet
    --> Set.fromList [ 0, 1, 2 ]

-}
keySet = ...

which will generate a test with

Dict.fromList [ ( 0, "A" ), ( 1, "B" ), ( 2, "C" ) ]
    |> keySet
    |> Expect.equal (Set.fromList [ 0, 1, 2 ])

why?

Finding broken or incorrect examples in the documentation is confusing and frustrating to new users. At the same time, these examples quickly get out of sync with your API. Now, how do you find all the places where things changed for your examples? The compiler certainly doesn't check them which makes it easy to miss some

setup

• elm install lue-bird/elm-review-documentation-code-snippet
  

  then add the rule to your review/src/ReviewConfig.elm

  import Review.Rule
  import Review.Documentation.CodeSnippet
  
  config : List Review.Rule.Rule
  config =
      [ Review.Documentation.CodeSnippet.check
      ]

  or if you don't want to install it, yet

  elm-review --template lue-bird/elm-review-documentation-code-snippet/example
  

• add a minimal tests/DocumentationCodeSnippetTest.elm which the rule can overwrite. Something like

  module DocumentationCodeSnippetTest exposing (tests)
  tests =
      tests

  You can add this file to .gitignore.

I suggest running it in the background

elm-review --rules Review.Documentation.CodeSnippet --watch --fix-all-without-prompt

while from time to time keeping an eye on possible reported syntax errors and failing/non-compiling generated tests.

thanks

• Christoph Hermann (stoeffel) for elm-verify-examples which established the core ideas and syntax in a nice command line tool
• dillonkearns for elm-markdown of which parts are used as a base for finding code blocks
• Rupert Smith for elm-syntax-dsl which can pretty print a whole elm file and is compatible with elm-syntax
• miniBill for elm-fast-dict

what could we add in the future?

• fuzzy check syntax. Something like

  --* xs is list unit
  List.take 3 xs |> List.length
  --> Basics.maximum 3 (xs |> List.length)
  
  --* xs, ys is list unit
  List.map2 Tuple.pair xs ys |> List.length
  --> Basics.minimum (xs |> List.length) (ys |> List.length)

  where list unit is interpreted as Fuzz.list (Fuzz.constant ())
• compare imports with used module names and automatically add missing imports as a fix
• ✨ your idea

elm-review-documentation-code-snippet's Issues

Code generation picks the wrong function

Hello again! I think I've found a bug:

Here's an excerpt from a code example in the docs for the record function in my Control module:

    myRecordControl =
        record
            (\name age ->
                { name = name
                , age = age
                }
            )
            |> field .name string
            |> field .age int
            |> endRecord

Generated code:

myRecordControl__Control__record_0 =
    Control.record (\name age -> { name = Control.name, age = age })
        |> Control.field .name Control.string
        |> Control.field .age Control.int
        |> Control.endRecord

The rule seems to get confused and think that the variable name in the anonymous function refers to the top-level definition name from the Control module, but actually it should refer to the argument supplied to the anonymous function.

Generate code from examples even if no `-->` is found

Hi lue-bird!

This is a brilliant elm-review rule, thanks for making it.

I noticed that if you define a function/constant in a documentation code example, the rule will generate the code for that function/constant in the test file. But this only happens if the code example ends with a -->.

I have some docs where I provide examples of how to use the API of my package, but it doesn't really make sense to show the reader the value that results from the code, so I don't add a --> line. For example:

myDict = 
    dict int string

Where dict, int and string are functions in my package, and the value of myDict is a fairly complex opaque type that contains a function, which the user of the package doesn't need to know about.

It would be really useful if the rule would still generate the code for myDict in the test file, because then if I rename one of my API functions but forget to update the name in the code example, I'll get a compilation error in the test file.

What do you think? Would this be possible/desirable? Or maybe I should rethink the way I write my doc examples so that they always end in a -->?