Run Test Suite classes with method per test about vba-test

[timhall]

I'm heading towards method-per-test too, I've been bitten by variables / errors from other tests causing issues too. I have some plans for how to build the process into vba-test and will definitely take a deep look at your suggestions. I'll get back to you on this

[robodude666]

@timhall good to hear! Looking forward to it 👍

@connerk I originally tried something similar but the issue I have with it is you need to remember to add the "sub" test suite. The benefit of discovery (as outlined above) is all you need to do is add a public sub and the test will automatically run.

[connerk]

Good plan fellas.
I got something similar working in a different way last December
starting at 4c34095
and improved through 0d8b936

basically, I added a collection of specsuites to the specsuite class, allowing nesting. it's not explicitly one test per method but you could do that if you wanted to.

it made reporting of large numbers of tests more organized in the workbook reporter as well by allowing for collapsible, nested reports.

how to structure the code is in the Readme

example WorkbookReporter output

food for thought at least I hope. 😄

[connerk]

any consideration on using comment markers?
Rubber duck calls them Annotations
NUnit calls them Attributes

[robodude666]

@timhall btw, what were you thinking of?

@connerk The way RubberDuck handles Annotations via an ANTLR grammer makes it really easy on the implementation. However, if we do it from VBA you can't really do that very well. In VBA you can very easily iterate through the CodeModule fairly efficiently and get the name of the procedure (via ProcOfLine).

It is technically possible to use the same method and then parse the comments before the beginning of the procedure, but I donno.

I do like the idea through of supporting skipping tests or marking them as expecting to fail, but I don't like the idea of needing to add extra stuff to make a test run. Perhaps a combination of both? Using signature to detect "TestMethod" and annotations to mark tests for skip/fail?

[timhall]

Generally, I see vba-test as a low-level testing library that can provide the backing for various code generation, annotation, and other approaches. With #27 I think both approaches presented here are doable. I imagine @robodude666 example like the following:

Public Sub RunTests()
  Dim Suite As New TestSuite
  Suite.Name = "Project Name"

  Dim Reporter As New WorkbookReporter
  Reporter.ConnectTo ThisWorkbook.Worksheets("TestRunner")
  Reporter.ListenTo Suite

  Dim DebugReporter As New ImmediateReporter
  DebugReporter.ListenTo Suite

  ' From external library
  LoadTestsFromModule Suite.Group("Module Name"), "ModuleName"
  LoadTestsFromClass Suite.Group("Class Name"), "ClassName"
End Sub

' Example

Public Sub LoadTestsFromModule(Suite As TestSuite, Name As String)
  ' Load CodeModule for Name
  ' Iterate through module looking for ' #[test] attribute
  ' -> Get test name (and modifiers)
  On Error Resume Next  

  For Each TestName In FoundTests
    Set Test = Suite.Test(TestName)
    Application.Run Name & "." & TestName, Test
    Test.NotError
    
    Err.Clear
  Next TestName  
End Sub

Public Sub LoadTestsFromClass(Suite As TestSuite, Name As String)
  ' Similar to above, but with CallByName
End Sub

The point being that LoadTestsFromModule can happen outside of vba-test. With that said I plan on integrating test generation into vba-blocks (once I get it working and released). That will generate and inject the test runner code based on ' #[test] "attributes". This would generate output like the following:

Public Sub RunTests()
  Dim Suite As New TestSuite
  Suite.Name = "Project Name"

  Dim Context As New Tests_Context ' <- Generated by vba-blocks test
  
  With Suite.Group("TestCase")
    Context.TestCase_ListenTo .Self
        
    Context.TestCase_ShouldDoA .Self.Test("should do a")
    Context.TestCase_ShouldDoB .Self.Test("should do b")
  End With
    
  With Suite.Group("TestSuite")
    Context.TestSuite_ListenTo .Self

    Context.TestSuite_ShouldDoC .Self.Test("should do c")
    Context.TestSuite_ShouldDoD .Self.Test("should do d")
  End With
End Sub

This would allow writing tests in modules or classes like the following:

' #[test.before_each]
Public Sub BeforeEach(Test As TestCase)
  ' ...
End Function

' #[test]
Public Sub ShouldAddTwoNumbers(Test As TestCase)
  ' ...
End Function

' #[test.skip]
Public Sub SkippedTest(Test As TestCase)

End Function

' #[test("override test name").only]
Public Sub ShouldAddAnyNumberOfNumbers(Test As TestCase)
  ' ...
End Function

' #[test.group]
Public Sub SubGroup(Suite As TestSuite)
  With Suite.Test("...")
    ' ...
  End With
End Sub

With all that said, I think vba-test will stay somewhat minimal, with either separate packages finding and running tests or via an outside tool like vba-blocks generating test runners