[Joss Review] Tests about seg1d

[cadop]

Your point is well taken. There are definitely more tests I can add to ensure coverage/completeness.

I haven't found that this project benefits much from using specifically the unittest framework over doctest as the algorithms are written pretty functionally so the nice features like magic inputs on some combination haven't been missed so far. Maybe you did track it down, but I am using doctest for more than just the function and class docstrings. There are extra test files that are run (there are over 200 tests run). Many of the examples were made/written as code tests.

So far, relying on doctest forces the documentation to be clear and comprehensive, as pretty much every case that I would want to check I make an example of how it is expected to perform. In my experience, unittests for things like 'assert some function returns a non-empty list' are not as useful as expected output examples, because I have had lists returned that are not empty, but not the expected output.

So I am humbly clear though; it's not to say I think the testing I have is perfect or exhaustive, there is and always will be room for improvement. I could see some students helping and contributing to the project through the testing as a first step on it as well.

Sorry if I missed your point, "happy-path regression/integration tests" is fair, was there other types of tests you expect?

[cadop]

I also setup some github actions to at least make the commits more consistent for checking fc41445

[AKuederle]

Sry for the silience from my side. Just let me give a little bit of context to my issue.

First of all, compared to many other packages out there the testing of seg1d is really solid. However, I think lack of proper testing is one of the biggest issue with many smaller scientific codebases, I wanted to provide some further feedback.

Unittesting: Unittesting is a concept and not really tied to any framework like unittest or pytest. Unittesting is just the concept to isolate small components of you codebase (e.g. a single class or single function) and test the full functionality of this part of the code. Im comparison to testing programs end-to-end, is that focusing on a single function often makes it easier to come up with proper testcases, as the overall complexity is lower.

Happy Path Testing: Happy Path Testing refers to the pitfall of only testing a function in the way you expect an experience user to use it correctly. This often leads to not testing combinations of inputs, that might not be reasonable, not testing that exception are thrown correctly, what happens if functions are called with wrong input data types. In particular the last one is a typical Python problem. Of course it is not feasible to assert and test every function with any possible combinations for input types, but in particular with scientific python there are a couple of things that I usually like to test explicitly: What happens when your function is called with a pandas dataframe instead of an numpy array. Sometimes this leads to different results without throwing an error message. Similarly, I test what happens if float values are passed instead of integer values (or the other way around). Due to the differences how native Python and numpy deal with datatypes during mathematical operations, this can sometimes highlight a couple of edge cases of your function. In general, it is up to the developer to be aware of potential edgecases of the code and write explicit tests for this to make sure that these edge cases are properly handled.

Regarding your tests, I was missing tests that for example check, if exceptions are thrown in the correct way, or if unexpected results occur if invalid input combinations are used. These things are far easier to test, if you isolate a single function in a unittesting concept. This is why I suggested it.

[AKuederle]

As this is not a blocking issue, but just FYI, I will close it