Comments (11)
I think it's a good idea. The unit test need some cleanup (see Issue #27). Testing all the error cases is especially important (where the error happening and being caught is the success).
from json-fortran.
Do you have any preferences for directory structure and naming convention if we split json_example.f90 into individual tests? I’m gonna tackle this now I think.
from json-fortran.
Maybe a tests
directory? With file names test1.f90
, test2.f90
, etc.
from json-fortran.
Under src, or along side?
from json-fortran.
Hmmm.. not sure. Maybe under src?
from json-fortran.
Right now this is what I’m thinking in terms of splitting out the tests:
- Move
json_example.f90
tojson_unit_tests.f90
and make it a module rather than a program. - Retain the executable program portion of
json_example.f90
andUSE
thejson_unit_tests
module
to pull in the test procedures. This means that very little modification of the build systems is required, and the example program is still fully functional, but split across two files. - This is where we have some choices
- I’m tempted to use CMake’s ability to automatically generate test drivers which would:
- require a C compiler be installed to run the CMake unit tests—this is almost guaranteed if a Fortran compiler is installed, but I have mixed feelings about this
- automatically generate the test driver program, and compile and link in the fortran tests (with
iso_c_binding
wrappers) - make writing the test effortless
- allow us to create things like a CDash dashboard
- mean that the Scons and
build.sh
build/test scripts would still just run unified tests injson_example.f90
but CMake and CTest would have more fine grained test control and results. The fine grained CMake/CTest test control can be implemented either way, but will be more work without using the automatic test driver build.
- OR we could just create a number of stand alone programs, which, for now, call the procedures in the
json_unit_tests
module.- We’ll have to script each of them to be called in each build system
- We’ll need some way in each build system to check for success or failure
- Could use regexp matching
- Comparison with known valid output?
- I’m tempted to use CMake’s ability to automatically generate test drivers which would:
Any feedback appreciated esp. CC: @jacobwilliams
from json-fortran.
I would prefer the simpler option of just having stand-alone programs, with each one a separate test. And then the master unit test script (Python maybe) calls each of them and determines if each one passes or fails. I know there are unit-testing frameworks out there, I'm just not familiar with the different options.
I would choose whatever option is the simplest, easy to use and add new tests, and easiest for a poor Fortran man to understand!
from json-fortran.
Well… the cmake one just requires you to write a bind(c) function for each test, put it in its own file having the same name as the function, and add it to a list of test sources in CMake… CMake builds and links etc for you and you can run all test with make test
or ctest --verbose
so it is actually quite simple for the end user… they just need a c compiler, but they don’t need to know how to use it.
btw do you have any insight on updating the SConstruct build in the case that json_example.f90 has been spit into json_unit_tests.f90
(module) and json_example.f90
(main program)
from json-fortran.
OK, I have thought more about this, and I have come up with what I think is a much better and simpler solution than the one I proposed above:
- Move
build.sh
—>build-and-test.sh
, building and running tests unless environment variable$JF_SKIP_TESTS
is set totrue
- Completely refactor
json_example.f90
into individual test programs and get rid ofjson_example.f90
altogether—people looking for usage examples can just look at the unit tests directly. - Each test subroutine from
json_example.f90
will now be its own program stored in a file namedjf_test_*.f90
. This will let us give tests descriptive or numeric names, while allowing us to vacuum up all test programs in the tests directory with some simple file globs. This will also make it easy to run the tests with CMake. - Each of these test programs will use
stop 1
statements to let the driver script know an error occurred. Both GFortran and Intel compilers will set$?
, the return value, of programs according to thestop
statement. Withset -e
inbuild-and-test.sh
this means that the travis-ci will stop and report failures as soon as the first test encounters an error. There are ways to change this so that it will attempt all the tests, which could be desirable (although will take longer) and we can change this if that’s what you prefer. - All test programs print informational messages to stderr and any json output to stdout, so that it may be redirected into jsonlint or similar for validation.
- If, in the future, we want to call the tests from some test harness/driver we can always have each source file contain a module with the test subroutine and then use conditional compilation with preprocessing to determine whether the stand alone program should be compiled (to avoid multiple definitions of
_main
):
module js_test_strprint_mod
use json_module
implicit none
! constants etc here
contains
subroutine test_strprint(err_count)
integer, intent(out) :: err_count
! do tests
end subroutine
end module
#ifndef NO_STAND_ALONE_TESTS
program jf_test_strprint
use jf_test_strprint_mod
implicit none
integer :: errors
call test_strprint(errors_
if (errors /= 0) stop 1
end program
#endif
@jacobwilliams do you think this is simple/KISS enough for “fortran poor man”? 😄
from json-fortran.
That sounds great!! I especially like the idea of being able to add a new jf_test_*
file to the tests directory, and it is automatically included in the test run, rather than having to update all the various build files.
Also, it might be good for the build script to have a command line flag like -skiptests
if you don't want to run the tests. A poor fortran man doesn't know how to set environment variables. Ha ha!
from json-fortran.
HAHA, well this “poor fortran man” is a bit rusty on bash command line flags (I know you use getopt
or something like that…) so I may end up leaving that for someone else to implement… we’ll see.
from json-fortran.
Related Issues (20)
- null_to_integer_mode mode
- Available in Fedora 35/36 HOT 2
- Cross-platform? HOT 2
- On Definition of operator [] HOT 1
- How to get a derived-type table? HOT 2
- Make unicode support optional for homebrew HOT 1
- Rounding a real when printing HOT 1
- Incorrect mod locations when using FetchContent HOT 3
- Fails to build on PPC due to missing ieee_arithmetic in gfortran HOT 2
- Tests fail with nvfortran compiler HOT 10
- Many tests unable to find executable, several tests fail (PowerPC): what am I doing wrong here? HOT 12
- Check if keyword is valid HOT 1
- Add test to FPM file
- Unable to build json-fortran with nvfortran HOT 1
- libdir and includedir empty in json-fortran.pc HOT 2
- All tests fail with errors like "Could not find executable ./jf_test_45" HOT 2
- json parse failed HOT 1
- Can not find some variables in json file HOT 1
- Write out 2d array to JSON file HOT 3
- Question: dealing with duplicate named objects HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from json-fortran.