tgutu / clunit Goto Github PK
View Code? Open in Web Editor NEWCLUnit: A Common Lisp Unit Testing Framework.
Home Page: http://tgutu.github.com/clunit
License: MIT License
CLUnit: A Common Lisp Unit Testing Framework.
Home Page: http://tgutu.github.com/clunit
License: MIT License
The documentation here: http://tgutu.github.io/clunit/#clunit_4 describes a test case:
;; Define a test called TEST-INT1
(deftest test-int1 (IntegerSuite)
(assert-true (= 1 -1))
(assert-equality 4 (+ 2 2)))
that fails when run because of:
(4 (+ 2 2)) can't be destructured against the lambda list (CLUNIT::TEST CLUNIT::VALUE CLUNIT::EXPRESSION &BODY CLUNIT::FORMS), because it does not contain at least 3 elements.
Just a brief feature request: I'd love if there were a way to list defined tests. Or, given a suite, to list all tests within that suit.
I want to inspect test results programmatically (for the sake of https://github.com/cl-test-grid/cl-test-grid/ project).
In my code I use clunit::test-reports, clunit::test-name, clunit::passed-p.
It would be nice to have them exported from the clunit package.
For programmatically inspect the test result, I would like at least one method to check the run-suite is successful or not. It could use following condition:
(and (zerop failed)
(zerop errors))
Some other methods to inspect the clunit-report
object is appreciated.
The following does not work:
> (flet ((foo () 5)) (deftest test () (assert-equalp 5 (foo)))) ; looks for (foo) in global env
> (let ((a 4)) (deftest test () (assert-equalp a 4))) ; same issue - a in global environment
Is this a bad practice? This is an "issue" in lisp-unit
as well.
It looks like you're depending on with-gensyms
without having alexandria
in the :depends-on
in clunit.asd or package.lisp. This was giving me errors:
Undefined function CLUNIT::RESULT called with arguments () .
Once I imported/used alexandria, everything worked fine.
It would be more convenient to write
(deftest foo (bar)
(assert-equality #'my-fancy-equality value1 expression1)
(assert-equality #'my-fancy-equality value2 expression2)
(assert-equality #'my-fancy-equality value3 expression3))
as something like
(deffixture bar (@body)
(let ((*assert-same-test* #'my-fancy-equality))
@body))
(deftest foo (bar)
(assert-same value1 expression1)
(assert-same value2 expression2)
(assert-same value3 expression3))
The wrapper could also be something like
(deffixture bar (@body)
(with-assert-same-test (#'my-fancy-equality)
@body))
Obviously the function/macro/variable names are just examples and it is very likely that you can come up with something better. This functionality is present in other libraries, eg lift:*lift-equality-test*
.
Under CCL (Clozure) the following code:
(deftest vector-addition (arithmetic)
(assert-true (= 1 -1))
(assert-equality #'eq 4 (+ 2 2)))
(deftest vector-equality (arithmetic)
(assert-equality #'eq 4 (+ 2 2)))
demonstrates the (assert-equality) fails for the vector-addition test, but passes in the vector-equality one, despite having the same conditions.
Is this a bug, or do I misunderstand how this is supposed to work?
In Clozure, and perhaps other lisps (I couldn't try), the report format differs from that in the tutorial. Newlines are missing, e.g.:
(cls-tests:run)
TESTS: (Test Suite)
COMPOUND: (Test Suite)
ARITHMETIC: (Test Suite)
VECTOR-ADDITION: F.
FAILURE DETAILS:================TESTS -> COMPOUND -> ARITHMETIC: (Test Suite)
VECTOR-ADDITION: Expression: (= 1 -1)
Expected: T
Returned: NIL
SUMMARY:========Test functions:Executed: 1Skipped: 0Tested 2 assertions.Passed: 1/2 ( 50.0%)Failed: 1/2 ( 50.0%)
Running a test with the same code, and the same output settings will result in the same output regardless of if it has been run from within the REPL or from the command line.
Detailed output is provided in the REPL but the output produced when run from the command line includes only the bare minimum of info required to know if something passed or failed
Because how else are you going to do continuous testing as you develop?
Have something watch the dir that includes the tests (as you update them) and the code (as you update it)... have it kick off your test suite every time it's run.
Here's the output that is produced when run from within clisp REPL:
PROGRESS:
=========
TESTSUITE: (Test Suite)
TEST-CLI-OUTPUT: F
FAILURE DETAILS:
================
TESTSUITE: (Test Suite)
TEST-CLI-OUTPUT: Expression: (EQUAL 1 2)
Expected: 1
Returned: 2
Expected to fail
SUMMARY:
========
Test functions:
Executed: 1
Skipped: 0
Tested 1 assertion.
Failed: 1/1 (100.0%)
Here is the output that's produced when you run the same tests from the command line.
PROGRESS:
=========
TESTSUITE: (Test Suite)
TEST-CLI-OUTPUT: F
Here's the content of the file used to produce this output:
#! /usr/local/bin/clisp
#-quicklisp
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp" (user-homedir-pathname))))
(when (probe-file quicklisp-init)
(load quicklisp-init)))
;; WHEN IN THE REPL JUST RUN THE FOLLOWING
(ql:quickload "clunit")
(use-package :clunit)
(defsuite TestSuite ())
(deftest test-cli-output (TestSuite)
(assert-equal 1 2 "Expected to fail"))
(run-suite 'TestSuite)
SBCL (1.1.3) warns about an undefined variable when loading the library:
; file: /home/tamas/src/lisp/clunit/src/functions/rerun-failed-tests.lisp
; in: DEFUN CLUNIT:RERUN-FAILED-TESTS
; (SETF CLUNIT::QUEUED-TESTS (LIST)
; CLUNIT::LAST-CLUNIT-REPORT CLUNIT::CLUNIT-REPORT)
; --> PROGN SETF
; ==>
; (SETQ CLUNIT::QUEUED-TESTS (LIST))
;
; caught WARNING:
; undefined variable: CLUNIT::QUEUED-TESTS
; file: /home/tamas/src/lisp/clunit/src/functions/test-case-functions.lisp
; in: DEFUN CLUNIT:RUN-TEST
; (SETF CLUNIT::QUEUED-TESTS (LIST)
; CLUNIT::LAST-CLUNIT-REPORT CLUNIT::CLUNIT-REPORT)
; --> PROGN SETF
; ==>
; (SETQ CLUNIT::QUEUED-TESTS (LIST))
;
; caught WARNING:
; undefined variable: CLUNIT::QUEUED-TESTS
;
; compilation unit finished
; Undefined variable:
; CLUNIT::QUEUED-TESTS
; caught 2 WARNING conditions
As you probably know, when WARINIG is signalled by SBCL, ASDF considers it a failure
(acccording to asdf:compile-file-failure-behaviour)
Unfortunately old ASDF looses some warnings due to cl:with-compilation-unit around the whole ASDF system.
New ASDF (recently released 2.31.8) fixes this and can detect all the warnings.
To avoid breaking existing systems this improved warnings detection is still disabled in ASDF, but very
soon it is going to be enabled.
In short, there is a warning(s) in clunit which is ignored now, but soon will break the compilation of the system.
Here is the first warning breaking the compilation:
(ql:quickload :clunit :verbose t)
[lot of output]
; file: C:/Users/anton/quicklisp/dists/quicklisp/software/clunit-20130128-git/src/functions/test-suite-functions.lisp
; in: DEFUN CLUNIT:RUN-SUITE
; (SETF CLUNIT::*QUEUED-TESTS* (LIST)
; CLUNIT::*LAST-CLUNIT-REPORT* CLUNIT::*CLUNIT-REPORT*)
; --> PROGN SETF
; ==>
; (SETQ CLUNIT::*QUEUED-TESTS* (LIST))
;
; caught WARNING:
; undefined variable: CLUNIT::*QUEUED-TESTS*
;
; compilation unit finished
; Undefined variable:
; CLUNIT::*QUEUED-TESTS*
; caught 1 WARNING condition
debugger invoked on a ASDF/LISP-BUILD:COMPILE-FAILED-ERROR in thread #<THREAD "main thread" RUNNING {23D98F31}>: Lisp compilation failed
To reproduce it yourself get the recent ASDF (>= 2.31.8) and call (asdf:enable-deferred-warnings-check).
Then (ql:quickload :clunit :verbose t) will fail as shown above.
Tested at quicklisp 2013-01-28, clunit-20130128-git.
I see you have closed issue #3 two month ago, but the problem still happens, and the specials.lisp file doesn't define the variable queued-tests, it defines only queued-tests-reports
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.