The documentation here: http://tgutu.github.io/clunit/#clunit_4 describes a test case:

;; Define a test called TEST-INT1
(deftest test-int1 (IntegerSuite)
(assert-true (= 1 -1))
(assert-equality 4 (+ 2 2)))

that fails when run because of:

(4 (+ 2 2)) can't be destructured against the lambda list (CLUNIT::TEST CLUNIT::VALUE CLUNIT::EXPRESSION &BODY CLUNIT::FORMS), because it does not contain at least 3 elements.

Just a brief feature request: I'd love if there were a way to list defined tests. Or, given a suite, to list all tests within that suit.

I want to inspect test results programmatically (for the sake of https://github.com/cl-test-grid/cl-test-grid/ project).

In my code I use clunit::test-reports, clunit::test-name, clunit::passed-p.

It would be nice to have them exported from the clunit package.

https://notabug.org/cage/clunit2

http://quickdocs.org/clunit2/

For programmatically inspect the test result, I would like at least one method to check the run-suite is successful or not. It could use following condition:

(and (zerop failed)
     (zerop errors))

Some other methods to inspect the clunit-report object is appreciated.

The following does not work:

> (flet ((foo () 5)) (deftest test () (assert-equalp 5 (foo)))) ; looks for (foo) in global env
> (let ((a 4)) (deftest test () (assert-equalp a 4))) ; same issue - a in global environment

Is this a bad practice? This is an "issue" in lisp-unit as well.

It looks like you're depending on with-gensyms without having alexandria in the :depends-on in clunit.asd or package.lisp. This was giving me errors:

Undefined function CLUNIT::RESULT called with arguments () .

Once I imported/used alexandria, everything worked fine.

It would be more convenient to write

(deftest foo (bar)
  (assert-equality #'my-fancy-equality value1 expression1)
  (assert-equality #'my-fancy-equality value2 expression2)
  (assert-equality #'my-fancy-equality value3 expression3))

as something like

(deffixture bar (@body)
  (let ((*assert-same-test* #'my-fancy-equality))
    @body))

(deftest foo (bar)
  (assert-same value1 expression1)
  (assert-same value2 expression2)
  (assert-same value3 expression3))

The wrapper could also be something like

(deffixture bar (@body)
  (with-assert-same-test (#'my-fancy-equality)
    @body))

Obviously the function/macro/variable names are just examples and it is very likely that you can come up with something better. This functionality is present in other libraries, eg lift:*lift-equality-test*.

See quicklisp/quicklisp-projects#443

Under CCL (Clozure) the following code:

(deftest vector-addition (arithmetic)
(assert-true (= 1 -1))
(assert-equality #'eq 4 (+ 2 2)))

(deftest vector-equality (arithmetic)
(assert-equality #'eq 4 (+ 2 2)))

demonstrates the (assert-equality) fails for the vector-addition test, but passes in the vector-equality one, despite having the same conditions.

Is this a bug, or do I misunderstand how this is supposed to work?

In Clozure, and perhaps other lisps (I couldn't try), the report format differs from that in the tutorial. Newlines are missing, e.g.:

(cls-tests:run)

PROGRESS:

TESTS: (Test Suite)

    COMPOUND: (Test Suite)

        ARITHMETIC: (Test Suite)
            VECTOR-ADDITION: F.

FAILURE DETAILS:================TESTS -> COMPOUND -> ARITHMETIC: (Test Suite)
VECTOR-ADDITION: Expression: (= 1 -1)
Expected: T
Returned: NIL
SUMMARY:========Test functions:Executed: 1Skipped: 0Tested 2 assertions.Passed: 1/2 ( 50.0%)Failed: 1/2 ( 50.0%)

Expected Behavior

Running a test with the same code, and the same output settings will result in the same output regardless of if it has been run from within the REPL or from the command line.

Actual Behavior

Detailed output is provided in the REPL but the output produced when run from the command line includes only the bare minimum of info required to know if something passed or failed

Why from the command line?

Because how else are you going to do continuous testing as you develop?
Have something watch the dir that includes the tests (as you update them) and the code (as you update it)... have it kick off your test suite every time it's run.

Here's the output that is produced when run from within clisp REPL:

PROGRESS:
=========

    TESTSUITE: (Test Suite)
        TEST-CLI-OUTPUT: F

FAILURE DETAILS:
================

    TESTSUITE: (Test Suite)
        TEST-CLI-OUTPUT: Expression: (EQUAL 1 2)
          Expected: 1
          Returned: 2
          Expected to fail


SUMMARY:
========
    Test functions:
            Executed: 1
            Skipped:  0

    Tested 1 assertion.
            Failed: 1/1 (100.0%)

Here is the output that's produced when you run the same tests from the command line.

PROGRESS:
=========

    TESTSUITE: (Test Suite)
        TEST-CLI-OUTPUT: F

Here's the content of the file used to produce this output:

#! /usr/local/bin/clisp

#-quicklisp
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp" (user-homedir-pathname))))
    (when (probe-file quicklisp-init)
        (load quicklisp-init)))

;; WHEN IN THE REPL JUST RUN THE FOLLOWING

(ql:quickload "clunit")

(use-package :clunit)

(defsuite TestSuite ())

(deftest test-cli-output (TestSuite)
    (assert-equal 1 2 "Expected to fail"))

(run-suite 'TestSuite)

SBCL (1.1.3) warns about an undefined variable when loading the library:

; file: /home/tamas/src/lisp/clunit/src/functions/rerun-failed-tests.lisp
; in: DEFUN CLUNIT:RERUN-FAILED-TESTS
; (SETF CLUNIT::QUEUED-TESTS (LIST)
; CLUNIT::LAST-CLUNIT-REPORT CLUNIT::CLUNIT-REPORT)
; --> PROGN SETF
; ==>
; (SETQ CLUNIT::QUEUED-TESTS (LIST))
;
; caught WARNING:
; undefined variable: CLUNIT::QUEUED-TESTS

; file: /home/tamas/src/lisp/clunit/src/functions/test-case-functions.lisp
; in: DEFUN CLUNIT:RUN-TEST
; (SETF CLUNIT::QUEUED-TESTS (LIST)
; CLUNIT::LAST-CLUNIT-REPORT CLUNIT::CLUNIT-REPORT)
; --> PROGN SETF
; ==>
; (SETQ CLUNIT::QUEUED-TESTS (LIST))
;
; caught WARNING:
; undefined variable: CLUNIT::QUEUED-TESTS
;
; compilation unit finished
; Undefined variable:
; CLUNIT::QUEUED-TESTS
; caught 2 WARNING conditions

As you probably know, when WARINIG is signalled by SBCL, ASDF considers it a failure
(acccording to asdf:compile-file-failure-behaviour)

Unfortunately old ASDF looses some warnings due to cl:with-compilation-unit around the whole ASDF system.
New ASDF (recently released 2.31.8) fixes this and can detect all the warnings.

To avoid breaking existing systems this improved warnings detection is still disabled in ASDF, but very
soon it is going to be enabled.

In short, there is a warning(s) in clunit which is ignored now, but soon will break the compilation of the system.
Here is the first warning breaking the compilation:

(ql:quickload :clunit :verbose t)

[lot of output]

; file: C:/Users/anton/quicklisp/dists/quicklisp/software/clunit-20130128-git/src/functions/test-suite-functions.lisp
; in: DEFUN CLUNIT:RUN-SUITE
;     (SETF CLUNIT::*QUEUED-TESTS* (LIST)
;           CLUNIT::*LAST-CLUNIT-REPORT* CLUNIT::*CLUNIT-REPORT*)
; --> PROGN SETF
; ==>
;   (SETQ CLUNIT::*QUEUED-TESTS* (LIST))
;
; caught WARNING:
;   undefined variable: CLUNIT::*QUEUED-TESTS*
;
; compilation unit finished
;   Undefined variable:
;     CLUNIT::*QUEUED-TESTS*
;   caught 1 WARNING condition

debugger invoked on a ASDF/LISP-BUILD:COMPILE-FAILED-ERROR in thread #<THREAD "main thread" RUNNING {23D98F31}>: Lisp compilation failed

To reproduce it yourself get the recent ASDF (>= 2.31.8) and call (asdf:enable-deferred-warnings-check).
Then (ql:quickload :clunit :verbose t) will fail as shown above.

Tested at quicklisp 2013-01-28, clunit-20130128-git.

I see you have closed issue #3 two month ago, but the problem still happens, and the specials.lisp file doesn't define the variable queued-tests, it defines only queued-tests-reports