bluebird75 / luaunit Goto Github PK

Being able to skip test is a common feature of unit-tests framework.

Tyipcal usage are :

1. within the test, discover that the test does not apply and skip it by calling skip()
2. within the test, skip the test if some condition does not apply by calling skipIf( condition )
3. mark a test function with an equivalent skipIf( ) so that the whole test function is skipped under some conditions.

For 1. and 2. the behavior of calling skip is the following :

• test is interrupted, control is resumed to the luaunit framework
• test is reported as skipped in the final report

Case 3. is really a nice to have, but it provides a nice coding challenge. Typical usage could be

skipIf( "os" = "windows", def f() ....  )

Report of the test suite should be adjusted in the following way :

• if some tests are skipped in the test suite, add "x tests skipped" in the report
• if no tests are skipped, don't add anything
• do not count the number of skipped tests in the "run" ?

Example for test output

Ran 15 tests in 0.005 seconds, skipped 3 tests
FAILED (run=15, failures=12, skipped=3)

Meaning, total of 18 tests exists, 15 run, 3 skipped.

TAP output :

# Ran 15 tests in 0.012 seconds, 3 successes, 12 failures, 3 skipped

Although I am not totally sure this is the right way of reporting skipped tests. Since a test may be partially run then skipped, it's in a grey area.

For the XML reports, I remember that there is a conflict between where to report skipped tests between ANT and Jenkins. Need to verify that again.

> local prettystr = (require "luaunit").prettystr
> prettystr{ [true] = true } -- okay
> prettystr{ [{}] = {} } -- okay
> prettystr{ string = "string", [true] = true } -- okay
> prettystr{ string = "string", [{}] = {} } -- okay
> prettystr{ string = "string", 1 } -- okay
> prettystr{ 1, [true] = true } -- error: attempt to compare boolean with number
> prettystr{ 1, [{}] = {} } -- error: attempt to compare table with number
> prettystr{ [true] = true, [{}] = {} } -- error: attempt to compare boolean with table

It looks like if the set of the types of keys contains more than one of { number, boolean, table } then table.sort throws an error.

luaunit.private.__genSortedIndex splits the table into two parts - those with string keys, and those with all other keys - and then sorts those two parts separately. table.sort can't handle sorting on heterogeneous keys.

The stack trace (for all of these errors):

stack traceback:
    [C]: in function 'sort'
    ./luaunit.lua:73: in function '__genSortedIndex'
    ./luaunit.lua:105: in function 'for iterator'
    ./luaunit.lua:353: in function <./luaunit.lua:333>
    (...tail calls...)
    ./luaunit.lua:404: in function 'prettystr'
    stdin:1: in main chunk
    [C]: in ?

I use assertIs()/assertNotIs() to verify if two object references (actually two tables with protected metatables) are the same / different. When it fails, I get error cannot change a protected metatable in _table_raw_tostring() function. This is caused by prettystrPairs() function calls when assertIs()/assertNotIs() fails. Here is the example code:

local lu = require "luaunit"

-- To make it easier to reproduce this error, enable `PRINT_TABLE_REF_IN_ERROR_MSG`
lu.PRINT_TABLE_REF_IN_ERROR_MSG = true

local Object = {}
function Object:new(...)
    return setmetatable({
        class = self,
    }, {
        __metatable = "private",
        __index = self,
    })
end

function testA()
    local obj1 = Object:new()
    local obj2 = Object:new()
    lu.assertIs(obj1, obj2) -- will fail
end

local runner = lu.LuaUnit.new()
runner:setOutputType("text")
os.exit(runner:runSuite())

P.S.
When I use assertIs() on tables, I don't really care about the contents in the tables. I do only care whether they are the same. assertEquals() may be a better choice when I care about the contents.Is it possible to show only table references in fail messages of assertIs()/assertNotIs()?

It's a nice feature which I have seen in other test frameworks : provide a command line option to run tests in a random order. This increases the chances of finding some inter-test dependency issues.

Proposal :

• -r / --random command line option
• TestClass order are randomized
• tests within a testclass are randomized

Task is up for adoption

Proposal for extension:
As a developer I want to check in my test case if a table does (not) contain a certain value.

It could look like this in the code:

table = some_function_to_be_tested(...)
luaunit.assertContains(table, "some value")
luaunit.assertNotContains(table, "other value")

This is my current workaround:

tablex = require('pl.tablex')
function contains(table, value)
  return tablex.find(table, value) ~= nil
end

luaunit.assertTrue(contains(table,"some value"))
luaunit.assertFalse(contains(table,"other value"))

In example below, the assertion fails however tables have the same structure

local a={}
local b={}
a.x = b; b.x = a
local a1={}
local b1={}
a1.x = b1; b1.x = a1
lu.assertEquals(a, a1)

I am not familiar with code base of luaunit and decisions that was made during the development of this project, but possibly assertEquals can be extended with approach correlated with code below. This is some kind of prototype that works but maybe it is not optimal. The idea beyound this is if two tables have the same structure and we maintain current path to subtable that currenlty compared, we can check a new table gotten as a value from current subtable on existance in this path. If this table is in the path we asume that the table in the table we compare against should have correlated table on the same position in the path, if not assert fails.

local function assertEqualsWithCycles(op1, op2)
    local compared = {}
    local function check_in_compared(v1)
        for _, v in ipairs(compared) do
            if v.v1 == v1 then
                return v.v2
            end
        end
    end

    local function recursive_compare(op1, op2)
        table.insert(compared, {v1=op1, v2=op2})
        print(op1, op2)
        for k, v1 in pairs(op1) do
            local v2 = op2[k]
            if type(v1) == "table" and type(v2) == "table" then
                local v2_old = check_in_compared(v1)
                if v2_old ~= nil then
                    if v2_old ~= v2 then
                        return false
                    end
                else
                    if not recursive_compare(v1, v2) then
                        return false
                    end
                end
            else
                if v1 ~= v2 then
                    return false
                end
            end
        end
        table.remove(compared)
        return true
    end
    return recursive_compare(op1, op2)
end

When comparing very long strings where only a small part of it is different it is quite difficult to see in the output what the difference is.

• It may be useful to omit some part of the string at the beginning and at the end of the string that is equal
• mark the part of the string that is different with [...] for example
• use colors if terminal supports it? Or enable colors with commandline param

FYI JUnit does it this way (note the three dots and braces):

assertEquals("1111111111111111111111111111111111111111111114111111111111111","1111111111111111111111111111111111111111111111411111111111111");
expected:<...11111111111111111111[41]11111111111111> but was:<...11111111111111111111[14]11111111111111>

Something similar would be very useful when comparing big tables.

Hi I've currently written tests using the lua busted framework but turns out there isn't any maven plugins for busted. I came across luaunit and you claim that I can use luaunit in maven. I tried looking for the luaunit maven plugin but couldn't find any. Could you tell me how I could incorporate this luaunit test on maven. Thank you.

Hi guys,

I stumbled across a funny issue: I'm trying to nest testcases using multiple modules and tying them together into a test case collection by "require"ing the other modules. However there seems to be some environment-fu involved which not seem to allow me gluing my test cases together.

NB: Luaunit is not invoked by the Lua interpreter started from command line directly but by using a test driver written in C which sets up the environment via luaL_dostring (L, "luaunit = require ('luaunit')"), executes the testcases via luaL_dofile (L, execfile) and then fires of the testing using:

local lu = luaunit.LuaUnit.new ();
lu:setOutputType (__outputtype);
lu:setFname (__fname);
__err = lu:runSuite ()

All of that works fine when calling the luaL_dofile on the test case directly.

But If I attempt to run the luaL_dofile on something like:

TestFoo = require "test_foo"
TestBar = require "test_bar"

the testdriver will simply run zero tests without any errors. When inspecting the any of the tables returned by "require" which are supposed to contain my test functions I will get something which looks an awful lot like the table of of the luaunit module:

LINE_LENGTH 80
VERBOSITY_QUIET 0
assertIsString  function: 0x7fd7e8609a90
assertNotAlmostEquals   function: 0x7fd7e8609280
PRINT_TABLE_REF_IN_ERROR_MSG    false
assertIs    function: 0x7fd7e8609be0
prettystr   function: 0x7fd7e8608710
assertNotNil    function: 0x7fd7e8609910
LuaUnit table: 0x7fd7e8608a90
VERBOSITY_LOW   1
assertEquals    function: 0x7fd7e8609940
VERBOSITY_DEFAULT   10
assertTrue  function: 0x7fd7e8609880
assertItemsEquals   function: 0x7fd7e8609c60
assertFalse function: 0x7fd7e86098b0
assertStrIContains  function: 0x7fd7e86092f0
assertIsNumber  function: 0x7fd7e8609a30
VERSION 3.1
assertIsNil function: 0x7fd7e8609b20
assertNotStrIContains   function: 0x7fd7e8609370
assertErrorMsgMatches   function: 0x7fd7e86099b0
assertNotEquals function: 0x7fd7e8609240
NodeStatus  table: 0x7fd7e860afe0
assertNotIs function: 0x7fd7e8609c20
assertIsThread  function: 0x7fd7e8609bb0
assertIsCoroutine   function: 0x7fd7e8609bb0
assertIsUserdata    function: 0x7fd7e8609b80
assertIsFunction    function: 0x7fd7e8609b50
assertErrorMsgEquals    function: 0x7fd7e8609420
USAGE   Usage: lua <your_test_suite.lua> [options] [testname1 [testname2] ... ]
Options:
  -h, --help:             Print this help
  --version:              Print version information
  -v, --verbose:          Increase verbosity
  -q, --quiet:            Set verbosity to minimum
  -o, --output OUTPUT:    Set output type to OUTPUT
                          Possible values: text, tap, junit, nil
  -n, --name NAME:        For junit only, mandatory name of xml file 
  -p, --pattern PATTERN:  Execute all test names matching the lua PATTERN
                          May be repeated to include severals patterns
                          Make sure you esape magic chars like +? with %
  testname1, testname2, ... : tests to run in the form of testFunction, 
                              TestClass or TestClass.testMethod

assertStrMatches    function: 0x7fd7e86093b0
assertNotStrContains    function: 0x7fd7e8609330
assertStrContains   function: 0x7fd7e86092b0
assertErrorMsgContains  function: 0x7fd7e8609450
assertIsTable   function: 0x7fd7e8609ac0
assertAlmostEquals  function: 0x7fd7e8609210
assertError function: 0x7fd7e8609850
assertIsBoolean function: 0x7fd7e8609af0
VERBOSITY_VERBOSE   20
assertNil   function: 0x7fd7e86098e0
ORDER_ACTUAL_EXPECTED   true
private table: 0x7fd7e8607e70

Any ideas how to get this done?

Hi there,

Does luaunit have an equivalent to Python's test discovery functionality, as described here?

Currently I have a "runner" file that must list the individual tests (see here), and I want to avoid this.

Sorry, I'm a bit of a Lua newb!

Thanks,
Adrian

There are a few minor issues.

1. CDATA section is closed incorrectly in addFailure, missing >
2. failure type attribute is closed incorrectly, missing trailing "
3. various attribute values require protection for reserved XML characters:
   < and > are used in testsuite and testcase name attribute for global test functions, and
   " and ' are used in certain failure type (e.g., errorMsg generated by assertEquals for a string is prettyPrinted). For completeness, you probably want to handle & in the failure type as well since this includes error messages provided by user code.

It would be interesting to see if we have some 64bits number assumptions in luaunit. For what I know, compiling lua with 32bits floats is common in the embedded world. Worth checking !

Hi, I am testing with v. 3.2.1-1

I think luaunit should be able to assert errors that are raised as tables:

luaunit = require('luaunit')
luaunit.assertErrorMsgEquals({id=100}, function() error({id=100}) end)
stdin:1: LuaUnit test FAILURE: Exact error message expected: {id=100}
Error message received: {id=100}

stack traceback:
[C]: in function 'error'
/usr/local/share/lua/5.1/luaunit.lua:608: in function 'failure'
/usr/local/share/lua/5.1/luaunit.lua:613: in function 'fail_fmt'
/usr/local/share/lua/5.1/luaunit.lua:796: in function 'assertErrorMsgEquals'
stdin:1: in main chunk
[C]: ?

I think it should work similarly to assertItemsEquals but against the raised error.

Hey, while we are at having fun with Travis CI, why not take advantage of their MacOs support ?

I don't think it brings any new stuff to luaunit, as it is a pure lua module, platform independant by design but I find the idea of covering the 3 major platform pleasant.

Goal : add MacOs build to travis-ci for luaunit.

This happens when I try to clone:

raphael@r0fls ~/Desktop/code/r0fls $ git clone [email protected]:bluebird75/luaunit.git
Cloning into 'luaunit'...
Warning: Permanently added the RSA host key for IP address '192.30.252.121' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Here it works for a different repo:

Please make sure you have the correct access rights
and the repository exists.
raphael@r0fls ~/Desktop/code/r0fls $ git clone https://github.com/orangeduck/tgc.git
Cloning into 'tgc'...
remote: Counting objects: 42, done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 42 (delta 14), reused 40 (delta 12), pack-reused 0
Unpacking objects: 100% (42/42), done.
Checking connectivity... done.

I tried using https:// instead of git@ for installing luaunit but it still did not work. Are the access rights correct for this repository?

In luaunit test, I would like to force the test to pass if an assert do not cause an error.
For example if i am testing an error case like this:
'assert(msg == expected_error)'
If the test asserts here, fine - there error message is correct.
If not, I would like to force the test to pass and not continue to other lines of code.

Thanks in advance

Hi,

I would like to know how and why you use luaunit !

If you can, please also list the following information :

• since when you have been using LuaUnit
• number of tests / classes
• purpose of your project
• if open source, link to the project repository
• why you liked luaunit

Cheers,

Philippe

This code section https://github.com/bluebird75/luaunit/blob/master/luaunit.lua#L373-L382 can be remove, because pairs include number index key

Context : LuaUnit version 3 should be backward compatible with LuaUnit v2 which was forked .

LuaUnit v2 supports different names for setup / teardown methods ( see https://github.com/rjpcomputing/luaunit ). LuaUnit v3 should support all the methods :

• names to support : 'setUp' or 'Setup' or 'setup'. 'tearDown' or 'TearDown' or 'teardown'.

Unit-test should be added to verify that this is properly supported.

If I try any other function listed in the documentation besides assertEquals, I get attempt to call global 'assertX (a nil value).

This is probably related, but I'm running the script by using torch, as follows:

$ th unittest.lua

If I try running it with lua unittest.lua I get an error saying that I don't have lua installed. I found this odd, because if I type print(_VERSION) in the torch REPL, I get Lua 5.1.

Hi @bluebird75,

as the Debian Maintainer of luaunit I would like to be able to track the releases with the proper tags.

I've created some at https://github.com/linuxmaniac/luaunit/releases

Please, at least push the tag for the new 1.6 release when it's done.

Thanks,
Victor

It would be very nice to create custom assertions like this:

local luaunit = require('luaunit')

local msg = "Failed assertion on data set #%s"
local data = { ... }

for k, v in pairs(data) do
    -- the fourth parameter is custom assertion message
    luaunit.assertEquals(v.actual, v.expected, true, string.format(msg, k))
end

I am preparing a v3.2 release for soon.
@AbigailBuccaneer @n1tehawk if you have some pending modifications which you would like to push, now is the good time !

Cheers,

Philippe

There are several locations where "local" seems to be missing:

Line 1824: name, instance = v[1], v[2]
Line 1852: name, instance = v[1], v[2]
Line 2016: testNames = options['testNames']

Found using http://www.lua.org/extras/5.3/strict.lua

Hello,

Since reviewing this (https://www.lua.org/wshop06/Belmonte.pdf) document, I've been using table (class instances, really) as my favorite "error message". I do implement __tostring in these so it is fully possible to convert these messages to strings which luaunit can perform sensible comparisons against.

I'll submit a PR in the next few minutes to show you what I'm thinking of.

Thanks,
Doug

Would it be possible to add documentation about using luaunit with travis-ci or circleci? Or maybe it's already here somewhere...

With the recent commits I'm observing Travis CI build failures for the up-to-date 2.1.0-beta2 branch of LuaJIT, see e.g. https://travis-ci.org/bluebird75/luaunit/builds/133499924. These weren't present when creating the corresponding pull request originally (end of April), and seem to indicate a LuaJIT-2.1 regression. Errors now even show up when rebuilding older commits that definitely worked before, see https://travis-ci.org/n1tehawk/luaunit/builds/123109273 for an example.

This is currently not critical, as our Travis CI configuration allows the most recent ("bleeding edge") version of LuaJIT 2.0 and 2.1 to fail without breaking the build. Nevertheless it should be investigated. I'll take a closer look at it soon, but need to set up a test environment first - as currently I'm unable to reproduce the problem locally.

(My impression is we're not alone with this, and LuaJIT 2.1 is struggling a bit as of lately: https://github.com/LuaJIT/LuaJIT/issues?q=is%3Aissue+label%3A2.1+created%3A2016-04-20..2016-05-31)

Hi, I'm using luaunit to write unit tests for my android app. Think about I write a HTTP module, which send requests asynchronously by Java and handle responses in a lua callback function.

And I write a test like this.

function testGet()
    http.get("google", function(response)
        if response.status_code ~= 200 then
            luaunit.fail("status code should be 200")
        end
    end)
end
runner:runSuite()

The test suite ends before the callback is called. So the test result is not collected.

Any idea about how to write test cases in this situation?

Caused by the response to #101

Test case:

file 1 "room.lua"

local _M = {}
local Room = {}
_M.Room = function(o)
  o = o or {}
  setmetatable(o, Room)
  return o
end
function Room:__tostring()
end
return _M

file 2: "tests\test_room.lua"

local Room = require('../room').Room
local lu = require('luaunit')
test_room = {}

function test_room:test_constructor()
  local rm = Room()
  lu.assert_nil(rm, 'this line gets the strange error instead of one failing test')
end
os.exit( lu.run() )

expected result:

>lua tests/test_room.lua
F
Failed tests:
-------------
1) test_room.test_constructor
tests/test_room.lua:10: this line gets the strange error instead of one failing test
expected: nil, actual:
stack traceback:
        tests/test_room.lua:10: in function 'test_room.test_constructor'

Ran 1 tests in 0.000 seconds, 0 successes, 1 failure

actual result:

>lua tests/test_room.lua
E
Failed tests:
-------------
1) test_room.test_constructor
...in/../share/luarocks/rocks/share/lua/5.1/luaunit.lua:234: attempt to index local 'text' (a nil value)
stack traceback:
        (tail call): ?

Ran 1 tests in 0.000 seconds, 0 successes, 1 error

Reason for the problem - no return from the tostring method.

Possible fix:

Line 890 becomes

result = strsplit( '\n', tostring(tbl) or '<<malformed __tostring>>')

I don't know if this is really the best solution, but it does solve this issue. I ran across it when trying to build my tests first, and then was very confused trying to understand why I was getting a luaunit error.

I'm using this to pick up on #21 (comment) - @bluebird75 wrote:

If you feel like you have motivation for further improvements, one next big step is to detect "failures" ( = expected errors) from errors. It's not that difficult, instead of just returning a string with a typical error function when an assertion fails, one needs to return a full object (to be defined) containing the error description (msg, line, file) ... If error still returns a string, it's a true error.

Once that is done, the general framework needs to be improved to report separately errors from failures. The junit-xml would use this info for example. One difficulty is to properly test that within your own unittest, there is a small problem of failing luaunit execution within a luaunit execution, but nothing difficult to solve I believe.

For example, given the script

local lu = require("luaunit")

TestErrorVsFailure = {}

function TestErrorVsFailure:TestB0rken()
    local a = nil
    a.foo = "bar" -- (this will error out)
    lu.assertEquals(a.foo, "foobar")
end

function TestErrorVsFailure:TestFail()
    local a = {}
    a.foo = "bar"
    lu.assertEquals(a.foo, "foobar")
end

lu.LuaUnit.run()

current LuaUnit will report both tests as "failures"; while users might expect to see an actual "error" status on TestB0rken() - possibly with LuaUnit exiting, instead of continuing with other test(suite)s.

Sorry, I'm new in Lua world, but luarocks is installing version 2.1, are there a way to update it?

Thanks!

Please add a way to manually fail tests w/o using an assertFoo. The usecases would be any scenario where the conditions that indicate failure can't be accurately described with assertions.

For example, a scenario similar to assertNotEquals(), where failure is the default path and the function logic checks for success.

Remaining work before releasing v3.3 :

• add assertIsPlusInf and assertIsMinusInf in addition to assertInf
• add documentation about using luaunit for scientific computing
• document the new list comparison analysis

Of course, I will add whatever improvements are also available and consistent to the release.

Line #1725 of current luaunit.lua looks wrong to me, and actually got me an error ("variable 'fullErrMsg' is not declared").

I think the line
local ok=true, fullErrMsg, stackTrace, errMsg, t
should actually be
local ok, fullErrMsg, stackTrace, errMsg, t = true

Regards, NiteHawk

Today, luaunit supports setup/teardown to be executed before and after each test. Sometimes, it's convenient to also have per suite setup/teardown function and per class setup/teardown functions.

Proposal :

• names : setupSuite, teardownSuite, setupClass, teardownClass
• setupClass is always executed before running any test of the class
• teardownClass is always executed after running all tests of the class
• any failure or error in setupClass marks all tests as failed and does not execute them
• any failure or error in tearDownClass marks all tests as failed (only the successful tests, other are already failed)
  • note : it would be convenient to be able to report multiple error/failures for one test, to display a potential test failure, test teardown failure, test class teardown failure
• same principles for setupSuite and teardownSuite :
  • always executed, once per luaunit run invocation
  • failures put all tests into failures

Up for adoption...

assertAlmostEqual() allows a zero margin, while assertNotAlmostEqual() doesn't:

This is not caught by the unit test in

I assume that we want to adjust assertAlmostEqual() to refute a zero margin, just like assertNotAlmostEqual() does, right?

Regards, NiteHawk

Problem

Currently, luaunit assumes that error() function always takes a string
value as its first argument and performs some formattings based on the
assumption.

The problem is that we often pass non-string values to error() function
(e.g. passing details of the error using a table) and it breaks the execution
of luaunit.

Example

Let's assume we have the following error call:

error({message = "xxx", code = 111})

This error() call will break the test execution with the following error:

$ luajit test_example.lua
luajit: /usr/local/share/lua/5.1/luaunit.lua:1911: attempt to call method 'gsub' (a nil value)
stack traceback:
    /usr/local/share/lua/5.1/luaunit.lua:1911: in function 'protectedCall'
    /usr/local/share/lua/5.1/luaunit.lua:1967: in function 'execOneFunction'
    /usr/local/share/lua/5.1/luaunit.lua:2062: in function 'runSuiteByInstances'
    /usr/local/share/lua/5.1/luaunit.lua:2122: in function 'runSuiteByNames'
    /usr/local/share/lua/5.1/luaunit.lua:2185: in function 'run'
    test/run-test.lua:12: in main chunk
    [C]: at 0x55942f0111d0

Desired behaviour

No error should be raised on non-string error values.

When writing a test that should verify a table, I used luaunit.assertEquals(table1, table2)

A stack overflow occurs because table1 contains a key with a value set to table1. The issue is a result of logic in the _is_table_equals() function in luaunit.lua (currently line 510).

While debugging, I have added a print statement just above this recursive call at line 527:

print("_is_table_equals Testing:", actual, k, v)
if not _is_table_equals(v, expected[k]) then
    return false
end

resulting in the following log output:

_is_table_equals Testing: table: 0AA87F28 removeEventListener function: 0A4F5070
_is_table_equals Testing: table: 0AA87F28 addEventListener function: 0A4F52F0
_is_table_equals Testing: table: 0AA87F28 __index table: 0AA87F28
_is_table_equals Testing: table: 0AA87F28 removeEventListener function: 0A4F5070
_is_table_equals Testing: table: 0AA87F28 addEventListener function: 0A4F52F0
_is_table_equals Testing: table: 0AA87F28 __index table: 0AA87F28
_is_table_equals Testing: table: 0AA87F28 removeEventListener function: 0A4F5070
_is_table_equals Testing: table: 0AA87F28 addEventListener function: 0A4F52F0
_is_table_equals Testing: table: 0AA87F28 __index table: 0AA87F28
_is_table_equals Testing: table: 0AA87F28 removeEventListener function: 0A4F5070
_is_table_equals Testing: table: 0AA87F28 addEventListener function: 0A4F52F0
_is_table_equals Testing: table: 0AA87F28 __index table: 0AA87F28
_is_table_equals Testing: table: 0AA87F28 removeEventListener function: 0A4F5070
_is_table_equals Testing: table: 0AA87F28 addEventListener function: 0A4F52F0
_is_table_equals Testing: table: 0AA87F28 __index table: 0AA87F28
_is_table_equals Testing: table: 0AA87F28 removeEventListener function: 0A4F5070
_is_table_equals Testing: table: 0AA87F28 addEventListener function: 0A4F52F0
_is_table_equals Testing: table: 0AA87F28 __index table: 0AA87F28
_is_table_equals Testing: table: 0AA87F28 removeEventListener function: 0A4F5070
_is_table_equals Testing: table: 0AA87F28 addEventListener function: 0A4F52F0
_is_table_equals Testing: table: 0AA87F28 __index table: 0AA87F28
ERROR
stack overflow

Here, you can see that the actual table has the same memory address as the __index variable (this is a 3rd party library and I presume they have done this to set the metatable as the table itself).

A simplistic fix would be to add a check before the recursive call that verifies whether v or expected[k] are the same as actual. Example:

if v ~= actual and expected[k] ~= actual then
    if not _is_table_equals(v, expected[k]) then
        return false
    end
end

And, of course, a corresponding check in the next for loop (for expected):

if v ~= expected and actual[k] ~= expected then
    if not _is_table_equals(v, actual[k]) then
        return false
    end
end

However, I'm not sure if this has any further implications.

Additional Info:

• My current workaround is to use luaunit.assertTrue(table1 == table2)
• This issue existed at least as far back as d80ebece38398866111a0aa197830cc486c3ba90

The Appveyor CI tests at https://ci.appveyor.com/project/bluebird75/luaunit use a private configuration. It might be preferable to have an appveyor.yml file included in the code base instead.

The build currently pulls in the "GnuWin" package via cinst gnuwin. This slows down the build considerably and is not needed at all; the initial cinst lua is already sufficient to provide a functional Lua interpreter (which runs LuaUnit just fine).

The Appveyor test differs from the Travis one in that it doesn't use run_unit_tests.sh. This in not a real concern, as run_functional_tests.sh includes the unit tests. Still it might make sense to keep both setups similar.

I've been using Appveyor on LuaUnit for quite a while now, with a build matrix covering several (binary) versions of Lua and LuaJIT. You might want to check https://github.com/n1tehawk/luaunit/blob/nitehawk/appveyor.yml and https://github.com/n1tehawk/luaunit/blob/nitehawk/.appveyor/install-lua.cmd for some ideas. (See https://ci.appveyor.com/project/n1tehawk/luaunit for actual results.)

Regards, NiteHawk

When working with luajit, a function will have a different implementation whether it is run for the first time or jitted if run enough number of times.

Experiences from people working with luajit is that it is important when running tests to repeat them long enough to trigger the jit compiler.

Proposal :

• command-line option to trigger execution of each test many times
• final report of test execution shall show success + number of execution or index of the execution that failed + failure report
• when test execution fails, stop repeating the test
• it should be possible to mark tests individually for repetition
• luajit typically detects a hot loop after 56 executions, so this would be a typical number
• there might be luajit specific trick to trigger explicitly the jit of a test or function under test

Up for adoption

Hi!

Given the following script:

#!/usr/bin/env lua

luaunit = require("luaunit")

TestLuaUnit = {}

function TestLuaUnit:testNonSeqArray()
    local actual = {1, nil, 2}
    local expected = {1, [3]=2}

    luaunit.assertEquals(actual[1], expected[1])
    luaunit.assertEquals(actual[2], expected[2])
    luaunit.assertEquals(actual[3], expected[3])
    luaunit.assertEquals(actual, expected)
end

luaunit.LuaUnit.run()

The last assert fails with the following output:

Started on Fri Apr 26 17:41:40 2019
    TestLuaUnit.testNonSeqArray ... FAIL
./luaunit_non_seq_array.lua:14: expected: {1, 3=2}
actual: {1, 3=2}
=========================================================
Failed tests:
-------------
1) TestLuaUnit.testNonSeqArray
./luaunit_non_seq_array.lua:14: expected: {1, 3=2}
actual: {1, 3=2}
stack traceback:
	./luaunit_non_seq_array.lua:14: in upvalue 'TestLuaUnit.testNonSeqArray'

Ran 1 tests in 0.001 seconds, 0 successes, 1 failure

So, the elements are equal, yet the arrays are not.

The reason why it fails is most probably that when checking the equality of arrays the length operator is used, which isn't reliable if the arrays are non-sequential (contains 'nil' somewhere "in the middle").

Another issue for me here is, that according to the output, the arrays seem to be equal, so no clue is given, why the test has failed.

If you define a table with a __tostring metamethod (common with "objects") LuaUnit will still display the "raw" table structure thanks to its (awesome!) table-tostring logic. It would be nice if LuaUnit respected tables that have defined their own tostring behavior.

Example:

t = setmetatable({function() end}, {__tostring=function() return "special table!" end})
print(t)
lu.assertNil(t)

Outputs:

special table!
lua: /tmp/luaunitdemo.lua:6: LuaUnit test FAILURE: expected: nil, actual: {function: 0x7fec63c2bff0}

It might make sense to add a PRINT_TABLE_STRUCTURE_ALWAYS (or similar) parameter similar to PRINT_TABLE_REF_IN_ERROR_MSG so this behavior can be configured, but I think in general the __tostring result will be more informative, if it exists.

I'm happy to send a PR if this makes sense. Apologies if this has already been considered, I didn't see anything in the issue history.

It would be super awesome to be able to install this into my project with npm install or bower install. I know this isn't javascript, but I use gulp to build my libraries and being able to pull this in with a simple npm install would be awesome.

assertErrorMsgMatches() currently uses the strMatch() function to test for the expected message:

For example, when testing for a luaL_checknumber failure, it's not sufficient to have
assertErrorMsgMatches("^bad argument.*number expected", ...).
You have to use
assertErrorMsgMatches("^bad argument.*number expected.*", ...).
instead. (Note the .* padding at the end.)

I would like to suggest changing the match to a more natural error_msg:match(expectedMsg), following the standard Lua pattern semantics. While this is a somewhat incompatible change over previous LuaUnit versions, it should also accept existing patterns. If a narrower / "full" match is what the user desires, it's easy enough to anchor the Lua pattern with ^ and $ (http://www.lua.org/manual/5.1/manual.html#5.4.1).

It would be nice to add a "data provider" like in phpunit

Failing tests break LuaUnit on Lua 5.1.4.

For example running:

LuaUnit:run( 'test1_withFailure' )

from example_with_luaunit.lua will cause:

lua: ./luaunit.lua:92: attempt to index local 't' (a function value)
stack traceback:
    ./luaunit.lua:92: in function '(for generator)'
    ./luaunit.lua:551: in function 'runTestClass'
    ./luaunit.lua:594: in function <./luaunit.lua:573>
    (tail call): ?
    example_with_luaunit.lua:96: in main chunk
    [C]: ?