Comments (6)
I like the idea in general, and would consider this a very useful feature. However, upon closer investigation I notice some potential problems:
- Changing the error type of failed LuaUnit tests might impact the usefulness of 'standalone' LuaUnit, i.e. scenarios where users would rely on calling assertions directly - like
lu.assertEquals(true, false)
. If we return a failure "object" (non-string error) there, and without error protection, some Lua versions will no longer report anything useful and display(error object is not a string)
instead. - As you have already indicated above, LuaUnit's own internal testing also relies on string comparisons, especially when verifying error messages.
A possible solution to both these problems could be introducing a __tostring
metamethod. My experiments suggest that Lua 5.3 and current LuaJIT respect/use this, but unfortunately Lua 5.1 and 5.2 do NOT. (See http://lua-users.org/lists/lua-l/2011-11/msg00693.html)
My impression is that we would probably be better off (and avoid these kind of pitfalls) if we just add a prefix (string) to the error message to indicate failures. If the prefix is specific enough (e.g. "LuaUnit test FAILURE: "), it can be filtered and acted upon accordingly within execOneFunction()
; while keeping the error still valid as a standard string.
Regards, NiteHawk
from luaunit.
For a 'sneak preview' of what I have in mind, look at 0434c73 and dbf98d5 ...
from luaunit.
Hi Nitehawk,
Le 14/12/2015 22:42, NiteHawk a écrit :
I like the idea in general, and would consider this a very useful
feature. However, upon closer investigation I notice some potential
problems:
- Changing the error type of failed LuaUnit tests might impact the
usefulness of 'standalone' LuaUnit, i.e. scenarios where users
would rely on calling assertions directly - like
|lu.assertEquals(true, false)|. If we return a failure "object"
(non-string error) there, and without error protection, some Lua
versions will no longer report anything useful and display |(error
object is not a string)| instead.I suppose that it would actually display a table ref but yes, I got the
point. However, this is not a use case that I am aware of.
- As you have already indicated above, LuaUnit's own internal
testing also relies on string comparisons, especially when
verifying error messages.Test suites are here to evolve...
Pulling information from an object might actually make the test suites
more robust instead of relying on text. The TestError object would
actually contain in attributes the errore message, the stack trace, the
type of failure, and any additional information we might want to put
there in the future.
A possible solution to both these problems could be introducing a
|__tostring| metamethod. My experiments suggest that Lua 5.3 and
current LuaJIT respect/use this, but unfortunately Lua 5.1 and 5.2 do
/NOT/. (See http://lua-users.org/lists/lua-l/2011-11/msg00693.html)My impression is that we would probably be better off (and avoid these
kind of pitfalls) if we just add a prefix (string) to the error
message to indicate failures. If the prefix is specific enough (e.g.
/"LuaUnit test FAILURE: "/), it can be filtered and acted upon
accordingly within |execOneFunction()|; while keeping the error still
valid as a standard string.
I already do this for pulling the stracktrace, so why not...
See :
function M.LuaUnit:protectedCall( classInstance , methodInstance,
prettyFuncName)
local function err_handler(e)
return debug.traceback(e..SPLITTER, 3)
end
local ok=true, fullErrMsg, stackTrace, errMsg, t
ok, fullErrMsg = xpcall( function () methodInstance() end,
err_handler )
if ok then
return ok
end
t = strsplit( SPLITTER, fullErrMsg )
errMsg = t[1]
stackTrace = string.sub(t[2],2)
...
I would like to use this same mechanism to also provide the ability to
skip test, to mark an early test success, to force a failure without an
assertion.
So ... up to you, feel free to experiment.
Cheers,
Philippe
from luaunit.
It would also be nice to somehow distinguish between user test errors and internal errors - they both current lead to a failure in the exact same way. (I've just encountered this in #32.)
from luaunit.
Thanks for merging the above PR! I'll now prepare another pull request that should address this issue.
from luaunit.
Well done !
from luaunit.
Related Issues (20)
- testing without globals HOT 4
- JUnit output doesn't escape class or test names HOT 4
- assertNoError? HOT 5
- Is there a way to modify the level of the error reported at the time of assertion HOT 8
- assertEquals not matching some values HOT 4
- support for Lua 5.4 HOT 4
- Table driven tests support? HOT 3
- assert for function returning, nil/msg? HOT 2
- Inaccurate Time HOT 4
- assertError does not work? HOT 2
- Mock globals, other function and tables HOT 2
- test_luaunit.lua fails on Debian build for lua-unit_3.4-1 HOT 16
- can i search test_xxx from _ENV HOT 2
- Recommendation to add "local testNames" on line 13.
- Running tests across multiple files HOT 2
- issues in LuaUnit’s documentation
- luaunit conflicts with strict HOT 2
- [feat req] `assert_almost_equals()` margin relative to precision of expected number
- Bad argument to randomseed
- Strings are not properly escaped in JUnit XML reports HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from luaunit.