Comments (6)
This was discovered when looking at #43.
from test2-suite.
FWIW, Test::Deep has the same problem, and they just documented it.
I'll try to come up with some implementation that does not suffer from combinatorial explosion.
from test2-suite.
I have rewritten Bag in a maybe-too-complicated way. It builds a matrix of inputs ร checks, then looks at it to see what went wrong.
It handles the degenerate cases (empty input array, empty bag) up front.
My main problem at the moment: I don't know what kind of deltas to return in some cases. Suggestions very welcome.
from test2-suite.
First off, those commits are failing in travis, so that needs to be fixed.
I have not yet done a full code review, I will when I have a chance.
I would like an explanation of how it handles things after the matrix is created. Say 1 check matches 2 items in the array, but no other checks match either item, will it pass or fail? What about more complicated scenarios where all items in the array have matched at least 1 check, but some checks matched multiple items, but there are less checks than matched items and 'end' was used?
I am actually not 100% sure what the correct way to handle these scenarios are, so right now I am interested in how you handle them.
from test2-suite.
Re: failing, yes, I know, the tests that are failing are those that check the deltas returned in cases where I'm not sure what we should be returning.
My current logic is:
- each check must match at least one input element, always
- if
end
was used ($closed
, in my code):- each input element must match at least one match
- the number of input elements must be the same as the number of checks
- this is actually not wholly implemented, because I have no clear idea how to report it, but there's a
$inputs_with_matches
counter to use for that purpose
- this is actually not wholly implemented, because I have no clear idea how to report it, but there's a
Regarding deltas:
- if an input element matches 0 checks, I return the deltas from each check against that element
- if a check matches 0 input elements, I return
{dne=>'got',id=>[ARRAY=>'*'],check=>$the_check}
(I read that as "there's no input anywhere matching this check") - if there are more matching elements than checks, I return
{id=>[ARRAY=>$idx],got=>$list[$dx],check=>$check}
for each check matching that element (this is horrible, since I'm abusing Delta to report a successful match)
So the open questions are:
- is my basic logic sound?
- how do we report "these elements matched, but you weren't expecting them to"?
- how do we report "these checks matched multiple times, but you weren't expecting them to"?
- does my approach to implementing the logic make sense?
from test2-suite.
Yes, it makes sense. Once again I have not looked at the code. That said I worked it out, and here is how I think the deltas should work:
First check that the array has at least as many elements as the bag, if it does not then add a delta that indicates we expected more items:
| PATH | GOT | CHECK |
| [0] <count> | 5 | 8+ |
If end is specified we want to do the same for the opposite, more array items than bag items:
| PATH | GOT | CHECK |
| [0] <count> | 8 | 5 |
After those deltas add the ones for any checks that didn't match an item
| PATH | GOT | CHECK |
| [0][*] | <DOES NOT EXIST> | x |
if end() is specified add deltas for any extra array items, these should indicate we expected them to not exist.
| PATH | GOT | OP | CHECK |
| [0][0] | a | !exists | <DOES NOT EXIST> |
if end() is not specified, but we have deltas for any failure, add deltas for all unmatched items, this should NOT indicate we expected them to not exist, the 'expected' and 'op' columns should be empty. This is just informative diagnostics
| PATH | GOT | OP | CHECK |
| [0][0] | a | | |
To summarize
- Delta for the count if there are more items in the bag than the array, should expect the count of the bag or more (bag_count+)
- Delta for the count if the array is bigger than the bag and 'end' is used
- Delta for any unmatched check, always
- Delta for extra/unmatched array items if end is used
- Delta for any extra/unmatched array items if end is not used, but there are unmatched checks, or count mismatch (this does not indicate that the items should not exist, it just prints them for diag purposes)
from test2-suite.
Related Issues (20)
- Possible incompatibility with Sub::Name v0.26 HOT 5
- Test2::Tools::Warnings a test case which isn't caught as a warning. HOT 1
- FR: Implement debug flag HOT 3
- Return true from Test2::Manual modules.
- Possible to mock CORE sub? HOT 7
- FR: Provide a way to test arrays in reverse HOT 1
- Test2's is() needs better handling of unicode, and some differences
- Tests are consistently failing on perl 5.8 HOT 6
- Add 'stringify_got' attribute to Test2::Compare::Custom HOT 2
- perltidyrc doesnโt match code style, seems invalid and not be be applied or enforced HOT 3
- Test2::Mock should optionally track HiRes timestamps for calls HOT 1
- Should dies() warn when called in void context? HOT 1
- Add ability to call orig subroutine when we override it HOT 2
- Document how to use `in_set` builder HOT 4
- T() and F() do not handle references properly
- Refcount assertions HOT 2
- async_subtest output incompatible with TAP14 spec HOT 3
- context level is incorrect when mocks are used HOT 3
- Improve documentation for Test2::Tools::Subtest and Test2::AsyncSubtest HOT 1
- Dumper-on-fail for `is`, `isnt`, etc HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from test2-suite.