test-more / test2-suite Goto Github PK

sta 'method' and sta method => $val

Call 'method' on the check being constructed. If no second argument is given then use '1' as the argument. Otherwise use the specified argument.

Only call methods listed in a hash returned by the 'dsl_attributes' class method on the check, calling an unlisted method is an exception.

I don't think there's any benefit in the "same terms as Perl itself" wording, and can only cause confusion. If it's being licensed as Artistic License 2.0, then I think it makes sense to go ahead and specify it.

http://use.perl.org/use.perl.org/_Shlomi%2BFish/journal/36050.html has some discussion of this.

st Type => ...

the Type must be in 'Test2::Compare::Check::Type' unless a '+' prefix is given to mean absolute path

The arguments after the first must either be a value (so if type is 'Array' it should be an arrayref) or a builder subref.

The first argument passed into the builder must be the check being constructed so that you can directly manipulate it if desired.

Back when I was first creating Test2::Suite I had a vision for how it could move forward when breaking changes were needed. I mostly forgot, and even started to move away from that path... I think I need to return to the original plan, and I will document it here as best I can.

With Test2::Suite tools, plugins, and bundles are all treated as disposable. This is why tools and plugins are separated out by functionality, and are very small units. With exception of Test2::Tools::Compare nearly everything in Test2::Tools::* is very tiny.

The idea was that if Test2::Tools::Foo turned out to have serious problems, or be a bad idea, we could replace it by writing Test2::Tools::Bar and recommend people stop using ::Foo. The consequences are minimal, people already using ::Foo don't break, and people writing new tests can use ::Bar instead.

Bundles are also supposed to work this way, if a bundle includes a bunch of bad tools then simply write a new bundle, it should be easy since bundles have minimal logic and simply re-export tools and plugins.

Right now there are 2 modules in particular that we are having trouble with:

• Test2::Tools::Compare
• Test2::Bundle::Extended

The reason we are having trouble with Test2::Tools::Compare is because it really did break away from the small and disposable model. The extended bundle has a problem because the compare tools have a problem, but also because it is the "recommended" bundle, so replacing it is strange.

One of the things I as working on to try to address moving forward was the use of '+v#' pins. Another idea that is similar, but less crazy and more maintainable is the use of Foo::V# variants, like what perl5i uses.

I have a problem with this system. This works with perl5i because it is a single module the consumer uses. Test2::Suite is a ton of modules, and the consumer can use any number/combination of them. Having V variants of each requires a lot of overhead for the consumers. Even if we make it so that they all get a new variant at once with the same number it is still a lot of cognitive overhead.

I don't like either system here for most things, though I do still want to adopt the perl5i ::V# system for one specific thing: Recommended bundle.

I think we should create Test2::Bundle::V and Test2::Bundle::V1. Stop reommending 'Extended' and start recommending Test2::Bundle::V. Loading bundle V will load the latest V# variant, but warn that the user should use V# (whatever is latest) instead. If someone uses V1 and we create V2 they will not break because we will not change V1 in a breaking way once it is created.

What about Test2::Tools::Compare?

Boils down to this: I think that including the compare functions is, like, etc and the dsl in a single tools module is/was a mistake.

• The dsl functions should be split into a new Tools::* module (maybe Test2::Tools::Compare::DSL).
• We should deprecate (via a warning) the import of the dsl functions from Test2::Tools::Compare (but not break them!)
• The new DSL tools should be written in the new styles have been discussing, breaking compatability here is fine because it is a new package, people have to choose to switch
• The V1 bundle will export the new tools, the 'Extended' bundle will continue to provide the old.

What about is() and implicit end?

The more I think about it, the more I think not having an implicit end() was a bug, plain and simple. I suspect I was the only one who knew, and intentionally used the lack of implicit end as a feature. So lets just fix the damn bug. Things that break from fixing this bug break because a bug was letting them pass when they should not have. And the fix is easy, add an etc call to the structure.

That said, I also think we should give extra information when a test fails due to implicit end() to explain the situation better (this previously passed, but fails now, here is why....)

Since this is the only change to the non-dsl tools in Test2::Tools::Compare we will not need to replace Test2::Tools::Compare or create and variants for it.

So to summarize:

• Fix the implicit end bug
• Write a DSL tool module with our new DSL ideas (still in the works)
• Deprecate the dsl tools in Test2::Tools::Compare
• Recommended bundle is now Test2::Bundle::V and numeric variants as we need to make breaking changes.

Unparsable version '2.9e-05' for prerequisite Test2::Suite treated as 2.9 at /home/ron/perl5/perlbrew/perls/perl-5.20.2/lib/site_perl/5.20.2/CPAN/Meta/Requirements.pm line 136.
Warning: prerequisite Test2::Suite 2.9 not found. We have 0.000029.

First of all, thank you for the herculean work you've done. Test2 is much nicer to work with than Test::Builder could ever be.

Now, I found myself in need of the functionality of Test::Deep::bag, and I couldn't find something equivalent for Test2. So I wrote it: Test2::Compare::Bag, Test2::Tools::MoreCompare.

The latter also provides a call_list which behaves like call but evaluates the method in list context. list_call $method => $result is a shortcut for call(sub{[$_->$method]},$result,$method)

Should I release those packages to CPAN, or should I provide them here as a pull request? If the former, does anyone have a better name than Test2::Tools::MoreCompare?

They were only separate cause Workflow was not stable when Suite was released.

run this script

use Test2::Tools::ClassicCompare;# or use Test2::Bundle::More;
is_deeply({},{})

then I get module load error.

Can't locate object method "new" via package "Test2::Compare::Hash" (perhaps you forgot to load "Test2::Compare::Hash"?) at lib/perl5/Test2/Compare.pm line 88.
A context appears to have been destroyed without first calling release().
Based on $@ it does not look like an exception was thrown (this is not always
a reliable test)

This is a problem because the global error variables ($!, $@, and $?) will
not be restored. In addition some release callbacks will not work properly from
inside a DESTROY method.

Here are the context creation details, just in case a tool forgot to call
release():
  File: ./basic_sample.pl
  Line: 2
  Tool: Test2::Tools::ClassicCompare::is_deeply

However, Test2-Suite/t/modules/Tools/ClassicCompare.t is a success.
Because this test loads Test2::Bundle::Extended.
(
Test2::Bundle::Extended loads Test2::Tools::Compare. Test2::Tools::Compare loads Test2::Compare::Hash, Test::Compare::Array, and so on
)

The D() quick check is useful, but I find myself wanting the opposite - U(). As it doesn't exist yet, the following works but is a bit tedious:

not_in_set(D())

#!/var/perl/bin/perl

use warnings;
use strict;

use Test2::Bundle::More;

is_deeply( { foo => 1 }, { bar => 2 } );
done_testing();

generates this

$ prove Dev/test2.t
[11:03:15] Dev/test2.t .. Can't locate object method "new" via package "Test2::Compare::Hash" (perhaps you forgot to load "Test2::Compare::Hash"?) at /var/perl5.20.3/lib/site_perl/5.20.3/Test2/Compare.pm line 87.
A context appears to have been destroyed without first calling release().
Based on $@ it does not look like an exception was thrown (this is not always
a reliable test)

This is a problem because the global error variables ($!, $@, and $?) will
not be restored. In addition some release callbacks will not work properly from
inside a DESTROY method.

Here are the context creation details, just in case a tool forgot to call
release():
  File: Dev/test2.t
  Line: 8
  Tool: Test2::Tools::ClassicCompare::is_deeply

Cleaning up the CONTEXT stack...
# No tests run!
# Looks like your test exited with 255 after test #0.
[11:03:15] Dev/test2.t .. Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run 
[11:03:15]

Test Summary Report
-------------------
Dev/test2.t (Wstat: 65280 Tests: 0 Failed: 0)
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
Files=1, Tests=0,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.05 cusr  0.00 csys =  0.07 CPU)
Result: FAIL

Similar results when doing array comparisons.

$ cat Dev/test2.t
#!/var/perl/bin/perl

use warnings;
use strict;

use Test2::Bundle::More;

is_deeply( [1], [2] );
done_testing();

$ prove Dev/test2.t
[11:04:16] Dev/test2.t .. Can't locate object method "new" via package "Test2::Compare::Array" (perhaps you forgot to load "Test2::Compare::Array"?) at /var/perl5.20.3/lib/site_perl/5.20.3/Test2/Compare.pm line 84.
A context appears to have been destroyed without first calling release().
Based on $@ it does not look like an exception was thrown (this is not always
a reliable test)

This is a problem because the global error variables ($!, $@, and $?) will
not be restored. In addition some release callbacks will not work properly from
inside a DESTROY method.

Here are the context creation details, just in case a tool forgot to call
release():
  File: Dev/test2.t
  Line: 8
  Tool: Test2::Tools::ClassicCompare::is_deeply

Cleaning up the CONTEXT stack...
# No tests run!
# Looks like your test exited with 255 after test #0.
[11:04:16] Dev/test2.t .. Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run 
[11:04:16]

Test Summary Report
-------------------
Dev/test2.t (Wstat: 65280 Tests: 0 Failed: 0)
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
Files=1, Tests=0,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.04 cusr  0.01 csys =  0.08 CPU)
Result: FAIL

So people don't think seeding with the date is wrong.

use strict;
use warnings;

use Test2::Bundle::Extended;
use Test2::Tools::Subtest;

subtest_buffered(
    'parent',
    sub {
        subtest_buffered(
            'buffered',
            sub {
                ok(1, 'b1');
                ok(1, 'b2');
            },
        );
        subtest_streamed(
            'streamed',
            sub {
                ok(1, 's1');
                ok(1, 's2');
            },
        );
    }
);

done_testing;

Outputs:

# Seeded srand with seed '20160917' from local date.
ok 1 - parent {
    ok 1 - buffered {
        ok 1 - b1
        ok 2 - b2
        1..2
    }
    # Subtest: streamed
    ok 2 - Subtest: streamed
    1..2
}
1..1

Note that using the subtest provided by Test::Builder has the same issue (and may require a separate fix).

This sub essentially replaces 'prop()' main difference is that 'this' is always an available property.

PR #113 broke Test2::Tools::EventDumper. Specifically it makes compare objects all false in boolean context, which is not right. I have reverted the PR until it can be corrected.

Test2::Tools::EventDumper caught the problem because it has a test where it constructs a compare object by first dumping what the code should be, eval'ing that code, and running conditionals, it considers the eval returning false to be a failure, since the compare objects all have boolean-false overloading all tests failed.

Call cmp_ok with integers and all is good.

$ perl -MTest2::Tools::ClassicCompare -E'cmp_ok( 1, ">", 4 )'
not ok 1
# Failed test at -e line 1.
# +-----+----+-------+
# | GOT | OP | CHECK |
# +-----+----+-------+
# | 1   | >  | 4     |
# +-----+----+-------+

But call it with decimals and you get this:

$ perl -MTest2::Tools::ClassicCompare -E'cmp_ok( 1.23456, ">", 4.5678 )'
not ok 1
# Failed test at -e line 1.
# +------+---------+----+--------+
# | TYPE | GOT     | OP | CHECK  |
# +------+---------+----+--------+
# | num  | 1       | >  | 4      |
# | orig | 1.23456 |    | 4.5678 |
# +------+---------+----+--------+

http://www.cpantesters.org/cpan/report/6e2476ea-3a17-11e6-9fc5-14bfbd15da07

This is more common in older perls, and only seems to happen on runs after the first. Old File::Temp versions only try a couple names before giving up. Since this test uses SRand it always has the same temp files/dirs.

The easy fix is to stop using SRand in this test. However it may be worth it to make a more complete fix (not sure how, maybe require a newer File::Temp?)

The docs say:

Note: This will fail if the recieved value is undefined, it must be defined.

So when using string() as a quick check, it confesses when not defined. Shouldn't it simply return false if it is not a string?

Start with explaining that you have to use Dist::Zilla, not EU::MM, and how to use Dist::Zilla to do common tasks. Not everyone uses it, and without a Makefile.PL, someone unfamiliar with Dist::Zilla will not understand what to do.

When your test suite is a Moose class (for example, if you use Test::Class::Moose) then the exported meta function conflicts with both the inherited meta method and the meta that Moose exports, which causes problems:

package Hello;

use Test::Class::Moose bare => 1;
use Test2::Bundle::Extended;

sub test_reverse_dtrt {
    is(scalar(reverse('dlroW olleH')), 'Hello World', 'Hi');
}

__PACKAGE__->meta->make_immutable;
1;

Mark@travis:~/tmp/tcm$ prove -v runner.pl
runner.pl ..
# Seeded srand with seed '20160706' from local date.
Prototype mismatch: sub Hello/meta: none vs (&) at Hello.pm line 4.
Can't use string ("Hello") as a subroutine ref while "strict refs" in use at /opt/markperl/lib/site_perl/5.22.0/Test2/Compare.pm line 57.
 at /opt/markperl/lib/site_perl/5.22.0/Test2/Compare.pm line 59.
    Test2/Compare/build("Test2/Compare/Meta", "Hello") called at /opt/markperl/lib/site_perl/5.22.0/Test2/Tools/Compare.pm line 142
    Test2/Tools/Compare/meta("Hello") called at Hello.pm line 10
    require Hello.pm called at (eval 5) line 1
    Test/Class/Moose/Load/BEGIN() called at Hello.pm line 0
    eval {...} called at Hello.pm line 0
    eval 'use Hello ()' called at /opt/markperl/lib/site_perl/5.22.0/Test/Class/Moose/Load.pm line 54
    Test/Class/Moose/Load/_load("Test/Class/Moose/Load", "./Hello.pm", ".") called at /opt/markperl/lib/site_perl/5.22.0/Test/Class/Moose/Load.pm line 68
    Test/Class/Moose/Load/__ANON__() called at /opt/markperl/lib/5.22.0/File/Find.pm line 447
    File/Find/_find_dir(HASH(0x7fd55c99e718), ".", 4) called at /opt/markperl/lib/5.22.0/File/Find.pm line 235
    File/Find/_find_opt(HASH(0x7fd55c99e718), ".") called at /opt/markperl/lib/5.22.0/File/Find.pm line 759
    File/Find/find(HASH(0x7fd55c99e718), ".") called at /opt/markperl/lib/site_perl/5.22.0/Test/Class/Moose/Load.pm line 73
    Test/Class/Moose/Load/import("Test/Class/Moose/Load", ".") called at runner.pl line 6
    main/BEGIN() called at Hello.pm line 0
    eval {...} called at Hello.pm line 0
Compilation failed in require at (eval 5) line 1.
BEGIN failed--compilation aborted at (eval 5) line 1.
BEGIN failed--compilation aborted at runner.pl line 6.
# No tests run!
# Looks like your test exited with 255 after test #0.
Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run

Test Summary Report
-------------------
runner.pl (Wstat: 65280 Tests: 0 Failed: 0)
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
Files=1, Tests=0,  1 wallclock secs ( 0.01 usr  0.01 sys +  0.26 cusr  0.03 csys =  0.31 CPU)
Result: FAIL

None pof isa_ok, can_ok, or DOES_ok allow a test description. This makes for a weird and easily forgettable API, where they are the outliers among all the test functions.

This wasn't a good idea for Test::More, and I'd hate to see Test2::Suite repeat that mistake. I think it'd be preferable to accept a single class or arrayref of classes (or methods, etc) and allow the third argument to be the test description.

default/v1 must not change, we must not break existing tests.

I am working on updating all Test2::Suite packages that export symbols. All of them will be updated to use an upcoming feature in Importer called 'pins'. A pin is a way to version exports from a module, the main example is Compre:

use Test2::Tools::Compare ':v1';

vs

use Test2::Tools::Compare ':v2';

The first will import is() that does not automatically add bounds-checking to checks built with the dsl. The second one will import the updated is() that does bound checking. providing no pin will default to ':v1' for back-compat, don't want to break things.

Question is, should we add a warning whenever someone loads a bundle/tool without specifying a pin? I think we should, that way everyone will need to add :v1 or :v2, and will be forced to know about pins. This way they can also always check if there is a newer pin that provides updated features.

I do not think we should warn when people are using older pins, that almost defeats their purpose, though I think an env var to turn on such warnings would be useful.

I am leaning towards warning on no-pin with bundles, I am not so sure on tools:: packages..

I've got a function that I've been calling bool_eq that compares truthiness. Consider this example, probably in the middle of a big loop that goes over a bunch of different test cases.

my $thingy = find_a_thingy();  # Either returns a Thingy object or undef.
if ( $expect_to_find_a_thingy ) {  # Are we expecting to find one?
    ok( $thingy, 'Found a thingy' );
}
else {
    ok( !$thingy, 'Should not find a thingy' );
}

bool_eq lets that be rewritten as:

bool_eq( find_a_thingy(), $expect_to_find_a_thingy );

Or something like:

bool_eq( scalar @errors, $should_have_errors );

The diagnostics on failure would tell what values came in and how they evaluated:

#   Failed test 'Should have no errors'
#   at foo.t line 8.
#          got: true (5)
#     expected: false (0)

#   Failed test 'Should have found at least one user'
#   at foo.t line 10.
#          got: false (0)
#     expected: true (1)

#   Failed test 'Should not find a Mech object'
#   at foo.t line 9.
#          got: true (WWW::Mechanize=HASH(0x3cca328))
#     expected: false (0)

https://metacpan.org/source/EXODIST/Test2-Suite-0.000016/lib/Test2/Require/Perl.pm#L43

=head1 SYNOPSIS

    # Skip the test unless perl 5.10 or greater is installed.
    use Test2 Perl 'v5.10';

    # Enable 5.10 features.
    use v5.10;

This test fails when using perl compiled on Windows using Visual Studio tools because paths are separated using \ instead of /. Using Perl's builtin $0 instead of the hard coded file name avoids the spurious failure.

In the old system, I have many tests that are of the form

is_nonblank( $foo->{name} );

which is effectively

ok( !ref($foo->{name}) && length($foo->{name}) > 0 );

And we should be able to do

is( $foo, hash { field name => nonblank } );

Most of my code lives in checks. This works nicely, especially when validating deep hashes.

However, I've found that it's a bit awkward when it comes to arrays - currently, array checks can only be called manually. This doesn't work when you don't know how many items in an array before executing.

What I'd like, is something along the lines of:

my $is_below_ten = validator(sub {
  my (%params) = @_;

  return undef if $params{got} < 10;
});

is(
  [3, 4, 5],
  array {
    all_items $is_below_ten;
  }
);

If I expect something to live, I really want to get diagnostic of the exception when it doesn't. The lives or note idiom is really suboptimal.

Even if you don't want to add this, I'd at least update the docs to suggest something like is( exception { ... }, undef, ... )

It appears to be a problem with Test2::Compare::Bag::deltas but I couldn't devote further time to fixing it.

use Test2::Bundle::Extended;
use Test2::Tools::Compare qw( is bag item );

my $check = bag {
    item 'a';
    item 'b';
    end(); # Ensure no other elements exist.
};

is( [ ], $check, 'All of the elements from bag found!'); # passes but shouldn't

done_testing;

Tools and bundles should use pins (from Exporter.pm) we need to document that.

Function prototypes are a huge benefit when writing tests, because it's so easy to get the arguments goofed up. You're thinking cmp_ok when writing is, and you do this:

is( 'dog', '==', 'cat', 'Are dogs the same as cats?' );

Under Test2, the test will fail but the code will execute.

Using Test::More I would get:

Too many arguments for Test::More::is at foo.pl line 7, near "'Are dogs the same as cats?' )"
Execution of foo.pl aborted due to compilation errors.

Anything we can do to catch errors at compile time is a big win.

In Debian we are currently applying the following patch to
Test2-Suite.
We thought you might be interested in it too.

Description: fix spelling
Origin: vendor
Author: gregor herrmann <[email protected]>
Last-Update: 2016-07-30

The patch is tracked in our Git repository at
https://anonscm.debian.org/cgit/pkg-perl/packages/libtest2-suite-perl.git/plain/debian/patches/spelling.patch

Thanks for considering,
gregor herrmann,
Debian Perl Group

Maybe I'm doing something wrong, but I would have expected that all of these tests should pass:

#!/usr/bin/perl

use strict;
use warnings;

use Test2::Bundle::Extended;

my $check = hash {
  field first   => 42;
  field second  => undef;
  field third   => DNE();
  field fourth  => in_set(42, undef);
  field fifth   => in_set(42, undef);
  field sixth   => in_set(42, DNE());
  field seventh => in_set(42, DNE());
};

is(
  {
    first   => 42,
    second  => undef,
    # third DNE
    fourth  => 42,
    fifth   => undef,
    sixth   => 42,
    # seventh DNE <- fails :(
  },
  $check
);

Test2::Plugin::UTF8, among other things, sets a :utf8 layer on the global STDOUT and STDERR handles. While it will usually be used at a top level in a test file via Test2::Bundle::Extended or similar, it still doesn't seem like the most compatible idea. For example, if a test file uses Test2::Bundle::Extended as well as a library which is using Test::More (or another Test::Builder based tool), that library will assume that its test output needs to be encoded to UTF-8, resulting in double-encoded output. There may also be other failure cases I'm not thinking of, due to the global nature of the handles.

I don't know enough about the architecture to know the feasibility, but would it be possible to instead have Test2::Plugin::UTF8 set something internally in Test2 which then causes any output to be encoded before being passed to STDOUT/STDERR? Or is there another solution that would work without affecting the global handles?

Plugin that turns anything that is printed to the global STDOUT and STDERR into events. This will primarily be used by harnesses, but can also help with the order-mangling of printing to both STDERR and STDOUT.

stc is pretty simple, it takes exactly 1 or 2 arguments, and passes both to $check->add_component(...) for the check currently being built.

Test2-Suite-0.000065 moved a code into a separate Term-Table distribution, but forgot to remove a dependency on List::Util module (Makefile.PL:23) that is not used in Test2-Suite anymore.

#56 and #62 bring up interesting questions about the behavior of is and like. The docs say is is the "strict checker" and like is the "relaxed checker". And I've been thinking of them this way. is is an equality check and like says we expect it to contain the selected things.

But upon reading the docs there's a lot more than that and some of them are quite surprising.

use Test2::Bundle::Extended;

use v5.10;

# This is "foo" =~ qr/foo/
like "foo", qr/foo/, "like regex";               # pass

# This is "foo" eq qr/foo/ ... or something?
is "foo",   qr/foo/,   "is regex";               # fails

# This uses the code ref to check 'foo'
like "foo", sub { $_ eq 'foo' }, 'like coderef'; # pass

# This checks if "foo" is the code ref?
is   "foo", sub { $_ eq 'foo' }, 'is coderef';   # fails

done_testing;

These is features are for extremely rare cases. They complicate understanding is vs like. The like ones are pretty useful, being able to check strings with regexes and evaluating complicated nested conditions with code refs. It sucks that you can only get them with like. You can't choose "strict checking but also let me use regexes and code".

I propose...

• Make is and like behave as similar as possible
  • Eliminate the code ref and regex and any other special cases
  • Replace them with coderef and regex DSLs (see below).
• is checks that the two sides are the same
• like checks that the LHS contains the RHS

The special case behavior of is is replaced by adding to the DSL. Then you can use it with like if you want.

is $code, codref $some_code_ref;  # check that $code is $some_code_ref
is $regex, regex qr{foo};  # check $regex is the same as qr{foo}

Yes, this does call into question what is really the difference between is and like.

Yes, I'm increasingly thinking of Test2 as a DSL rather than a collection of comparison functions.

The current state of master applies the backcompat breaking change to all code. I think we need to use the planned v# export tag system instead, that is what it is there for!

• Update Test2::Tools::Compare to use Importer
• Current defaults, including old-style is() without implicit declare as ':v1' tag (and default when no tag specified)
• ':v2' tag will have a new is() with implicit end, for now it will have extra diag when implicit end causes a failure to help people migrating, the diag will eventually go away.

lintian (Debian's package checker) says about Test2::Suite:

I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Bundle::More.3pm.gz seperate separate
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Bundle::More.3pm.gz compatability compatibility
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Bundle::Simple.3pm.gz seperate separate
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Compare::Base.3pm.gz overriden overridden
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Compare::Number.3pm.gz recieved received
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Compare::String.3pm.gz recieved received
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Mock.3pm.gz overriden overridden
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Mock.3pm.gz overriden overridden
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Mock.3pm.gz orignal original
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Plugin::UTF8.3pm.gz wether whether
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Require.3pm.gz recieve receive
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Require::Fork.3pm.gz mimick mimic
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Suite.3pm.gz compatability compatibility
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Tools.3pm.gz seperate separate
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Tools::Basic.3pm.gz seperator separator
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Tools::Basic.3pm.gz seperator separator
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Tools::ClassicCompare.3pm.gz isnt isn't
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Tools::Compare.3pm.gz isnt isn't
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Tools::Compare.3pm.gz specfied specified
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Tools::Mock.3pm.gz existance existence
I: libtest2-suite-perl: spelling-error-in-manpage usr/share/man/man3/Test2::Util::Table.3pm.gz anomolies anomalies

W: libtest2-suite-perl: manpage-has-errors-from-pod2man usr/share/man/man3/Test2::Util::Grabber.3pm.gz:218

the latter being (perldoc output)

POD ERRORS
    Hey! The above document had some coding errors, which are explained
    below:

    Around line 125:
        '=item' outside of any '=over'

    Around line 163:
        You forgot a '=back' before '=head1'

Maybe someone (@karenetheridge?) can add author tests like in Test::Simple's xt/author/ here as well?

Thanks,
gregor, Debian Perl Team

All my subtests have plans.

subtest foo => sub {
    plan tests => 14;
}

In order to use Test2::Tools::Basic, I would have to change all of those to:

subtest foo => sub {
    plan 14;
}

If you'd allow either

plan tests => 14;
plan 14;

then it would save me from changing 1500 plans in 400 different files.

It would also be nice if we could use

use Test2::Bundle::Extended plan => 14;

just like we currently do

use Test::More plan => 14;

As discussed in Slack and on #56, add etc. This explicitly states that an array or hash is not the complete specification regardless of being used by is or like.

Examples...

# After #56, both of these will be better written as `like`.
is [1,2,3], array { item 1; item 2; etc };
is { foo => 23, bar => 42 }, hash { field foo => 23; etc };

This opens the opportunity for etc to appear in other parts of an array.

# It contains 2 and 3 in that order
is [1,2,3,4,5], array { etc; item 2; item 3; etc; };

# It starts with 1 and ends with 5
is [1,2,3,4,5], array { item 1; etc; item 5; };

Test2::Tools::Compare says right in the SYNOPSIS that is is "strict checking, everything must match".

    # Strict checking, everything must match
    is(
        $some_hash,
        {a => 1, b => 2, c => 3},
        "The hash we got matches our expectations"
    );

But with array, bag, hash that expectation doesn't hold true.

use Test2::Bundle::Extended;

my $array = [1,2,3,4];

is $array, array { item 1; item 2; item 3; end; }, "array w/end";          # fail
is $array, [1,2,3],                                "array ref";            # fail
is $array, array { item 1; item 2; item 3; },      "array wo/end";         # pass

is $array, bag { item 2; item 3; item 1; end; },   "bag w/end";            # fail
is $array, bag { item 2; item 3; item 1; },        "bag wo/end";           # pass


my $hash = { foo => 23, bar => 42, baz => 99 };

is $hash, { foo => 23, bar => 42 },                "hash ref";             # fail
is $hash, hash { field foo => 23; field bar => 42; end; }, "hash w/end";   # fail
is $hash, hash { field foo => 23; field bar => 42 },       "hash wo/end";  # pass

done_testing;

• This is a trap for the user. It's unexpected for the writer and the reader. is is for equality. like is for matches. Having an is that does a match breaks that expectation.
• The mistake is invisible. When the user inevitably forgets end the test will silently pass. They won't be aware of your mistake.
• It breaks the expectation that [1,2,3] and array { item 1; item 2; item 3; } are equivalent.

It's caught me plenty of times, probably all my code that uses hash or array forgot to use end.

To avoid this, I would suggest that is add an implicit end to any comparison which can take it. This avoids a very common mistake, and it removes a very common exception to the behavior of is vs like.

As for backwards compatibility, I think we'll find that most folks are screwing this up too. It's early enough in Test2's lifespan that we can fix this before it spreads.

Instead of adding a ton of subs, with possible inconsistencies, we should define these 5 base dsl subs. They should be used consistently, and Compare::XXX types should know what to do with them (consistent!!!)

is(
    $structure,
    st Hash => sub {
        my $st = shift;
        # or
        my $st = st_this;

        stc key1 => 'val1';
        stc key2 => 'val2';

        stc nested => st Array => { sta empty => 1 };

        stc non_empty_string => st String => { sta empty => 0 };

        stc price => st Number => 12; 

        sta 'end'; # Or to be more explicit:
        sta end => 1; # or stp etc => 1,
    },  
    "name"
);

st - define a structure

st Type Val
st Type &Builder
to define a Test2::Compare::Type check

stc - structure component;

stc key => 'val' # like 'field'
stc idx => 'val' # for arrays (like item with an index)
stc 'val' # for arrays (like item with no idx)
stc $val # String/number/etc.
to define components inside the structure

stcs - structure values

stcs ...; # Specify several components at once

sta - structure attributes

sta 'attr' - toggle attribute on
sta attr => $val - set attribute to value
This is essentially calling a method on the check when used in a builder to toggle attributes

stp - structure property

stp what => val
This would be like a 'prop' to check properties of the thing you got.

The 4th test of t/modules/Util/Table.t fails iff my terminal has less than 80 columns. I've attached the output from make test TEST_FILE='t/modules/Util/Table.t'.

Looking at the end of term_size() the behaviour looks intentional, but it was surprising to me:

...
    return 80 if !$total;
    return 80 if $total < 80;
    return $total;
}

testoutput.txt

#30 https://github.com/Test-More/Test2-Suite/blob/master/t/modules/Tools/Compare.t#L412

Confused to what the behavior here should be, Operator is '' when checking comparison operator of a string. So no OP is returned in the message.

stc is pretty simple, pass all arguments to $check->add_components(...) for the check currently being built.

Some notable differences ...

• Works around some brain damage in earlier Perl versions
• Exports a test_needs sub you can use in subtests, which is really nice

I think that's it, but @haarg might know other differences.