pgTAP output is reported as failures about tap-junit

[dhershman1]

Thanks for the report!

I'm super sorry for the late reply. Work was hectic!

I'll start digging into this soon as I can.

[villelahdenvuo]

No worries, meanwhile I can use a workaround to report without failing the CI:

pg_prove --ext .sql -v test | tee out.tap
cat out.tap | npx tap-junit -o junit || true
grep -q 'All tests successful' out.tap

[dhershman1]

@villelahdenvuo So, is that 4th ok at the end always there? The problem is that the parser is reading that as a 4th test, which I'm assuming it shouldn't be because it looks like the test count is 3.

If so, I'll have to think up a snazzy method to counter this problem since I can't just mark empty oks to be ignored.

[villelahdenvuo]

@dhershman1 Seems like pgTAP always add the final ok. Maybe a CLI option to allow a final empty ok?

Edit: Looks like pg_prove actually prints the extra ok as it's supposed to print filename.....ok but the test results get in the middle. I will see if I can fix it with some arguments.

[dhershman1]

If it printed off the filename...ok I think you'd be good, that is causing a miscount of your tests, tap thinks you have 4 tests and is only expecting 3.

[villelahdenvuo]

t/group.sql ....... ok     
t/logging.sql ..... ok     
t/rls_admin.sql ... ok     
t/rls_normal.sql .. ok     
t/security.sql .... ok   
t/service.sql ..... ok   
t/user.sql ........ ok     
t/utils.sql ....... ok   
All tests successful.

This is the output without --verbose and here it is with --verbose:

...
ok 23 - find_user_by_email_in_user_pools should find user in multiple user pools
ok
t/utils.sql ....... 
1..3
ok 1 - slugify function test 1
ok 2 - slugify function test 2
ok 3 - slugify function test 3
ok
All tests successful.

So the problem indeed is in the output format of pg_prove. I didn't find any command line option for it to not output extra ok for each file. I guess I will just keep my workaround. You can close this issue if you want.

[dhershman1]

Though it may be against convention, I can see about adding an option like --expect-hanging-ok or something to account for this.

It may not be great but the more outputs tap-junit can support the more helpful it can be!

[villelahdenvuo]

If you can implement something like that it would be great, but the workaround has been working fine so it's not a big issue for us. Thanks for everything!