Recent change seems to have rendered documented way to call `biber` not working about l3build

[josephwright]

I'd forgotten we'd documented that run could be used in a conditional - sorry about that.

[josephwright]

One option would be to change back to running any extra tasks ignoring the exit code of the main run.

I'm going to go with this plan - I think on balance it's safest.

[josephwright]

I was using 'filler' text here as it seemed safe, but we can change that. But that still leaves the issue that if you are trying in the hook to save state changes, we can't check if it's got an empty return value. Should we document something?

[gusbrs]

I'm not sure I understand the reason for the commit, but if the comment:

This avoids an unrequired shell call.

is the reason, shouldn't you use the same arguments as the actual call?

    if errlevels[i] == 0 and runtest_tasks(jobname(lvtfile),i) ~= "" then
      local errorlevel =
        runcmd(preamble .. runtest_tasks(jobname(lvtfile),i),testdir)
      if errorlevel ~= 0 then errlevels[i] = errorlevel end
    end

I'm not sure what you were trying to do, or what you mean by "if you are trying in the hook to save state changes" (I assume you have a use case in mind). I just know the change rendered the documented way of doing things not to work anymore, and that's what I reported.

[gusbrs]

Thank you!

[moewew]

I just (well, the test started failing a few days ago, but I just got round to looking at it) ran into a very similar issue with my biblatex-ext tests. Evidently Biber is not being called. I didn't manage to get any sensible redefinition of runtest_tasks going. Is there something else that changed here?

---

I even tried

function runtest_tasks (name,run)
  return "biber " .. name
end

but no joy.

[gusbrs]

@moewew Theoretically, it should be working, as long as you have the latest update. The fix did work. However, in my experience, after I got the update, most of my biblatex related tests were back to normal (meaning biber is running), but some of them did not. Tracking it down, it turned out that biber was not running if some error occurred along the way. In my case I had some stray \counterwithin*{postnote}{chapter} in tests in the article class. They were completely unrelated to the tests, occurring before \START with no real effect on the document, but the error they generated prevented somehow biber from running (I don't understand why really, or why this behavior changed...). Once I fixed those errors, the remaining tests were back to working (see gusbrs/postnotes@cd5f164). Perhaps you have something of the sort on your side?

[josephwright]

@moewew, @gusbrs What's happening here is that we needed to split the test run into the main part and (optionally) the extra tasks; in the past the extra tasks always ran. So if the main run gives a non-zero errorlevel, we now don't run the extra tasks. That doesn't seem unreasonable - if you need post-run tasks, they likely need the main run to finish correctly.

[gusbrs]

So if the main run gives a non-zero errorlevel, we now don't run the extra tasks. That doesn't seem unreasonable - if you need post-run tasks, they likely need the main run to finish correctly.

My first reaction was in agreement, and I'm happy I corrected the blatant errors in my test files. But isn't deliberately generating an error a legitimate technique in tests (e.g. the l3build documented \ERROR macro)?

[moewew]

Hmm, I'm using lots of \showbox which I think halts the TeX run and probably lets it end with a non-zero exit code. Is there an alternative for that?

[u-fischer]

@moewew you could redirect the box content into an external file and read that in again. Imho that shouldn't led a non-zero exit code:

\documentclass{article}
\input{regression-test}
\begin{document}
\START
\newwrite\boxout
\immediate\openout\boxout{box.txt}
\showstream\boxout
\setbox0\hbox{duck}
\showbox0
\setbox0\hbox{bär}
\showbox0
\immediate\closeout\boxout
\SHOWFILE{box.txt}
xxx
\end{document}

[FrankMittelbach]

If that works, we could provide a command like \SHOWBOX that encaps that workflow, couldn't we?

[moewew]

@u-fischer Thanks. That appears to work and is a very clever trick. I hope you don't mind me saying that it doesn't feel particularly elegant, but since I've packed it up into a macro, I don't have to worry about that too much.

[FrankMittelbach]

@moewew that's why I suggested that the test support code should offer such a macro so that one has a clean interface for such testing. I therefore reopen that issue, to consider this as an enhancement

[muzimuzhi]

The problem described in (current) title of this issue has been resolved. Maybe trace the new/follow-up problem reported in #327 (comment) in a new issue?

Hmm, I'm using lots of \showbox which I think halts the TeX run and probably lets it end with a non-zero exit code. Is there an alternative for that?

Update: I opened #336.

[josephwright]

One option would be to change back to running any extra tasks ignoring the exit code of the main run.

I included the check as I'd imaged that people using an external tool would arrange that any tests only took place after the external tool had run, so something like

\IfFileExists{output-of-tool}
  {\def\testoutput#1{Some-useful-payload}}
  {\def\testoutput#1{}}

but that was likely naïve.

Almost all of the time there are no extras, so simply running irrespective of errorlevel is not going to impact on almost all users of l3build.