Nested capture groups are not passed to custom param definition transformer about cucumber-expressions

[mpkorstanje]

This applies to every language implementation. But I don't see how this can be fixed without potentially breaking everything for everyone.

Other than Ecmascript non-compliance, do you have a concrete example of how this prevents you from doing something?

[notaphplover]

Hi @mpkorstanje, yeah, sometimes both root and non root capturing groups are convenient to have.

As an example, consider a list of names whose delimiter is the comma (,) char. The last elements would be separated by using the and keyword. A possible regex would be:

((\w+)(?:, (\w+))*(?: and (\w+))?)

Consider a root group is added due to the fact the group 0 is not passed to the transformer.

Consider this link to have a playground. Including nested capturing groups leads to a different implementation that not having them. Nested capturing groups, in this example, allow us to know the last two elements separated by the and delimiter if present. I am aware parser implementations can be proposed to correctly parse all the elements, but they would have to do additional checks to determine wheter or not there're two last elements separated by the keyword and. More sophisticated examples can be proposed, but I find this one simple enough to illustrate my issue.

Other than Ecmascript non-compliance, do you have a concrete example of how this prevents you from doing something?

Well, I would hazard to say all regex specs shares this basic nested capturing group feature, not only Ecmascript one.

But I don't see how this can be fixed without potentially breaking everything for everyone.

Breaking or fixing? I can't find a place in the docs in which it's said this is the expected behavior. If this is the expected behavior I would prefer to close this issue in favor of a new one with a proposal to change it, so everyone knows this is not a bug, but a feature instead

[mpkorstanje]

Thanks for the explanation. That is a good usecase indeed.

The fix would be a breaking change for anyone who currently has declared a regex with nested capture groups but only utilizes the top level groups programmatically. And even though not explicitly documented, it is a fairly observable part of the API for any user so we'd run into Hyrum's Law.

This would need some accommodation to allow the different cucumber implementations to introduce this as a non-breaking change. Something along the lines of Argument.getValueFromCaptureGroups. But I haven't checked the exact usage in the Cucumber implementations.

[notaphplover]

Hey @mpkorstanje, thank you so much for giving feedback.

The fix would be a breaking change for anyone who currently has declared a regex with nested capture groups but only utilizes the top level groups programmatically. And even though not explicitly documented, it is a fairly observable part of the API for any user so we'd run into Hyrum's Law.

I understand your concerns. I cannot think in a non breaking fix simple enough that would be better than releasing a major version so, would fixing this whenever you guys consider it's time for a major version make sense?

[mpkorstanje]

Not so lightly. While we could make a breaking change here and increase the major version, it would also be a breaking change for the different Cucumber implementations that use this library. They're each on their own independent life cycle -- coordinating this would be rather difficult. The alternative would be maintaining multiple versions of cucumber-expressions which isn't ideal either.

[notaphplover]

But that's going to happen with every bug / feature you have, right? Do you have any process to accomplish this? How do you deliver a new feature / fix a bug in this project?

[mpkorstanje]

For this specific problem, typically by introducing a second code path as a non-breaking change. And then deprecating the bugged code path, eventually removing it once all the Cucumber implementations have gone through their life cycle.

[notaphplover]

Then, what about having a RecursiveGroup extends Group class that is able to recursive look for capture groups and adding options on the user entrypoint (I don't know which one/s would be) for either using RecursiveGroup or Group? This way we could fix this that way and deprecate those options in favor to always use RecursiveGroup