qsv tojson sometimes coerces non-boolean fields to true/false about qsv

[jqnatividad]

I see your point @tmtmtmtm.

I'll add an option to explicitly turn off boolean inferencing. I'll also fine-tune the inferencing heuristic a bit more to further minimize false positives:

• if there are only two records, turn it off
• add more conditions to the inferencing logic

[jqnatividad]

Yes. This is by design @tmtmtmtm

If the cardinality of a column is 2, and the two unique values are "truthy" - i.e. the first character is 1/0, t/f, y/n case-insensitive, then its inferred to be a boolean field.

In the real world - this heuristic should be good enough.

Did you find this behavior in a "real-world" dataset?

[jqnatividad]

I improved it a bit to minimize True/False boolean inferencing false positives.

Lemme know how it works for you. Feel free to reopen.

[tmtmtmtm]

Did you find this behavior in a "real-world" dataset?

Yes. I don't think I'd have stumbled across it otherwise :)

A large part of my use of qsv is in a pipeline of handling changes to data. In the case in question here, exactly two records had changed, but this time the names triggered this false inference, so when the CSV was turned into JSON and passed through to the next stage of the pipeline, everything went badly awry.

Am I reading the code correctly the new failure case would be if the names started with "Tr" and "Fa" rather than just "T" and "F"? If so, that's definitely better, but it's still not one I'm particularly comfortable in presuming would never happen.

Perhaps there could be an option to turn this behaviour off? It smells too much like all those scenarios where Excel silently mangles some dates on import etc.