Filter files that do not match a size predicate about duff

[elmindreda]

It'd be silly for duff to replicate all the options not only in find but all other file-finding tools. That's why you can use duff as a filter. There are even examples of this in the manpage. To use one of your examples above:

find -size +1G | duff

If your file names contain whitespace, you need to use null-termination:

find -size +1G -print0 | duff -0 | xargs -n1 -0 echo

The customizable header (-f) is intended to help with the building of pipes at the other end. For example, if you set it to just output the number of files in each cluster, you get data easily iterable from scripts. See the bundled join-duplicates.sh for an example of this.

[sarnobat]

You're right, this is easily pipable. I'm closing this ticket.

I must have been thinking of some other non-pipable functionality. I do have one question (probably not supposed to be in this ticket) - how does duff buffer the list of files that are fed in from find? Is it by number of bytes, and does the shell control that rather than duff? What I do know is that duff doesn't buffer the entire input before beginning the duplicate detection - thankfully. Though as a consequence can't identify all duplicates.

[elmindreda]

Duff processes files a little and performs an initial sort as they come in, but most of the work cannot be started until all files have been collected, exactly because it does report all duplicates.

[sarnobat]

Oh, I didn't know it batches them all first. I thought I've seen behavior where the buffering prevented the full report and that it streams results instead. I must be wrong.

[elmindreda]

The process_args function provides a nice high-level overview of what duff does and in what order.