match_files with negated path spec about python-pathspec

[cpburnz]

The most convenient way to find files under a directory, and exclude some based upon gitignore patterns is to treat the files as sets, and perform operations from there:

import pathspec
import pathspec.util

ignore_spec = pathspec.GitIgnoreSpec.from_lines([
    "*.txt",
    # ...
])

all_files = set(pathspec.util.iter_tree_files("/some/directory"))
ignore_files = set(ignore_spec.match_files(all_files))
keep_files = all_files - ignore_files

I think your 3rd suggestion of adding a negate parameter to PathSpec.match_tree_files() would be the safest way to make this easier. Gitignore has peculiar edge cases that I don't think would behave properly if every match was simply negated. Implementing the logic at the tree level would allow using the file iterator intelligently without having to consume the whole thing in 2 or 3 sets.

[oprypin]

Some ideas on how it could be achieved:

1. Add a method to PathSpec to negate the whole spec and return a spec with everything flipped.
   A bit similar to

   python-pathspec/pathspec/pathspec.py

   Line 80 in 933dd7d

   def __add__(self: Self, other: "PathSpec") -> Self:

2. Add a method to PathSpec to negate the whole spec and return a wrapper object that has a negated match_file method. (note: until this functionality is in the library, making such a wrapper myself seems quite feasible in the meantime)

3. Add a "negate" parameter to match_tree_files
   https://github.com/cpburnz/python-pathspec/blob/933dd7da982551300a584c98570993402a56bc27/pathspec/pathspec.py#L267-L268C19
   pass it on to match_files and negate the condition accordingly
   

   python-pathspec/pathspec/pathspec.py

   Line 213 in 933dd7d

   if self._match_file(use_patterns, norm_file):

[oprypin]

4. Add a "negate" parameter to from_lines which flips the include of each parsed line

[oprypin]

Yes, a negate parameter seems to be the right approach, and indeed I was running into all kinds of edge cases otherwise.

The current approach with scanning the directory tree twice does not seem optimal to me :(

Meanwhile I also realized that there is a decently convenient way in the current state as well, without pulling too much of internals with it:

def match_files_negated(exclude: pathspec.PathSpec, root: str) -> Iterable[str]:
    for path in pathspec.util.iter_tree_files(root):
        if not exclude.match_file(path):
            yield path

[cpburnz]

The negate parameter for PathSpec.match_files() and related methods has been implemented and is in the new v0.11.2 release.