Exclusion patterns work only if processed after the inclusion ones. about python-pathspec

[cpburnz]

@ftrofin @ofek

I've now tested this and git handles the patterns as they appear in the ".gitignore" file. It does not run inclusion patterns before exclusion patterns. I'd recommend to sort the patterns after they're compiled. I.e.,

import pathspec
spec = pathspec.PathSpec.from_lines('gitwildmatch', ...)

# This will only work if the lines passed into `.from_lines()` is a `list` or a
# non-collection iterable which will be consumed as a `list`.
spec.patterns.sort(key=lambda p: -p.include)  

# This is more reliable.
spec.patterns = sorted(spec.patterns, key=lambda p: -p.include)  

---

Test 1 - git

Files:

• .gitignore
• a.txt
• b.txt
• keep.txt

.gitignore:

!keep.txt
*.txt

Files matched by git:

• .gitignore

Test 1 - pathspec

source_files = [
	".gitignore",
	"a.txt",
	"b.txt",
	"keep.txt",
]

ignore_spec = pathspec.PathSpec.from_lines('gitwildmatch', [
	"!keep.txt",
	"*.txt",
])

remove_files = list(ignore_spec.match_files(source_files))
print(remove_files)
# ['keep.txt', 'b.txt', 'a.txt']

keep_files = sorted(set(source_files).difference(remove_files))
# ['.gitignore']

Test 2 - git

Files:

• .gitignore
• a.txt
• b.txt
• keep.txt

.gitignore:

*.txt
!keep.txt

Files matched by git:

• .gitignore
• keep.txt

Test 2 - pathspec

source_files = [
	".gitignore",
	"a.txt",
	"b.txt",
	"keep.txt",
]

ignore_spec = pathspec.PathSpec.from_lines('gitwildmatch', [
	"*.txt",
	"!keep.txt",
])

remove_files = list(ignore_spec.match_files(source_files))
print(remove_files)
# ['b.txt', 'a.txt']

keep_files = sorted(set(source_files).difference(remove_files))
# ['.gitignore', 'keep.txt']

[ftrofin]

Here's another way of fixing it:

---
 pathspec/util.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/pathspec/util.py b/pathspec/util.py
index bcba878..c040f5a 100644
--- a/pathspec/util.py
+++ b/pathspec/util.py
@@ -7,6 +7,7 @@ import os
 import os.path
 import posixpath
 import stat
+from operator import attrgetter
 
 from .compat import Collection, Iterable, string_types, unicode
 
@@ -268,7 +269,10 @@ def match_files(patterns, files):
 	"""
 	all_files = files if isinstance(files, Collection) else list(files)
 	return_files = set()
-	for pattern in patterns:
+
+	# We need to apply the include patterns first
+	sorted_patterns = sorted(patterns, key=attrgetter('include'), reverse=True)
+	for pattern in sorted_patterns:
 		if pattern.include is not None:
 			result_files = pattern.match(all_files)
 			if pattern.include:
-- 

[ftrofin]

Let me know which way you would prefer and I can try to create a PR (or you can apply the changes yourself if you prefer.)

[ftrofin]

A workaround until this is fixed in the library itself is be to sort the patterns on the client side before calling pathspec:

	specs = sorted(specs, key=lambda s: s.startswith('!'))

[ofek]

@cpburnz Any update on this?