dart-lang / glob Goto Github PK

It appears that you can't match files in parent directories using Glob.

  Unable to spawn isolate: Error: Cannot run with sound null safety as one or more dependencies do not
  support null safety:

   - package:glob

Do you accept Pull Requests on that?

@nex3 @munificent @bwilkerson

Asking users to think about differences between Posix and Windows styles is too much.
Especially when path separators are already / for any OS.

I don't also quite understand the requirement for the leading / to match absolute paths.
It seems confusing.

The test test/test_list.dart has 6 subtests failing on Windows. The failures can be seen at
https://build.chromium.org/p/client.dart.packages/builders/packages-windows-glob/

It's deprecated!

Hi there,

Because Glob depends on the current working directory as determined by the path library, and path bypasses IOOverrides to determine this value (dart-lang/sdk#39796), libraries using glob become more difficult to test when using mocked filesystems.

Today we leak the FileSystemEntity api from dart:io or package:node_io respectively out to users, even though they don't all need this functionality. This also means the public api of this package actually changes on the node platform, which is less than ideal.

I propose that we migrate the list api to use package:file, and accept an optional require a file system as a parameter. We would also rename these apis to listFileSystem By default it would use the dart:io file system and if not available it would throw.

We will also provide an extension method in a separate library in this package, which can be used to glob the local file system without providing it explicitly, and this would match exactly the current api (list, listSync).

This would be a breaking change, and would require a new dependency on package:file. We would release this with null safety. The premise is that package:file is at least a more general package, and better than leaking a node specific dep to all downstream users.

cc @nex3 for thoughts as well since I believe that Sass is the primary user of the Node functionality.

glob 1.2.0 needs publishing, for @nex3 . I do not have publish permissions.

When using glob-2.1.1, I kept getting the following errors (with glob being a transitive dependency for build_runner)

/pub.dartlang.org/build_runner-2.3.2/lib/src/server/server.dart:381:34: Error: The method 'Glob' isn't defined for the class 'AssetHandler'.
...
/pub.dartlang.org/build-2.3.1/lib/src/asset/reader.dart:37:30: Error: Type 'Glob' not found.
...
/pub.dartlang.org/glob-2.1.1/lib/glob.dart: Error: The control character U+0000 can only be used in strings and comments.
...
/pub.dartlang.org/analyzer-5.2.0/lib/src/lint/linter.dart:153:8: Error: Type 'Glob' not found.
...
/pub.dartlang.org/build-2.3.1/lib/src/builder/build_step_impl.dart:109:30: Error: Type 'Glob' not found.
...
/pub.dartlang.org/build_runner-2.3.2/lib/src/server/asset_graph_handler.dart:73:39: Error: The method 'Glob' isn't defined for the class 'AssetGraphHandler'.
...
/pub.dartlang.org/build_runner_core-7.2.7/lib/src/asset_graph/node.dart:258:9: Error: Type 'Glob' not found.
...
/pub.dartlang.org/glob-2.1.1/lib/list_local_fs.dart: Error: The control character U+0000 can only be used in strings and comments.
...
/pub.dartlang.org/build_runner-2.3.2/lib/src/package_graph/build_config_overrides.dart:21:25: Error: Method not found: 'Glob'.
...

Going back to 2.1.0, the errors disappeared, which leads me to believe that

/pub.dartlang.org/glob-2.1.1/lib/glob.dart: Error: The control character U+0000 can only be used in strings and comments.
/pub.dartlang.org/glob-2.1.1/lib/list_local_fs.dart: Error: The control character U+0000 can only be used in strings and comments.

are the relevant errors that caused the other packages not to be able to resolve Glob.

So it seems that glob.dart and list_local_fs.dart contain some sort of illegal character?

I'm working on windows, using Dart SDK 2.19.0-255.2.beta.

I was running unit tests which are including Glob(...).matches(...), and it fails to match files when the mock filesystem is Windows-style, and the actual machine OS is POSIX-style. Even passing fileSystem.path as the context to Glob(...) does not help. When I had a closer look I found that in case of **\*a*, the class constructor appends {,/**} to the pattern. And although this should not produce any harm, I believe, it is not working as expected. I suspect, the machine's directory separator is used instead of the one for the file system. The similar issue happens to Glob(...).listFileStsemSync(...).

Run the tests with --preview-dart-2 (using Dart, like, > 2.0.0-dev.46 or something), and see failures:

$ dart --preview-dart-2 test/list_test.dart
...
00:01 +48 -12: sync lists a subdirectory that sometimes exists [E]
  type 'SubListIterable<String>' is not a subtype of type 'List<String>' where
    SubListIterable is from dart:_internal
    String is from dart:core
    List is from dart:core
    String is from dart:core

  package:glob/src/ast.dart 169:33                              SequenceNode.split
  package:glob/src/list_tree.dart 75:31                         new ListTree
  package:glob/glob.dart 150:44                                 Glob.listSync
  test/list_test.dart 340:12                                    syncAndAsync.<fn>.<fn>
  test/list_test.dart 287:24                                    main.<fn>.<fn>

Hello,

I upgraded to glob 2.0.0 today and found that I had to import package:glob/list_local_fs.dart to use Glob.list() and Glob.listSync(). Please update your readme example to reflect this slight change.

Thank you.

Hello,
I have this behavior on Windows with Glob 1.1.0 and I want to understand if this is me doing something wrong.

import 'dart:io';
import 'package:glob/glob.dart';

main() {
  new File('c:/myfolder/a.txt').createSync(recursive: true);

  String pattern = r'c:/myfolder/**';
  print(new Glob(pattern).listSync().length);
  print(new Glob(pattern, caseSensitive: false).listSync().length);
  print(new Glob(pattern, caseSensitive: true).listSync().length);
}

It prints:
0
0
1

Why is caseSensitive: true necessary here?
Is it normal to have more result with caseSensitive: true than false?

While writing my second Builder, I encountered the following:

$ dart run build_runner build
Building package executable...
Built build_runner:build_runner.
[INFO] Generating build script completed, took 235ms
[WARNING] Invalidated precompiled build script due to missing asset graph.
[INFO] Precompiling build script... completed, took 4.6s
[SEVERE] Nothing can be built, yet a build was requested.
[INFO] Initializing inputs

You have hit a bug in build_runner
Please file an issue with reproduction steps at https://github.com/dart-lang/build/issues


FileSystemException: Directory listing failed, path = 'c:\<path to project>\not\*' (OS Error: The system cannot find the path specified.
, errno = 3)
Instance of '_StringStackTrace'

Steps to reproduce

1. Create a new dart package (e.g. using dart create -t package-simple bug_report)
2. Add build_runner: to dev_dependencies in pubspec.yaml
3. Add the build.yaml below
4. Invoke build_runner (e.g. using dart run build_runner build) on Windows

# build.yaml
targets:
  $default:
    sources:
      include:
      - $package$
      - lib/$lib$
      - not/exists/**

Expected Results

Nothing. <path to project>/not doesn't exist and shouldn't cause anything. I.e. exactly what happens when includeing just not/**. Under WSL/Ubuntu, the same example package doesn't crash.

What builder(s) you are using (or writing yourself).

I'm writing a builder that is supposed to take data from a JSON file and turn it into dart classes at build time.

Context: We're looking for a way to turn ISO alpha-2 codes into localized country names using CLDR data without using a plugin (which limits platform choices) or one of the many packages with vendored lists (usually limited to english names). Essentially, something akin to how date_symbol_data_local is generated.

In practice I'm npm installing https://www.npmjs.com/package/cldr-localenames-modern as a lightweight alternative to vendoring https://github.com/unicode-org/cldr-json/tree/main/cldr-json/cldr-localenames-modern as files or a git submodule.

I'm currently debugging why the JSON files (located in <path to project>/sub_package/node_modules/...) are not passed to the builder instance created (it prints a log statement) for the $default target in sub_package when running build_runner build on the root project. I still don't know why not, but that's a different issue (also hit dart-lang/build#2835 in the process).

However, includeing sub_package/node_modules/** as test on whether the working directory was somehow wrong is how I encountered this bug.

Dart SDK Version

$ flutter pub --version
Flutter 2.10.4 • channel stable • https://github.com/flutter/flutter.git
Framework • revision c860cba910 (4 weeks ago) • 2022-03-25 00:23:12 -0500
Engine • revision 57d3bac3dd
Tools • Dart 2.16.2 • DevTools 2.9.2

What package(s) from this repo you are using, and the version

build: 2.3.0
build_config: 1.0.0
build_runner: 2.1.10

Whether you are using Windows, MacOSX, or Linux

Windows 11

I am trying to glob a bunch of images under a folder:

final avatars = new Glob('assets/images/avatars/**.png');

Unfortunately, I get:

Directory listing failed, path = './assets/images/avatars/' (OS Error: No such file or directory, errno = 2)

Same error with the following:

Glob('lib/**.png')

I cannot seem to figure out what is actually glob-able, any direction here?

There is an error in case of using relative paths: search with caseSensitive: true returns more results then caseSensitive: false.

Here is an example. Consider the following directories:

.
├── dir1
│   └── file1.txt
└── dir2
    └── file2.txt

Run the following program from the dir1:

import 'package:glob/glob.dart';
import 'package:glob/list_local_fs.dart';

void main(List<String> arguments) {
  print('1: ${Glob('**.txt', caseSensitive: null).listSync()}');
  print('2: ${Glob('**.txt', caseSensitive: false).listSync()}');
  print('3: ${Glob('**.txt', caseSensitive: true).listSync()}');

  print('4: ${Glob('../dir2/**.txt', caseSensitive: null).listSync()}');
  print('5: ${Glob('../dir2/**.txt', caseSensitive: false).listSync()}');
  print('6: ${Glob('../dir2/**.txt', caseSensitive: true).listSync()}');

  print('7: ${Glob('..\\dir2\\**.txt', caseSensitive: null).listSync()}');
  print('8: ${Glob('..\\dir2\\**.txt', caseSensitive: false).listSync()}');
  print('9: ${Glob('..\\dir2\\**.txt', caseSensitive: true).listSync()}');
}

Output on Linux:

1: [LocalFile: './file1.txt']
2: [LocalFile: './file1.txt']
3: [LocalFile: './file1.txt']
4: [LocalFile: './../dir2/file2.txt']
5: []
6: [LocalFile: './../dir2/file2.txt']
7: []
8: []
9: []

Output on Windows:

1: [LocalFile: '.\file1.txt']
2: [LocalFile: '.\file1.txt']
3: [LocalFile: '.\file1.txt']
4: []
5: []
6: [LocalFile: '.\..\dir2\file2.txt']
7: []
8: []
9: []

4 is OK, because caseSensitive defaults to false when context is Windows and true otherwise.

The real problem is that more strict condition caseSensitive: true (6) produces more then caseSensitive: false (5).

banner.dart
banner.parse.dart
banner.widget.parser.dart
banner.widget.parser.parse.dart
tony.dart
tony.parse.dart
tony.widget.parser.dart
tony.widget.parser.parse.dart

import 'package:glob/glob.dart';

//final dartFile = new Glob("lib/source_widget/*.{!parse*,!widget*}");
//Return empty

final dartFile = new Glob(r"lib/source_widget/*.{parse.dart,widget.parse.dart}",caseSensitive: true);
//Return 
//./lib/source_widget/tony.parse.dart
//./lib/source_widget/banner.parse.dart
//./lib/source_widget/banner.widget.parser.parse.dart
//./lib/source_widget/tony.widget.parser.parse.dart

// Recursively list all Dart files in the current directory.

void main(List<String> arguments) {
  for (var entity in dartFile.listSync()) {
    print(entity.path);
  }
}

But I just want *.dart for the build.yaml

Please advice.

Hello!

It appears that there are currently no way of excluding certain matches.
For example, a glob may want to match against foo.dart but reject foo_test.dart

The Glob syntax seems to support *[!_test].dart, but this does not appear to be supported by this package.

This is basically ready to go just need to land it in the Dart & Flutter SDKs first, which has been harder than it should be due to infra issues of adding a new dependency.

Consider the following:

// Returns a [Glob] that matches all .dart files not in test/.
new Glob("**.dart") - new Glob("test/**.dart")

// Returns a [Glob] that matches all .dart files in lib/ or bin/ or web/.
new Glob("bin/**.dart") + new Glob("lib/**.dart") + new Glob("web/**.dart")

Yes, you can do this with regular expressions, but:

• Not everyone is great with these (I'm not)
• This makes it easier to "build" a glob from many inputs

I imagine implementation would be something like:

class _CompositeGlob implements Glob {
  final List<Glob> _includes;
  final List<Glob> _excludes;

 _CompositeGlob(this._includes, this._excludes);
}

Consider:

Glob('**.dart');

If you are within:

/private/var/folders/68/t6gjyz1j7t5gzhvzbzwczpr80000gn/T/8kbZAb

and check against:

/Users/remirousselet/dev/whatever.dart

then glob.matches returns false.

This seems to be because:

• The ** ast returns canMatchAbsolute => false
• This causes Glob.matchAsPrefix to merge the current and tested paths, in this case:

  ../../../../../../../Users/remirousselet/dev/whatever.dart
  

• the ** ast doesn't matches against ../ paths as per _toRegExp's comment:

  // Double star shouldn't match paths with a leading "../", since these paths
  // wouldn't be listed with this glob. We only check for "../" at the
  // beginning since the paths are normalized before being checked against the
  // glob.
  

On Windows, the path /foo/bar (or equivalently \foo\bar) matches the path foo\bar relative to the current drive root. Globs should support this behavior when matching and when listing—so for example, /** should list all files on the current drive.

See #2

The methods list() and listSync() of the Glob class seem to be replaced by listFileSystem() and listFileSystemSync() but the example used in the readme still uses the old methods.

The glob listing functionality depends on dart:io, so package:glob/glob.dart can't be used in the browser. This could be resolved either by moving the listing functionality to a separate library within the package, or by using a file system abstraction like files.

@nex3 @bwilkerson

Currently globs are always case sensitive.
But it is not convenient.
In Dart Analysis Server we want to analyze both /User/user/proj/foo.dart and /User/user/proj/bar.DaRt files.
We could apply toLowerCase() externally to both pattern and paths, but why, if Glob could do this?

I'm not sure if this is a valid pattern, but while looking at #52880 I tried to use "{**/,}foo.dart", but it resulted in an exception:

print(Glob('{**/,}foo.dart').matches("c:\\foo\\bar.txt"));

Unhandled exception:
Bad state: No element
#0      List.first (dart:core-patch/growable_array.dart:343:5)
#1      SequenceNode.canMatchAbsolute (package:glob/src/ast.dart:60:38)
#2      OptionsNode.canMatchAbsolute.<anonymous closure> (package:glob/src/ast.dart:371:59)
#3      ListBase.any (dart:collection/list.dart:114:15)
#4      OptionsNode.canMatchAbsolute (package:glob/src/ast.dart:371:40)
#5      SequenceNode.canMatchAbsolute (package:glob/src/ast.dart:60:44)
#6      Glob._patternCanMatchAbsolute (package:glob/glob.dart:70:46)
#7      Glob.matchAsPrefix (package:glob/glob.dart:162:9)
#8      Glob.matches (package:glob/glob.dart:154:32)
#9      main (file:///C:/Dev/Test%20Projects/blogtest/bin/blogtest.dart:5:32)
#10     _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:294:33)
#11     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:189:12)

If the pattern is invalid and exception is expected, it should be more descriptive than this.

It would make upgrades easier from glob 1 to 2 if there was a release on the 1x release that had the same named entry point as the backwards compatibility file in the 2x releases.
In glob 2 you can import 'package:glob/list_local_fs.dart'; but you can't add this import while on the 1x line since that file doesn't exist.
Can I request a release of version 1.2.1 that simply has an empty lib/list_local_fs.dart to ease the upgrade to version 2?

For context we have a large dependency graph so this would make it possible to

• add the additional import where needed while on 1.2.0
• raise the allowed glob dependency version '>=1.0.0 <3.0.0'
• fix any other dependencies that might prevent actually resolving to glob 2x
• release and consume packages all the way through
• raise the minimum glob version to ^2.0.0
• We could then go back and fix the list calls and remove the list_local_fs imports

https://build.chromium.org/p/client.dart.packages/builders/packages-windows-glob/builds/888/steps/pub%20run%20test/logs/stdio

00:01 +107 -1: test\list_test.dart: async lists an absolute glob
  ScheduleError:
  | Expected: equals ['foo\\baz\\bang', 'foo\\baz\\qux'] unordered
  |   Actual: []
  |    Which: has too few elements (0 < 2)
  
  Stack chain:

00:02 +132 ~1 -2: test\list_test.dart: sync lists an absolute glob
  ScheduleError:
  | Expected: equals ['foo\\baz\\bang', 'foo\\baz\\qux'] unordered
  |   Actual: []
  |    Which: has too few elements (0 < 2)
  
  Stack chain:

unawaited is now in dart:async.