expanddirectories applied only on the patterns in the patterns options. Directories defined in the ignore options should be expanded too.

https://github.com/isaacs/node-glob#events

Use case: I want to check why my globs are running slowly.

If the specified pattern contains parentheses, no match is ever found, even when the path exists.

const globby = require('globby');

(async () => {
  const glob = './any(directory)/file.txt';
  const paths = await globby(glob);

  console.log(paths); //always logs an empty array
})();

Tested environments...

macOS 10.13.3
globby 8.0.0
node 7.10.0
npm 5.6.0

Windows 10.0.16299 Pro
globby 8.0.0
node 9.5.0
npm 5.6.0

Currently, globby or rather glob returns both directories and files. You can turn off returning directories using the nodir: true glob option, but that's not what you want. You'd rather want to expand directories into files. That's what this option should do. So globby.sync('foo') where foo is a directory would be expanded into all the files in that directory that doesn't match any of the ignore patterns. Same with globby.sync('foo/**'). There also needs to be a way to specify an array of extensions. In many cases you wouldn't want the expansion to happen for all files, but, for example, just .js files. It should override the nodir option if specified by the user.

This would be useful and simplify code in AVA, XO, cpy, imagemin-cli, and other tools expecting files. For example: https://github.com/avajs/ava-files/blob/master/index.js#L212-L271

@UltCombo @schnittstabil @jamestalmage @SamVerschueren @novemberborn Thoughts?

If you have a match like ['parent/**','!/parent/level1/level2/test.txt']
currently the parent directories of test.txt will remain in the matching list:

'parent/'
'parent/level1/'
'parent/level1/level2/'

If you want to use f.e. https://github.com/sindresorhus/del to delete these matches test.txt will be removed too. To delete a directory, but preserve some subset, the parent directories of any negative match need to be removed too.
See also sindresorhus/del#3

Can't make globby to search file inside this folder: C:\Users\me\Desktop\github.com + Globby.
I've tryed noext and other options, but without result.

Windows 10, Node 10

This finds a lot of files with node_modules in their path.

const paths = await globby(['..'], { ignore: ['**/node_modules/**'] });

Same with this:

const paths = await globby(['..', '!**/node_modules/**'] );

This doesn't include such files:

const paths = await globby(['..'], { ignore: ['../**/node_modules/**'] });

Is this behavior expected or is it a bug?

Why does globby return normalized slashes (/), even for paths with non-ASCII characters in Windows? According to the slash module, those paths would not be usable in Windows.

If you bump the version to at least node >= 4, you can get rid of the pinkie-promise and object-assign dependencies.
I would be happy to submit a PR if you would like.

I have had issues with big directories and globby.

When globby searches huge directories it takes very long (> 20min) to return results.
In the mentioned comments below I describe the following behavior.

proj
  - path/to/code/src (3 directories, 1 level nesting, 20 files)
  - assets (10 directories, 3 level nesting, 100 files)
  - dirt (100 directories, 10 level nesting, 1000 files)

globby('path/to/code/src/**/*.js')
// let's wait for 20 mintes
.then(files => {

})

I stumbled upon this when using Ava and XO in a big project.

Metions:
xojs/xo#212 (comment)
avajs/ava#1329 (comment)

If more info is needed I will be happy to provide.

Gulp might want to depend on this. Accommodate what's needed to make that happen.

gulpjs/glob-stream#27 (comment)

TODO:

• #2
• #3
• #4

I think this stopped working with cf55e5c.

This

We probably have to use deep equality or improve the handling some other way. The readability of the code could also be improved.

// @kevva

kevva earned $40.00 by resolving this issue!

• Checkout the Issuehunt explorer to discover more funded issues.
• Need some help from other developers? Add your repositories on Issuehunt to raise funds.

It appears that there is a cwd option, but it doesn't seem to be documented in the README file yet.

If we're treating this as a single glob, it should be more performant to reuse the cache properties between glob calls. There are cache, statCache, symlinks and realpathCache options exposed - https://github.com/isaacs/node-glob#properties.

problem

on my centos7 system, globby doesn't pick up files. it does pick up files on my osx box

example

$ node
> g = require('globby')
> g.sync(`/home/cdieringer/src/proj/packages/*`)
[]
>
(To exit, press ^C again or type .exit)
> // ^C
$ ll /home/cdieringer/src/proj/packages/
total 28
drwxrwxr-x.  7 cdieringer cdieringer 4096 Aug  9 13:51 ./
drwxrwxr-x. 10 cdieringer cdieringer 4096 Aug  9 13:52 ../
drwxrwxr-x.  8 cdieringer cdieringer 4096 Aug  9 13:53 api/
drwxrwxr-x.  5 cdieringer cdieringer 4096 Aug  9 13:52 common/
drwxrwxr-x.  7 cdieringer cdieringer 4096 Aug  9 13:52 e2e/
drwxrwxr-x.  8 cdieringer cdieringer 4096 Aug  9 13:53 ui/

as you can see, my user cannot find the files from globby, but can from bash.

bummer!

I noticed in globby 7 that nodir is true by default. It would be nice if it would auto-flip to false if the pattern has a trailing "/" which is used to indicate matching directories.

Would you accept a pull request for a final progress callback parameter? This function would be called each time a file or directory is checked for a match, whether the match is successful or not.

We have a case there the result doesn't match the expected outcome
It could be a bug in "glob" itself

---test---

test( 'glob - async (ignore pattern) ', async t => {
	// Setup
	// fs.mkdirSync( 'tmp' );
	fs.writeFileSync( 'tmp/my.comp.spec.js', 'content' );
	fs.writeFileSync( 'tmp/a.spec.js', 'content' );
	fs.writeFileSync( 'tmp/c.spec.js', 'content' );

	// console.log( await m( 'tmp/*[!.comp].spec.js' ));
	t.deepEqual(await m( 'tmp/*[!.comp].spec.js' ), [ 'tmp/a.spec.js', 'tmp/c.spec.js']);
});

I got curious about your uses of this file, on further investigation I found some nice post on SO. Please have a look since I find it more reasonable.

I am attempting to create some workflow functions with gulp that look something like this:

gulp.task('clean:docs', function () {
  console.log('current directory: ', require('process').cwd());
  console.log('files:', globby.sync(['./docs/**/*']));
  return del(['docs/**/*']);
});

In the base directory of my project I have a docs directory that I am trying to clear before regenerating documentation. The output of the above function is:

current directory: f:\sites\project-name-here
files: []

and the files remain untouched.

Since version 8.0.0 this library or one of its dependencies needs the "git" executable on npm install, which breaks several of our CI builds. I downgraded to 7.1.1 for now.

Is it possible to remove this dependency to git?

Node: 8.9.4
NPM: 5.6.0
OS: Windows Server 2016

if you get to it before i do:
https://github.com/jonschlinkert/micromatch

See this benchmark comparing glob-stream with globby. Although glob-stream has the overhead of creating streams for the matched files (which is not really significant as there is only 1 matched file in my test), glob-stream is still over 2 times faster than globby.

This is because globby seeks the paths that would be matched by the negative globs generating too many unnecessary file system calls, while glob-stream uses minimatch to handle negative globs.

TODO add some kind of benchmarking.

function handleMatch (individualMatch) {
  // return a promise
}

globby.map([...patterns], handleMatch).then(function (arrayOfPromiseResults)) {

});

essentially the equivalent of:

globby.map([...patterns]).then(function (arrayOfMatches) {
  return Promise.all(arrayOfMatches.map(handleMatch));
});

But it provides better async behavior by not requiring you wait for globby to be done before handling a match.

My project structure

/project
    /src
        - index.js (`globby` is used in this code)
    /files
        - foo.json
        - bar.json

Behavior in v7.1.1

const matches = await globby(['../files/foo.json', '../files/bar.json']);
console.log(matches);
// Output: ['../files/foo.json', '../files/bar.json']

Behavior in v8.0.1

const matches = await globby(['../files/foo.json', '../files/bar.json']);
console.log(matches);
// Output: [ ]

I know these aren't exactly globs but sometimes pure file paths get passed in and I need to globby to return them if they exist. Is this intended behavior, to only support files at the same or lower levels of the file structure?

Node version: 8.9.4

If array-union gets an upped version, it might be worthwhile to up this one as well, since this is part of where I found the issue in array-union.

This issue is dedicated to gathering ideas on how we can improve performance.

Some things that might help is to do filtering in the correct order to ensure minimal work is done, and cache and memoize as much as possible. The most important thing is that we skip ignored directories like node_modules as early as possible. It's the source of a lot of performance problems. The hard drive is the bottleneck, so we should ensure we interact with it as little as possible.

Some more ideas in #44 and mrmlnc/fast-glob#15

This Node.js feature request would also help a lot: nodejs/node#15699

Currently, the "cwd option" unit test is just an exact copy of the test above.

Upgrading from 6 to 8 (fast-glob change) forced me to use the following options to discover folders with dot:

{dot: true, onlyDirectories: true}

My code now looks like:

await globby('.mutode/mutode-*', {dot: true, onlyDirectories: true})

This is to help those who run into the issue.

Would you mind exposing the hasMagic function in globby.

https://github.com/isaacs/node-glob#globhasmagicpattern-options

globby([
'User/me/workspace/test-project',
'!node_modules/**'
]).then(res) // result always match the node_modules file.

I want to use an absolute path in patterns like the above code, and how could I know which directory does globby start to match file?

It would be great to see results as they are found. By implementing JavaScript async iteration support, we could be able to have results stream in as they are located.

Filing as a feature request here, because this is my favorite glob library. 🤩 Thanks!!!

I noticed a performance loss when updating to version 8.0.1 in globby.sync().
I try to find a file in some defined folders in a tree of 500 documents.
Using the sync method in this example is about 10 times faster with version 7.1.1

Switching to "fast_glob" does not seem to be faster in any case,
or maybe you see potential to optimize the used patterns or options?

globby.sync([ 'test/file-1.php', 'test/file-2.php' ])

returns:
[ 'test/file-2.php' ] but should return both files

(both file exists)

• add expandDirs option to "glob"
• add Globs to "globs" here
• add memoized dedupe to "globs" here
• use Globs to stream file paths, optionally performing other actions such as filtering junk

It looks like the latest version behaves differently from 7.1.1 for negated patterns.

['src\scss\\*.{scss,sass}', '!src\scss\\_*.{scss,sass}']

works as expected in v7.1.1 but does not work in v8 at the moment.

Hi. with different cwd, expands directories is not working.

└── a/
  ├── b
  └── c/
    ├── d
    └── e

// in directory 'a'

globby('c') // ['c/d', 'c/e']
globby('c', { cwd: process.cwd() }) // ['c/d', 'c/e']
globby('a', { cwd: path.resolve(process.cwd(), '..') }) //  []
globby('a/**/*', { cwd: path.resolve(process.cwd(), '..') }) // ['a/b', 'a/c/d', 'a/c/e']

thanks

There is a $20.00 open bounty on this issue. Add more on Issuehunt.

• Checkout the Issuehunt explorer to discover more funded issues.
• Need some help from other developers? Add your repositories on Issuehunt to raise funds.

There is no validation on the patterns. It is assumed that they can be used, and the code throws an error if they can't instead of rejecting the promise and triggering the catch.

Example:

globby([null]).then(...).catch(function(err) {
  // expect error to happen
});

This throws here, where the expectation is that it can read the pattern as a string when it is not a string.

Im working with globby 8.0.1
I'm passing this paths [ '.', '!docs', '!package-lock.json', '!public', '!node_modules' ] to globby.

In Unix works fine.

But in windows, the ignored paths doesn't work, and all paths that should be ignored pass through globby.

What am I doing wrong?

Right now, globby only returns the matched paths after all the globbing work is done. It would be nice to have an evented or streaming API to be notified as soon as a file is matched.

I don't understand what this is supposed to do. At what point do I install and use this? Obviously, I have to create my git repository first, but in what order do I use globby, before my first commit or after? How does this work exactly, within the work flow? I'm using node/npm, with webpack, and vanilla js, and a few other npm packages. I am not using gulp or grunt at this point.

I was hoping this would take care of the .gitignore problem, giving me a way to easily exclude it from my public github repository.

Thanks! :)

Currently only gitignores from cwd and down are taken into account for gitignore filtering. What about those who are higher in a hierarchy?

Use case: monorepo repositories (like babel for example), where gitignore is placed in root and individual packages do not have their own.

There is a $80.00 open bounty on this issue. Add more on Issuehunt.

• Checkout the Issuehunt explorer to discover more funded issues.
• Need some help from other developers? Add your repositories on Issuehunt to raise funds.

The task is to find all node_modules in the project tree, that are not inside other node_modules.

I tried unsuccessfully something like:

• ('**/node_modules/', {ignore: '**/node_modules/**/node_modules/'})

• ('**/node_modules/', {ignore: '**/node_modules/**'})

Any advice on this?

hello,
how are you?
i have a function that run every minute. what is weird is that it looks like to me it finds some data, that is already moved to another location. it must be globby doing it as it not getting any other data.
it looks like this:

incomingInvoiceDirIncomingFiles = await globby(`${incomingInvoiceDir.incoming}/*.pdf`)

Is there any kind of cache in globby ? if so, how can i disable it?

globby currently uses async.reduce which runs in series, I believe it would be better to run in parallel.

I see this passed in CI so it must just be a windows issue only

Running at 5d3d4de

(expansion is working fine in latest 7.X (pending #63))