yuanchuan / node-watch Goto Github PK

Hi @yuanchuan,

You might be planning to already, but would be nice to release the recent commits soon for use in qunitjs/qunit.

I've already landed use of [email protected], but would feel more comfortable with the adoption if these recent changes are also applied there before releasing the next version of QUnit.

Thanks for everything!

Example Codes:

watch('myCustomDir',{
    filter: function(filename){
        return /\.js$/.test(filename);
    }
},function(filename){
    // do something else ...
})

What I mean is to watch all the js file in the custom directory, but the function just return with nothing.

I walk through the source code, and found these lines at the beginning of the watch function:

  var skip = watcher.closed || !options.filter(fpath) || (
    is.sym(fpath) && !(options.followSymLinks && options.maxSymLevel--)
  );
  if (skip) return;

Which means: The path given must suit the filter, or it will return directly.

But should the filter here only affect only on the changed files/directories ? In other words, removing the !options.filter(fpath) condition may be a better choice here ?

Test platform: Windows 7.
When delete a file in nodejs 0.12.7 , a TypeError is thrown:

path.js:233
      throw new TypeError('Arguments to path.join must be strings');
      ^
TypeError: Arguments to path.join must be strings
    at Object.win32.join (path.js:233:13)
    at FSWatcher.<anonymous> (E:\work\game_bible_cms_proj\document\tools\node_mo
dules\node-watch\lib\watch.js:257:26)
    at FSWatcher.emit (events.js:110:17)
    at FSEvent.FSWatcher._handle.onchange (fs.js:1159:12)

This error happend because the filename arguments is not always provided, see the doc here.
So the path.join(fpath, fname) will raise that error. It usually occurs when deleting a file.

However, it works fine on node 4.5.0 and node 6.6.0. It may need a fallback logic or friendly warning, or simply drop the support for low version nodejs.

the file change event doesn't be triggered in recursive directory if having filter. eg. dir is /root, change under /root is triggered, but change under /root/doc is not triggered.

`
options =
recursive: true
filter: (name) ->
path.extname(name) is '.jade'

watch dir, options, (eventType, filename) =>
console.log "#{eventType}: #{filename}"
`

Hi
How i can run the watch when new folder is added in that watched folder or any folder is removed in it

Sample case:

------------ test script -----------

events = require 'events'
event  = new events.EventEmitter()
watch  = require 'watch'
fileExists = (stat) ->
    return false unless stat
    return stat.dev or stat.mode
BASEDIR = "./root"
watch.watchTree BASEDIR, { ignoreDotFiles: on }, (f, curr, prev) ->
    if typeof(f) is 'object'
        for path of f
            event.emit '#set', path
        return
    event.emit '#del', f if fileExists(prev) and not fileExists(curr)
    event.emit '#set', f

return event.on '#del', (file) -> console.log "del #{file}"
event.on '#set', (file) -> console.log "set #{file}"

------------ test script end --------

How to reproduce:

1. create a test directory, say ~/testwatch
2. cd ~/testwatch
3. put the test script in case1.coffee (or its js equivalent using coffee -c .)
4. mkdir "./root"
5. run coffee case1.coffee
6. while it's running, do mkdir -p ./root/FOO/BAR/BAZ/BUZ

Expected output:

./root/FOO/BAR/BAZ/BUZ
./root/FOO/BAR/BAZ
./root/FOO/BAR
./root/FOO

Actual output

set ./root
set root/FOO
set root/FOO

Problem: seems like some events are getting triggered twice. Furthermore, watch didn't pick up the subfolders.

Thanks
JM

The : should be a ; on the following line in the second FAQ question:

fn(filename):

It looks like the recursive option for fs.watch is available only on macos and Windows:

if (!(common.isOSX || common.isWindows))
  common.skip('recursive option is darwin/windows specific');

• https://github.com/nodejs/node/blob/bf6ce47259ebfe9f6349c223cc1735726aee062d/test/parallel/test-fs-watch-recursive.js#L5-L6

If we know this, why not just check the platform quickly, instead of doing an elaborate runtime check? The nodejs tests use these helpers:

exports.isWindows = process.platform === 'win32';
exports.isOSX = process.platform === 'darwin';

The code for hasNativeRecursive is pretty extensive:

• https://github.com/yuanchuan/node-watch/blob/master/lib/has-native-recursive.js

Is there a reason why the quick checks aren't used?

Thanks!

The hardcoded delay of 200ms in has-native-recursive.js means that on Linux, there are is no watcher existing during the first 200ms after calling watch(dir, options).

This is okay, but I think it would be good for users of node-watch to not have to hardcode this number in their own code.

Instead, perhaps we can emit a ready event once this check has finished and consumers can expect events to fire.

During #73, I was adding on('ready') to various tests, and they worked everywhere, except for the two tests that use composeWatcher.

Waiting for ready on a composed watcher times out, because I forgot to add support for it in #71. I did not realise...

I can fix this, but wanted to focus on #69 first, so I'm writing it down here for later :)

The following snippet will give me double change events if I change a file in the files directory:

const watcher = watch([".", "files/"], { recursive: true });

Is this by design or a bug? Both the patterns will match on a file in the files directory, but I would expect that duplicate events are removed.

There's a specific node package called "blade" that uses a single circular reference in

node_modules/blade/meteor/node_modules/blade/

where the last folder is

-> ../../

this causes watch to endlessly recurse until the stack is blown. There should probably be an option to ignore symlinks, stop at a certain depth, or (better yet) recognize circular symlinks.

I have tried creating a symlink like this:
── example
│   ├── sym.jade -> ../../temp/symlink.jade
│   └── test.jade

and then I watched example/
however, instead of test.jade, when I modified sym.jade, watch.js didn't trigger file changed.

my watch.js is like this:

watch('example', { recusive: true, followSymLinks: true, maxSymLevel: 3 },
    filter(/\.jade$/, function(filename) {
        console.log(filename, ' changed.');
}));

Hi,

The watcher callback is triggered even if the .close() method is called. To be exact the callback is triggered after the configured delay.

Not sure if you schedule this callback with a timeOut or something else, but we expect .close() method to clean the timer when triggered.

We detected the problem on our Config package (reported by one user).
SlimIO/Config@5228597

You should be able to reproduce with a case like that:

const watch = require('node-watch');

const config = { delay: 500 };
const watcher = watch('yourfile', config, function() {
  console.log('triggered');
});
// Write your file here to trigger the watcher
watcher.close();
// Wait 500ms or more here
// Watcher callback triggered even if closed...

Best Regards,
Thomas

The following code

watch('foo.bar',function(fn){
   console.log(fn,'first listener');
});
watch('foo.bar',function(fn){
   console.log(fn,'second listener');
});

Will result:

foo.bar first listener

second listener not firing

I im having ffmpeg that is generating upon node start in tmp folder m3u8 files that holds indexes for ts segment files..so i need to watch m3u8 file to file change and then extract indexes from that file.

node-watch works excellent in terms of low cpu usage and instantly it get triggered from OS when m3u8 file changes...that works flawlessly.

But problem is when i start program ffmpeg needs sometime 2seconds to generate m3u8 file sometimes 10seconds...so time creation of m3u8 file is random.

And when i call on start node.js application and run watch for file 1.m3u8 i get this error:

Error: /tmp//function (req, res, next) {
app.handle(req, res, next);
}_.m3u8 does not exist.

So i need if you could rewrite code in watch.js line 371 to return to watch function and to call it again in for example one second...when file will be generated...or if you have better idea you can add it to this line...

here is original code that throws error when file not exists...maybe adding check if file exists and then settimeout to call function again?

if (!is.exists(fpath)) {
watcher.emit('error',
new Error(fpath + ' does not exist.')
);
}

So in short description problem is that watcher exits with error when file does not exists...if you could fix this to if file not exists call watcher again in one second...if it exists continuo to execute code like now

I've been considering switching over from chokidar to this due to a few issues, and having 0 dependencies is actually somewhat appealing to me.

But I'm missing a few features:

• Initial Scan (report found files as when starting the watcher)
• A "create" event (for when files are created)
• When a directory is removed in a recursive watch, report it's contents recursively as individual 'remove' events as well

Currently I have a few dirty workarounds for some of the things above, but it'd be nice if node-watch can have these built-in.

Using an async function as callback doesn't works anymore, the function never get called (no warnings/errors either).
This used to work with v0.5.5

node v8.5.0

not always emit "change" because is.file return "undefined" - exception: wrong file path
replace
normalizeCall(**fname**, options, cb, watcher);
with
normalizeCall(**fpath**, options, cb, watcher);

It seems in watch.js on like 23 throws an error in situtation where files are moving at light-speed:

      if (fs.existsSync(fpath)) {
          memo.push(fpath, method);
          return stat(fpath)['is' + shortcuts[method]]();  

We get ENOENT errors even though the existsSynch succeed. It happens very seldom but it does happen. When it does happen, the application crashes complete. Should a simple try{} catch(){} around the watch function capture this and not affect watch from continuing?

Hi. Is there any possibility to write something like that:

watch('myDir/*/', filter(/\.js$/, function(filename) {
    // 
}));

It means that i've got many folders in myDir and i want to watch for js-files in them. Can i use a pattern in myDir value?

I think there needs to be a way to remove existing recursive watch and I do not see anything like this in the API. Please advise.

@yuanchuan I can't even get this running on windows. I tried the examples from the readme. Nothing happends.

I tried on Mac, and it works. However I noticed

• removing a folder you are watching, throws errors
• no checks for absolute path
• eating a lot of CPU
• rename a sub-folder inside a folder you are listen too, didn't invoke any events

The watch function returns a fs.FSWatcher like object as the same as fs.watch

let watcher = watch('file', () => {});

watcher.on('close', () => {
   // this never gets called
   console.log('close event');
});

watcher.close();

See nodejs/lib/internal/fs/watchers.js#L183

Hey!
Great tool by the way.

So this is my script:

'use strict';

// Imports
const watch = require('node-watch');

// Start the watcher
const watcher = watch('**/*');

console.log("Starting server watch...");

// Wait for changes...
watcher.on('change', file => {
    console.log("foo");
});

// Was there an error?
watcher.on('error', err => console.error(err));

This is the script I'm trying to run.
Is watcher supposed to be left open? At the moment, with this script, the watcher doesn't actually "watch" it just closes right at the end of the script. Is there any option to keep the watcher open?

When a Watcher for home (recursive) receives an event about home/c/sub being removed (file or directory), it calls close(fullPath).

The logic in Watcher.prototype.close is mostly correct. It understands fullPath and only closes fs.FSWatcher objects that correlate to the fullPath. And, if the removed thing was just a file, then it means there is no FSWatcher object for a directory, and nothing happens.

So far so good.

Then, the last line in the close() function is:

  this._isClosed = true;
  process.nextTick(emitClose, this);

After this, the internal FSWatcher stays open (which is good, because the directory was not really removed, only a file inside of it). But, the Watcher now thinks it is closed!

This problem only affects Linux, because on macOS and Windows, the recursive watcher works natively, and this logic is never used.

I found this problem because of a combination with a problem of Node.js internally.

• The internal problem is nodejs/node#25301. In a nut shell: When removing directory home/c, there are three events:

  1. event for home/c/c renamed (wrong)
  2. event for home/c/c renamed (wrong)
  3. event for home/c renamed (correct)

• In my bug fix for #69, I separate the internal delay from the external delay. That means the callback in Watcher.prototype.add runs immediately when an event happens.

• The first event is for home/c/ which thinks home/c/c was removed. It checks if home/c/c exists, sees that it doesn't exist, and calls close(fullPath='home/c/c'), which sets _isClosed = true.

• The second event ...

• The third event is for home/ which thinks home/c was removed. This is the event we want. But... the callback checks isClosed() and then returns early. There is no emit('change').

This caused the unit test for watch for directories / should watch directories inside a directory to fail, because no event for home/c is emitted.

So before I can fix #69, I will try to reproduce and fix this one first :)

On Mac OS X El Capitan I listen for a directory "upload" recursively.

Using finder (also tried to do it via console), I copied "abbrev" folder with some files in it to "upload/node_modules" directory.

node-watch only detected that upload/node_modules/abbrev changed (no file changes inside abbrev directory)

But sometimes node-watch also detects files inside copied directory

Hi,
I want to know is there a way I can pass config from json file ?
I wrote json file however not sure how to pass the regular expression from json file.
When I use below code it is not working

    "watchConfig": {
        "recursive": true,
        "filter": "/\\.tests.ts$/"
    },

When you use recursive: true and watch a file together with some folders, it seems the lstat is throwing an error..

If you get ENOSPC error, but you actually have free disk space - it means that your OS watcher limit is too low and you probably want to recursively watch a big tree of files.

Follow this description to increase the limit:
https://confluence.jetbrains.com/display/IDEADEV/Inotify+Watches+Limit

@yuanchuan my suggestion would be to notice this somewhere in README to avoid confusion.

It is not possible to Watch changes to 'files' or 'directoriies' in the onewire directory used by owfs. FS has the same problem, but files can be read and written to without problmes (in fs).
Any explanation?

I get the following error if I delete a directory that is within the directory I am watching. It crashes the node server. I'm on Windows 8 64bit.

events.js:71
    throw arguments[1]; // Unhandled 'error'event

Error: watch EPERM
    at errnoException (fs.js:806:11)
    at FSEvent._handle.onchange (fs.js:824:26)

It happens on both hard delete and deleting to recycle bin, and whether the directory is empty or not.

When using pure fs.watch it does not happen, although oddly fs.watch reports a rename event, not a delete or remove.

Sample case:

------------ test script -----------

events = require 'events'
event  = new events.EventEmitter()
watch  = require 'watch'
fileExists = (stat) ->
    return false unless stat
    return stat.dev or stat.mode
BASEDIR = "./root"
watch.watchTree BASEDIR, { ignoreDotFiles: on }, (f, curr, prev) ->
    if typeof(f) is 'object'
        for path of f
            event.emit '#set', path
        return
    event.emit '#del', f if fileExists(prev) and not fileExists(curr)
    event.emit '#set', f

return event.on '#del', (file) -> console.log "del #{file}"
event.on '#set', (file) -> console.log "set #{file}"

------------ test script end --------

How to reproduce:

1. create a test directory, say ~/testwatch
2. cd ~/testwatch
3. put the test script in case1.coffee (or its js equivalent using coffee -c .)
4. mkdir "./root"
5. touch "./root/FOO"
6. run coffee case1.coffee

Expected output:

set ./root
set ./root/FOO

Actual output

set ./root
set root/FOO

Problem: watch is trimming "./", but it's not even consistent. It would be better to keep whatever the user supplied.

Thanks
JM

When we use this library (which is great), we still have to add some code to fs.stat the file once an inotify event occurs.

We must do this because larger files when copied come across the TCP pipe in smaller buffers until the file is completely written. We created some timeout events to make this work. Perhaps some attention can be put into addressing the idea of creating a custom EventEmitter or something of that nature when a file is completely written.

It seems on Linux, once a new file is created, changes to it are not reported correctly.

I encountered this issue when attempting to use node-watch inside QUnit (for its CLI watch mode), as some of our unit tests weren't passing.

NodeJS's fs.watch supports a { persistent: false } flag which can be used to stop the watcher stalling the event loop (and stopping the program from exiting).

node-watch should support passing persistent:false through its options to fs.watch

Is there a way to avoid following symbolic links from the directory being watched?

Hi, on my system I'm doing a quick fix to allow for unwatching and figured you might be interested in what I'm doing. I'm just returning the fs.watch so the user can perform a close on that. We might as well have the function return something.

Is it somehow possible to know, when deleting a directory, the list of files and subdirectories deleted with it, or at least if it was a file or a directory?

Do you have any plans to implement command line interface for the library? I've been searching for a good cli tool for watching files matching a certain pattern.

If you don't have plans for this, would you accept a cli as pull request?

It would be great to have options like select and reject.

select is a regular expression or an array of file extensions to be watched (by default any)
reject is a regular expression or an array of file extensions to be not watched (by default none).

Is there any option can be used to configure the dirs I don't want to watch like node_modules?

fs.watch issue

http://stackoverflow.com/questions/21378627/watching-for-file-changes-on-complete-file-transfer