howeyc / fsnotify Goto Github PK

The function addWatch must be public so, we can pass it the flags that we desire.

https://github.com/howeyc/fsnotify/blob/master/fsnotify_bsd.go#L89

Note: the another files like fsnotify_linux.go also should be changed.

Reading about the subject online, I came across a recommendation that files be opened with the O_EVTONLY flag instead of O_RDONLY, when watching is desired -- is that applicable here? I'm not sure if the system performs any optimizations with that, but the only functional benefit is that it will then not prevent an unmount of the volume that the file is on. (This is not directly applicable to my use case, just thought I'd notify you of my finding)

https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/open.2.html

I am seing this behavior on OS X. Is this intended or a bug?

For example when watching /tmp/foo and doing the following:

cd /tmp/foo
mkdir blah
touch bar

I get 3 events:

"/tmp/foo/blah": CREATE
"/tmp/foo/bar": CREATE
"/tmp/foo/blah": CREATE

instead of just the first 2, which is what I would have expected.

If you touch(1) a file inside a watched folder and the file already exists, it will not trigger a modify event. If the file does not exist, it will trigger a create event. If you watch the file instead of the folder, touching an existing file will trigger a modify event.

This seems inconsistent to me. Is there a specific reason for this?

It seems to me, that if you watch a folder, the files inside it gets "degraded", thus ends up with lesser flags than FSN_ALL, which the folder and its subfolders gets. Why is that? Why doesn't the files also get FSN_ALL?

I'm testing this on Mac OS X 10.8.

Updated:
I'd like to understand the reason behind this behavior because it's entirely possible that the current behavior is the desired behavior and that I'm just not getting it.

To reproduce this, run tests with the -race flag. I didn't paste it here because there are more than 50 races (about 4200 lines), but here comes a gist: https://gist.github.com/fsouza/4722172.

If i watch a file and a dir, without the wd how can i distinguish file and dir?

This is a request that came up earlier that I almost forgot about. Creating issue so I remember.

Desire is to have the library send events/errors using select so that the receiving application can be monitoring the channels without the use of a separate goroutine.

I've been running https://github.com/azer/boxcars locally, which uses fsnotify to watch a single configuration file. Today I noticed that it's taking up 0.2% CPU even when idle, which struck me as odd.

Digging in, I found that it was due to the 100ms timeout that fsnotify uses for a timeout on the kevent() system call. As an experiment, I increased it to 1 second, and even 10 seconds. In both cases: CPU usage drops to 0%, and events still trigger as soon as the file modification happens.

Other than for tests, which should run quickly, is there any reason NOT to increase the syscall timeout? The kevent call is in a goroutine, so it doesn't matter if it's blocked. I'd rather that my programs sleep until events are ready instead of retriggering the syscall 10 times per second.

In any case, I'm keeping my local modification, as it works for me. :-) Thanks for this nice and clean little library!

When I watch a file and do edit it and save it i get 2 MODIFY events. I should receive only one MODIFY event.

I'm using fsnotify on Linux.

Hello,
When I do 'echo something > foo' I get two modify events for foo. Just curious. Perhaps that is just what you get from the kernel, and I have to deal with it somehow.
Also, after a rename of a watched file, fsnotify no longer watches either the old or new inodes. Is this normal, and if so, is there way to continue with the new inode? I tried to remove then re-add, but that didn't work either.

thanks for sharing your project!

If I watch a directory:

w, _ := fsnotify.NewWatcher()
go w.Watch("/Users/robfig/code/gocode/watched")
for {
    select {
    case ev := <- w.Event:
        log.Println(ev)
    case err := <- w.Error:
        log.Println(err)
    }
}

I do not get any events when creating a file in that directory. (e.g. "touch test.txt"). Further modifications to that file do fire events, though, so it seems like it may be a bug.

OSX 10.8.2

Thanks!

On Linux 64bit, I'm getting double events on all events. I'm not sure if other platforms are problematic.

Here's the code that I used to test:

package main

import (
    "fmt"
    "github.com/howeyc/fsnotify"
    "os"
)

func main() {
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }

    err = watcher.WatchFlags("/home/larz/Desktop/test.go", fsnotify.FSN_MODIFY)
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }

    for {
        select {
        case event := <-watcher.Event:
            fmt.Println(event)
        case err := <-watcher.Error:
            fmt.Println(err)
            os.Exit(1)
        }
    }
}

Here's the output while running:

%:Desktop ⵗ go run test.go                                                 
"/home/larz/Desktop/test.go": MODIFY
"/home/larz/Desktop/test.go": MODIFY 

I only modified it once, so I'm not sure why it would be doing it twice.

I've been seeing a few cases where a single modification to a file on Mac OS X causes two modification events to be sent over the chan. As first I assumed this was my editor doing something clever, but even a simple touch filename would trigger it.

I did some tests, altering TestFsnotifyAttrib to check if attribReceived.value() != 1 instead of checking for zero. This updated test case fails roughly half the time for me.

Digging even further it looks like the extra update update is being caused by mds (the daemon responsible for updating the file metadata used by spotlight). See lines 6 onwards from the output of fs_usage, which shows the name of any process performing file operations:

$ sudo fs_usage | grep TestFsnotifyAttrib.testfile
11:46:35  open              _test/TestFsnotifyAttrib.testfile                                                0.000065   fsnotify.tes
11:46:35  lstat64           _test/TestFsnotifyAttrib.testfile                                                0.000010   fsnotify.tes
11:46:35  open              _test/TestFsnotifyAttrib.testfile                                                0.000009   fsnotify.tes
11:46:35  chmod             _test/TestFsnotifyAttrib.testfile                                                0.000024   fsnotify.tes
11:46:35  access            /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000008   dbfseventsd 
11:46:35  getattrlist       /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000192   mds         
11:46:35  lstat64           /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000018   mdworker    
11:46:35  stat64            /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000013   mdworker    
11:46:35  lstat64           /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000009   mdworker    
11:46:35  getattrlist       /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000116   mdworker    
11:46:35  open              /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000024   mdworker    
11:46:35  getattrlist       /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000041   mdworker    
11:46:35  getattrlist       /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000116   mdworker    
11:46:35  getattrlist       go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile/.DS_Store    0.000009   mdworker    
11:46:35  getattrlist       go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile/.DS_Store    0.000006   mdworker    
11:46:35  getattrlist       go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile/.DS_Store    0.000005   mdworker    
11:46:35  getattrlist       go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile/.DS_Store    0.000005   mdworker    
11:46:35  getattrlist       /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000024   mds         
11:46:35  getattrlist       /Users/ph/go/src/github.com/howeyc/fsnotify/_test/TestFsnotifyAttrib.testfile    0.000010   mds         
11:46:35  unlink            _test/TestFsnotifyAttrib.testfile                                                0.000131   fsnotify.tes

(It's also worth noting that this test case consistently passes as of 2c4a662, because mds ignores temporary directories. But the problem is still there.)

I don't think this is a bug in fsnotify, but I do wonder if it's something you might want to document somewhere, possibly as part of the FAQ in the README? I'm happy to put together a pull request if you think that would be useful?

found via Go1.1's go test -race:

=== RUN TestFsnotifyFakeSymlink
==================
WARNING: DATA RACE
Read by goroutine 4:
  github.com/howeyc/fsnotify.TestFsnotifyFakeSymlink()
      /Users/sridharr/go/src/github.com/howeyc/fsnotify/fsnotify_symlink_test.go:76 +0x993
  testing.tRunner()
      /usr/local/Cellar/go/1.1/src/pkg/testing/testing.go:353 +0x12f
  gosched0()
      /usr/local/Cellar/go/1.1/src/pkg/runtime/proc.c:1218 +0x9f

Previous write by goroutine 8:
  github.com/howeyc/fsnotify.func·002()
      /Users/sridharr/go/src/github.com/howeyc/fsnotify/fsnotify_symlink_test.go:46 +0x19d
  gosched0()
      /usr/local/Cellar/go/1.1/src/pkg/runtime/proc.c:1218 +0x9f

Goroutine 4 (running) created at:
  testing.RunTests()
      /usr/local/Cellar/go/1.1/src/pkg/testing/testing.go:433 +0xaef
  testing.Main()
      /usr/local/Cellar/go/1.1/src/pkg/testing/testing.go:365 +0xab
  main.main()
      github.com/howeyc/fsnotify/_test/_testmain.go:63 +0xda
  runtime.main()
      /usr/local/Cellar/go/1.1/src/pkg/runtime/proc.c:182 +0x91

Goroutine 8 (running) created at:
  github.com/howeyc/fsnotify.TestFsnotifyFakeSymlink()
      /Users/sridharr/go/src/github.com/howeyc/fsnotify/fsnotify_symlink_test.go:51 +0x4db
  testing.tRunner()
      /usr/local/Cellar/go/1.1/src/pkg/testing/testing.go:353 +0x12f
  gosched0()
      /usr/local/Cellar/go/1.1/src/pkg/runtime/proc.c:1218 +0x9f

Not sure what causes it, haven't looked into it. But when I set up multiple tailers, only the first file in the list works.

Works fine in OS X, though.

In your readme you say: Do I have to watch the Error and Event channels in a separate goroutine?
As of now, yes. Looking into making this single-thread friendly.

But the example code right above it only runs a single goroutine:

go func() {
    for {
        select {
        case ev := <-watcher.Event:
            log.Println("event:", ev)
        case err := <-watcher.Error:
            log.Println("error:", err)
        }
    }
}()

What gives?

I'm currently working on an internal event processing pipeline that will replace purgeEvents(), calling a series of functions to handle recursive watches (#56), throttling (#62), along with basic filtering and logging. I would prefer to keep the pipeline itself internal, at least initially, but there are still some API changes.

• improve testability
• surface new options
• aesthetics

fsnotify.Event interface

In order to unit test pipeline steps, I would like to fake the event being passed in, which can be accomplished with an interface:

type Event interface {
    IsCreate() bool
    IsDelete() bool
    IsModify() bool
    IsRename() bool
    Path() string
}

As an interface cannot contain a field, the pipeline expects a Path() method, rather than a Name field.

The Watcher API still provides a concrete Event chan *FileEvent that contains a Name field in addition to Path(). Even so, I think it would be good to export the Event interface and to mark the Name field as deprecated.

New Options

These options don't necessarily correspond one-to-one with the steps in the pipeline, but they do represent the configuration those steps need.

type Options struct {
    Recursive        bool
    Hidden      bool
    Pattern          string
    Throttle         bool
    ThrottleDuration time.Duration
    Triggers         Triggers
    Verbose          bool
}

In order to specify these options, a new WatchPath method is introduced. The options have reasonable defaults, mostly using zero-initialization, though ThrottleDuration and Triggers don't have zero value defaults.

err = watcher.WatchPath("./", &fsnotify.Options{
    Recursive:        true,
    WatchHidden:      false,
    Pattern:          "*.go,*.c",
    Triggers:         fsnotify.Create | fsnotify.Modify,
    Throttle:         true,
    ThrottleDuration: 1 * time.Second, // default
    Verbose:          true,
})

Watch & WatchFlags work as before, but are marked as deprecated. Of note, they continue to watch hidden files/directories (WatchHidden: true).

It may be more appropriate if some of these options were specified at a "global" level rather than per-Watch, but I would rather not change the NewWatcher() API. I suppose we could add some accessors to the Watcher instead?

A number of options don't really make sense when watching a single file. I wonder if anyone is using fsnotify for that purpose, and whether or not FSEvents (#54) supports single file watches. If so, I'm sure we can come up with a solution beyond "don't use these options in this case", perhaps a WatchFile() method?

Triggers type

When looking through the Go standard library, I've seen constants in ALL_CAPS and TitleCase. I'm not aware of an official reason to choose one over the other, but I find the following looks a little nicer on a page than FSN_CREATE, etc.

type Triggers uint32

const (
    Create Triggers = 1 << iota
    Modify
    Delete
    Rename

    allEvents Triggers = Modify | Delete | Rename | Create // default
)

Outside of introducing a type for the constants, this is mostly an aesthetic change. The internal representation is exactly the same.

Comments

While working away on a first pull request, I would appreciate any early feedback on the naming and approach described here. Thanks!

I've had to implement this several times now, and I really think that a recursive option should be part of the core API because it really is needed in many scenarios.

My usual implementation is to use filepath.Walk on the root directory and call watcher.Watch on every directory; I think it's not obvious and I also have to remember to handle directories specially on a Create event so that new directories are also watched.

I'd be happy to create a pull request for this functionality if there's a chance that you'll accept it. If you don't want to add such an option, perhaps we could provide an example that shows how to do a recursive watch?

Hi,

One chap submitted a pull request to watch the entire GOPATH worth of source:
https://github.com/robfig/revel/pull/153

That seems to work in inotify (and I believe windows as well) -- however, kqueue chokes on it due to the number of file descriptors required.

This article suggests that FSEvents would be the mechanism to use for directory-level monitoring:
http://stackoverflow.com/questions/1772209/file-level-filesystem-change-notification-in-mac-os-x

Have you thought about integrating this sort of functionality into your package? I think it would be valuable (for revel in particular!).

Thanks again!
Rob

Note that this error is not defined in syscall/ztypes_windows.go for some reason, you can declare as follows:

const (
    ERROR_MORE_DATA syscall.Errno = 234
)

When GetQueuedCompletionStatus returns ERROR_MORE_DATA and lpOverlapped is not null, the buffer is full but n will still be 0 (really it should be the buffer size but that's Windows for you).

In my case it's is sufficient to just modify n and carry on, though it carries the risk of running off the end of the buffer during event processing:

case ERROR_MORE_DATA:
    if watch == nil {
        w.Error <- errors.New("ERROR_MORE_DATA with null ov")
    } else {
        n = 4096
    }

However a proper solution would be to fetch the additional data (presumably by looping while ERROR_MORE_DATA and accumulating buffers) before we start processing events.

If the test is interrupted, subsequent runs may fail because the test directories or files already exist.

The tests should create a unique dir using os.TempDir().

Linux-64bit, GO1.0.3

Code: https://gist.github.com/nbkolchin/5143162

Command:
$ for i in seq 0 4 ; do touch $i ; done

Output:

• 2013/03/12 17:51:36 363416 385912 2686976 364 96
• 2013/03/12 17:51:36 363608 386976 2686976 387 115
• 2013/03/12 17:51:36 363784 388136 2686976 408 135
• 2013/03/12 17:51:36 363832 389056 2686976 428 154
• 2013/03/12 17:51:36 363880 389976 2686976 448 173

On each loop iteration, the amount of AllocatedAndUsed memory increases and never goes down. In "real life" application, it exceeded 1G.

"Reproducable" on MacOSX, but as I understand this is 'expected' behaviour on that platform.

the fsnotify FSN_RENAME can't distinguish IN_MOVED_TO AND IN_MOVE_FROM

Possibly the most useful (for us) use of inotify on Linux is to monitor for files arriving in a directory (IN_MOVED_TO and IN_CLOSE_WRITE). The fsnotify.go:purgeEvents() method rather forcefully looks only at the os-agnostic events, as does fsnotify_linux.go:watch().

Is there any plan or desire to extend fsnotify so people who do want to make fullest use of their pet OS can do so? I appreciate that the eventual goal is to get fsnotify merged back into the kernel, but being able to specify "custom" events with either a "not implemented" error or the event just not firing for a given OS would be very helpful to us.

Before doing any big changes (#56), I would like to have an environment setup to test the kqueue/inotify/etc. implementations. I've looked into various CI solutions, but nothing has been that promising (outside of the tools used by Go itself). Testing locally with Vagrant could work quite well during development.

We should be able to run vagrant ssh -c "go test ./..." and have it run tests against multiple operating systems in VirtualBox.

• Ubuntu 12.04 LTS (provided)
• BSD, I have yet to find an existing BSD box that has the VirtualBox tools setup properly and mounts a shared folder (checked those on http://www.vagrantbox.es and did some Googling).
• Windows, I suspect we could use these VMs provided by Microsoft, and there appear to be some "plugins" for Vagrant to support Windows, but I haven't figured them out yet.

There probably isn't a good solution for testing OS X from other operating systems, but right now it's kind've the same as BSD (until #54) and at least some of us (myself included) are using OS X for development.

If I have a dead symlink in a directory, and I ask to watch that directory, the watcher.Watch(dir) call never returns.

To reproduce setup a directory like:

mkdir foo
cd foo
ln -s x y

And then try to watch this directory.

I have tried fsnotify on both linux and windows env. it works pretty good, however it doesnt show events of directory changes for all the nested folders and files deep inside.
say I watch on ./tmp
it show changes for ./tmp/x and ./tmp/y and not for ./tmp/x/z/ directory.

windows has http://msdn.microsoft.com/en-us/library/windows/desktop/aa365261%28v=vs.85%29.aspx support

We are going to want a place to list new APIs and other changes, so we might as well make a changelog.

I'm not sure how far we want to go back in time, perhaps pretending there were monthly releases, or something like that?

MacOSX 10.8.2, go1.0.3.

n.WatchFlags(".", fsnotify.FSN_CREATE)

$ touch a
$ rm a

Under Linux:

• event: "./a": CREATE

Under MacOSX:

• event: "a": CREATE
• event: "a": DELETE

Hi, I was trying to create a new watcher after I got the event from the first watcher, but when I try to read from the new watcher's .Event chan, I get deadlock. Am I doing something wrong? The relevant code snip is below. As is, it runs. If I uncomment the "rescan" bit, I get the deadlock.

Am I doing something dumb?

func getWatcher(buildpath string) (watcher fsnotify.Watcher, err error) {
watcher, err = fsnotify.NewWatcher()
// adds a bunch of directories to the watcher
addToWatcher(watcher, buildpath, map[string]bool{})
return
}
/...*/
watcher, err := getWatcher(buildpath)
if err != nil {
log.Fatal(err)
}

for {
    we := <-watcher.Event
    var installed bool
    installed, errorOutput, _ = install(buildpath, errorOutput)
    if installed {
        log.Print(we.Name)
        runch <- true
        /* rescan */
        // watcher.Close()
        // watcher, err = getWatcher(buildpath)
        // if err != nil {
        //  log.Fatal(err)
        // }
    }
}

I get intermittent nil pointer panics when running my own tests using fsnotify from readEvents.

The issue appears to be that Close() cleans up associated os.FileInfo instances code which readEvents expects to exist code.

One solution might be to fiddle with the mutex synchronisation between the two functions, but it looks like one could also make the done channel unbuffered. This would have the desired effect -- not closing watches until the internal event queue has stopped -- but may also violate the contract of Close() to become a blocking operation.

I'm working on setting up a consistent repro case, but here's the stack trace:

throw: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
github.com/howeyc/fsnotify.(*Watcher).watchDirectoryFiles(0x422d8de0, 0x423ed420, 0x26, 0x4224d690)
        /Users/naitik/usr/go/src/github.com/howeyc/fsnotify/fsnotify_bsd.go:259 +0x284
github.com/howeyc/fsnotify.(*Watcher).addWatch(0x422d8de0, 0x423ed420, 0x26, 0x10000002b, 0x0, ...)
        /Users/naitik/usr/go/src/github.com/howeyc/fsnotify/fsnotify_bsd.go:135 +0x422
github.com/howeyc/fsnotify.(*Watcher).Watch(0x422d8de0, 0x423ed420, 0x400000026, 0x100000001, 0x424109f0, ...)
        /Users/naitik/usr/go/src/github.com/howeyc/fsnotify/fsnotify_bsd.go:152 +0x44
main.main()
        /Users/naitik/usr/go/src/github.com/nshah/go.onchange/main.go:165 +0x3c3

goroutine 2 [syscall]:
created by runtime.main
        /usr/local/Cellar/go/1.0.1/src/pkg/runtime/proc.c:221

goroutine 3 [chan send]:
github.com/howeyc/fsnotify.(*Watcher).readEvents(0x422d8de0, 0x0)
        /Users/naitik/usr/go/src/github.com/howeyc/fsnotify/fsnotify_bsd.go:236 +0x617
created by github.com/howeyc/fsnotify.NewWatcher
        /Users/naitik/usr/go/src/github.com/howeyc/fsnotify/fsnotify_bsd.go:67 +0x23d

The source for that application is here: https://github.com/nshah/go.onchange/blob/master/main.go. In my test run, I end up adding these via Watch and the last one is where it usually hangs and occasionally throws with the deadlock above:

2012/05/02 15:27:40 Watching /Users/naitik/usr/go/src/github.com/nshah/go.rell
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/expvar
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/bytes
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/errors
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/io
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/sync
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/unicode
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/encoding/json
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/encoding/base64
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/strconv
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/math
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/unsafe
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/fmt
2012/05/02 15:27:40 Watching /usr/local/Cellar/go/1.0.1/src/pkg/os

I was using exactly the same code in CentOS 5 and it worked flawlessly. Now, I moved my application to CentOS 6.4 and no events are being triggered by my watcher.

I'm on OS X

Removing with "rm -r" a watched dir which includes a watched file causes an error.
I expected rather, that delete events for each item and the containing folder would be generated.

Steps to reproduce

• Watch a directory
• Watch a file in the directory
• Rm -rf the directory

A repro case:

https://gist.github.com/3972447

i put the example code in main,but it seems does not work.here is my code
env: win7 golang1.1

package main

import (
    "log"
    "github.com/howeyc/fsnotify"
 )

func main(){
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        log.Fatal(err)
    }



go func() {
    for {
        select {
        case ev := <-watcher.Event:
            log.Println("event:", ev)
        case err := <-watcher.Error:
            log.Println("error:", err)
        }
    }
}() 

err = watcher.Watch("E:/mysite/golang_code/cmd")
if err != nil {
    log.Fatal(err)
}
}

when i run go run xx.go ,the program run and exit,nothing happended!,any idea?

MacOSX 10.8.2, GO1.0.3

Code: https://gist.github.com/nbkolchin/5137058

I.e. we get Create event and immediately delete file.

$ touch a
$ touch a
$ touch a

Linux:

• 2013/03/11 23:28:21 event: "./a": CREATE
• 2013/03/11 23:28:21 removing ./a
• 2013/03/11 23:28:25 event: "./a": CREATE
• 2013/03/11 23:28:25 removing ./a
• 2013/03/11 23:28:29 event: "./a": CREATE
• 2013/03/11 23:28:29 removing ./a

MacOSX:

• 2013/03/11 23:49:56 event: "a": CREATE
• 2013/03/11 23:49:56 removing a

I.e. we lost two events and will never get event about file "a" again.

Hello,

This issue is a little hard to report as it is quite specific. I run /tmp on a tmpfs and using the example code in this package it appears to go into an infinite loop

watching, for example, /home/dfc, then touching /home/dfc/t generates the usual

lucky(~/devel/src/watcher) % go run watcher.go
2012/04/07 19:03:36 event: "/home/dfc/t": CREATE
2012/04/07 19:03:36 event: "/home/dfc/t": MODIFY

watching /tmp as a tmpfs gives

lucky(~/devel/src/watcher) % go run watcher.go 2>&1 | head -n20
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vteAA4WBW": MODIFY
2012/04/07 19:04:39 event: "/tmp/vte763WBW": MODIFY

I'm not sure it this is something specific to my installation (ubuntu 11.10 / amd64)

On OS X 10.8.2 with go 1.0.3 watcher.Watch gives the following error:

operation not supported on socket

How can I fix that?

Is there a particular reason IN_MOVED_TO isn't supported for rename events in the inotify/linux backend? Without it, watching for files being moved into a directory is not possible..

I'm a big fan of this package, especially how it just-works even under Windows! ;)

So no probs in my local dev environment, Win7, Go 1.1 64-bit.

Now I tried to upload a simple web app to Google App Engine, which runs on Linux 64-bit but sadly only Go 1.0.3. My app won't compile because 6g fails to compile the fsnotify package:

11:04 PM Compilation starting.
11:04 PM Compilation: 35 files left.
11:04 PM Error 422: --- begin server output ---
Compile failed:
2013/06/04 14:04:26 go-app-builder: Failed building app: failed running 6g: exit status 1

github.com/howeyc/fsnotify/fsnotify.go:20: undefined: Watcher
--- end server output ---

So I'm wondering whether this is due to GAE using Go 1.0.3, or just due to the sandboxed restrictions at GAE... @howeyc have you ever used this on GAE or know any fsnotify users who successfully have?

Let's say we have two directories, A and B. Directory B is within directory A.

1. Watching directory A places a DELETE watch on directory B
2. Subsequently watching directory B (all events) does not invoke watchDirectoryFiles.
3. Therefore, changes within directory B are not reported.

The suggested fix is to call watchDirectoryFiles even if found = true. (Pull request forthcoming)

Additionally, a different but related bug:

1. Watch directory B
2. Watch directory A, which downgrades the filter on directory B to DELETE, from all events.

The suggested fix here is to check if the directory is already watched. If so, either avoid calling addWatch for it if its flag includes DELETE, or always call addWatch with DELETE|current_flags. If it is not already watched, do the current thing.

The second fix is a little more involved, and I wasn't sure how to do it immediately..

fsnotify looks well structured, but there is one thing which makes it not very useful at the moment: it's not possible to get the rename target, nor the raw notify events. I guess that this is to have a consistent interface across operating systems.

The problem with not telling me the rename target is that gedit (a text editor) actually doesn't modify the target file directly, but creates a new file and when it is complete replaces the target by renaming over it.

At the moment fsnotify outputs these events when I hit save:

event: "/path/to/dir/.goutputstream-7U75JW": CREATE
event: "/path/to/dir/.goutputstream-7U75JW": MODIFY
event: "/path/to/dir/.goutputstream-7U75JW": MODIFY
event: "/path/to/dir/.goutputstream-7U75JW": RENAME

The end result being that I don't know what file was modified by the editor.

After problems with Hot Code Reload of Revel on a fresh Fedora 19 installation, I tried the following test:

package main

import (
    "log"
    "github.com/howeyc/fsnotify"
)

func main() {
    log.Print("Creation of watcher")
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        log.Fatal(err)
    }
    log.Print("Watcher created")

    done := make(chan bool)

    // Process events
    go func() {
        for {
            select {
            case ev := <-watcher.Event:
                log.Println("event:", ev)
            case err := <-watcher.Error:
                log.Println("error:", err)
            }
        }
    }()

    log.Print("Start watching /tmp")
    err = watcher.Watch("/tmp")
    if err != nil {
        log.Fatal(err)
    }

    <-done
    log.Print("Done!")

    watcher.Close()
}

Output, with the same error found on Revel:

$ go run testwatcher.go
2013/10/10 14:31:50 Creation of watcher
2013/10/10 14:31:50 Watcher created
2013/10/10 14:31:50 Start watching /tmp
2013/10/10 14:31:50 no space left on device
exit status 1

MacOSX 10.8.2, GO1.0.3

Code: https://gist.github.com/nbkolchin/5137058

$ for i in seq 0 20000 ; do touch $i ; done

Output (MacOSX):
2013/03/12 02:08:13 event: "tmp/17928": CREATE
2013/03/12 02:08:13 removing tmp/17928
2013/03/12 02:08:13 event: "tmp/17929": CREATE
2013/03/12 02:08:13 removing tmp/17929
2013/03/12 02:08:13 error: open tmp: too many open files
2013/03/12 02:08:13 error: open tmp: too many open files
2013/03/12 02:08:13 error: open tmp: too many open files
2013/03/12 02:08:13 error: open tmp: too many open files

Works fine on Linux for ~2.000.000 files.

When attempting to recursively watch directories, it would be useful to be able to receive notifications when directories are created/removed. For instance, if I am trying to watch all files contained recursively in a directory, I would need to know when directories are added in order to begin watching them as well.

I know this is supported by some existing inotify interfaces (pyinotify at least), but am not sure how easy this would be on the Windows side.

On OS X 10.8.5 I sometimes see the following test failure:

--- FAIL: TestFsnotifySubDir (0.70 seconds)
    fsnotify_test.go:558: event received: "/var/folders/9g/167chryj3pgbrgx74j0464q00000gn/T/7358485684450452136/sub": CREATE
    fsnotify_test.go:558: event received: "/var/folders/9g/167chryj3pgbrgx74j0464q00000gn/T/7358485684450452136/sub": DELETE
    fsnotify_test.go:558: event received: "/var/folders/9g/167chryj3pgbrgx74j0464q00000gn/T/7358485684450452136/TestFsnotifyFile1.testfile": DELETE
    fsnotify_test.go:611: incorrect number of create events received after 500 ms (1 vs 2)
FAIL
FAIL    github.com/howeyc/fsnotify  5.903s

I haven't seen it on BSD yet.

This seems much more complete than exp/inotify, maybe you could propose it for inclusion in the main distribution?

Great work, thanks!

I didn't see a mention of which license you're releasing this code under in your readme or from a quick glance at the code.

Would be great if you could release it under some sort of BSD style license :-)

I'm not really sure what TextEdit is doing behind the scenes, but here is what I observe.

1. Watch a directory with a text file. (See code in my previous issue)
2. Edit the text file with TextEdit.app, and Save
3. Observe a DELETE event get fired for that file.
4. Make another edit and save.
5. Observe a CREATE event get fired.
6. Make subsequent edits / saves.
7. Observe that it fires a "MODIFY" event for the directory, but no more events on the file thereafter.

I have not traced through why this happens, but fsnotify stops firing events after the DELETE/CREATE events, which seems like a problem.

As an aside, before editing with TextEdit, I can successfully trigger MODIFY events on the file using echo 'xyz' >> test1.txt, but not afterwards.