haskell-fswatch / hfsnotify Goto Github PK

View Code? Open in Web Editor NEW

133.0 133.0 39.0 378 KB

Unified Haskell interface for basic file system notifications

License: BSD 3-Clause "New" or "Revised" License

Haskell 100.00%

hfsnotify's People

Contributors

Stargazers

Watchers

hfsnotify's Issues

Watching parent directories?

Here's something I want to implement, but I'd like to have some feedback, just in case I'm missing something or there's a better solution.

This is related to #32. Suppose I want to watch a path. As far as I understand, neither Windows nor Linux allow to watch a path. Windows only allows to watch a directory, and Linux allows to watch a file (identified by the inode) but not the path.

So, in either case we'll have to also watch the parent directory.

Now, the immediate parent directory itself may be replaced or even not exist when we start watching. So the question is, should we watch every single directory up the tree?

Here's an example of what I mean. Remember, I'm interested in whatever happens
under a certain path. Let's say it's /foo/bar/baz, and only /foo exists at
the moment. /foo/bar/baz may appear as a result of the following:

mkdir /foo/bar
touch /foo/bar/baz

To catch this, we must first watch for /foo, and after /foo/bar is created,
check and watch for /foo/bar/baz. Moreover, the following may happen later:

rm -rf /foo
mkdir -p /foo/bar
touch /foo/bar/baz

Which means that we must either watch every single directory up the tree, or
readjust the watchlist dynamically if things get deleted/moved (i.e. start
watching / when /foo is deleted, to catch creation of a new /foo).

Am I overcomplicating things here? I feel that watching a path is what the user
usually means, and we should expose a simple and intuitive API, even at the cost
of internal complexity (which is caused by os-level API being too low-level).

Build failure on Windows

I was testing the new 0.3.0.0 release on Windows, and I got the following build failure:

    Preprocessing library for fsnotify-0.3.0.0..
    Building library for fsnotify-0.3.0.0..
    [1 of 7] Compiling System.FSNotify.Path ( src\System\FSNotify\Path.hs, .stack-work\dist\7d103d30\build\System\FSNotify\Path.o )
    [2 of 7] Compiling System.FSNotify.Types ( src\System\FSNotify\Types.hs, .stack-work\dist\7d103d30\build\System\FSNotify\Types.o )
    [3 of 7] Compiling System.FSNotify.Listener ( src\System\FSNotify\Listener.hs, .stack-work\dist\7d103d30\build\System\FSNotify\Listener.o )
    [4 of 7] Compiling System.FSNotify.Polling ( src\System\FSNotify\Polling.hs, .stack-work\dist\7d103d30\build\System\FSNotify\Polling.o )

    C:\Users\Michael Snoyman\AppData\Local\Temp\stack1764\fsnotify-0.3.0.0\src\System\FSNotify\Polling.hs:64:17: warning: [-Wname-shadowing]
        This binding for `path' shadows the existing binding
          bound at src\System\FSNotify\Polling.hs:58:22
       |
    64 |     pathAndInfo path = handle (\(_ :: IOException) -> return Nothing) $ do
       |                 ^^^^
    [5 of 7] Compiling System.FSNotify.Win32 ( src\System\FSNotify\Win32.hs, .stack-work\dist\7d103d30\build\System\FSNotify\Win32.o )

    C:\Users\Michael Snoyman\AppData\Local\Temp\stack1764\fsnotify-0.3.0.0\src\System\FSNotify\Win32.hs:62:34: error:
        Not in scope: `WNo.fILE_NOTIFY_CHANGE_FILE_NAME'
        Module `System.Win32.Notify' does not export `fILE_NOTIFY_CHANGE_FILE_NAME'.
       |
    62 |   let fileFlags = foldl (.|.) 0 [WNo.fILE_NOTIFY_CHANGE_FILE_NAME, WNo.fILE_NOTIFY_CHANGE_SIZE, WNo.fILE_NOTIFY_CHANGE_ATTRIBUTES]
       |                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    C:\Users\Michael Snoyman\AppData\Local\Temp\stack1764\fsnotify-0.3.0.0\src\System\FSNotify\Win32.hs:62:68: error:
        Not in scope: `WNo.fILE_NOTIFY_CHANGE_SIZE'
        Module `System.Win32.Notify' does not export `fILE_NOTIFY_CHANGE_SIZE'.
       |
    62 |   let fileFlags = foldl (.|.) 0 [WNo.fILE_NOTIFY_CHANGE_FILE_NAME, WNo.fILE_NOTIFY_CHANGE_SIZE, WNo.fILE_NOTIFY_CHANGE_ATTRIBUTES]
       |                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^

    C:\Users\Michael Snoyman\AppData\Local\Temp\stack1764\fsnotify-0.3.0.0\src\System\FSNotify\Win32.hs:62:97: error:
        Not in scope: `WNo.fILE_NOTIFY_CHANGE_ATTRIBUTES'
        Module `System.Win32.Notify' does not export `fILE_NOTIFY_CHANGE_ATTRIBUTES'.
       |
    62 |   let fileFlags = foldl (.|.) 0 [WNo.fILE_NOTIFY_CHANGE_FILE_NAME, WNo.fILE_NOTIFY_CHANGE_SIZE, WNo.fILE_NOTIFY_CHANGE_ATTRIBUTES]
       |                                                                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    C:\Users\Michael Snoyman\AppData\Local\Temp\stack1764\fsnotify-0.3.0.0\src\System\FSNotify\Win32.hs:63:33: error:
        Not in scope: `WNo.fILE_NOTIFY_CHANGE_DIR_NAME'
        Module `System.Win32.Notify' does not export `fILE_NOTIFY_CHANGE_DIR_NAME'.
       |
    63 |   let dirFlags = foldl (.|.) 0 [WNo.fILE_NOTIFY_CHANGE_DIR_NAME]
       |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Not in scope: `forkFinally' (Debian 7.5, Haskell Platform 2012.2.0.0)

This is what I did:

$ mkdir test-fsnotify && cd $_
$ cabal sandbox init && cabal install fsnotify-0.1.0.1

And this is what I get:

Configuring fsnotify-0.1.0.1...
Building fsnotify-0.1.0.1...
Preprocessing library fsnotify-0.1.0.1...
[1 of 7] Compiling System.FSNotify.Path ( src/System/FSNotify/Path.hs, dist/build/System/FSNotify/Path.o )
[2 of 7] Compiling System.FSNotify.Types ( src/System/FSNotify/Types.hs, dist/build/System/FSNotify/Types.o )
[3 of 7] Compiling System.FSNotify.Listener ( src/System/FSNotify/Listener.hs, dist/build/System/FSNotify/Listener.o )
[4 of 7] Compiling System.FSNotify.Linux ( src/System/FSNotify/Linux.hs, dist/build/System/FSNotify/Linux.o )
[5 of 7] Compiling System.FSNotify.Polling ( src/System/FSNotify/Polling.hs, dist/build/System/FSNotify/Polling.o )
[6 of 7] Compiling System.FSNotify  ( src/System/FSNotify.hs, dist/build/System/FSNotify.o )

src/System/FSNotify.hs:215:3: Not in scope: `forkFinally'
Failed to install fsnotify-0.1.0.1
cabal: Error: some packages failed to install:
fsnotify-0.1.0.1 failed during the building phase. The exception was:
ExitFailure 1

Same error outside the Cabal sandbox as well.

Occasional test failure

I'm seeing the following test failure in some circumstances. I think it's just a race condition in the tests, but I'm not certain.

Unpacking to fsnotify-0.1.0.2/
Resolving dependencies...
Configuring fsnotify-0.1.0.2...
Building fsnotify-0.1.0.2...
Preprocessing library fsnotify-0.1.0.2...
[1 of 7] Compiling System.FSNotify.Path ( src/System/FSNotify/Path.hs, dist/build/System/FSNotify/Path.o )
[2 of 7] Compiling System.FSNotify.Types ( src/System/FSNotify/Types.hs, dist/build/System/FSNotify/Types.o )
[3 of 7] Compiling System.FSNotify.Listener ( src/System/FSNotify/Listener.hs, dist/build/System/FSNotify/Listener.o )
[4 of 7] Compiling System.FSNotify.Linux ( src/System/FSNotify/Linux.hs, dist/build/System/FSNotify/Linux.o )
[5 of 7] Compiling System.FSNotify.Polling ( src/System/FSNotify/Polling.hs, dist/build/System/FSNotify/Polling.o )
[6 of 7] Compiling System.FSNotify  ( src/System/FSNotify.hs, dist/build/System/FSNotify.o )
[7 of 7] Compiling System.FSNotify.Devel ( src/System/FSNotify/Devel.hs, dist/build/System/FSNotify/Devel.o )
[1 of 7] Compiling System.FSNotify.Path ( src/System/FSNotify/Path.hs, dist/build/System/FSNotify/Path.p_o )
[2 of 7] Compiling System.FSNotify.Types ( src/System/FSNotify/Types.hs, dist/build/System/FSNotify/Types.p_o )
[3 of 7] Compiling System.FSNotify.Listener ( src/System/FSNotify/Listener.hs, dist/build/System/FSNotify/Listener.p_o )
[4 of 7] Compiling System.FSNotify.Linux ( src/System/FSNotify/Linux.hs, dist/build/System/FSNotify/Linux.p_o )
[5 of 7] Compiling System.FSNotify.Polling ( src/System/FSNotify/Polling.hs, dist/build/System/FSNotify/Polling.p_o )
[6 of 7] Compiling System.FSNotify  ( src/System/FSNotify.hs, dist/build/System/FSNotify.p_o )
[7 of 7] Compiling System.FSNotify.Devel ( src/System/FSNotify/Devel.hs, dist/build/System/FSNotify/Devel.p_o )
In-place registering fsnotify-0.1.0.2...
Preprocessing test suite 'test' for fsnotify-0.1.0.2...
[1 of 2] Compiling EventUtils       ( test/EventUtils.hs, dist/build/test/test-tmp/EventUtils.o )

test/EventUtils.hs:70:14: Warning:
    This binding for ‘poll’ shadows the existing binding
      imported from ‘Control.Concurrent.Async’ at test/EventUtils.hs:7:1-31

test/EventUtils.hs:74:33: Warning:
    Defaulting the following constraint(s) to type ‘Integer’
      (Num b0) arising from the literal ‘5’ at test/EventUtils.hs:74:33
      (Integral b0) arising from a use of ‘^’ at test/EventUtils.hs:74:32
    In the second argument of ‘(^)’, namely ‘5’
    In the second argument of ‘(*)’, namely ‘10 ^ 5’
    In the ‘confPollInterval’ field of a record

test/EventUtils.hs:88:14: Warning:
    This binding for ‘poll’ shadows the existing binding
      imported from ‘Control.Concurrent.Async’ at test/EventUtils.hs:7:1-31

test/EventUtils.hs:97:1: Warning:
    Top-level binding with no type signature:
      expectEventsHere :: (?timeInterval::Int) =>
                          Bool -> [EventPattern] -> IO () -> Assertion

test/EventUtils.hs:97:18: Warning:
    This binding for ‘poll’ shadows the existing binding
      imported from ‘Control.Concurrent.Async’ at test/EventUtils.hs:7:1-31

test/EventUtils.hs:98:1: Warning:
    Top-level binding with no type signature:
      expectEventsHereRec :: (?timeInterval::Int) =>
                             Bool -> [EventPattern] -> IO () -> Assertion

test/EventUtils.hs:98:21: Warning:
    This binding for ‘poll’ shadows the existing binding
      imported from ‘Control.Concurrent.Async’ at test/EventUtils.hs:7:1-31
[2 of 2] Compiling Main             ( test/test.hs, dist/build/test/test-tmp/Main.o )

test/test.hs:11:1: Warning:
    The import of ‘Text.Printf’ is redundant
      except perhaps to import instances from ‘Text.Printf’
    To import instances alone, use: import Text.Printf()

test/test.hs:24:1: Warning:
    Top-level binding with no type signature: main :: IO ()

test/test.hs:34:1: Warning:
    Top-level binding with no type signature: tests :: Bool -> TestTree

test/test.hs:41:21: Warning:
    Defaulting the following constraint(s) to type ‘Integer’
      (Num b0) arising from the literal ‘6’ at test/test.hs:41:21
      (Integral b0) arising from a use of ‘^’ at test/test.hs:41:20
    In the second argument of ‘(^)’, namely ‘6’
    In the second argument of ‘(*)’, namely ‘10 ^ 6’
    In the expression: 2 * 10 ^ 6

test/test.hs:42:21: Warning:
    Defaulting the following constraint(s) to type ‘Integer’
      (Num b0) arising from the literal ‘5’ at test/test.hs:42:21
      (Integral b0) arising from a use of ‘^’ at test/test.hs:42:20
    In the second argument of ‘(^)’, namely ‘5’
    In the second argument of ‘(*)’, namely ‘10 ^ 5’
    In the expression: 5 * 10 ^ 5

test/test.hs:54:44: Warning:
    Defaulting the following constraint(s) to type ‘Integer’
      (Num b0) arising from the literal ‘6’ at test/test.hs:54:44
      (Integral b0) arising from a use of ‘^’ at test/test.hs:54:43
    In the second argument of ‘(^)’, namely ‘6’
    In the second argument of ‘($)’, namely ‘10 ^ 6’
    In the second argument of ‘when’, namely ‘(threadDelay $ 10 ^ 6)’

test/test.hs:75:5: Warning:
    This binding for ‘filename’ shadows the existing binding
      imported from ‘Filesystem.Path.CurrentOS’ at test/test.hs:8:1-32
      (and originally defined in ‘Filesystem.Path’)
Linking dist/build/test/test ...
Building fsnotify-0.1.0.2...
Preprocessing library fsnotify-0.1.0.2...
In-place registering fsnotify-0.1.0.2...
Preprocessing test suite 'test' for fsnotify-0.1.0.2...
Running 1 test suites...
Test suite test: RUNNING...
Tests
  Native
    Non-recursive
      Right here
        new file:                FAIL
          Unexpected event.
            Expected :[Added FilePath "/home/ubuntu/haskell/stackage/runtests/fsnotify-0.1.0.2/testdir/test.6435/testfile",Modified FilePath "/home/ubuntu/haskell/stackage/runtests/fsnotify-0.1.0.2/testdir/test.6435/testfile"]
            Actual: [Modified (FilePath "/home/ubuntu/haskell/stackage/runtests/fsnotify-0.1.0.2/testdir/test.6435/testfile") 2014-08-07 17:45:59.012689 UTC,Added (FilePath "/home/ubuntu/haskell/stackage/runtests/fsnotify-0.1.0.2/testdir/test.6435/testfile") 2014-08-07 17:45:59.012681 UTC]
        modify file:             OK
        delete file:             OK
        directories are ignored: OK
      In a subdirectory
        new file:                OK
        modify file:             OK
        delete file:             OK
        directories are ignored: OK
    Recursive
      Right here
        new file:                FAIL
          Unexpected event.
            Expected :[Added FilePath "/home/ubuntu/haskell/stackage/runtests/fsnotify-0.1.0.2/testdir/test.6435/testfile",Modified FilePath "/home/ubuntu/haskell/stackage/runtests/fsnotify-0.1.0.2/testdir/test.6435/testfile"]
            Actual: [Modified (FilePath "/home/ubuntu/haskell/stackage/runtests/fsnotify-0.1.0.2/testdir/test.6435/testfile") 2014-08-07 17:46:03.164445 UTC,Added (FilePath "/home/ubuntu/haskell/stackage/runtests/fsnotify-0.1.0.2/testdir/test.6435/testfile") 2014-08-07 17:46:03.164444 UTC]
        modify file:             OK
        delete file:             OK
        directories are ignored: OK
      In a subdirectory
        new file:                OK
        modify file:             OK
        delete file:             OK
        directories are ignored: OK
  Polling
    Non-recursive
      Right here
        new file:                OK
        modify file:             OK
        delete file:             OK
        directories are ignored: OK
      In a subdirectory
        new file:                OK
        modify file:             OK
        delete file:             OK
        directories are ignored: OK
    Recursive
      Right here
        new file:                OK
        modify file:             OK
        delete file:             OK
        directories are ignored: OK
      In a subdirectory
        new file:                OK
        modify file:             OK
        delete file:             OK
        directories are ignored: OK

2 out of 32 tests failed
Test suite test: FAIL
Test suite logged to: dist/test/fsnotify-0.1.0.2-test.log
0 of 1 test suites (0 of 1 test cases) passed.

Reuseable WatchManager

It would've been quite convenient if WatchManager could be reused after closeManager in the same way as http-conduit's Manager.

Fails to report events on Windows without polling

On Windows 8, I've used the minimal test example which prints events. Without using confUsePolling=True, it never reports any events, though it does with polling.

Any tips on debugging this would be appreciated.

Option to ignore filesystem changes caused by current process

hfsevents supports it. I don't know whether the APIs on other platforms do, so I'd include a caveat that the option is expected to be a no-op on non-Darwin OSes.

Make a command line application for this

Current implementations of fswatch don't quite cut it for me.

I'm glad to have found a Haskell project that does that job watch directories but to showcase its functionality (and to serve as example code...?) there should really be a command line tool built with it.

fsnotify-0.2 does not compile on Mac

Configuring fsnotify-0.2...
Building fsnotify-0.2...
Preprocessing library fsnotify-0.2...

src/System/FSNotify/OSX.hs:22:8:
    Could not find module ‘Filesystem’
    Use -v to see a list of the files searched for.

src/System/FSNotify/OSX.hs:23:8:
    Could not find module ‘Filesystem.Path’
    Use -v to see a list of the files searched for.

DISCUSSION: Breaking changes

I'm working on a Haskell project that requires cross-platform filesystem notifications, and unfortunately, hfsnotify doesn't currently provide all the features that I need.

I'm going to begin work on a fork that will include the following breaking changes:

Add Moved FilePath FilePath UTCTime constructor to Event
Rename eventPath to eventOrigin
Add eventDestination
Update fsnotify to report on changes made to directories
Update reporting logic to account for the tracking of changes to directories

If these are changes you'd like included in hfsnotify, I'm open to discussing changes to their semantics, however I require both the "move" and "directory" changes for what I'm working on, so they come as a set. If you want just one of them, you'll have to backport it from my fork.

I'll make a pull request when I believe it's stable on Linux, OS X, and Windows. This ticket is just a heads-up so that there is more time to discuss the implications/semantics of these changes.

master doesn't compile on Linux

I was able to fix all the syntax/scoping errors, but a bunch of type errors followed.

It would be nice to track work in progress in a separate branch to keep master compiling, and also to have releases tagged.

Cheers!

only added events show up on OSX

Tried to start using this on OS X. I didn't seem to get any events for the action version. For the Chan version I only got added events. First step will be to make sure the unit tests are working properly for all event types on OS X.
Here is the code I am using. I think we should include the guardExt function in the package.

{-# LANGUAGE OverloadedStrings #-}

import Prelude hiding (FilePath)
import Data.Text
import System.IO.FSNotify
import Filesystem.Path.CurrentOS
import Filesystem
import System.Cmd (rawSystem)
import Control.Concurrent

main :: IO ()
main = do
  wd <- getWorkingDirectory
  withManager $ \man -> coffee man wd
  _<-getLine
  void

coffee :: WatchManager -> FilePath -> IO ()
coffee man dir =
  guardExt man dir "coffee" "js"
    (\fp -> rawSystem "coffee" ["-c", encodeString fp] >> void)

void :: IO ()
void = return ()

-- | assumes you are running a compile function that produces a file in the same directory but with a different extension
guardExt :: WatchManager
         -> FilePath -- ^ Directory to watch
         -> Text -- ^ old extension
         -> Text -- ^ new extension
         -> (FilePath -> IO ()) -- ^ compile action to run on file
         -> IO ()
guardExt man dir oldExt newExt action =
  watchTreeAction man dir pred compile
  where
    actionWrapper f = do
      print f
      print $ convert f
      action . convert $ f
    extFilter = flip hasExtension oldExt
    convert = flip replaceExtension newExt
    compile event =
      case event of
        Added    f -> actionWrapper f
        Modified f -> actionWrapper f
        Removed  f -> void

    pred event =
      case event of
        Added    f -> extFilter f
        Modified f -> extFilter f
        Removed  f -> extFilter f

-- | for debugging
guardExtChan :: WatchManager
         -> FilePath -- ^ Directory to watch
         -> Text -- ^ old extension
         -> Text -- ^ new extension
         -> IO ()
guardExtChan man dir oldExt newExt = do
  chan <- newChan
  watchTreeChan man dir pred chan
  e <- readChan chan
  print e
  where
    extFilter = flip hasExtension oldExt

    pred event =
      case event of
        Added    f -> extFilter f
        Modified f -> extFilter f
        Removed  f -> extFilter f

`watchTree*` and creating new subdirectories

New subdirectories aren't immediately watched:

#!/usr/bin/env stack
-- stack --resolver lts-7.4 --install-ghc runghc --package fsnotify --package directory --package temporary

import Control.Concurrent
import System.FSNotify
import System.FilePath
import System.Directory
import System.IO.Temp

main = 
  withSystemTempDirectory "fsnotify" $ \baseDir -> 
  withManager $ \mgr -> do
    _ <- watchTree mgr baseDir (const True) print
    writeFile (baseDir </> "file.txt") "whuzza"
    let childDir = baseDir </> "child"
    createDirectoryIfMissing True childDir
    -- threadDelay (10*1000)
    writeFile (childDir </> "child.txt") "whoop"
    threadDelay (1000*1000)

This prints only one line, instead of the expected two when that line is uncommented.

Also, this works as expected on OS X, but not on my Linux VM.

Expose eventPath and eventTime

I ended up implementing these myself since the type of event is basically useless on OS X, but I noticed that you had them already in the source. It'd be convenient if these were exposed.

When removing a file, Added event fired

Hey guys, I'm playing with the library's default example and every time I remove a file, I got back an Added event, and sometimes, the expected Removed event. To undestand the problem better, I configured the manager with NoDebounce, and when removing a file, I get something like this:

Removed "/Users/santios/Documents/xxxx/hola.hs" 2017-07-04 21:23:16.182179 UTC
Added "/Users/santios/Documents/xxxx/hola.hs" 2017-07-04 21:23:16.182179 UTC

I'm not sure why the library is giving me back an Added event when removing a file. Any help is appreciated.

This is the code for reference and I'm on a Mac. OSX(10.12.3)

main :: IO ()
main = withManagerConf pollingConf $ \mgr -> do
  watchTree
    mgr
    "."
    (const True)
    print
  forever $ threadDelay 1000000

pollingConf :: WatchConfig
pollingConf = WatchConfig
      { confDebounce = NoDebounce
      , confPollInterval = 10^(6 :: Int) -- 1 second
      , confUsePolling = False
      }

Thanks!

Add kqueue backend to avoid event types weirdness from hfsevents on OS X

OS X's FSEvents API has some odd behavior which I still don't entirely understand. It often provides several flags for each event. For example (output produced using trace from hfsevents), if I do this:

$ touch myfile
$ mv myfile myfilex
$ mv myfilex myfile
$ rm myfile

I get this log output:

id:    18147469115959082481
path:  /Users/mfowler/Code/test/hfsevents/test/myfile
flags: ItemCreated ItemIsFile

id:    18147469115959083733
path:  /Users/mfowler/Code/test/hfsevents/test/myfile
flags: ItemCreated ItemRenamed ItemIsFile

id:    18147469115959083734
path:  /Users/mfowler/Code/test/hfsevents/test/myfilex
flags: ItemRenamed ItemIsFile

id:    18147469115959088855
path:  /Users/mfowler/Code/test/hfsevents/test/myfilex
flags: ItemRenamed ItemIsFile

id:    18147469115959088856
path:  /Users/mfowler/Code/test/hfsevents/test/myfile
flags: ItemRenamed ItemIsFile

id:    18147469115959090377
path:  /Users/mfowler/Code/test/hfsevents/test/myfile
flags: ItemRemoved ItemIsFile

Take a look at the flags for the second entry.

Consider also this command:

$ touch x && echo test > x && rm x

Which yields this log:

id:    18147469115959120388
path:  /Users/mfowler/Code/test/hfsevents/test/x
flags: ItemCreated ItemInodeMetaMod ItemModified ItemIsFile

id:    18147469115959120391
path:  /Users/mfowler/Code/test/hfsevents/test/x
flags: ItemCreated ItemRemoved ItemInodeMetaMod ItemModified ItemIsFile

Notice the fact that each event contains flags from other recent events. This explains the anomalies I've seen regarding the types of events generated by System.FSEvents on OS X. (I ended up just ignoring the event type totally for now.) There definitely needs to be a change to how System.FSEvents decides what type of event to generate. It seems impossible to do this correctly in a stateless fashion, which is unfortunate. (Though one option is just to generate duplicate events and give the user of the library the responsibility to deduplicate them if they need to.)

removeWatch function

Resources can be released with stopManager, but a finer grained removeWatch would be useful.

Does not "work" properly on windows

Hi,

I've tried the library on Windows 8, and it does "work". More precisely only a few events are really triggered.

If I watch a directory and I add a new file, nothing happens. If I add 10 files, 3
(sometimes 4-5) events are triggered. Same for deletion.

So it does work, but it definitely does not work correctly on windows.

code:

module Main (
main
) where

import System.FSNotify
import Filesystem.Path.CurrentOS
import Control.Concurrent

fileWatcher man = do
watchDir man (decodeString "D:/watching/") (\f -> True) (\e -> do
print "something")

main = do
man <- startManager
forkIO $ fileWatcher man
getLine >>= print
stopManager man

Simpify System.FSNotify.readEvents

readEvents does some locking. The purpose of that seems to be to disallow multiple callbacks to overlap.

Is that necessary? It seems to me like a user's responsibility.

Please support the latest version of hinotify

fsnotify currently supports only hinotify version 0.3.2, but the latest version of that package is 0.3.5. Could you please update fsnotify to support that version, too?

Not triggered for `git checkout`

This issue probably belongs to this repo:

→ schell/steeloverseer#28 ←

We have an automated /bin/sh test case there, which you can run.

What could be the reason for this behavior?

Thanks a lot!

Update CHANGELOG?

Hi there! It's nice to see all the work happening on hfsnotify. With the recent 0.3 release it would be nice to know what has changed since version 0.2.x, to give me a better idea of what I might need to do to upgrade to the newest version. (See also commercialhaskell/stackage#3678 .) Thanks!

Re-export `ActionPredicate`, `Action`, and `EventChannel` types from `System.FSNotify`

Looking at the Haddock docs for System.FSNotify I have no idea how I am supposed to call a function like watchDir, since the documentation looks like this:

watchDir :: WatchManager -> FilePath -> ActionPredicate -> Action -> IO ()

but the definitions of ActionPredicate and Action are nowhere to be seen, so I do not know how to construct arguments of those types. Looking at the source code, I can see that in fact ActionPredicate and Action (and EventChannel) are in fact type synonyms defined in System.FSNotify.Types, which is not exported by the package. It would be nice to re-export these type synonyms from System.FSNotify module so that they show up in the Haddock documentation.

fsnotify-0.2.1 test suite failure

Citing from http://hydra.cryp.to/build/1109590/log/raw:

Running 1 test suites...
Test suite test: RUNNING...
Tests
  Native
    Non-recursive
      Right here
        new file:                OK (0.56s)
        modify file:             FAIL (0.52s)
          Unexpected number of events.
            Expected: [Modified "/tmp/nix-build-haskell-fsnotify-0.2.1.drv-0/fsnotify-0.2.1/testdir/test.2380/testfile"]
            Actual: []
        delete file:             OK (0.50s)
        directories are ignored: OK (0.50s)
      In a subdirectory
        new file:                OK (0.50s)
        modify file:             OK (0.50s)
        delete file:             OK (0.50s)
        directories are ignored: OK (0.50s)
    Recursive
      Right here
        new file:                OK (0.54s)
        modify file:             OK (0.50s)
        delete file:             OK (0.50s)
        directories are ignored: OK (0.50s)
      In a subdirectory
        new file:                OK (0.50s)
        modify file:             OK (0.50s)
        delete file:             OK (0.50s)
        directories are ignored: OK (0.51s)
  Polling
    Non-recursive
      Right here
        new file:                OK (2.01s)
        modify file:             OK (2.00s)
        delete file:             OK (2.00s)
        directories are ignored: OK (2.00s)
      In a subdirectory
        new file:                OK (2.00s)
        modify file:             OK (2.00s)
        delete file:             OK (2.00s)
        directories are ignored: OK (2.00s)
    Recursive
      Right here
        new file:                OK (2.00s)
        modify file:             OK (2.00s)
        delete file:             OK (2.00s)
        directories are ignored: OK (2.00s)
      In a subdirectory
        new file:                OK (2.01s)
        modify file:             OK (2.01s)
        delete file:             OK (2.00s)
        directories are ignored: OK (2.00s)

1 out of 32 tests failed (40.29s)
Test suite test: FAIL

CloseHandle: invalid argument on Windows

When working on the fsnotify-conduit test suite, I ran into seemingly unrelated errors on Windows. I was able to reduce it to the following test case:

#!/usr/bin/env stack
-- stack --resolver lts-11.8 script
import System.FSNotify
import System.Directory
import UnliftIO (tryIO, withSystemTempDirectory)

main :: IO ()
main =
  withSystemTempDirectory "fsnotify-test" $ \dir ->
  withManager $ \man -> do
    stop <- watchDir man dir (const True) print
    stop

This results in the output:

 CloseHandle: invalid argument (The handle is invalid.)

Events are executed concurrently

Hi, the watchDir/watchTree docs say:

No two events pertaining to the same FilePath will be executed concurrently.

However, I've found that not to be the case:

import Control.Concurrent
import System.FSNotify

main = withManager $ \wm -> do
    watchDir wm "." (const True) $ \ev -> do
        putStrLn ("Handling " ++ eventPath ev)
        threadDelay 2000000
        putStrLn "Done"
    _ <- getLine
    pure ()

Firing up a terminal and running touch foo; touch foo; within two seconds, I see:

Handling /Users/mrosen/junk/foo
Handling /Users/mrosen/junk/foo
Done
Done

but I'd expect to see

Handling /Users/mrosen/junk/foo
Done
Handling /Users/mrosen/junk/foo
Done

Am I misunderstanding the docs here? Thanks.

Using Debounce can result in missing critical events

Description

Under certain conditions, such as rapidly deleting and writing a file, hfsnotify may fail to report on critical events, potentially leaving the program running hfsnotify in a broken state.

Steps to Reproduce

Run the following Bash script:
```
mkdir -p /tmp/example
```

Run the following Haskell program:

{-# LANGUAGE OverloadedStrings #-}

import System.FSNotify
import Control.Concurrent
import Control.Monad

main :: IO ()
main = withManagerConf defaultConfig { confDebounce = Debounce 1 } $ \mgr -> do
   _ <- watchDir mgr "/tmp/example" (const True) print
   forever (threadDelay maxBound)

While the Haskell program is running, run the following Bash script:

cd /tmp/example

touch foo
sleep 1

rm foo
touch foo
sleep 1

rm foo

Expected Output

Added (FilePath "/tmp/example/foo") <TIMESTAMP>
Removed (FilePath "/tmp/example/foo") <TIMESTAMP>
Added (FilePath "/tmp/example/foo") <TIMESTAMP>
Removed (FilePath "/tmp/example/foo") <TIMESTAMP>

Actual Output

Added (FilePath "/tmp/example/foo") <TIMESTAMP>
Removed (FilePath "/tmp/example/foo") <TIMESTAMP>
Removed (FilePath "/tmp/example/foo") <TIMESTAMP>

Discussion/Proposal

I was a little hesitant to call this a "bug" since hfsnotify is technically behaving as advertised.

I decided to label it as such since I believe that most users of the library expect it to accurately keep track of the current state of the filesystem, and in that regard, this would be considered a failure to behave as expected.

My proposal to correct this behavior is to accumulate change events over the specified debounce time interval and apply the following logic to them:

Note that "Impossible" errors cannot be reached, and the "File already exists" and "File does not exist" errors can only be reached if the notification manager is returning bad results (ex: two Added events in a row).

The following is an example implementation of the proposed logic:

-- It is assumed that all events passed into this function have the same eventPath
combineEvents :: Bool -> [Event] -> Maybe Event
combineEvents fileExists = snd . foldl logic (fileExists, Nothing)
  where
    -- Combining logic
    logic (True  , Nothing            ) (Modified fp t) = (True  , Just (Modified fp t))
    logic (True  , Nothing            ) (Removed  fp t) = (False , Just (Removed  fp t))
    logic (False , Nothing            ) (Added    fp t) = (True  , Just (Added    fp t))
    logic (True  , Just (Added    _ _)) (Modified fp t) = (True  , Just (Added    fp t))
    logic (True  , Just (Added    _ _)) (Removed   _ _) = (False , Nothing             )
    logic (True  , Just (Modified _ _)) (Modified fp t) = (True  , Just (Modified fp t))
    logic (True  , Just (Modified _ _)) (Removed  fp t) = (False , Just (Removed  fp t))
    logic (False , Just (Removed  _ _)) (Added    fp t) = (True  , Just (Modified fp t))

    -- If you see these, it means your notification manager is broken
    logic (True  , _                  ) (Added     _ _) = error "File already exists"
    logic (False , _                  ) (Modified  _ _) = error "File does not exist"
    logic (False , _                  ) (Removed   _ _) = error "File does not exist"

    -- If you see these, it means this function's logic is broken
    logic (True  , Just (Removed  _ _)) _               = error "Impossible"
    logic (False , Just (Added    _ _)) _               = error "Impossible"
    logic (False , Just (Modified _ _)) _               = error "Impossible"

While these changes don't have a significant impact on small debounce time intervals, this would allow time intervals that are seconds, minutes, or even hours long to be specified while still reporting the correct state of the filesystem.

The only downside to this approach is that you have to wait the specified debounce time interval before first receiving a change notification, however, I think this change would be considered acceptable in almost all use cases.

uniqueid dependency is deprecated

According to http://hackage.haskell.org/package/uniqueid, @sebfisch has deprecated the uniqueid package in favor of of value-supply. Would it be possible to update fsnotify to that new package instead of uniqueid?

I'm packaging fsnotify for NixOS, and I'm having trouble because uniqueid won't compile with GHC 7.8.2 any more (http://hydra.cryp.to/build/84337/nixlog/1/raw).

Watching files

The library is marketed as "Cross platform library for file creation, modification, and deletion notification", yet there is no function to watch a single file.

touch command does not create events

testing on OS X, touching a file is not triggering events.

Potential Cleanup for Duplication - Native and Polling

I find the repeated 'either nativeImpl pollingImpl' code in System.FSNotify a little irritating. Fortunately, it would be easy to eliminate, e.g. by shifting away from a typeful model to something closer to:

data Session wd = Session 
    { killSession :: IO ()
    , killListener :: wd -> IO ()
    , listen :: WatchConfig -> FilePath -> ActionPredicate -> EventChannel -> IO wd
    , listenRecursive :: WatchConfig -> FilePath -> ActionPredicate -> EventChannel -> IO wd
    }

Then using Rank2Types to hide wd.

type Listener = forall wd. Session wd
initNative :: IO (Maybe Listener)

Potentially, we can also eliminate listenRecursive since I'm working on a new Event type that carries isDir information. We'd need to make the Session type independent of channels. But we can simplify it to:

data Session wd = Session 
    { killSession :: IO ()
    , killListener :: wd -> IO ()
    , listen :: FilePath -> (Event -> IO ()) -> IO wd
    }

And then introduce channels and Debounce externally.

None of this would be observable to a client of FSNotify.

perfomance in large projects

on OS X I have a project with 16274 files in 1993 directories. With a predicate filter, performance is great, but with const True as the filter and print as the action, performance is very slow.

My predicate is a simple extension filter.

Build failures with GHC 7.0.4

Hi, is there any chance that the following dependencies can be relaxed?

containers >=0.4.2.1, directory >=1.1.0.2, time >=1.4

These restrictions prevent fsnotify from being built with GHC 7.0.4, because these are base libraries shipped with the compiler, and in turn this prevents every other library which depends on fsnotify from being built with that compiler. It would be great if those versions of the dependencies could still be supported.

stopManager not stopping listeners

I am trying to use stopManager, but it does not seem to have any effect. We should have a test case for this in the project. I will narrow down my use to something reproducible when I get a chance.

no notifications in a forked thread

Thanks for fsnotify. The example works for me, but when I move it into a thread (forkIO) or bound thread (forkOS), I see no events. The executable is built with -threaded and GHC 8 on OSX.

Memory leak on OS X

There seems to be a memory leak when using this library on OS X. I am not sure if it comes from code in System.FSNotify or from the underlying System.OSX.FSEvents, though.

Interest in switching to filepath?

It seems like filepath has been shipping with GHC for a long time and that system-filepath has been deprecated in favor for it. It looks like it goes back to using a String alias, so I wanted to check with the project and see if there was any interest in a PR that would switch to the filepath package.

Using NativeManager on Linux, moving a watched directory results in incorrect behavior

Description

After moving a directory that hfsnotify is watching, hfsnotify reports changes inside that directory as if it had not been moved.

Only the NativeManager on Linux is affected.

I have not tested the NativeManager on OS X or Windows.

Steps to Reproduce

Run the following Bash script:
```
mkdir -p /tmp/example
```

Run the following Haskell program:

{-# LANGUAGE OverloadedStrings #-}

import System.FSNotify
import Control.Concurrent
import Control.Monad

main :: IO ()
main = withManager $ \mgr -> do
   _ <- watchTree mgr "/tmp/example" (const True) print
   forever (threadDelay maxBound)

While the Haskell program is running, run the following Bash script:

cd /tmp/example

mkdir foo
touch foo/test

mv foo bar

touch bar/test
rm -rf bar

Expected Output

Added (FilePath "/tmp/example/foo/test") <TIMESTAMP>
Removed (FilePath "/tmp/example/foo/test") <TIMESTAMP>
Added (FilePath "/tmp/example/bar/test") <TIMESTAMP>
Modified (FilePath "/tmp/example/bar/test") <TIMESTAMP>
Removed (FilePath "/tmp/example/bar/test") <TIMESTAMP>

Actual Output

The following is printed from the Haskell program:

Added (FilePath "/tmp/example/foo/test") <TIMESTAMP>
Modified (FilePath "/tmp/example/foo/test") <TIMESTAMP>
Removed (FilePath "/tmp/example/foo/test") <TIMESTAMP>

Check rtsSupportsBoundThreads to ensure threaded runtime present on Windows

I tried the API, but it breaks the stdout stream in some weird way. Using the example at the top of the docs, plus a call to set the stdout to NoBuffering, when run from runhaskell I get each event printed (with some output interleaved, but that's expected without buffering or locking). When compiled and run I see M after about 5 modifications, which likely is the start of Modified. In previous larger examples I've had it print out 1 character on each change, even though each change generates a line of output.

Doesn't work on macOS unless -N is specified

When using forever $ threadDelay 1000000, I needed both -threaded and -with-rtsopts=-N to get it to work. When using forever $ getLine, I only needed -threaded. Perhaps this should be mentioned in the docs.

    forkIO $ FSNotify.withManager $ \manager -> do
        directory <- takeDirectory <$> makeAbsolute (settingsRoot settings)
        let predicate = const True
        FSNotify.watchDir manager directory predicate print
        forever $ threadDelay 1000000

Question about implementing folder events

I saw in the docs folder events aren't supported, but it seems like the underlying Linux, macOS, and Windows file watching libs do so is it just a matter of adding it?

Asking before I embark on this just in case you folks know something about this that I don't :-)

Doesn't work on OS X

I tried and failed to get the minimal example from the documentation working on OS X 10.9.5. Here are the exact steps to reproduce:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.9.5
BuildVersion:   13F34
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.8.3
$ cabal --version
cabal-install version 1.20.0.3
using version 1.20.0.2 of the Cabal library 

$ mkdir /tmp/fsnotify
$ cd /tmp/fsnotify
$ cabal sandbox init
$ cabal install fsnotify-0.1.0.3
$ echo '-- Minimal example from http://hackage.haskell.org/package/fsnotify-0.1.0.3/docs/System-FSNotify.html#description
{-# LANGUAGE OverloadedStrings #-} -- for FilePath literals

import System.FSNotify
import Control.Concurrent (threadDelay)
import Control.Monad (forever)

main =
  withManager $ \mgr -> do
    -- start a watching job (in the background)
    watchDir
      mgr          -- manager
      "."          -- directory to watch
      (const True) -- predicate
      print        -- action

    -- sleep forever (until interrupted)
    forever $ threadDelay maxBound
' > Main.hs

$ cabal exec runhaskell Main.hs
fsnotify.hs: c_poll: invalid argument (Invalid argument)
fsnotify.hs: ioManagerWakeup: write: Bad file descriptor
fsnotify.hs: ioManagerWakeup: write: Bad file descriptor

No modification notifications for symlinks

this has been copied over from #66 (comment) by @Prillan

Setup

/tmp$ tree fsnotify-test
fsnotify-test
├── files
│   └── test
└── links
    └── test -> ../files/test

Start the watcher in /tmp/fsnotify-test/links: watchTree mgr "/tmp/fsnotify-test/links/" (const True) print

The following commands illustrate this point

$ cd fsnotify-test/
$ touch files/test
<no output from watcher>
$ echo "ASDF" >> files/test
<no output from watcher>
$ echo "ASDF" >> links/test
<no output from watcher>
$ ln -s files/test links/test2
Added "/tmp/fsnotify-test/links/test2" 2016-08-09 21:09:36.370309 UTC
$ rm files/test
<no output from watcher>
$ rm links/test
Removed "/tmp/fsnotify-test/links/test" 2016-08-09 21:10:05.512831 UTC

Note that the only actions output were the ones affecting the links themselves.

build failure, directory bounds need updating

fsnotify-0.3.0.1 (eg) still depends on directory (>=1.1.0.0), but it uses doesPathExist which was introduced in directory-1.2.7.0.

Terminating steeloverseer+tmux triggers "Error removing watch:"

See schell/steeloverseer#34 for original issue. I've found that hfsnotify swallows errors from hinotify at System.FSNotify.Linux.hs#L113 which originally to require me to kill my tmux server as a workaround and then switch to entr. The bug can be seen here: https://asciinema.org/a/Em8NVKMw5gdxutQWU2NiiLApZ

I'm now finding that this outside of tmux has the same output -- even when I kill the tmux server. Somehow hfsnotify fails to catch the removal of watches. Currently building on NixOS with kernel 4.19.5. ~~Originally, schell/steeloverseer#34 was present on Ubuntu.~~ Also present in Ubuntu 18.04, kernel 4.15.0.

directory events & isDirectory

I need to see the creation of directories, e.g. for discovery purposes, to display in a file browser. Is there a reason directory events are blocked? (Hmm. From the code, it seems that directory events are blocked for Win32 and Linux but not for OSX. I suspect the latter wasn't intentional.)

Also, it seems Linux, OSX, and Win32 all provide an 'isDirectory' capability on the incoming events, and it would be trivial to obtain that information for polling. I would like to add this information to the common FSNotify event flags, i.e. such that developers can make it part of their ActionPredicate.

Any counter-positions? If not, I'll develop this capability and submit another pull request.

@gregwebs, @mdittmer

Broken symlinks break directory watching

It seems stat is called on every file in a directory tree prior to much else. If there's a dead symlink somewhere, it'll cause watchTree & friends to throw an exception:

$ ln -s /tmp/foo /tmp/bar

// in ghci
> mgr <- startManager
> watchTree mgr "/tmp" (const True) print
*** Exception: /tmp/bar: getFileStatus: does not exist (No such file or directory)

~~Somehow this didn't make it into the changelog for that package.~~ because it's not yet released :)

haskell-fswatch / hfsnotify Goto Github PK

hfsnotify's People

Contributors

Stargazers

Watchers

Forkers

hfsnotify's Issues

Description

Steps to Reproduce

Expected Output

Actual Output

Discussion/Proposal

Description

Steps to Reproduce

Expected Output

Actual Output

Recommend Projects

Recommend Topics

Recommend Org