read_events does:

    # get event queue size
    if fcntl.ioctl(self._fd, termios.FIONREAD, buf_, 1) == -1:
        return

I'm not sure what this is trying to do, but you probably meant either:

    if fcntl.ioctl(self._fd, termios.FIONREAD, buf_, 1) == 0:
        return

or to just use a real nonblocking inotify fd.

The WatchManager.add_watch(...) method has a boolean argument "do_glob" — what exactly does this do? Is it for excluding certain paths? How is the glob set?

Hello Sebastien,
thank you for pyinotify, I find it very useful.

I believe many users could benefit from a simplified interface to configure pyinotify, e.g.

from pyinotify import monitor, threaded_monitor

# blocking 
monitor(('*.py', 'tests/*.py', 'docs/*.rst'), run=myfunction)

# threaded
threaded_monitor(('*.py', 'tests/*.py', 'docs/*.rst'), run=myfunction)

Also, various applications - especially editors like vim and emacs - do some clever write/rename/delete operations to save files safely that triggers the annoying error "The pathname ... has probably changed and couldn't be updated, so it cannot be trusted anymore."

In some cases the user might not be interested in filtering for some specific events and simply wants to run some command when some in a directory have been modified.
Maybe the only solution would be to monitor a directory rather than each file?

Thanks!

When I run the ThreadedNotifier example, I get warnings:

eric@ubuntu:~/seb-m-pyinotify-2c7e8f8/python2/examples$ python tutorial_threadednotifier.py
[2011-07-23 11:16:18,423 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=9 >
[2011-07-23 11:16:18,424 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=4 >
[2011-07-23 11:16:18,424 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=5 >
[2011-07-23 11:16:18,424 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=1 >

Not sure if this is a real problem or not.

I monitor a creation of files over nfs , with
wdd = wm.add_watch('/mounts/', mask, auto_add=True, rec=True, exclude_filter=excl)

files are created in directories with year/month/day of the file , and one time per month fails , failed in 1 of july with first file and now in first of august

at do you advice to avoid fails use :
notifier = pyinotify.Notifier(wm, handler)
or
notifier = pyinotify.ThreadedNotifier(wm, handler)
or
notifier = pyinotify.AsyncNotifier(wm, handler) ?

Thanks,

Hey,
my script watch 5 directorys if new file is create.
Everythink works great for a few hours, but then inotify freez.
i have no idea what is the problem, other parts of the script works.

wm = pyinotify.WatchManager() # Watch Manager
mask = pyinotify.IN_CREATE # watched events

class EventHandler(pyinotify.ProcessEvent):
def process_IN_CREATE(self, event):
print "Find File:", event.pathname

handler = EventHandler()
notifier = pyinotify.Notifier(wm, handler)

hanse_inv = wm.add_watch('/home/123', mask, rec=True)
hanse_umlagerung = wm.add_watch('/home/456', mask, rec=True)
hanse_wa = wm.add_watch('/home/789', mask, rec=True)
hanse_we = wm.add_watch('/home/012', mask, rec=True)
intime = wm.add_watch('/home/345', mask, rec=True)

notifier.loop()

Works nice for a few hours but then it stop working.
i tryed notifier.start & stop and daemonized mode, everyhing the same.

Is there any reasone?

The included example:

import pyinotify
wm = pyinotify.WatchManager()  # Watch Manager
mask = pyinotify.IN_DELETE | pyinotify.IN_CREATE  # watched events
class EventHandler(pyinotify.ProcessEvent):
    def process_IN_CREATE(self, event):
        print "Creating:", event.pathname
    def process_IN_DELETE(self, event):
        print "Removing:", event.pathname
notifier = pyinotify.ThreadedNotifier(wm, EventHandler())
notifier.start()
wdd = wm.add_watch('/tmp', mask, rec=True)
print wdd
wm.rm_watch(wdd.values())
print wdd
notifier.stop()

gives this output

kato examples # python tutorial_threadednotifier.py 
 {'/tmp/.wine-1000/server-804-2c05c2': 7, '/tmp': 1, '/tmp/kde-satarsa': 16, '/tmp/kde-root': 18, '/tmp/ksocket-root': 4, '/tmp/pulse-eS4A6sQMNHw5': 2, '/tmp/.X11-unix': 11, '/tmp/akonadi-satarsa.ufjcQc': 3, '/tmp/kde-satarsa/krArc2011-04-13T15_13_08': 17, '/tmp/ksocket-satarsa': 13, '/tmp/mc-satarsa': 12, '/tmp/mc-root': 10, '/tmp/.ICE-unix': 9, '/tmp/.esd': 5, '/tmp/pulse-Y91gmxOGS7dL': 8, '/tmp/hsperfdata_satarsa': 15, '/tmp/.org.chromium.1fkBv2': 14, '/tmp/.wine-1000': 6}
[2011-04-13 22:06:14,885 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=7 >
[2011-04-13 22:06:14,885 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=1 >
[2011-04-13 22:06:14,885 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=16 >
[2011-04-13 22:06:14,885 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=18 >
[2011-04-13 22:06:14,886 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=4 >
[2011-04-13 22:06:14,886 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=2 >
[2011-04-13 22:06:14,886 pyinotify WARNING] Unable to retrieve Watch object associated to <_RawEvent cookie=0 mask=0x8000 name='' wd=11 >
{'/tmp/.wine-1000/server-804-2c05c2': 7, '/tmp': 1, '/tmp/kde-satarsa': 16, '/tmp/kde-root': 18, '/tmp/ksocket-root': 4, '/tmp/pulse-eS4A6sQMNHw5': 2, '/tmp/.X11-unix': 11, '/tmp/akonadi-satarsa.ufjcQc': 3, '/tmp/kde-satarsa/krArc2011-04-13T15_13_08': 17, '/tmp/ksocket-satarsa': 13, '/tmp/mc-satarsa': 12, '/tmp/mc-root': 10, '/tmp/.ICE-unix': 9, '/tmp/.esd': 5, '/tmp/pulse-Y91gmxOGS7dL': 8, '/tmp/hsperfdata_satarsa': 15, '/tmp/.org.chromium.1fkBv2': 14, '/tmp/.wine-1000': 6}

Thus, watch objects are not deleted from the dict.

Hi,

Pyinotify version: 0.9.1
Kernel version: 2.6.18-274.7.1.el5

I'm running pyinotify to monitor a directory for changes recursively with auto_add on:
pyinotify -r -a -f -e IN_CREATE,IN_CLOSE_WRITE,IN_DELETE,IN_MOVED_FROM,IN_MOVED_TO /basedir

When I move directories around inside the watched directory, delete and create events will always show paths for the original directory instead of the current one.

Example:

1. pathname=/basedir/folderA/mypic.jpg, maskname=IN_CREATE ( create the file )
2. pathname=/basedir/ffolderA, maskname=IN_MOVED_FROM|IN_ISDIR
3. pathname=/basedir/ffolderB/folderA, maskname=IN_MOVED_TO|IN_ISDIR ( folderA moved to folderB)
4. pathname=/basedir/folderA/mypic.jpg, maskname=IN_DELETE (delete the file)

As you can see the path on row 4. is incorrect since the folder has been moved.

So is this intended behavior, a bug, or kernel bug?

Is it possible to interrupt check_events() if no timeout is supplied?

I have another thread, which calls watcher thread's method that calls notifier.stop() - but the watcher thread seems to hang on notifier.check_events().

Hello,

An exemple is better than a long explanation :

1. Create a dir called < /share/test > and add a watch on it - with the parameters and <auto_add> set to True
2. Create a subdir in < /share/test > called < subdir1 >
3. Move the subdir < subdir1 > to < /share > ( the subdir is always watched )
4. Create a subdir to < subdir1 > called < subsubdir1 >

After the point 4, this error appairs :
[2011-08-10 11:34:15,034 pyinotify ERROR] add_watch: cannot watch _FALSE PATHNAME HERE_ WD=-1, Errno=No such file or directory (ENOENT)

Here is the code I use :

    mask = pyinotify.IN_DELETE  | pyinotify.IN_CLOSE_WRITE | pyinotify.IN_MOVED_FROM | pyinotify.IN_MOVED_TO | pyinotify.IN_CREATE
    wm = pyinotify.WatchManager()
    wd = wm.add_watch( flick.IMAGE_DIR, mask, rec=True, auto_add=True )

    class EventHandler(pyinotify.ProcessEvent):
        def process_IN_DELETE(self, event):
                            print event.pathname

        def process_IN_CLOSE_WRITE(self, event):
                            print event.pathname

        def process_IN_MOVED_FROM(self, event):
                            print event.pathname

        def process_IN_MOVED_TO(self, event):
                            print event.pathname

        def process_IN_CREATE(self, event):
                            print event.pathname

    notifier = pyinotify.ThreadedNotifier(wm, EventHandler())
    notifier.start()

hi,
here is the directory structure of '/tmp/site/'

├── data                      
│   ├── da2                   
│   │   └── fs                
│   └── img1                  
│       ├── a.jpg             
│       ├── b.jpg             
│       └── ff                
│           └── fd  
│      ..
│      .... many many directorys  
│
├── data2
│   └── a1                    
└── data3
      └── b1                    

data has thousands of subdirectory in it .
I want to monitor /tmp/site/ data2 and data3 except data

e = ['^/tmp/site/data']
# it will be exclude all ,no problem

e = ['^/tmp/site/data$']
''' `data2` and `data3` will be watched ,and it exclude 'data'.
    But the subdir of 'data' will be watched too.
'''
e = ['^/tmp/site/data$','^/tmp/site/data/.*']
''' I add another rule `/tmp/site/data/.*` 
    it's work as i expected ;
'''
vm.add_watch('/tmp/site/', mask, exclude_filter=pyinotify.ExcludeFilter(e))

i read the source class WatchManager
add_watch() will walk all subdir first (may be take very longtime ),then check if it needs to be excluded.

is it possible that if i exclude data ,subdir of data wouldn't walk() anymore
then subdir wouldn't be watched.

Because of this condition in process_events():

        watch_ = self._watch_manager.get_watch(raw_event.wd)
        if watch_ is None:
            # Not really sure how we ended up here, nor how we should
            # handle these types of events and if it is appropriate to
            # completly skip them (like we are doing here).
            log.warning("Unable to retrieve Watch object associated to %s",
                        repr(raw_event))
            continue

events of type IN_Q_OVEFLOW (which always have wd==-1) are never processed, not even by _SysProcessEvent.

The commit that added the above code doesn't go into much detail as to why it was added. Perhaps a misguided optimization attempt? It might be the case that it can be removed entirely. (If not, the best option would probably be to make a special case for raw_event.mask & IN_Q_OVERFLOW or maybe raw_event.wd == -1.)

The pid file is not created with secure permissions (0600) which may result in a world-writable PID file.
This results in a security hole as calling loop() with daemonize arguments again will kill whatever pid is listed in that file.
And depending on what you use your program for it might be even killed from a process running with root privileges which is certainly bad.

Hello, I've found a strange behaviour: IN_MODIFY event is launched two times when I edit a single file.

I'm using the tutorial_notifier.py example, monitoring a folder inside /tmp and every time I edit a file inside it, 2 IN_MODIFY events shows up.

I'm on a Ubuntu 12.04.1 box with the kernel 3.2.0-31-generic-pae.

3.4.6 and 3.4.7 Kernel compiled with the same config as my 2.6.32 that works.

CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y

Both give the following error:

InotifyBindingNotFoundError: Couldn't find any inotify binding

Any clues?

Hi,

still in my inotify things, I noticed that with 17220 watches, my daemon process takes 49Mb, which is much more than I may allow it to consume.

I modified the Watch class to use slots, and now the process takes only 32Mb, which I find a huge enhancement, for a few-lines modification. I probably have more room for improvements in other parts of my code, but watches are by far the biggest number of instance of one class I have in the process.

Here is the patch:

--- pyinotify2.py   2011-03-17 11:46:20.496813005 +0100
+++ pyinotify.py    2011-03-17 11:46:03.536813000 +0100
@@ -1390,6 +1390,7 @@
     Represent a watch, i.e. a file or directory being watched.

     """
+    __slots__ = ('wd', 'path', 'mask', 'proc_fun', 'auto_add', 'exclude_filter', 'dir')
     def __init__(self, wd, path, mask, proc_fun, auto_add, exclude_filter):
         """
         Initializations.
@@ -1425,7 +1426,7 @@
         s = ' '.join(['%s%s%s' % (Color.field_name(attr),
                                   Color.punctuation('='),
                                   Color.field_value(getattr(self, attr))) \
-                      for attr in self.__dict__ if not attr.startswith('_')])
+                      for attr in self.__slots__ if not attr.startswith('_')])

         s = '%s%s %s %s' % (Color.punctuation('<'),
                             Color.class_name(self.__class__.__name__),

best regards,
Olivier

Hi,
The del_watch() method can raise an exception which is logged through the logging facility. Here is the code :

try:
del self._wmd[wd]
except KeyError, err:
log.error(str(err))


But in that case "err" contains only the key that is missing from the dictionnary. For example, my logs contain several instance of

[2011-09-23 14:37:15,922 pyinotify ERROR] 16
[2011-09-23 14:3X:XX,9XX pyinotify ERROR] 23
[2011-09-23 14:3X:XX,9XX pyinotify ERROR] 39

It's a bit cryptic and took me a while before tracing the origin of the issue.

Thanks !

Hi,

First try with pyinotify and first fail (lastest stable version from today). I've launched the command line "pyinotify -r -a -e IN_CLOSE_WRITE /xfs".
After a while, I got the message :
Traceback (most recent call last):
File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
exec code in run_globals
File "/usr/lib/python2.6/site-packages/pyinotify-0.9.3-py2.6.egg/pyinotify.py", line 2298, in
command_line()
File "/usr/lib/python2.6/site-packages/pyinotify-0.9.3-py2.6.egg/pyinotify.py", line 2252, in command_line
wm = WatchManager()
File "/usr/lib/python2.6/site-packages/pyinotify-0.9.3-py2.6.egg/pyinotify.py", line 1708, in init
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)

My /xfs is a little big (20TB enterprise storage, millions of files). I've found my kernel settings were too low. So I have ajusted to fs.inotify.max_user_watches=2000000000 (64bit kernel). First, it still does not work with the same message. Worst : if I launch the same command on an empty path, it still tells me "Errno=Too many open files (EMFILE)"!!!!

Have I missed something or this soft is really broken?

I know this probably isn't part of the normal inotify functionality, but I had to implement a means of telling when a directory had finished copying.

http://code.google.com/p/archivematica/issues/detail?id=482

I created a second recursive watch directory, on the directory getting copied(being populated) to check for updates in the created directory, and timer class to act when modifications had stopped occurring.

http://code.google.com/p/archivematica/source/diff?spec=svn1364&r=1364&format=side&path=/trunk/src/MCPServer/lib/archivematicaMCP.py

I was curious if anyone else had faced this, and what solutions they came up with.

There is a tricky part in the FAQ that leaves room for interpretation:

use IN_CLOSE_WRITE because if emitted the all changes on the appropriate file are safely written inside the file

Does it mean

1. if emitted, all the changes are safely inside the file
2. emitted before all the changes are safely inside the file

It seems to be the former, but I am experiencing a situation in which IN_CLOSE_WRITE is triggered, but when I open the file - it doesn't contain any changes yet. So I thought I would first make sure that I understand the manual correctly.

just as title showed

Following code is from pyinotify, questionable line: 136
class UnicodeLogRecord(logging.LogRecord):
def init(self, name, level, pathname, lineno,
msg, args, exc_info, func=None):
py_version = sys.version_info
# func argument was added in Python 2.5, just ignore it otherwise.
if py_version[0] >= 2 and py_version[1] >= 5:

I believe that it should be changed to:
if py_version[0] > 2 or py_version[0] == 2 and py_version[1] >= 5:

The initial set up of watches takes exponential time in the number of directories to such a degree it would take approximately 30 hours to set up watches on 100,000 directories on my laptop.

I can't test it right now, but I suspect the culprit is line 1855 in pyinotify.py (under python2) (1842 under python3):

if self.get_wd(rpath) is not None:
    # We decide to ignore paths already inserted into
    # the watch manager. Need to be removed with rm_watch()
    # first. Or simply call update_watch() to update it.
    continue

where it checks if a watch descriptor already exists for the directory.

It makes good sense to do when adding watches to dirs that could have been moved from elsewhere in a recursively watched dir, but in the initial set up of watches, if no moving/creating/deleting is being done, it just slows it down immensely. Should be doable in linear time.

So instead of removing this check altogether (if it is the culprit) I suggest a "static" flag or similar to disable the check on the initial watches setup when it is known that the hierarchy won't change during.

Hello,

auto_add will not work as intended when the mask of the watch does not include IN_CREATE.

It would be nice if setting auto_add=True automatically included IN_CREATE in the mask, or if it was explicitely stated in the doc that auto_add requires IN_CREATE.

Thanks!

Hi, tnx for this piece of code, is really useful!

I'm trying to watch a dir and launch an external sh script with the full path of the file. I've searched and adapted various piece of code, this is the result, the name for this file is monitor.py
import pyinotify
import sys
import os.path
from subprocess import call
import signal

log_file = open("/var/log/monitor.log", "a")

def log(text):
log_file.write(text + "\n")
log_file.flush()

def signal_handler(signal, frame):
log("Fine Monitoraggio Cartella Vari")
sys.exit(0)

log("Avvio Monitoraggio Cartella Vari")

signal.signal(signal.SIGTERM, signal_handler)

allowed_exts = ["mp4", "flv", "avi", "divx", "mpg", "mpeg", "mkv"]

wm = pyinotify.WatchManager() # Watch Manager
mask = pyinotify.IN_CLOSE_WRITE | pyinotify.IN_MOVED_TO # watched events

class EventHandler(pyinotify.ProcessEvent):
def init(self):
self.modified_files = set()

def process_IN_MOVED_TO(self, event):
    self.do_index_command(event)

def process_IN_CLOSE_WRITE(self, event):
    self.do_index_command(event)

def do_index_command(self, event):
    if self.is_allowed_path(event.pathname, event.dir):
        log("Analizzo %s " % (event.pathname))
        call(["/home/xbmc/scripts/filebot_script.sh", event.pathname])

        # Remove from list of modified files.
        try:
            self.modified_files.remove(event.pathname)
        except KeyError, e:
            # Don't care.
            pass
    else:
        log("%s non e' da monitorizzare!" % event.pathname)

def is_allowed_path(self, filename, is_dir):
    # Don't check the extension for directories
    if not is_dir:
        ext = os.path.splitext(filename)[1][1:].lower()
        if ext not in allowed_exts:
            return False
    if filename.find("@eaDir") > 0:
        return False
    return True

handler = EventHandler()
notifier = pyinotify.Notifier(wm, handler)
wdd = wm.add_watch(["/media/TwoTera/Download/vari"], mask, rec=True, auto_add=True)

try:
notifier.loop(daemonize=True, pid_file='/var/run/monitor.pid')

except pyinotify.NotifierError, err:
print >> sys.stderr, err

the filebot_script.sh take the full path of the file and use it to make his stuff :)
filebot also "make" his own log, so if I want to check the filebot activity I watch his log

The "problem", I think that is only my mistake, is that:
If I launch the script through the command line "python monitor.py" all works fine

Now I also found a init.d script to run the script at boot, here the script:

!/bin/sh

case "$1" in
start|"")

start the monitoring daemon

python /home/xbmc/scripts/monitoraggio_file.py
;;
restart|reload|force-reload)
echo "Error: argument '$1' not supported" >&2
exit 3
;;
stop)
kill cat /var/run/monitor.pid
;;
*)
echo "Usage: fileswatch.sh [start|stop]" >&2
exit 3
;;
esac

Now, If I run the script, monitor.py works, but filebot don't make any log!

Any idea why?

Tnx, Jo

Hello i love your library, i just wanted to recommend a feature.

When you create a notifier with stats and start the notifier, the notifier.start() causes the stat object to populate _stats.
So you get something like this: {'IN_OPEN': 41, 'IN_CLOSE_NOWRITE': 41, 'IN_ISDIR': 82} but this events didn't actually occur they were caused by notifier.start().
At the moment i can do stat._stats = {} and its solved, but i think it should be added as the last task in notifier.start().
It can be reproduced using the stats example on the wiki.

I am not sure if you agree.
Thank You

Hey guys,

I downloaded pynotify from the Git repo, and the recursiveness doesn't work for me.
in one window:

git clone git://github.com/seb-m/pyinotify.git
cd pyinotify/python2/
cp examples/tutorial_notifier.py .

python tutorial_notifier.py

second window:
crisgal@ubuntu:~$ mkdir /tmp/dir1
output: Creating: /tmp/dir1

crisgal@ubuntu:~$ touch /tmp/file1
output: Creating: /tmp/file1

but these, don't generate any output:
crisgal@ubuntu:$ mkdir /tmp/dir1/dir2
crisgal@ubuntu:$ touch /tmp/dir1/file2

I'm running an up to date Ubuntu Lucid Lynx:
Linux ubuntu 2.6.32-24-generic #43-Ubuntu SMP Thu Sep 16 14:17:33 UTC 2010 i686 GNU/Linux
Python 2.6.5

any idea what am I doing wrong?

$ grep rec tutorial_notifier.py
wdd = wm.add_watch('/tmp', mask, rec=True)

thanks!

when an existing file modified, pyinotify emmits deleted, created, modified events. But it just should trigger modified event.

I've had to extend Notifier for use in a multiprocessing environment, specifically to handle the case where to do an orderly shutdown in a related group of processes, the notifier loop needs to be stopped from another process.

In this instance, even using a ThreadedNotifier, the stop() method once it has called _self.stop_event.set() and written 'stop' to _self.pipe[1] in order to stop the loop, goes on to try to cleanup, which fails.

The cleanup code would be better done at the end of loop(), so that it is in the same process context as the loop() method, rather than the process context of the stop() method. Also the _.stop_event attribute needs to be a multiprocessing.Event(), not a threading.Event().

I propose either adding the following MultiprocessNotifier, or somehow detecting the presence of multiprocessing, and altering the behaviour of Notifier, but this may be a bit 'hidden'. (What I don't like about this code is that it's almost a carbon-copy of ThreadedNotifier with a few subtle changes - seems inefficient!)

import os
import time
import select
from multiprocessing import Event

from pyinotify import Notifier

class MultiprocessNotifier(Notifier):
    """
    This notifier inherits from Notifier, but can be used in a multiprocessing environment.

    Note that every functionality provided by this class is also provided
    through Notifier class. Moreover Notifier should be considered first because
    it is not threaded and could be easily daemonized.

    The crucial difference between this and the Notifier and ThreadedNotifier is that
    .stop() can be called in a different process to the one running .loop() 
    """
    def __init__(self, watch_manager, default_proc_fun=None, read_freq=0,
                 threshold=0, timeout=None):
        """
        Initialization, initialize base classes. read_freq, threshold and
        timeout parameters are used when looping.

        @param watch_manager: Watch Manager.
        @type watch_manager: WatchManager instance
        @param default_proc_fun: Default processing method. See base class.
        @type default_proc_fun: instance of ProcessEvent
        @param read_freq: if read_freq == 0, events are read asap,
                          if read_freq is > 0, this thread sleeps
                          max(0, read_freq - timeout) seconds.
        @type read_freq: int
        @param threshold: File descriptor will be read only if the accumulated
                          size to read becomes >= threshold. If != 0, you likely
                          want to use it in combination with an appropriate
                          value set for read_freq because without that you would
                          keep looping without really reading anything and that
                          until the amount of events to read is >= threshold. At
                          least with read_freq you might sleep.
        @type threshold: int
        @param timeout:
           see http://docs.python.org/lib/poll-objects.html#poll-objects
        @type timeout: int
        """
        # Stop condition
        self._stop_event = Event()
        # Init Notifier base class
        Notifier.__init__(self, watch_manager, default_proc_fun, read_freq,
                          threshold, timeout)
        # Create a new pipe used for thread termination
        self._pipe = os.pipe()
        self._pollobj.register(self._pipe[0], select.POLLIN)

    def stop(self):
        """
        Stop notifier's loop. Stop notification.
        Remainder of the cleanup is done at the end of loop (in the original
        process)
        """
        self._stop_event.set()
        os.write(self._pipe[1], 'stop')

    def loop(self):
        """
        Notifier's main loop.

        Events are read only once time every min(read_freq, timeout)
        seconds at best and only if the size of events to read is >= threshold.
        """
        # When the loop must be terminated .stop() is called, 'stop'
        # is written to pipe fd so poll() returns and .check_events()
        # returns False which triggers the while's stop condition
        # ._stop_event.isSet() which stops the loop execution.
        # Following the termination of the while loop, it is this process
        # which must do the cleanup. 
        while not self._stop_event.is_set():
            self.process_events()
            ref_time = time.time()
            if self.check_events():
                self._sleep(ref_time)
                self.read_events()
        Notifier.stop(self)
        self._pollobj.unregister(self._pipe[0])
        os.close(self._pipe[0])
        os.close(self._pipe[1])

I am running pyinotify on my readynas device. When I do and watch a directory, where I rename a file from "test1.txt" to "test.txt" it will show up as "test.txto" as the event.pathname. It seems like in the c-code, the null terminator isn't getting reset and maybe just a string length is being passed with the pathname?

The readynas runs:
$ uname -r -v -m -s -p
Linux 2.6.17.14ReadyNAS #1 Wed Sep 22 04:42:09 PDT 2010 padre unknown

The output is:
[2011-03-27 15:30:02,628 pyinotify DEBUG] Event queue size: 64
[2011-03-27 15:30:02,637 pyinotify DEBUG] <_RawEvent cookie=4269 mask=0x40 name=test.txt wd=1 >
[2011-03-27 15:30:02,648 pyinotify DEBUG] <_RawEvent cookie=4269 mask=0x80 name=test1.txt wd=1 >
Been modified: testdir/test.txt:testdir/test1.txt
[2011-03-27 15:30:15,527 pyinotify DEBUG] Event queue size: 64
[2011-03-27 15:30:15,537 pyinotify DEBUG] <_RawEvent cookie=4270 mask=0x40 name=test1.txt wd=1 >
[2011-03-27 15:30:15,547 pyinotify DEBUG] <_RawEvent cookie=4270 mask=0x80 name=test.txto wd=1 >
Been modified: testdir/test1.txt:testdir/test.txto

the event handler:

class EventHandler(pyinotify.ProcessEvent):
movEvent = {}
def __init__(self, root):
    pyinotify.ProcessEvent.__init__(self)
    self.root = root
    self.root_dir = os.path.dirname(root)
def process_IN_CREATE(self, event):
    self.been_modified(event)
def process_IN_DELETE(self, event):
    self.been_modified(event)
def process_IN_MOVED_TO(self, event):
    self.been_modified(event)
def process_IN_CLOSE_WRITE(self, event):
    self.been_modified(event)
def process_IN_MODIFY(self, event):
    self.been_modified(event)
def been_modified(self, event):
    pathname = os.path.relpath(event.pathname, self.root_dir)
    src_pathname = os.path.relpath(event.src_pathname, self.root_dir) if hasattr(event, 'src_pathname') and event.src_pathname else None
    if not src_pathname:
      print "Been modified:", pathname
    else:
      print "Been modified:", src_pathname + ":" + pathname

When I run the module's own main:

python -m pyinotify -v testdir/

[2011-03-27 15:46:10,026 pyinotify DEBUG] Event queue size: 64
[2011-03-27 15:46:10,037 pyinotify DEBUG] <_RawEvent cookie=4280 mask=0x40 name=test1.txt wd=1 >
Traceback (most recent call last):
  File "/usr/lib/python2.6/logging/__init__.py", line 776, in emit
    msg = self.format(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 654, in format
    return fmt.format(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 436, in format
    record.message = record.getMessage()
  File "/usr/local/lib/python2.6/dist-packages/pyinotify.py", line 293, in getMessage
    msg = unicode(msg, sys.getfilesystemencoding())
  File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe0 in position 148: invalid continuation byte
<Event cookie=4280 dir=False mask=0x40 maskname=IN_MOVED_FROM name=test1.txt path=testdir pathname=/c/foobar/testdir/test1.txt wd=1 >
<Event cookie=4280 dir=False mask=0x80 maskname=IN_MOVED_TO name=test.txt path=testdir pathname=/c/foobar/testdir/test.txt src_pathname=testdir/test1.txt wd=1 >

When an event is triggered I get console logs like this:

<Event dir=False mask=0x2 maskname=IN_MODIFY name='' path=myfile.csv pathname=/home/me/myfile.csv wd=1 >

How do I turn off this logging behavior?

Sometimes you would like to do some IO to a directory you're watching but not handle the events. I would prefer not to rm_watch and then add it again or add special logic tagging every event. Would be nice to have a context manager or something similar to let you do any IO you want without triggering pyinotify events. I'm thinking of something like

with ignorePyinotify(wm):
do_stuff() # no pyinotify events are triggered

is this possible?

when I start the loop.py ,moniter all events on SEVER A which exports as a nfs server .

SERVER B mounted A and create dir,some files .

but the loop.py only print the create dir event ,none of the files.

what's the matter?

Thank you !

Please, change if err.errno == errno.EINTR: at line 1083 to if err.args[0] == errno.EINTR:

Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import pyinotify
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.6/dist-packages/pyinotify.py", line 134, in
load_libc()
File "/usr/local/lib/python2.6/dist-packages/pyinotify.py", line 119, in load_libc
LIBC = ctypes.CDLL(libc, use_errno=True)
TypeError: init() got an unexpected keyword argument 'use_errno'

dpkg -l | grep libc6
ii libc6 2.11.2-9 Embedded GNU C Library: Shared libraries
ii libc6-dev 2.11.2-9 Embedded GNU C Library: Development Librarie

When creating/adding a watch using pyinotify under pypy, I get the following warnings:

virtualenv/site-packages/pyinotify.py:232: RuntimeWarning: C function without declared arguments called
return self._libc.inotify_init()
virtualenv/site-packages/pyinotify.py:237: RuntimeWarning: C function without declared arguments called
return self._libc.inotify_add_watch(fd, pathname, mask)

It seems pyinotify should set argtypes for these ctypes calls?

with my python script running in the background: (wdd = wm.add_watch('/home/usr/public_html', mask, rec=True))

> touch public_html/file

Create: /home/usr/public_html/file
CloseWrite: /home/usr/public_html/file
> mkdir public_html/adir

Create: /home/usr/public_html/adir
> touch public_html/adir/file

> touch public_html/adir/file2

kill the script and restart:

> touch public_html/adir/file

CloseWrite: /home/usr/public_html/adir/file
> touch public_html/adir/file2

CloseWrite: /home/usr/public_html/adir/file2

(of course the Create: is no longer triggered)

I was hoping to find that
if IS_DIR and IN_CREATE and (rec==True) :
add_watch(NewDir, mask, rec=True)

immediately in the pyinotify.

Because if I call add_watch it in the EventProcessor and call the command tar -xzf project.tar.gz in the watched directory, then the watch will not be add until the queue reaches the create directory. That allows the system to write files to the directory before it's watched, possibly all the files.

Hi, seb-m.

Recently I am writing a script to monitor some files. I want to monitor a dir , and the dir has sub dir and in subdir there are somes files with .md. (maybe there are some other files, such as *.swp...)

I only want to monitor the .md files, I have read the doc, and there is only a ExcludeFilter, and in the issue : #31 says, only dir can be filter but not files.

Now what I do is to filter in process_* function to check the event.name by fnmatch.

So if I only want to monitor the specified suffix files, is there a better way? Thanks.

It would be nice, if recursive globs (where ** matches any number of dirs) would be supported/recognized.

One option would be to use Formic:
https://pypi.python.org/pypi/formic

(via http://stackoverflow.com/a/14823904/15690)

Otherwise it would be helpful to issue a warning, if such a glob is used - since it appears to be a common use case when watching for file changes, and would avoid confusion.

Would love to have some sort of tests that I could run against. Dependent applications using pyinotify would benefit from the extra peace of mind (but thanks for the great work so far).

Duplicated from Pyinotify Trac - Issue 8

Consider this case:

• /wdir: is a watched directory, where each new created directory is automatically watched
• /sdir: is a directory with several small files (file1, file2,..., file10)

Now, /sdir and its content are copied to /wdir:

 cp -a /sdir /wdir

Under these circumstances if we had started pyinotify like this:

 $ python pyinotify.py -a -e IN_CREATE /wdir

We would have observed the following output (assuming files1-10 are copied in that order):

 <Event dir=True mask=0x40000100 maskname=IN_CREATE|IN_ISDIR name=sdir path=/wdir pathname=/wdir/sdir wd=1 >
 # ... missed IN_CREATE events for file1 thru file3 ...
 <Event dir=True mask=0x40000000 maskname=IN_CREATE name=file4 path=/wdir/sdir pathname=/wdir/sdir/file4 wd=1 >
 <Event dir=True mask=0x40000000 maskname=IN_CREATE name=file5 path=/wdir/sdir pathname=/wdir/sdir/file5 wd=1 >
 <Event dir=True mask=0x40000000 maskname=IN_CREATE name=file6 path=/wdir/sdir pathname=/wdir/sdir/file6 wd=1 >

What happened is that there is a short laps of time between the moment /sdir is copied into /wdir and the moment where pyinotify effectively starts watching it. And during that time where /wdir/sdir is not yet watched the first files file1, file2, file3 are copied into this directory and mistakenly do not generate IN_CREATE events as expected.

(Thanks to Peter Smith for his initial report)

Removing old versions from pypi can break automated deployment scripts that depend on a specific version using e.g. pip requirement files. For example, the recent 0.9.1 release broke one of my deployment scripts - I had an explicit dependency on pyinotify==0.9.0, which pip can no longer find for installation.

Please consider leaving the old versions available on pypi for scenarios such as this.

Thanks,

Ryan

It's very annoying that pyinotify forces a specific Logger class on me globally. My application uses its own Logger class to inject extra information in log records, and importing pyinotify undoes all that. Is this really the only way to go about this? Couldn't pyinotify limit its modifications to its own loggers?

Hello,

I had installed pyinotify 0.9.2 via easy_install, took tutorial_notifier.py, enabled auto_add and started to test.
uname -r
2.6.38-8-generic

making tmpfs to not touch disks

mkdir /tmp/tin;mount -t tmpfs -o size=20000000,mode=1777,nr_inodes=200000,noatime,nodiratime none /tmp/tin

making queue large

echo 1000000 > /proc/sys/fs/inotify/max_queued_events

starting notifier

time python ./tutorial_notifier.py > tmp.log

starting tmpflood, code http://vorona.com.ua/pyinotify/tmpflood2.py

creating 25000 dirs under /tmp/tin and then removing them

./tmpflood2.py
Creating dirs ...
Dirs created. Wait till notifier stops eating CPU and press Enter to removing

waiting ~ 5 minutes till notifier process all events, noted cpu time 4:45 and pressing Enter

Removing dirs ...

waiting a couple seconds till notifier process all events

Then pressing Ctrl+C in notifier console and got CPU time

user 4m50.130s
sys 0m1.948s

So, I got 4:45 for processing creation and 5 secs for processing deletion.

After investigation I had found main slowdown in get_wd(). So, I simply added one more dictionary "path ->watch descriptor" to speedup this lookup - patch http://vorona.com.ua/pyinotify/pyinotify.py.patch
After that I run test again and got following results

user 0m57.088s
sys 0m0.660s
I got 5 secs for processing creation and 52 secs for processing deletion. Total speedup with patch was ~5x and can vary depending on dirs names, dirs count etc. In my tests I saw 10x speedup and more. Directory creation speedup ~57x, directory deletion slowdown ~10x. But there is no deletion without creation, so I do not see any drawbacks.

Regards,
Alex

This should be on PyPI and installable via pip install pyinotify

I see a new addition is the TornadoAsyncNotifier class. What is this? The wiki doesn't mention it and the example has one comment in the handle_read_callback function:

Just stop receiving IO read events after the first
iteration (unrealistic example).

The 'Recent developments' page doesn't mention it either, but it stops at version 0.9.3. The current version is 0.9.4. Google wasn't much help either.

I'm sure it is a nice feature, but have no idea what it is.

Lets say you start pynotify watching a dir with recursive and auto_add enabled.
Then you create a new dir and it gets added to the watched list.
Then you delete it and it gets removed.
Lastly you recreate it but this time it is not added to the watched list as expected.

Test case shown bellow:

$ mkdir /tmp/watch/  #create base directory to be watched

$ python -m pyinotify -r -a /tmp/watch/  #run in second terminal to monitor events

$ cd /tmp/watch/
$ mkdir dir  #create a new dir and it gets picked up
<Event dir=True mask=0x40000100 maskname=IN_CREATE|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000020 maskname=IN_OPEN|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000020 maskname=IN_OPEN|IN_ISDIR name='' path=/tmp/watch/dir pathname=/tmp/watch/dir wd=2 >
<Event dir=True mask=0x40000010 maskname=IN_CLOSE_NOWRITE|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000010 maskname=IN_CLOSE_NOWRITE|IN_ISDIR name='' path=/tmp/watch/dir pathname=/tmp/watch/dir wd=2 >
<Event dir=True mask=0x40000020 maskname=IN_OPEN|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000020 maskname=IN_OPEN|IN_ISDIR name='' path=/tmp/watch/dir pathname=/tmp/watch/dir wd=2 >
<Event dir=True mask=0x40000010 maskname=IN_CLOSE_NOWRITE|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000010 maskname=IN_CLOSE_NOWRITE|IN_ISDIR name='' path=/tmp/watch/dir pathname=/tmp/watch/dir wd=2 >

$ touch dir/file  #create a new file in the subdir and it gets picked up
<Event dir=False mask=0x100 maskname=IN_CREATE name=file path=/tmp/watch/dir pathname=/tmp/watch/dir/file wd=2 >
<Event dir=False mask=0x20 maskname=IN_OPEN name=file path=/tmp/watch/dir pathname=/tmp/watch/dir/file wd=2 >
<Event dir=False mask=0x4 maskname=IN_ATTRIB name=file path=/tmp/watch/dir pathname=/tmp/watch/dir/file wd=2 >
<Event dir=False mask=0x8 maskname=IN_CLOSE_WRITE name=file path=/tmp/watch/dir pathname=/tmp/watch/dir/file wd=2 >

$ rm -rf dir  #remove the directory
<Event dir=True mask=0x40000020 maskname=IN_OPEN|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000020 maskname=IN_OPEN|IN_ISDIR name='' path=/tmp/watch/dir pathname=/tmp/watch/dir wd=2 >
<Event dir=False mask=0x200 maskname=IN_DELETE name=file path=/tmp/watch/dir pathname=/tmp/watch/dir/file wd=2 >
<Event dir=True mask=0x40000200 maskname=IN_DELETE|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000010 maskname=IN_CLOSE_NOWRITE|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000010 maskname=IN_CLOSE_NOWRITE|IN_ISDIR name='' path=/tmp/watch/dir pathname=/tmp/watch/dir wd=2 >

$ mkdir dir  #recreate it and this time you can see that not every expected event is fired, like above
<Event dir=True mask=0x40000100 maskname=IN_CREATE|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >

$ touch dir/file  #'dir' is no longer watched for events!
// nothing happens //

$ rm dir/file  #same...
// nothing happens //

$ rm -rf dir
<Event dir=True mask=0x40000020 maskname=IN_OPEN|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000200 maskname=IN_DELETE|IN_ISDIR name=dir path=/tmp/watch pathname=/tmp/watch/dir wd=1 >
<Event dir=True mask=0x40000010 maskname=IN_CLOSE_NOWRITE|IN_ISDIR name=dir path=/tmp/watch pathname=/

This is with:

$ python -V
Python 2.7.3

$ python -m pyinotify -V
0.9.2

$ uname -r
3.2.0-32-generic-pae

I have a NFS mount to a storage server on server A. We have a web server on Server B that general users access to place updates/etc that is attached to the same NFS mount that Server A has access to.

When I put files onto Server A in the watched directory via ssh/sftp my basic pyinotify fires off every event that is happening. However when I put files on Server B (same NFS mount ie, same folder) it doesn't fire any events.

Server A path: /srv/sandbox
Server B path: /www/docs/sandbox

Here is my watcher code:

wm.add_watch('/srv/sandbox/', pyinotify.ALL_EVENTS, rec=True, auto_add=True)

I read through the issues on StackOverflow and read through the wiki/examples and couldn't find anything related - I could just be missing a param or is it because the mounts are mounted at different locations?

Yes, i like and use pyinotify. Thank you for the work.
While i've used it today for an application of me, i stumble across the following things.
For your information, i'm using Python 3.2 - pyinotify in the latest version of pypi (0.9.3).

When i stop the watcher with notifier.stop(), than i get the following:

File "/home/mono/.projects/sugarsync/SugarSyncNotifier.py", line 86, in stopNotifier
self.notifier.stop()
File "/usr/lib64/python3.2/site-packages/pyinotify-0.9.3-py3.2-linux-x86_64.egg/pyinotify.py", line 1459, in stop
os.write(self._pipe[1], b'stop')
OSError: [Errno 9] Bad file descriptor

"notifier" ist object of pyinotify.ThreadedNotifier.

Second thing:

File "/home/mono/.projects/sugarsync/SugarSyncNotifier.py", line 84, in stopNotifier
self.wm.rm_watch(f.getWatcher().values())
File "/usr/lib64/python3.2/site-packages/pyinotify-0.9.3-py3.2-linux-x86_64.egg/pyinotify.py", line 2041, in rm_watch
wd_ = self._inotify_wrapper.inotify_rm_watch(self._fd, awd)
File "/usr/lib64/python3.2/site-packages/pyinotify-0.9.3-py3.2-linux-x86_64.egg/pyinotify.py", line 156, in inotify_rm_watch
return self._inotify_rm_watch(fd, wd)
File "/usr/lib64/python3.2/site-packages/pyinotify-0.9.3-py3.2-linux-x86_64.egg/pyinotify.py", line 189, in _inotify_rm_watch
ret = inotify_syscalls.inotify_rm_watch(fd, wd)
TypeError: an integer is require

This happens, when i call the rm_watch-Method with a dict of integers instead of only one integer.
The help means that this is possible - so it is a bug or it the help not so up-to-date?

Thanks. Best Regards
~ monofox

I have a list of exclude patterns like this :
['..hg.', '.hg-check.']

This patterns aim to exclude mercurial files. The problem is the second pattern because it's a file and not a directory. Pyinotify does not ignore the file when it changed. I think that it's a bug ?

Suppose I create a directory named something like blablahg-checkblabla/ and if suppose Icreate a file into it named aaa.txt. If a modify the content of the aaa.txt, Pyinotify does not sent the IN_MODIFY event. It's good but it works because the pattern match the directory name.

Example of code to test my issue :

import pyinotify


class EventHandler(pyinotify.ProcessEvent):
    def __init__(self):
        pass

    def process_IN_CREATE(self, event):
        pass

    def process_IN_MODIFY(self, event):
        print 'modif !'
        pass

    def process_IN_DELETE(self, event):
        pass

vm = pyinotify.WatchManager()
mask = pyinotify.IN_CREATE | pyinotify.IN_MODIFY | pyinotify.IN_DELETE
handler = EventHandler()
e = ['^/etc/hosts']

vm.add_watch('/etc/', mask,
             exclude_filter=pyinotify.ExcludeFilter(e))

monitor = pyinotify.ThreadedNotifier(vm, handler)
monitor.coalesce_events()
monitor.start()

When I change the /etc/hosts file, the stdout print 'modif' !' :-(