mailprocessing / mailprocessing Goto Github PK

Maildir and IMAP folder processing with user-defined Python scripts

Home Page: https://mailprocessing.github.io/mailprocessing

License: GNU General Public License v2.0

Makefile 0.58% Shell 0.28% Python 99.14%

mailprocessing's Introduction

mailprocessing -- maildir and IMAP processors

GitHub: https://github.com/mailprocessing/mailprocessing
Documentation: https://mailprocessing.github.io/mailprocessing

maildirproc was written by Joel Rosdahl <[email protected]> and is
distributed under the terms of the GNU General Public Licence (see the
file LICENSE for details).

imapproc was written by Johannes Grassler <[email protected]> and is
distributed under the terms of the GNU General Public Licence (see the
file LICENSE for details).

maildirproc is a program that processes one or several existing mail
boxes in the maildir format. It is primarily focused on mail sorting
-- i.e., moving, copying, forwarding and deleting mail according to a
set of rules. It can be seen as an alternative to procmail, but
instead of being a delivery agent (which wants to be part of the
delivery chain), maildirproc only processes already delivered mail.

imapproc is a program that processes one or more folders on an IMAP
server. Much like maildirproc it sorts, e.g. moves, copies and forwards
email. It uses the same filter configuration mechanism as maildirproc: a
Python script that hooks into maildirprocs processing code.

For more information, some documentation and examples, see the docs
directory.

mailprocessing's People

Contributors

Stargazers

Watchers

Forkers

tgray edgewood romanofski lvsv jgrassler mook-as ggada jrosdahl prechelt penguinpee maxhbr

mailprocessing's Issues

move() method eats the first email moved to new folder when create=True

tl;dr: maildirproc eats the first email moved to a new folder when creating folders automatically with create=True. Do not use this feature until this is fixed.

The move() method for IMAP emails has an optional create parameter that creates nonexistent folders. If that parameter is True, imapproc (maildirproc is unaffected) will

Create all non-existent folders and their parent folders if told to move a message to a non-existent folder
Fail to copy the message to the target folder
Delete the original message anyway.

The net result is that the first message to be moved to a non-existent folder will get deleted.

Please do not use the create parameter for now. I will fix this tonight (CEST).

Error logging error due to invalid encoding

Hi! I appear to be hitting an error decoding one of my messages, which is causing a different error:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/mailprocessing/processor/imap.py", line 478, in _download_headers_batched
    s = s.decode(c if c else "ascii")
UnicodeDecodeError: 'iso2022_jp' codec can't decode bytes in position 5-6: illegal multibyte sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/imapproc", line 11, in <module>
    load_entry_point('mailprocessing==1.2.6', 'console_scripts', 'imapproc')()
  File "/usr/lib/python3.6/site-packages/mailprocessing/cmd/imap.py", line 326, in main
    processor = ImapProcessor(rcfile, log_fp, **processor_kwargs)
  File "/usr/lib/python3.6/site-packages/mailprocessing/processor/imap.py", line 114, in __init__
    self.set_folders(kwargs['folders'])
  File "/usr/lib/python3.6/site-packages/mailprocessing/processor/imap.py", line 176, in set_folders
    self._cache_headers()
  File "/usr/lib/python3.6/site-packages/mailprocessing/processor/imap.py", line 319, in _cache_headers
    self._cache_headers_file()
  File "/usr/lib/python3.6/site-packages/mailprocessing/processor/imap.py", line 354, in _cache_headers_file
    self._cache_headers_memory()
  File "/usr/lib/python3.6/site-packages/mailprocessing/processor/imap.py", line 339, in _cache_headers_memory
    folder, self.header_cache[folder]['uids'])
  File "/usr/lib/python3.6/site-packages/mailprocessing/processor/imap.py", line 559, in _update_cache
    messages = self._download_headers_batched(folder, uids_download)
  File "/usr/lib/python3.6/site-packages/mailprocessing/processor/imap.py", line 482, in _download_headers_batched
    self._processor.log_error(
AttributeError: 'ImapProcessor' object has no attribute '_processor'

I see that there was code to attempt to handle the decoding issue, but that is itself falling over.

imapproc 1.0.1 (from mailprocessing 1.2.6-lp151.6.1 from obs://server:mail; I understand you maintain the packages on OBS as well?). Unfortunately I can't figure out which message is causing the issue, so moving it out of the way is a bit hard.

Thanks for imapproc!

maildirproc does not allow empty folder prefix

It is impossible to set an empty folder prefix. Passing option -p "" to maildirproc does not work. I have found that adding the line

processor.prefix = ""

to the .rc file works, but access to this variable from the .rc file is undocumented and therefore unreliable.
Older versions of maildirproc (for example version 1.0.1) did allow the use of different Maildir names with no common prefix.
I do not see any reason to prevent such use, therefore I believe that preventing the user from setting an empty prefix is a bug, not a feature.

Maildir copy appears to add the prefix twice when creating folder

Hi! Just something I noticed while testing things:

mailprocessing/mailprocessing/mail/maildir.py

Lines 51 to 54 in 53e1b5b

    
           # Take care of prefix here and ensure create_folder() does not mess 
        
           # with it. 
        
           folder_list = self._processor.path_ensure_prefix(maildir, sep=sep) 
        
           maildir = self._processor.list_path(folder_list, sep=sep)

We call path_ensure_prefix in Maildir's mail.copy(). We then go into _copy():

mailprocessing/mailprocessing/mail/maildir.py

Lines 140 to 143 in 53e1b5b

    
           def _copy(self, maildir, **kwargs): 
        
               if kwargs.get('create', False) and not os.path.isdir(maildir): 
        
                   try: 
        
                       self._processor.create_folder(maildir, **kwargs)

We (can) call create_folder, in which we call path_ensure_prefix again:

mailprocessing/mailprocessing/processor/maildir.py

Line 109 in 53e1b5b

folder_list = self.path_ensure_prefix(folder, sep=self.separator)

This leads to errors for me, since I (think I) end up creating ..directory instead when calling copy(directory).

IMAP: Headers never updated

As far as I can tell, if --interval is used to re-process folders as new mail comes in, nothing ever calls _cache_headers() in __iter__(); this means it just sorts the same messages over and over, instead of actually processing new mail.

This can be see in the logs:

Dec 29 14:53:15 localhost imapproc[17242]: ==> Moving UID 633 to Lists/moo
Dec 29 14:53:15 localhost imapproc[17242]: ==> Copying 633 to Lists/moo
Dec 29 14:53:15 localhost imapproc[17242]: ==> Deleting 633
Dec 29 14:53:20 localhost imapproc[17242]: ==> Moving UID 633 to Lists/moo
Dec 29 14:53:20 localhost imapproc[17242]: ==> Copying 633 to Lists/moo
Dec 29 14:53:20 localhost imapproc[17242]: ==> Deleting 633
Dec 29 14:53:25 localhost imapproc[17242]: ==> Moving UID 633 to Lists/moo
Dec 29 14:53:25 localhost imapproc[17242]: ==> Copying 633 to Lists/moo
Dec 29 14:53:25 localhost imapproc[17242]: ==> Deleting 633
Dec 29 14:53:30 localhost imapproc[17242]: ==> Moving UID 633 to Lists/moo

The same message keeps getting moved every time the interval happens.

mailprocessing / mailprocessing Goto Github PK

mailprocessing's Introduction

mailprocessing's People

Contributors

Stargazers

Watchers

Forkers

mailprocessing's Issues

move() method eats the first email moved to new folder when create=True

Error logging error due to invalid encoding

maildirproc does not allow empty folder prefix

Maildir copy appears to add the prefix twice when creating folder

IMAP: Headers never updated

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	# Take care of prefix here and ensure create_folder() does not mess
	# with it.
	folder_list = self._processor.path_ensure_prefix(maildir, sep=sep)
	maildir = self._processor.list_path(folder_list, sep=sep)

	def _copy(self, maildir, **kwargs):
	if kwargs.get('create', False) and not os.path.isdir(maildir):
	try:
	self._processor.create_folder(maildir, **kwargs)