AttributeError: 'NoneType' object has no attribute 'get_newest_date' about alot

[pazz]

hmpf. for some reason my comment did not show up here.. might have to do with me being in a train. anyhow:

I can't reproduce this one, there must be something special with a particular thread in your initial search result.
Could you add a ui.logger.debug(tid) just before line 44 in widgets.py and look for particularities in the
thread?
thanks,
/p

[pazz]

Traceback (most recent call last):
  File "/usr/local/bin/alot", line 20, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/alot/init.py", line 114, in main
    args.colours,
  File "/usr/local/lib/python2.7/dist-packages/alot/ui.py", line 79, in __init__
    self.mainloop.run()
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 274, in run
    self.screen.run_wrapper(self._run)
  File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py", line 229, in run_wrapper
    return fn()
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 307, in _run
    self.event_loop.run()
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 1159, in wrapper
    rval = f(*args,**kargs)
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 358, in _update
    self.process_input(keys)
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 447, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/usr/local/lib/python2.7/dist-packages/alot/ui.py", line 44, in keypress
    urwid.Frame.keypress(self, size, key)
  File "/usr/local/lib/python2.7/dist-packages/urwid/container.py", line 641, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/usr/local/lib/python2.7/dist-packages/alot/buffer.py", line 53, in keypress
    return self.body.keypress(size, key)
  File "/usr/local/lib/python2.7/dist-packages/urwid/listbox.py", line 776, in keypress
    return self._keypress_page_down((maxcol, maxrow))
  File "/usr/local/lib/python2.7/dist-packages/urwid/listbox.py", line 1177, in _keypress_page_down
    widget, pos = self.body.get_next(pos)
  File "/usr/local/lib/python2.7/dist-packages/alot/walker.py", line 39, in get_next
    return self._get_at_pos(start_from + 1)
  File "/usr/local/lib/python2.7/dist-packages/alot/walker.py", line 61, in _get_at_pos
    widget = self._get_next_item()
  File "/usr/local/lib/python2.7/dist-packages/alot/walker.py", line 70, in _get_next_item
    next_widget = self.containerclass(next_obj, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/alot/widgets.py", line 38, in __init__
    self.rebuild()
  File "/usr/local/lib/python2.7/dist-packages/alot/widgets.py", line 45, in rebuild
    newest = self.thread.get_newest_date()
AttributeError: 'NoneType' object has no attribute 'get_newest_date'

[lmacken]

This issue is still preventing me from using alot. I cannot figure out how to view the thread that is causing problems. I'm running 'notmuch show ' and it's not displaying anything. Any ideas?

[lmacken]

Ah, I'm starting to track this issue down now...

[lmacken]

Ok, so the core of the problem here is with emails that have timestamps that are in the future.

Traceback (most recent call last):
  File "/home/lmacken/code/github.org/alot/alot/db.py", line 160, in get_thread
    return Thread(self, query.search_threads().next())
  File "/home/lmacken/code/github.org/alot/alot/db.py", line 200, in __init__
    self.refresh(thread)
  File "/home/lmacken/code/github.org/alot/alot/db.py", line 211, in refresh
    self._oldest_date = datetime.fromtimestamp(ts)
ValueError: year is out of range

In notmuch.vim, this message timestamp is displayed as (the future).

[pazz]

i knew the problem would solve itselve if we wait long enough :)
Seriously, this looks rather ugly. First of: I can't see the connection between your two tracebacks:
if alot.db.Thread.i__init__ raises a ValueError, the code in IteratorWalkter should pass that along as its
only testing for StopIteration. (thats _get_next_item). It should rather fail with
'next_widget' referenced before assignment than returning None.

Regarding the timestamp issue: Could you provide such a problematic timestamp?
the datetime.fromtimestamp docs say

Return the local date corresponding to the POSIX timestamp, such as is returned by time.time(). This may raise ValueError, 
if the timestamp is out of the range of values supported by the platform C localtime() function. 
It’s common for this to be restricted to years from 1970 through 2038.

but on my system (ubuntu 11.04) i get

In [42]: datetime.datetime.fromtimestamp(2217928174)
Out[42]: datetime.datetime(2040, 4, 13, 12, 9, 34)

Seemingly i can go as high up as the year 10000 before i get these errors.
Maybe this has somethig to do with me being on 64bit?
notmuch.thread.get_oldest_timestamp's docs say:

Return type:    c_unit64

So: a fix should look like this:

1. find out your local maximal value for timestamps
2. catch ValueErrors, and set the datetimeobject to represent your maximal date.

This way we won't have to hardcode that value. moreover, the threadline widget could test for exactly
this value and display "invalid" or so instead.

[pazz]

datetime doesnt seem to use time.localtime, as this works perfectly well here:

In [61]: time.localtime(503017928174)
Out[61]: time.struct_time(tm_year=17910, tm_mon=1, tm_mday=2, tm_hour=18, tm_min=16, tm_sec=14, tm_wday=6, tm_yday=2, tm_isdst=0)

[pazz]

datetime.datetime.max might be our friend

[lmacken]

Ok, so my last traceback is caused by me adding a 'logging.exception' when the Thread.init fails, so that is probably why you were confused.

When running 'notmuch show thread:ID', the date in the header is as follows: Date: 1314866652
Yet the thread.get_oldest_date() is returning 41493121656249600

The following patch works around the issue, but I have a feeling this is a bug somewhere else.

--- a/alot/db.py
+++ b/alot/db.py
@@ -204,8 +204,14 @@ class Thread(object):
         self._authors = thread.get_authors()
         self._subject = thread.get_subject()
         ts = thread.get_oldest_date()
-        self._oldest_date = datetime.fromtimestamp(ts)
-        self._newest_date = datetime.fromtimestamp(thread.get_newest_date())
+        try:
+            self._oldest_date = datetime.fromtimestamp(ts)
+        except ValueError: # year is out of range
+            self._oldest_date = datetime.min
+        try:
+            self._newest_date = datetime.fromtimestamp(thread.get_newest_date())
+        except ValueError: # year is out of range
+            self._newest_date = datetime.max
         self._tags = set([t for t in thread.get_tags()])
         self._messages = {}  # this maps messages to its children
         self._toplevel_messages = []
diff --git a/alot/message.py b/alot/message.py
index e00d62d..2fb41e0 100644
--- a/alot/message.py
+++ b/alot/message.py
@@ -43,7 +43,10 @@ class Message(object):
         self._id = msg.get_message_id()
         self._thread_id = msg.get_thread_id()
         self._thread = thread
-        self._datetime = datetime.fromtimestamp(msg.get_date())
+        try:
+            self._datetime = datetime.fromtimestamp(msg.get_date())
+        except ValueError: # year is out of range
+            self._datetime = datetime.max
         self._filename = msg.get_filename()
         self._from = msg.get_header('From')
         self._email = None  # will be read upon first use

[pazz]

@logging.exception: ahyes, ok makes sense, thx.

quick question: does fromtimestamp also throw a ValueError if the timestamp is below datetime.min ? if so,
returnung max here is not what we want.
It might be a better idea to jyst set the value to None, indicating 'invalid' (and adjusting the widgets of course).
thoughts?

Unfortunately, this might well be an error in the notmuch lib.

One more thing: could you have a look at the source of that message and post the date-header here?
That way, I could write this into a dummy mail for debugging. In case the timestamp is well formated,
this issue looks like a problem in the bindings/lib.

Is it only one single mail that causes the problem?

[pazz]

oh wait: Date: 1314866652 is the raw string in the mail? thats not valid!

[lmacken]

Yup, it's definitely not valid, and the spam checker headers in that message also mention that it is invalid.

[pazz]

ok: i'd favor something like this:

• set the datetime value to None,
• adjust the widgets that display this (Threadline, MessageHeader I believe)

Moreover we must take some care when designing the new™ Messagelist buffer that will replace threadbuffer
as to None values in dates when sorting.

It would be cool if we had some sort of validity checker for header (is there something in stdlib?). Then we could
highlight invalid headerlines in the MessageHeaderWidget.

I don't have much time atm, but will hack on a fix here soon if you don't beat me to it :)

[pazz]

found the time :)

this should fix the issue for you. Strangely, I still cant reproduce it as a call to notmuch.database.thread.get_newest_date() on a thread with your datestring
will return datetime.datetime(1970, 1, 1, 0, 59, 59) for me.