haystack / feedme-rss Goto Github PK

A lot of the code in bookmarklet.py builds html to print to the user.  This
HTML should live in a template

allow people to maintain recommendation groups that are recommended and
everyone in the group then receives an email

hypothesis: the system gets slower with the number of people you've
recommended to in the past

Type the email address and hit 'send' -- it doesn't fill in automatically.
 Should this be default behavior, or is it dangerous?

The bookmarklet overwrites greasemonkey's xmlhttprequest to use jquery, but
the callback in the greasemonkey script expects different arguments.  wrap
the ajax response from jquery so that greasemonkey doesn't get confused.

Receivers should get better default passwords than their email address:)

allow users to receive plain-text emails instead of rich-text ones

Our emails should include the RSS feed title.  This involves fixing the
Greasemonkey script to correctly mine that information.

currently the files on groups.csail... are just hanging around without
being versioned.

This at least happens in the unsubscribe email (in subscription.py), but we
shouldn't hard-code our domain/port in the code.  We should be able to ask
Django for this.

Sometimes recommendations come back with an AJAX error.  One such entry on
the slashdot feed that does this is

http://news.slashdot.org/article.pl?sid=09/05/07/1432253&from=rss

In share.py, we call nltk.clean_html on the text, which actually removes
the hrefs.  We should make sure that the links make it into the plaintext
email.

put an explicit checked checkbox near each person that's been selected for
recommmendation

If you click on a label instead of an individual feed, it thinks you're
looking at an oddly-formatted (but correct) RSS feed.  E.g.,
?tab=my#stream/user%2F12162841236402604720%2Flabel%2Fdesign

We need to fix the Greasemonkey script to realize when this is happening
and harvest instead from the "from" line in the posts.

rather than just having a send and like button, we should remove the like
button, and have a now and a later button to say whether to digest or
immediately share explicitly.

right now we say a person who you added manually has 0 recommendations
today.  if the person isn't on the recommended list but has had stuff
recommended to them today, and i just typed their name in to find them
again, we still say 0.  We should think of the best way to (prefetch?)
this data.

By default, recommended receivers are white.

If they are selected to become recipients, they should turn blue.

Once the sender clicks "Now" or "Later," they should turn from blue to
orange to grey.

In the future, if you revisit a post, the previous recipients should be
grey.  Grey recipients won't receive an email if the sender clicks "Now" or
"Later."  If the sender clicks on a grey person (just like on a person in
white), the user will become blue, as an active recipient of the next "Now"
or "Later" button press.

I think this makes sense -- they haven't actually received the Later
suggestions yet.

Create a new user.  Log in to feedme, and go to GReader.  Emptiness fills
your soul.

The reason is that wait-for-suggestions never gets removed if the
recommendations list has size 0.  We could fix this by moving:
$(".wait-for-suggestions", postToPopulate).removeClass("wait-for-suggestions")
outside the if(start_person < min_length), but then you get the "more"
button showing up when it shouldn't.

The slow portion of the recommendation engine is hitting the database to
retrieve the term vector of each friend.  We can instead cache these, since
the term vector is retrieved for every post you read, which has good
temporal locality during a greader session.

The task would be to:

set up memcached
check memcached for a friend's term vector
if the term vector exists, unpickle the list and use it
if the term vector does not exist, retrieve it from the DB, pickle it,
insert it into memcache, and use it for calculating cosine distance.

we need to escape any javascript in the code presented in the bookmarklet
to prevent XSS attacks.  example of javascript making it into the
bookmarklet: load the issues page for feedme-rss, which has JS in the body.
 Enable the bookmarklet, and look at the iframe's source.  You will find JS
throughout the code.

so we can actually test on the dev db

I think they're starting with ://www.google.com

do a UI lookover for the emails we send

allow users to share arbitrary webpages rather than just reader posts

either implement auto-update for greasemonkey script upon loading google
reader, or change to a firefox plugin.

Receivers should be able to say whether they like/dislike a shared item to
give sharers feedback.  The mechanism for doing this (how to notify
sharers) is unclear.  We might want to reverse engineer how google calendar
does this when you get a meeting invite in gmail where you can click
"Yes/No/Maybe"

Also do a UI pass on the accounts pages

Can't duplicate the bug reliably though.  Needs more testing.

COMMENT OUT lines 611-612
            // TODO: This is a horrible hack to replicate the functionality
of toggle_friend...
            $(".feedme-person:last", context).toggleClass("feedme-toggle");
            $(".feedme-controls", context).slideDown("normal");

UNCOMMENT line 619           
            // Because the following call generates a "Component is not
available" error when called!
            // It's called out of the jQuery code, apparently because the
elements in a GM script are
            // in an XPCNativeWrapper, and jQuery doesn't deal with this. 
A similar issue:
            //
http://stackoverflow.com/questions/564342/jquery-ui-dialog-throw-errors-when-inv
oked-from-greasemonkey
            // This seemed to break in FF 3.5 for me.
            //$(".feedme-person:last", context).click();

this won't block all sharing...we just won't recommend them to people. 
this means the unsubscribe text should tell them to contact the person
directly to explicitly stop emailing them otherwise

What steps will reproduce the problem?
1. Go to http://feedme.csail.mit.edu:8000/OMNOMNOMNOMNOM

Allow receivers to set 3-way choice: normal, digest-only, and stop
recommending.

[ How do you want to get emails?
- Individual emails and digest emails (as the sender recommends)
- Only digest emails
]
[ Do you want to be recommended for receiving posts?  If you say no, you
will only receive e-mails if a friend decides to add you directly; we'll
never suggest you as a possible recipient.
- Yes
- No, bitchez
]

it's kind of jarring that the comment box appears and disappears
immediately.  I think jQuery can smoothly scroll it for us.

- UI in Greasemonkey
- Django logic

Should also autocomplete out of people forwarded to previously.

Need to either use a queue or create a dirty bit.  This seems to happen if
the recipient has had a lot of stuff forwarded to them.

they currently stay orange, which means it's unclear whether the person
will receive a second  email if you hit send again.  also---the person
should NOT receive a second email if you hit send again, since they are now
white

Configuration parameters we'd like:
do you wan to be cc:'d when you share with people, or not?
how many recommendations per line (currently it's always 3, but maybe wide
screen folks want more)

Loading indicator needs to be added.  Except, iframe onload is returning
too early to get this to work?

If a URL is longer than 200 characters, django currently throws an
exception.  This is because URLFields have a 200 character limit.  We can
increase this, but mysql4 limits varchars to 255 characters.

Possible solutions:
1) URLField->TextField
2) Upgrade to mysql5
3) Use postgres 8.3

Some non-feed websites generate horrendous html.  we need to handle
poorly-built DOMs so that we can strip html tags from them.

Currently, the comment box pops up prominently when you select someone to
recommend an article to.  This implies you should comment.

We should (at least) put a greyed-out "(optional comments)" in the comment
box to suggest it's not required.  Additionally, we might want to make the
user explicitly push a button to open the comment box, so that they are
actively aware it's an option.

the UI should include a checkbox near each "Send" button.  The default
value for the checkbox should either be stored in greasemonkey or the
server side

(copied from email conversation)

The first time a receiver gets an email from FeedMe, prepend a message
highlighting the ability to opt out or change subscription settings.

Also, fill in the button with a red color if the person has asked not to
receive posts.

Or make visible their subscription settings, which is a signal of how much
they want to receive.

What steps will reproduce the problem?
1. Try running attached test runner

It dies if two different feeds have the same post URL.  This is bad because
whoever is the first to read slashdot using a weird feed (e.g.
/www.google.com/reader/view/#stream/user%xxxxxxxxxx%2Fstate%2Fcom.google%2Freadi
ng-list
will kill it for everyone else.

Seems like we need to make a post jointly unique on feed and url, not just
url.  How do we do this?

have an expand button, and get the next three people.

the django endpoint should initially just return EVERYONE that you've
shared with in the past, but in sorted order, to facilitate getting the
next 3 people.

ensure that users know what each button will do with alt text

consider the case where we start from scratch in how to properly calculate idf.

We should ask people for their names upon signup, and then in
recommendation emails, send the receivers "John Doe thought you might like
this..."

We might also want to collect gender, so that we can say "he" or "she" if
they identify themselves that way, but we can cleverly get around this with
the English language.