rehamaltamimi / wikipedia-map-reduce Goto Github PK

GraphParser's readGraph() method returns an ArrayList of Links.  Is the Graph 
class not necessary?

getContributors() uses a HashSet to weed out duplicate contributors, but it 
doesn't add all, even when they are non-unique.

Users info can be gathered from <revision> data or from <article> data.  If 
it's from <revision> data, we have either a uID/Name pair or just an IP.  If 
it comes from <article> data, we have a Name or IP and an aID, but no uID.

Need to refactor code to use new User subclasses.

Check all single-argument User() instantiations for validity.

Should be refactored to a wrapper class for a list of edges.  May not be 
needed in the end anyways, but it should still happen. 

One of the main issues with user-based analysis of Wikipedia is that it's hard 
to capture all user information.  In our system, if users don't have IDs 
attached to the, we have to ignore them; furthermore, we can't get information 
on a registered user in the first place unless they edit a page.  We can't rely 
on User: namespace pages, because not every user's User: page is defined.

This issue is a first gathering place for thoughts on how best to handle this 
problem.  If some sort of consensus starts to be had, we'll move everything 
over to a Wiki page.

Refactor calls to Revision() to include new boolean parameter.

Also create isVandalism handling in PageParser.

Each time a user is created, it can be done using `User(String id)` or 
`User(String name, String id)`.  For now, if the user only has an IP, not a 
uID, that is treated as the users ID.  Users should have either a uID/Name 
pair *or* an IP to be identified by.

Can have User superclass with RegisteredUser and AnonymousUser subclasses.