njfritter / myersbriggsnlpanalysis Goto Github PK

To Do:

• Make code to generate various actionable insights (exploratory analysis)
  • Hashtag frequency
  • Url frequency
  • N-gram frequency
  • Mention frequency
• Be able to get counts by user, average number per personality type, etc.

Need to generate this data so I can move on with the project; currently a bottleneck right now as the tokenization process takes a while

Had Raul review my project and he absolutely grilled me on stuff. So here's what I need to do:

• Redo formatting of scripts; some have 2 spaces as indent
• Remove all extraneous packages (i.e. using both pandas and csv modules)
• Reformat helper_functions, remove unnecessary ones, add purpose and inputs
• Redo tokenization section to account for special twitter stuffs

Will add on as I find more

To Do:

• Proper data subsetting and generation
• Fix plotting issues with type and word frequency using built in plotting methods
• Fix up helper_functions.py script as necessary
• Make function to return rows containing a specific string pattern for frequency analysis

Currently the main README contains all the results/information; would like to divide this up so the main one only has key takeaways.

Was thinking of moving results to the "scripts" folder, as well as moving information on the data to the "data" folder.

Will update if I have more ideas on this