nmoya / whatsapp-parser

Parser to the What's App log file.

whatsapp-parser

Parser to the What's App log file. Feel free to fork and modify :-)

Dependencies

• Python 2.*

Input

A Whatsapp chat history file and an optional list of regular expression patterns.

Output

• Several statistics about the chat history and the frequency of each pattern.
• A JSON file is created inside /logs/ folder

Usage Steps

• Export your Whatsapp chat history. (http://www.whatsapp.com/faq/general/23753886)
• Clone this repository.
• For simplicity, put the chat history file in the same folder of the cloned repository.
• Open your terminal and execute:

python chat.py -f chat.txt -p WhatsApp -n "Nikolas Moya" -r [regex. patterns]

Example

python chat.py -f chat.txt -p WhatsApp -n "Nikolas Moya" -r sleepy yes

• Expected output:

--PROPORTIONS
 total_messages -> 32773
 total_words -> 172760
 total_chars -> 899435
 messages
   Marilia Ferreira -> 16234
   ‪Nikolas Moya -> 16539
 words
   ‪Nikolas Moya -> 72797
   Marilia Ferreira -> 99963
 chars
   ‪Nikolas Moya -> 379642
   Marilia Ferreira -> 519793

--SHIFTS
 morning -> 2581
 afternoon -> 6593
 evening -> 9736
 latenight -> 13863

--WEEKDAY
 Wednesday -> 4155
 Thursday -> 4319
 Monday -> 4638
 Saturday -> 4650
 Tuesday -> 4730
 Friday -> 4874
 Sunday -> 5407

--AVERAGE MESSAGE LENGTH
 ‪Nikolas Moya -> 4.40153576395
 Marilia Ferreira -> 6.1576321301

--PATTERNS
  sleepy
   ‪Nikolas Moya -> 10
   Marilia Ferreira -> 5
 yes
   ‪Nikolas Moya -> 98
   Marilia Ferreira -> 87

--TOP 15 MOST USED WORDS (length >= 3)
não
amor
mais
hahaha
sim
muito
isso
amo
para
tudo
hahah
acho
agora
aqui
tava
./logs/Chat.txt.json

whatsapp-parser's Issues

Danish

false negatives in parsing messages in tests

hey,
the program wasn't working for me until i commented this loop
https://github.com/nmoya/whatsapp-parser/blob/master/parsers/whatsapp.py#L23

it gives false negatives when running with tests/testChat3.txt
can you please fix the loop. i think the len() comparison is not quite right.

thanks.

IndexError: list index out of range

# python wp_chat.py -f /abc/xpto.txt -n xpto -p WhatsApp -o xpto.json
Traceback (most recent call last):
  File "wp_chat.py", line 239, in <module>
    main()
  File "wp_chat.py", line 234, in main
    c.all_features(pattern_list=args['regexes'])
  File "wp_chat.py", line 110, in all_features
    self.features.compute_response_time_and_burst(self.messages, self.root, self.senders, initiation_thrs, burst_thrs, response_thrs)
  File "/root/whatsapp-parser/wp_parser/ChatFeatures.py", line 27, in compute_response_time_and_burst
    t0 = list_of_messages[0].datetime_obj
IndexError: list index out of range

Improve unit tests

Now that more people are collaborating, we should improve our unit tests to avoid regression.

update readme to include the new improvements

Is there a way to calculate response times

Hi, I found this parser very useful, I am doing a school project to analyze chat data. Thank you for contributing.

One thing I'd like to do is see the difference in response time between the root and their contact (much in the same way you calculate messages, characters, etc.)

Is there a way you recommend to add this functionality?

pip support

Hey,
I've finally found time to finish the work on making a pip installable package for this project.

I was hoping for you to test the pip package locally and get it merged soon.
There is some meta-data such as author name, email id which is required in setup.py Then you can edit some of the meta info in setup.py so we can publish this package on pypi easily.

Can't open 'chat.py'

Hi, I came across the below error when trying to run the program. It seems as though python is looking for 'chat.py' which doesn't seem to exist inside any of the directories..

Jacks-MBP-2:whatsapp-parser jackedmond$ python chat.py -f chat.txt -p WhatsApp -n "Nikolas Moya" -r sleepy yes
python: can't open file 'chat.py': [Errno 2] No such file or directory

Apologies if this is a stupid question.. I'm just getting started

Invalid date format?

Hi!

I'm using your library for a project I'm working on and I've been getting the following error when I try to use it:

ValueError: time data '2014-12-25' does not match format '%y-%m-%d'

Seeing as the format should match, I'm wondering if you could offer a fix for this.

Thanks!

Edit: I'm not quite sure where it is getting that format from at all considering in the log file it is: "12/25/14 2:57:18 PM:"

Weird output

Not working, gives weird output, show full sentences instead of word count.

and then dies with the following output:
--SHIFTS
evening -> 0
afternoon -> 4
latenight -> 11
morning -> 15

--WEEKDAY
Traceback (most recent call last):
File "main.py", line 225, in
main()
File "main.py", line 199, in main
output["weekdays"] = c.count_messages_per_weekday()
File "main.py", line 59, in count_messages_per_weekday
weekday = date.date_to_weekday(parsed_date)
NameError: global name 'date' is not defined

I am getting an error in partition command

Hi

I am getting the following message in the partition line of whatsapp.py file:

Traceback (most recent call last):
File "chat.py", line 239, in
main()
File "chat.py", line 227, in main
c.parse_messages()
File "chat.py", line 71, in parse_messages
self.senders, self.messages = p.parse()
File "C:\projetos\whatsapp-parser\parsers\whatsapp.py", line 19, in parse
msg_date, msg = l.partition(": ")
TypeError: a bytes-like object is required, not 'str'

Default path for storing basic_stats.json gives error in case logs directory doesn't exist

Got the following error in the end when running testChat2.txt file.

the hardcoded default JSON output file name: ./logs/basic_stats.json' gives error in case /logs folder is not present inside the root directory.

Traceback (most recent call last):
  File "Chat.py", line 227, in <module>
    c.save_features(args["output"])
  File "Chat.py", line 196, in save_features
    arq = open(output_name, "w")
IOError: [Errno 2] No such file or directory: './logs/basic_stats.json'

What does the -n "Nikolas Moya" parameter means

Hello

I would like to know what does the -n "Nikolas Moya" parameter means.

Is this code operational? I am getting a lot of errors even converting code to python 3.

Error in whatsapp.py

Hi, I came across this error when trying to run the program. I'm fairly new to Python so I only understand some parts of your code.

File "C:\Users\Darryl\Downloads\whatsapp-parser-master\wp_parser\parsers\whatsapp.py", line 19, in parse
    msg_date, sep, msg = l.partition(": ")
TypeError: a bytes-like object is required, not 'str'

Your help will be much appreciated :)

Where is the chat.py file

Hello everyone

Where is the chat.py file in a way I could execute the command line?

python chat.py -f myChat.txt -p WhatsApp -n "Nikolas Moya" -r sleepy yes

python3 support

since, the project has no dependencies, it should be relatively easy to extend support for python3.

something like 2to3 can come in really handy. https://docs.python.org/2/library/2to3.html