Comments (10)
Hi,
Interesting request. I think it could be possible. May I ask for which kind of purpose you would need to know the status of the last messages you sent?
The only use case I am able to think about is some sort of mass messaging where you would like to know which users have open / seen the message.
from dmarchiver.
Hi,
Thank you for your reply.
One use case is mass messaging, another is automated messaging when you want to know if the message is seen and then take some action (e.g., follow up, send another one...), and another is to have a more complete archive of your personal messages with their seen status.
I already gave it a try using your well-written code. Using the following line after reading the time stamp of the message.
ReadReceipt = dm_footer[0].cssselect('span.DMReadReceipt-statusView')
Although this variable returns the correct 'seen' or 'sent' status on the browser (using inspect), it always returns 'sent' using the script even when the message is actually seen!
I was wondering if the content receive by the script is slightly different than that of the browser. I tried to open 'https://twitter.com/messages/with/conversation?id=....' with the browser to see if the exact same content in the browser but it didn't work.
Thanks!
from dmarchiver.
Indeed, it seems that this is the associated text which is "Sent" by default. If you click on the blue tick, the DOM is updated with the text "Seen by everyone" / "Seen" and the "is-expanded" CSS class is added so the text is displayed.
It looks like you have to test for the presence of the CSS classes "is-seen" or "is-seenAnimated" with something like dm_footer[0].cssselect('span.is-seen')
. If the array is not empty, then the message is seen (or partially seen in a group?), otherwise it is sent (or partially seen in a group?).
Seen message:
<span class="DMReadReceipt-check is-seen is-seenAnimated">
<span class="Icon Icon--checkLight"></span>
</span>
Sent message:
<span class="DMReadReceipt-check">
<span class="Icon Icon--checkLight"></span>
</span>
I did not see other places that could give an indication about the message status. I did not check how the JS is able to update the status. Maybe it keeps a client-side status or maybe it also relies on the detection of the CSS class. 😀
from dmarchiver.
Awesome! Thanks a lot. This is exactly what I wanted. But now ran into another problem. The original script sends back the following error:
Expecting value: line 1 column 1 (char 0)
I looked at "response.content" and it returns:
b'<!DOCTYPE html>\n<html lang="en">\n<head>\n <meta charset="utf-8" />\n <title>Twitter</title>\n <style>\n body {\n background-color: #ffffff;\n }\n .link {\n color: #1da1f2;\n font-family: sans-serif;\n }\n </style>\n</head>\n<body>\n <noscript>\n \n <center><a href="/messages" class="link">Continue</a></center>\n </noscript>\n <script nonce="lYnW6pOTpZX+BQjh4Sh1/Q==">\n \n document.cookie = "app_shell_visited=1;domain=.twitter.com;path=/;max-age=30";\n \n location.replace(location.href);\n </script>\n</body>\n</html>\n'
I used the exact script from github which was working fine
Thanks!
from dmarchiver.
Hum, it was working previously? It looks like a recent update to prevent parsing.
Anyway, I have a workaround. 😀
Add these few additional headers and it should make the trick. 😉
ae151b0#diff-ebdd7cd1a345766481630db935d19b6b
from dmarchiver.
Thanks a lot for updating the script and being so supportive. But still the script cannot find anything related to "seen" status. For example, the content printed by the following test line doesn't have the word "seen" for a seen message.
response=_session.get(_twitter_base_url+'/messages/with/conversation?id=xxxx',headers=_ajax_headers )
print (response.content)
While in browser without clicking on the message class="DMReadReceipt-check is-seen is-seenAnimated" exists for seen messages.
Thanks again!
from dmarchiver.
You are right. I tried to find how the value was retrieved but I did not find yet.
I'm not sure this is something available in the HTML code of the tweet, because even using a browser, the "is-seen" class is not yet present in the HTTP response. It looks like it relies on another request to get this information and the DOM is updated afterward in JS.
from dmarchiver.
Thanks a lot for looking into this. Unfortunately this is not available through the twitter API either. Hope I could find a solution soon.
Thanks again!
from dmarchiver.
So I ended up resolving this using Selenium and it worked fine. Thanks for being so supportive.
from dmarchiver.
You're welcome. Thank you for the update. I also had to use Selenium for another parsing project. It's a bit slower but more reliable. Good luck for your project.
from dmarchiver.
Related Issues (20)
- add exception to attribute error HOT 5
- Windows detects Trojan:Win32/Ditertag.B HOT 2
- App closes when trying to use it + account suspension HOT 4
- -dv Videos won't download HOT 4
- Problems With LXML install on arch HOT 4
- Total newbie here. How do i run the $ commands?
- Kinda New. Just Need General Help?
- "Connection reset by peer" probably caused by legacy TLS HOT 4
- Add command to split in several files
- No longer working on Mac HOT 2
- Way to limit amount of dms pulled? HOT 1
- index out of range HOT 6
- image link is empty HOT 3
- Request: Optional Timestamps
- Conversation id error HOT 3
- Support to get reactions to message HOT 8
- A nit in the README HOT 1
- DMArchiver is broken HOT 24
- Dependabot couldn't authenticate with https://pypi.python.org/simple/
- wrong repo
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dmarchiver.