Coder Social home page Coder Social logo

Comments (10)

Mincka avatar Mincka commented on May 20, 2024

Hi,

Interesting request. I think it could be possible. May I ask for which kind of purpose you would need to know the status of the last messages you sent?

The only use case I am able to think about is some sort of mass messaging where you would like to know which users have open / seen the message.

from dmarchiver.

mmosleh avatar mmosleh commented on May 20, 2024

Hi,
Thank you for your reply.

One use case is mass messaging, another is automated messaging when you want to know if the message is seen and then take some action (e.g., follow up, send another one...), and another is to have a more complete archive of your personal messages with their seen status.

I already gave it a try using your well-written code. Using the following line after reading the time stamp of the message.

ReadReceipt = dm_footer[0].cssselect('span.DMReadReceipt-statusView')

Although this variable returns the correct 'seen' or 'sent' status on the browser (using inspect), it always returns 'sent' using the script even when the message is actually seen!

I was wondering if the content receive by the script is slightly different than that of the browser. I tried to open 'https://twitter.com/messages/with/conversation?id=....' with the browser to see if the exact same content in the browser but it didn't work.

Thanks!

from dmarchiver.

Mincka avatar Mincka commented on May 20, 2024

Indeed, it seems that this is the associated text which is "Sent" by default. If you click on the blue tick, the DOM is updated with the text "Seen by everyone" / "Seen" and the "is-expanded" CSS class is added so the text is displayed.

It looks like you have to test for the presence of the CSS classes "is-seen" or "is-seenAnimated" with something like dm_footer[0].cssselect('span.is-seen'). If the array is not empty, then the message is seen (or partially seen in a group?), otherwise it is sent (or partially seen in a group?).

Seen message:

  <span class="DMReadReceipt-check is-seen is-seenAnimated">
    <span class="Icon Icon--checkLight"></span>
  </span>

Sent message:

  <span class="DMReadReceipt-check">
    <span class="Icon Icon--checkLight"></span>
  </span>

I did not see other places that could give an indication about the message status. I did not check how the JS is able to update the status. Maybe it keeps a client-side status or maybe it also relies on the detection of the CSS class. 😀

from dmarchiver.

mmosleh avatar mmosleh commented on May 20, 2024

Awesome! Thanks a lot. This is exactly what I wanted. But now ran into another problem. The original script sends back the following error:

Expecting value: line 1 column 1 (char 0)

I looked at "response.content" and it returns:

b'<!DOCTYPE html>\n<html lang="en">\n<head>\n <meta charset="utf-8" />\n <title>Twitter</title>\n <style>\n body {\n background-color: #ffffff;\n }\n .link {\n color: #1da1f2;\n font-family: sans-serif;\n }\n </style>\n</head>\n<body>\n <noscript>\n \n <center><a href="/messages" class="link">Continue</a></center>\n </noscript>\n <script nonce="lYnW6pOTpZX+BQjh4Sh1/Q==">\n \n document.cookie = "app_shell_visited=1;domain=.twitter.com;path=/;max-age=30";\n \n location.replace(location.href);\n </script>\n</body>\n</html>\n'

I used the exact script from github which was working fine

Thanks!

from dmarchiver.

Mincka avatar Mincka commented on May 20, 2024

Hum, it was working previously? It looks like a recent update to prevent parsing.

Anyway, I have a workaround. 😀

Add these few additional headers and it should make the trick. 😉
ae151b0#diff-ebdd7cd1a345766481630db935d19b6b

from dmarchiver.

mmosleh avatar mmosleh commented on May 20, 2024

Thanks a lot for updating the script and being so supportive. But still the script cannot find anything related to "seen" status. For example, the content printed by the following test line doesn't have the word "seen" for a seen message.

response=_session.get(_twitter_base_url+'/messages/with/conversation?id=xxxx',headers=_ajax_headers )
print (response.content)

While in browser without clicking on the message class="DMReadReceipt-check is-seen is-seenAnimated" exists for seen messages.

Thanks again!

from dmarchiver.

Mincka avatar Mincka commented on May 20, 2024

You are right. I tried to find how the value was retrieved but I did not find yet.

I'm not sure this is something available in the HTML code of the tweet, because even using a browser, the "is-seen" class is not yet present in the HTTP response. It looks like it relies on another request to get this information and the DOM is updated afterward in JS.

from dmarchiver.

mmosleh avatar mmosleh commented on May 20, 2024

Thanks a lot for looking into this. Unfortunately this is not available through the twitter API either. Hope I could find a solution soon.

Thanks again!

from dmarchiver.

mmosleh avatar mmosleh commented on May 20, 2024

So I ended up resolving this using Selenium and it worked fine. Thanks for being so supportive.

from dmarchiver.

Mincka avatar Mincka commented on May 20, 2024

You're welcome. Thank you for the update. I also had to use Selenium for another parsing project. It's a bit slower but more reliable. Good luck for your project.

from dmarchiver.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.