scott-saunders / covid_data_bot Goto Github PK

The most common error occurs when the bot finds a tweet to reply to and then it tries to post a response with rtweet::post_tweet(status = 'some text', media = 'some_plot.png')

This step runs for minutes before failing like so:

curl::curl_fetch_memory(url, handle = handle) : 
  Empty reply from server

Others are having similar issues as detailed in this issue.

My best guess is that this is an issue with twitter itself. Here's what I know:

1. This error can occur locally as well as inside the github action. So far it seems like when the cloud has the issue, so does my local version.
2. This issue specifically occurs when trying to post media. Posting the same tweet with no media and only text from my local version rtweet::post_tweet('some text') runs instantly and posts correctly.
3. This issue seems to occur randomly and may not be a coding error, because the same code will repeatedly fail on the github action and then it will run perfectly fine.

Because of this behavior, I think this is an issue with twitter. Possibly this has to do with some media posting permissions or twitter's malicious / spam restrictions.

At one point I thought this issue might be explained by this restriction from twitter's POST function:

For each update attempt, the update text is compared with the authenticating user's recent Tweets. Any attempt that would result in duplication will be blocked, resulting in a 403 error. A user cannot submit the same status twice in a row.

Perhaps in rare cases the bot tries to post duplicate tweets. However, I tested this, and there is a different error thrown, not the empty reply error. Therefore I do not believe this is the issue.

This error is the single biggest problem limiting the reliability of covid data bot, so any help or insight anyone can offer would be much appreciated! Thanks

@scott-saunders putting in a thread here rather than the one on londonmapbot discussion.

You mentioned the setup of Actions taking a while, I've noticed this a lot with the setup-r helper in one of my repos. I happened to discover that macos-latest runner has recently been updated to have R pre-installed so you no longer need the line calling the setup-r action. You'll need to specify the repos argument of install.packages() as the runner doesn't set a repository in the global options. My narrowbotR repo uses the pre-installed version of R.

When looking at your workflow I also noticed that you set yourself as the author of the commits. Instead, I'd suggest setting the author as the GitHub actions bot, this will make it easier in your git log to identify any commits that are the result of an Action, it will also mean that if anyone forks your repo that if they activate the Action without amending the code then those commits aren't attributed to you. Example in one of my repos here.