tweetlord is an open source Twitter profile dumper (downloader) with the on-the-fly account swapping support for bypassing the rate limit restrictions. It is written in Python 3, uses the Twitter API and generates .xlsx
files at the output containing comprehensive information about the given profile.
++++ | Friends | Followers | Favorites | Timeline |
---|---|---|---|---|
Arg | 300 | 10000 | 2000 | 500 |
Actual | 74 | 8082 | 1637 | 326 |
From the table above it is seen why the progress bars in the 1st screenshot were not filled to the end: the actual number of items in each of the sections is less than it was specified in the arguments when running the tool. It's not a bug but a feature ๐ (one API request returns no more than 200 items so the pbar step = 200, btw).
- python3.x (or newer) interpreter
tweetlord makes use of the following external modules:
Resolve all Python dependencies with the pip
one-liner:
$ python3 -m pip install -r requirements.txt
The order of use is pretty straightforward:
โ ๏ธ First, you want to set your API keys (could be found here) in the credentials.py file for every Twitter account you want to involve in the procedure. It is needless to say that the more accounts you specify, the faster the dumping process will be (but nevertheless you can specify only one account). If a mistake is made when filling the credentials, the script will terminate with an unhandled tweepy exception, so keep that in mind.- Hmm... that's actually it! Feel free to run tweetlord as shown in the next section.
tweetlord.py [-h] (-u USER | -l) [-fr FRIENDS] [-fo FOLLOWERS]
[-fa FAVORITES] [-ti TIMELINE] [-o OUTPUT] [-w] [-e] [-d]
required arguments:
-u USER, --user USER set the user profile you want to dump: <USER> could be a screen name or an account ID (if it is an ID, you should start the string with the "id" prefix, e. g. "id859377203242426368")
OR
-l, --show-limits show the rate limit status (total โ remaining โ time_to_wait_till_reset) for each of the accounts you set when configuring the tool
optional arguments:
-fr N, --friends N set the number of friends to be dumped (if N == -1 then tweetlord will try to dump all friends)
-fo N, --followers N set the number of followers to be dumped (if N == -1 then tweetlord will try to dump all followers)
-fa N, --favorites N set the number of favorite tweets to be dumped (if N == -1 then tweetlord will try to dump all favorite tweets)
-ti N, --timeline N set the number of tweets from user's timeline to be dumped (if N == -1 then tweetlord will try to dump all timeline tweets)
-a, --all dump ALL the sections with ALL the items in each of them
-o NAME, --output NAME set the output filename (".xlsx" ending will be added)
-w, --wait-on-limit sleep if the rate limit is exceeded (the sleeping time will be printed)
-e, --tweet-extended get the whole tweet text but not only the first 140 chars
-d, --debug debug mode (extra info messages will be show when exceptions are caught)
-h, --help show help
If there's no rate limit left and you have specified the -w
flag, you can press Ctrl+C during the sleeping (waiting) process, but only during the waiting process (otherwise, you'll terminate the main program), to skip dumping current section and continue with the next one.
See more about the Twitter Rate Limiting.
tweetlord works great both on Windows and GNU/Linux systems, but the resulting .xlsx
dump files look prettier when opened in MS Excel app rather than in LibreOffice (and this is quite obvious because LibreOffice doesn't fully support .xlsx
format anyhow ๐).