maox-yu / weibo-spider

Weibo crawler

Environment: Python 3.7

---

Function

Crawl the personal information and historical blog posts of users who have published content on a topic

Document description

The function of the first.py file is to crawl users in a topic, and whether they are depressed users, and save them in the Weibo list file

The second.py file is used to enter the user's details page and crawl the user's personal information, such as the number of users' followers and fans, Number of tweets, gender, user profile and tag, then save them to the result.csv file, and crawl the user's tweet information Save it in the result file, and finally clean the file

The last.py file is used to write the file to the database and save it