Coder Social home page Coder Social logo

zhibo's Introduction

直播平台爬虫

项目环境配置

使用语言:python(2和3都兼容)
使用的库:urllib、re、pymysql、datetime 数据库:mysql

主要实现功能

  1. 可以对虎牙,斗鱼直播平台进行爬虫
  2. 抓取整个直播平台的全部游戏分区的名字和链接
  3. 抓取在各个分区下的主播名称,房间标题,当前观众数,房间链接
  4. 通过访问爬取到的房间链接,可以爬取主播的订阅数
  5. 使用伪装浏览器

将来要实现的功能

  1. 爬取主播房间里的贡献榜单,来实现对主播收入的简单统计
  2. 添加更多的平台如龙珠,熊猫TV等

使用截图

程序开始会自动爬取直播平台全部游戏分区
image

输入Y后
image
爬取某个分区后会提醒完成这个分区,然后开始爬取下个分区
image

结果展示

数据库link

image
id:主播名称、title:房间标题、js:在线观众、game:分区、link:房间链接、time:时间

数据库主播

image
id:主播名称、title:房间标题、count:观众订阅量、game:分区、link:房间链接、time:时间

zhibo's People

Contributors

lienjack avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.