Coder Social home page Coder Social logo

zhibo's Introduction

直播平台爬虫

项目环境配置

使用语言：python(2和3都兼容)
使用的库：urllib、re、pymysql、datetime 数据库：mysql

主要实现功能

  1. 可以对虎牙，斗鱼直播平台进行爬虫
  2. 抓取整个直播平台的全部游戏分区的名字和链接
  3. 抓取在各个分区下的主播名称，房间标题，当前观众数，房间链接
  4. 通过访问爬取到的房间链接，可以爬取主播的订阅数
  5. 使用伪装浏览器

将来要实现的功能

  1. 爬取主播房间里的贡献榜单，来实现对主播收入的简单统计
  2. 添加更多的平台如龙珠，熊猫TV等

使用截图

程序开始会自动爬取直播平台全部游戏分区
image

输入Y后
image
爬取某个分区后会提醒完成这个分区，然后开始爬取下个分区
image

结果展示

数据库link

image
id:主播名称、title：房间标题、js：在线观众、game：分区、link：房间链接、time：时间

数据库主播

image
id:主播名称、title：房间标题、count：观众订阅量、game：分区、link：房间链接、time：时间

zhibo's People

Contributors

lienjack avatar

Watchers

avatar avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.