Coder Social home page Coder Social logo

爬取页数限制 about jd_spider HOT 1 CLOSED

ramsayleung avatar ramsayleung commented on June 9, 2024
爬取页数限制

from jd_spider.

Comments (1)

ramsayleung avatar ramsayleung commented on June 9, 2024

是的，在我的博文两只蠢萌的爬虫里，我就有吐槽:

商品的评论也是以 sku-id 为参数通过异步的方式进行请求的，构造请求的方法跟价格类 似，也不需过多赘述。只是想要吐嘈一下的是，京东的评论是只能一页页向后翻的，不能 跳转。还有一点就是，即使某样商品有 10+w 条评论，最多也只是返回 100 页的数据。 略坑

这个限制是京东本身的接口限制，是没办法通过爬虫进行突破的

from jd_spider.

Related Issues (19)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.