dongfang是一个基于Scrapy-Selenium的股票信息爬虫，爬取了东方网千股千评的所有股票信息。
dongfang是一个基于Scrapy-Selenium的股票信息爬虫，爬取了东方网千股千评的所有股票信息。
爬取东方网千股千评的时候，其个股信息采用了js动态加载方式，然后分析其加载链接的规律，发现其链接返回的看似是一个JSON格式的文件，实际过程中无法用json.loads()获得其数据。试着用正则表达式获取其中的信息，也获取成功了，就是代码看着很复杂，遂采用selenium来获取。抓取过程中基本没有问题，而且对于这种获取数据量不大的情况下，selenium的速度慢影响不大。
在东方财富网的千股千评页面总共获取了3659条个股信息。每条信息中还包括了该股票的个股研报和其股吧的链接。结果由爬虫先存储在MongoDB中，再导出为Excle文件。部分数据如下截图:
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.
