wangshunping / weibo_spider Goto Github PK
View Code? Open in Web Editor NEWgraduate project, a weibo spider to find some interesting information such as "In social network , people tend to be happy or sad."
graduate project, a weibo spider to find some interesting information such as "In social network , people tend to be happy or sad."
你readme里面的豆瓣链接挂掉了
依赖项不全 用了一个mysqldb的包 在pypi中找不到 而且标准库是pymysql
另外还有一个pylab 也找不到
同时 在plot.py line119
plt.text(1500,60,u"安徽医科大学",color='black',size=36, \
horizontalalignment='right',verticalalignment='top')
plt.text(1500,40 ,u'性别与微博数量的比较',color='.75',size=12,\
ha='right',va='top')
plt.savefig('pic/sex2totalcontent.png')
plt.close()
这里 直接就写的安徽医科大学
您好,我今年也大四,我想问下您的代码中,id与cookie就是单纯是某个用户的登陆id与cookie吗?
另外重要的想问下,您是如何识别你抓取的用户就全都是安徽医科大的学生呢?
执行:python util.py,报:
BeautifulSoup([your markup], "lxml")
markup_type=markup_type))
Traceback (most recent call last):
File "util.py", line 134, in
main()
File "util.py", line 120, in main
firstInfo = getUniversityStudent(mainid, school)
File "util.py", line 58, in getUniversityStudent
personalInfo = divlabel[0].next_sibling.get_text('|',strip=True)
IndexError: list index out of range
Error: None
Traceback (most recent call last):
File "weiboSpider.py", line 37, in get_username
selector = etree.HTML(html)
File "src/lxml/lxml.etree.pyx", line 3161, in lxml.etree.HTML (src/lxml/lxml.etree.c:82353)
File "src/lxml/parser.pxi", line 1819, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:124533)
File "src/lxml/parser.pxi", line 1707, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:123074)
File "src/lxml/parser.pxi", line 1079, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:117114)
File "src/lxml/parser.pxi", line 573, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:110510)
File "src/lxml/parser.pxi", line 683, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:112276)
File "src/lxml/parser.pxi", line 624, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:111367)
XMLSyntaxError: None
Error: None
Traceback (most recent call last):
File "weiboSpider.py", line 51, in get_user_info
selector = etree.HTML(html)
File "src/lxml/lxml.etree.pyx", line 3161, in lxml.etree.HTML (src/lxml/lxml.etree.c:82353)
File "src/lxml/parser.pxi", line 1819, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:124533)
File "src/lxml/parser.pxi", line 1707, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:123074)
File "src/lxml/parser.pxi", line 1079, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:117114)
File "src/lxml/parser.pxi", line 573, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:110510)
File "src/lxml/parser.pxi", line 683, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:112276)
File "src/lxml/parser.pxi", line 624, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:111367)
XMLSyntaxError: None
Error: None
Traceback (most recent call last):
File "weiboSpider.py", line 86, in get_weibo_info
selector = etree.HTML(html)
File "src/lxml/lxml.etree.pyx", line 3161, in lxml.etree.HTML (src/lxml/lxml.etree.c:82353)
File "src/lxml/parser.pxi", line 1819, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:124533)
File "src/lxml/parser.pxi", line 1707, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:123074)
File "src/lxml/parser.pxi", line 1079, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:117114)
File "src/lxml/parser.pxi", line 573, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:110510)
File "src/lxml/parser.pxi", line 683, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:112276)
File "src/lxml/parser.pxi", line 624, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:111367)
XMLSyntaxError: None
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.