tauwu / weibo_daily_hotkey

Weibo's daily TOP5 hotkey. 自动爬取、筛选新浪微博每日热搜词 TOP5。https://github.com/TauWu/weibo_daily_hotkey/blob/master/data/data.md

WEIBO DAILY HOTKEY

This repo will get hotkeys for sina weibo, and upload it to this github repo.

本项目会定期从新浪微博中获取热搜关键词，并将数据回传到本项目。

NOTICE

本项目预期会在 WeiboDaily 进行重构，施工中🚧

DATA

Click Here to read daily hotkey data.

点击这里阅读每日微博热搜词数据。

Requirements

Softwares

apt-get install python3
apt-get install python3-pip
apt-get install redis-server

Python Modules

pip3 install requests
pip3 install GitPython
pip3 install lxml
pip3 install redis

Crawler Info

Base URL

https://s.weibo.com/top/summary?cate=realtimehot

Shouldn't login.

Parser

lxml module for python and regexp tools.

Database

Redis for daily data and MySQL for API data(//TODO).

Project Menu Tree

--
 |/conf 配置文件
 |/data 爬取数据
 |/util 工具函数
 |---->|/base 基础库
 |---->|/configcenter 配置中心
 |---->|/constant 常量
 |/cronjob 定时任务
 |---->|/datacenter 数据中心
 |-------->|/database 数据库数据
 |-------->|/cache 缓存数据
 |---->|/crawler 爬虫中心
 |---->|/tool 定时工具
 |/service //TODO 服务中心
 |cronjob.py 定时抓取、数据比对脚本
 |update.py 定时更新 repo 脚本
 |update_code.py 手动更新 github 代码脚本 //FIXME (git add . 的实现有 bug)
 -------------------------------

weibo_daily_hotkey's Issues

配置好conf.cfg报错如下

conf.cfg配置如下
[redis]
host=127.0.0.0
port=6379
passwd=123456
[file]
path=data/data.md
[git]
path=C:/Users/Administrator/Desktop/weibo_daily_hotkey-master
filepath=data/data.md

以下是执行企业微信自动推送需要的

[qywx]
tokens=
[monitor]
keys=123,456
hot_count=10000

Traceback (most recent call last):
File "C:/Users/Administrator/Desktop/weibo_daily_hotkey-master/cronjob.py", line 28, in
cfg = ConfigCenter.MonitorConfig()
File "C:\Users\Administrator\Desktop\weibo_daily_hotkey-master\util\configcenter\config_center.py", line 35, in MonitorConfig
cfg = ConfigReader.read_section_key('conf', 'base.cfg', 'monitor', *k)
File "C:\Users\Administrator\Desktop\weibo_daily_hotkey-master\util\base\config.py", line 40, in read_section_key
conf = Config(path_name=path_name, file_name=file_name, section_name=section_name)
File "C:\Users\Administrator\Desktop\weibo_daily_hotkey-master\util\base\config.py", line 12, in init
self.config.read(self.filepath)
File "C:\Anaconda3\lib\configparser.py", line 696, in read
self._read(fp, filename)
File "C:\Anaconda3\lib\configparser.py", line 1014, in _read
for lineno, line in enumerate(fp, start=1):
UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 194: illegal multibyte sequence

请问如何进行配置呢？运行不起来。。。

UnboundLocalError: local variable 'html' referenced before assignment

我又来了，又出现了一个报错：

谢谢大牛解答！