Comments (9)
就是firefox版本不对的原因。 你从这里下会有问题吗?https://ftp.mozilla.org/pub/firefox/releases/36.0/mac/zh-CN/
from wechat-spider.
@yijingping 您好,我下载了36的版本,但是下载后firefox 总是自动升级到最新版本,我选择了不让它自动更新,但不起作用,请问这个问题您是如何处理的呢?
from wechat-spider.
@yijingping 您好,我刚刚试出来的一个方法,将/Users//Library/Caches/Mozilla//updates/Applications/Firefox/ 路径下的文件改为root权限,同时将它们改为无权限(sudo chmod 000 *),这样Firefox就不能再升级了,谢谢,我再试一下能不能下载公众号的文章。
from wechat-spider.
@yijingping 您好在执行process.py脚本的时候报以下的错误:
EERFFWq/hFgADUMN4RzT6/OAAAAAElFTkSuQmCC"/> <p class="toast_content">已留言
Traceback (most recent call last):
File "bin/processor.py", line 57, in
processor.run()
File "bin/processor.py", line 52, in run
self.process(data)
File "bin/processor.py", line 37, in process
backend.process(data)
File "/Users/northleaf/git/bowenpay-weichat-spider/wechat/processors.py", line 48, in process
C.objects.update_or_create(uniqueid=params['uniqueid'], defaults=params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 422, in update_or_create
obj, created = self._create_object_from_params(lookup, params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 439, in _create_object_from_params
obj = self.create(**params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 348, in create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 710, in save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 738, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 822, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 861, in _do_insert
using=using, raw=raw)
File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 971, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 97, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1366, "Incorrect string value: '\xF0\x9F\x92\x97</...' for column 'content' at row 1")
这个错误是什么原因呢(我用的是Mysql)?
另外,如何显示留言呢?谢谢
from wechat-spider.
这说明文章已经爬取并解析好了,在存储的时候出错了。 有可能是你的数据库编码有问题。 你的mysql版本号是多少? 编码设为utf8mb4了吗?
from wechat-spider.
现在没有解析留言功能,需要改下配置规则文件和数据库结构,才能存。
from wechat-spider.
@yijingping 您好,编码功能我已调好,请问如何修改规则文件与数据库结构才能存储留言功能呢,谢谢!
from wechat-spider.
在 bin/extractor.py
中,修改变量 NORMAL_RULES
和 DETAIL_RULES
。 他们分别用于解析通过微信号爬取的文章 和 通过url爬取的文章。
1)在数据库中新增messages字段
2)修改变量 NORMAL_RULES
和 DETAIL_RULES
在变量列表中,仿照 avatar 字段的值,新增一个messages字段。 然后使用 xpath、 python规则,将留言中的头像、昵称、内容 都提取出来,用json字符串的形式存储在
3)展示的时候,你取数据库中的messages字段的json串 转化一下就行。
from wechat-spider.
@yijingping 谢谢,我按您说的试一下。
from wechat-spider.
Related Issues (20)
- 请求商务推广合作 HOT 1
- 你好downloader运行报这个错,能帮忙看下吗
- 你好,作者,你这个是个人项目,还是公司项目? HOT 2
- 可否远程协助安装
- 公公号是手动添加吗?能不能自动采集公共号进行添加
- runserver 捕捉到这样的错误
- selenium的一个错误 HOT 3
- 无 HOT 1
- 公众号搜索只有在首次部署后可以
- 大佬,看看我这个方式可以吗,https://mp.weixin.qq.com/s/C92OHAVl6h0NwHGzX5-IIg HOT 1
- Linux可以使用吗?
- 文章没有下载下来
- 这开源的项目还能不能用了呀?
- 代理状态: 异常
- 2020-06这个项目还能用吗,还在维护吗
- 搜狗微信在2019.10.29下线相关功能过后,这个项目就废了!!! HOT 4
- python bin/downloader.py 页面添加关键字报错nodename nor servname provided, or not known
- Project dependencies may have API risk issues HOT 1
- 爬的文章有水印吗?各位大佬 HOT 1
- 商业合作考虑吗?wechat:chenxiaolu5432 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wechat-spider.