为何下载时会报以下的错误呢？ about wechat-spider

[yijingping]

就是firefox版本不对的原因。 你从这里下会有问题吗？https://ftp.mozilla.org/pub/firefox/releases/36.0/mac/zh-CN/

[northleafup]

@yijingping 您好，我下载了36的版本，但是下载后firefox 总是自动升级到最新版本，我选择了不让它自动更新，但不起作用，请问这个问题您是如何处理的呢？

[northleafup]

@yijingping 您好，我刚刚试出来的一个方法，将/Users//Library/Caches/Mozilla//updates/Applications/Firefox/ 路径下的文件改为root权限，同时将它们改为无权限(sudo chmod 000 *)，这样Firefox就不能再升级了，谢谢，我再试一下能不能下载公众号的文章。

[northleafup]

@yijingping 您好在执行process.py脚本的时候报以下的错误：
EERFFWq/hFgADUMN4RzT6/OAAAAAElFTkSuQmCC"/> <p class="toast_content">已留言

<iframe src="http://mp.weixin.qq.com/mp/readtemplate?t=appmsg/cspsupport_test_tmpl&amp;level=1\" style="display: none;"></iframe><iframe src="http://mp.weixin.qq.com/mp/readtemplate?t=appmsg/cspsupport_test_tmpl&amp;level=2\" style="display: none;"></iframe>", "like_num": 0, "avatar": "http://mmbiz.qpic.cn/mmbiz_jpg/4huZu0wMGticNlDj6ibgMzPicwlEEyricULCiaeo36CQsMic57E1Gho1C0opdVNElHeQSDciadKAA6AU5y1ia7YderKqQw/0?wx_fmt=jpeg", "words": 3499, "wechat_id": 1, "read_num": 0}
Traceback (most recent call last):
File "bin/processor.py", line 57, in
processor.run()
File "bin/processor.py", line 52, in run
self.process(data)
File "bin/processor.py", line 37, in process
backend.process(data)
File "/Users/northleaf/git/bowenpay-weichat-spider/wechat/processors.py", line 48, in process
C.objects.update_or_create(uniqueid=params['uniqueid'], defaults=params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 422, in update_or_create
obj, created = self._create_object_from_params(lookup, params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 439, in _create_object_from_params
obj = self.create(**params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 348, in create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 710, in save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 738, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 822, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 861, in _do_insert
using=using, raw=raw)
File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 971, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 97, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1366, "Incorrect string value: '\xF0\x9F\x92\x97</...' for column 'content' at row 1")

这个错误是什么原因呢（我用的是Mysql）？
另外，如何显示留言呢？谢谢

[yijingping]

这说明文章已经爬取并解析好了，在存储的时候出错了。 有可能是你的数据库编码有问题。 你的mysql版本号是多少？ 编码设为utf8mb4了吗？

[yijingping]

现在没有解析留言功能，需要改下配置规则文件和数据库结构，才能存。

[northleafup]

@yijingping 您好，编码功能我已调好，请问如何修改规则文件与数据库结构才能存储留言功能呢，谢谢！

[yijingping]

在 bin/extractor.py 中，修改变量 NORMAL_RULES 和 DETAIL_RULES。 他们分别用于解析通过微信号爬取的文章 和 通过url爬取的文章。

1）在数据库中新增messages字段
2）修改变量 NORMAL_RULES 和 DETAIL_RULES
在变量列表中，仿照 avatar 字段的值，新增一个messages字段。 然后使用 xpath、 python规则，将留言中的头像、昵称、内容 都提取出来，用json字符串的形式存储在
3）展示的时候，你取数据库中的messages字段的json串 转化一下就行。

[northleafup]

@yijingping 谢谢，我按您说的试一下。