Cache breaks between python2 and python3 about howdoi

[gleitz]

Thanks for the report. It sounds like this is an issue with requests-cache. We're currently using version 0.3.0. If you get a chance would you mind upgrading to the latest (0.4.1) and seeing if you still see the issue?

A quick fix is to clear the cache when switching between versions with howdoi -C or disabling the cache by setting the environment variable HOWDOI_DISABLE_CACHE.

[kensington]

In 0.4.0, the issue persists. With 0.4.1, it doesn't work at all:

Traceback (most recent call last):
File "/usr/bin/howdoi-python2.7", line 9, in
load_entry_point('howdoi==1.1.4', 'console_scripts', 'howdoi')()
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 217, in command_line_runner
print(howdoi(args).encode('utf-8', 'ignore'))
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 176, in howdoi
return get_instructions(args) or 'Sorry, couldn't find any help with that topic\n'
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 143, in get_instructions
links = get_links(args['query'])
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 68, in get_links
result = get_result(url)
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 59, in get_result
return requests.get(url, headers={'User-Agent': random.choice(USER_AGENTS)}, proxies=get_proxies()).text
File "/usr/lib64/python2.7/site-packages/requests/api.py", line 55, in get
return request('get', url, *_kwargs)
File "/usr/lib64/python2.7/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, *_kwargs)
File "/usr/lib64/python2.7/site-packages/requests_cache/core.py", line 111, in request
hooks, stream, verify, cert)
File "/usr/lib64/python2.7/site-packages/requests/sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib64/python2.7/site-packages/requests_cache/core.py", line 91, in send
return send_request_and_cache_response()
File "/usr/lib64/python2.7/site-packages/requests_cache/core.py", line 85, in send_request_and_cache_response
self.cache.save_response(cache_key, response)
File "/usr/lib64/python2.7/site-packages/requests_cache/backends/base.py", line 40, in save_response
self.responses[key] = self.reduce_response(response), datetime.utcnow()
File "/usr/lib64/python2.7/site-packages/requests_cache/backends/storage/dbdict.py", line 168, in setitem
sqlite.Binary(pickle.dumps(item)))
cPickle.UnpickleableError: Cannot pickle <type 'zlib.Decompress'> objects

[gleitz]

It may be that the requests version needs to be upgraded as well. If that
doesn't work would you mind submitting this bug to the requests project?
On Aug 25, 2013 8:27 AM, "kensington" [email protected] wrote:

In 0.4.0, the issue persists. With 0.4.1, it doesn't work at all:

Traceback (most recent call last):
File "/usr/bin/howdoi-python2.7", line 9, in
load_entry_point('howdoi==1.1.4', 'console_scripts', 'howdoi')()
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 217, in
command_line_runner
print(howdoi(args).encode('utf-8', 'ignore'))
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 176, in
howdoi
return get_instructions(args) or 'Sorry, couldn't find any help with that
topic\n'
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 143, in
get_instructions
links = get_links(args['query'])
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 68, in
get_links
result = get_result(url)
File "/usr/lib64/python2.7/site-packages/howdoi/howdoi.py", line 59, in
get_result
return requests.get(url, headers={'User-Agent':
random.choice(USER_AGENTS)}, proxies=get_proxies()).text
File "/usr/lib64/python2.7/site-packages/requests/api.py", line 55, in get
return request('get', url, *_kwargs)
File "/usr/lib64/python2.7/site-packages/requests/api.py", line 44, in
request
return session.request(method=method, url=url, *_kwargs)
File "/usr/lib64/python2.7/site-packages/requests_cache/core.py", line
111, in request
hooks, stream, verify, cert)
File "/usr/lib64/python2.7/site-packages/requests/sessions.py", line 335,
in request
resp = self.send(prep, *send_kwargs)
File "/usr/lib64/python2.7/site-packages/requests_cache/core.py", line 91,
in send
return send_request_and_cache_response()
File "/usr/lib64/python2.7/site-packages/requests_cache/core.py", line 85,
in send_request_and_cache_response
self.cache.save_response(cache_key, response)
File "/usr/lib64/python2.7/site-packages/requests_cache/backends/base.py",
line 40, in save_response
self.responses[key] = self.reduce_response(response), datetime.utcnow()
File
"/usr/lib64/python2.7/site-packages/requests_cache/backends/storage/dbdict.py",
line 168, in *setitem
sqlite.Binary(pickle.dumps(item)))
cPickle.UnpickleableError: Cannot pickle objects

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/63#issuecomment-23229597
.

[kensington]

requests-cache-0.4.2 has just been released fixing the second issue mentioned, but the original issue still persists. Is that an upstream issue too?

[reclosedev]

requests-cache doesn't work when switching Python2 and Python3 with same database, due to unicode/str and pickle incompatibility.

Possible solution is to add major version to database name in client code.

[gleitz]

@kensington I would accept a pull request that attached a major version to the cache filename.