Comments (6)
Sorry a bit hard to guess at what you're asking without examples or more detail, but the main difference when using futures (not specifically related to requests-futures) is that exceptions will be thrown when you grab the result.
try:
resp = get('http://foo.bar/')
except Exception:
# error handling here
would become
future = get('http://foo.bar/')
...
try:
resp = future.result()
except Exception:
# error handling here
from requests-futures.
The take-away point, at least from my point of view, was to use add_done_callback
method of the returned future for registering a callback which will be called even when timeout exception is raised.
from requests-futures.
oh my apology if it lacked of detail. I am guessing of how to translate a sync code like,
try:
r = requests.get(url, timeout=5)
except requests.ConnectionError as e:
print e
to become something async like below,
api_one = session.post(url, json=payload, headers=headers, timeout=5)
api_two = session.get(url)
try:
r = api_one.result()
r2 = api_two.result()
except ConnectionError as e:
print(e.message)
return str(e.message)
I tried it tested but to no avail. I am also looking for detail doc something similar like requests.
Example above is for timeout but i also need case like ConnectionError and HttpError.
Thanks for the hint above it works but I need the case to be more specific. Would you help?
from requests-futures.
Below are examples of the cases you're talking about. There's no difference between the standard requests behaviors and requests-futures other than things moving to the results()
call. Your best bet is to look at the requests docs and questions/examples for it and move where the exception is thrown.
#!/usr/bin/env python
from pprint import pprint
from requests_futures.sessions import FuturesSession
from requests.exceptions import ConnectTimeout, HTTPError, ReadTimeout
session = FuturesSession()
future = session.get('http://httpbin.org/get', timeout=0.01)
try:
# This will throw a ConnectionTimeout exception b/c 0.01 isn't long enough
# to connect
resp = future.result()
except ConnectTimeout as e:
pprint(e)
future = session.get('http://httpbin.org/delay/5', timeout=1)
try:
# this will throw a ReadTimeout exception b/c 1s is enough to connect, but
# less time than we've asked httpbin to delay things
resp = future.result()
except ReadTimeout as e:
pprint(e)
future = session.get('http://httpbin.org/status/502')
# This will not throw an exception, b/c requests doesn't do that for http
# status codes (http errors)
resp = future.result()
try:
# This will throw an exception for 4xx and 5xx statues, it'd be a bit weird
# to call it and immediately catch it as I'm doing here, it'd probably be
# cleaner to just check the status code directly, if you want it to throw,
# but don't want to catch it right here then it'd make sense.
resp.raise_for_status()
except HTTPError as e:
pprint(e)
from requests-futures.
thanks a lot really appreciate.
from requests-futures.
The above helped me solve my problem, which is, I think, the same one the OP raised, namely how to do stuff to handle errors without blocking the caller. In my case the caller is a Kivy UI thread. So thanks for the clue.
The following may be a bit too verbose, but it demonstrates one way to get the job done. The whole business with re-raising the caught exception is just my way of visualizing the flow clearly in the context of the original post. In my real code I expect to map the exception classes to handlers and re-raise any unmapped exceptions in the caller's callback.
I also expect to add more context in the "pending" records so as to allow retry in an intelligent way (see #26 for more thoughts on that). My ultimate goal is to have reliable interaction with a server that might occasionally get balky, or when there is a short network issue (slightly unreliable wifi is involved in the application).
#!/usr/bin/env python
from pprint import pprint
import time
from requests_futures.sessions import FuturesSession
from requests.exceptions import ConnectTimeout, HTTPError, ReadTimeout
class Bar(object):
def __init__(self):
self.session = FuturesSession()
self.pending = []
def cb(self, future):
index = 0
callback = None
for pend in self.pending:
if pend['future'] == future:
discard = self.pending.pop(index)
callback = pend['callback']
break
index += 1
if callback is None:
raise Exception('logic error, no record of future')
exc = future.exception()
if exc is not None:
try:
print('got exception {}'.format(exc))
raise exc
except ConnectTimeout:
print('got connect timeout, increasing timeout for retry')
new_future = self.session.get('http://httpbin.org/get',
timeout=2.0)
new_future.add_done_callback(self.cb)
pend = dict(future=new_future, callback=callback)
self.pending.append(pend)
return
except ReadTimeout as e:
print('got read timeout, increasing timeout for retry')
new_future = self.session.get('http://httpbin.org/delay/1', timeout=3)
new_future.add_done_callback(self.cb)
pend = dict(future=new_future, callback=callback)
self.pending.append(pend)
return
except Exception as e:
callback(None, e)
else:
print('request got no exception')
res = future.result()
if res.status_code != 200:
print('bad status code {}'.format(res.status_code))
try:
res.raise_for_status()
except Exception as e:
callback(None, e)
return
callback(res)
def get_connect_timeout(self, callback):
future = self.session.get('http://httpbin.org/get',
timeout=0.01,)
future.add_done_callback(self.cb)
pend = dict(future=future, callback=callback)
self.pending.append(pend)
def get_read_timeout(self, callback):
future = self.session.get('http://httpbin.org/delay/5', timeout=1)
future.add_done_callback(self.cb)
pend = dict(future=future, callback=callback)
self.pending.append(pend)
def get_bad_status(self, callback):
future = self.session.get('http://httpbin.org/status/502')
future.add_done_callback(self.cb)
pend = dict(future=future, callback=callback)
self.pending.append(pend)
bar = Bar()
done = False
def cb(res, exc=None):
global done
if res:
print('got callback with res {}'.format(res))
else:
print('got callback with exception {}'.format(exc))
done = True
done = False
bar.get_connect_timeout(cb)
time.sleep(0.1)
while not done:
time.sleep(1)
print(time.time())
done = False
bar.get_read_timeout(cb)
time.sleep(0.1)
while not done:
time.sleep(1)
print(time.time())
done = False
bar.get_bad_status(cb)
time.sleep(0.1)
while not done:
time.sleep(1)
print(time.time())
Output:
$ python bar.py
got exception HTTPConnectionPool(host='httpbin.org', port=80): Max retries exceeded with url: /get (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x10a529ba8>, 'Connection to httpbin.org timed out. (connect timeout=0.01)'))
got connect timeout, increasing timeout for retry
request got no exception
got callback with res <Response [200]>
1520892929.684844
got exception HTTPConnectionPool(host='httpbin.org', port=80): Read timed out. (read timeout=1)
got read timeout, increasing timeout for retry
1520892930.7890759
1520892931.791595
request got no exception
got callback with res <Response [200]>
1520892932.794821
request got no exception
bad status code 502
got callback with exception 502 Server Error: BAD GATEWAY for url: http://httpbin.org/status/502
$
from requests-futures.
Related Issues (20)
- No module named 'requests_futures.sessions' HOT 3
- OSError Issues when used within celery task (gevent) HOT 2
- Adding attribute to response via hook (as seen in examples) fails HOT 1
- Cannot switch to a different thread HOT 3
- Possibility to accept old SSL versions? HOT 1
- Problem with data field in header request HOT 1
- Making a future request and letting callback work with result - some are missing HOT 2
- Error When there is a large parallel Requests go. HOT 2
- Provide wheels on PyPI HOT 1
- How do we pass arguments to the hooks? HOT 3
- AttributeError: 'Future' object has no attribute 'status_code' HOT 1
- 1.0.0: missing git tag HOT 5
- Response for batched requests HOT 6
- Add type annotations HOT 4
- Not showing as updated on PyPI HOT 3
- On exception it returns xx (Caused By None), whereas the requests module returns xx (Caused By xx) HOT 4
- The elapsed time hook example does not work HOT 3
- 1.0.0: pytest is failing in two units HOT 6
- Use pytest-httpbin? HOT 8
- Attribute set via response hooks does not work with ProcessPoolExecutor HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from requests-futures.