try and except about requests-futures

[ross]

Sorry a bit hard to guess at what you're asking without examples or more detail, but the main difference when using futures (not specifically related to requests-futures) is that exceptions will be thrown when you grab the result.

try:
    resp = get('http://foo.bar/')
except Exception:
    # error handling here

would become

future = get('http://foo.bar/')
...

try:
    resp = future.result()
except Exception:
    # error handling here

[pbasista]

The take-away point, at least from my point of view, was to use add_done_callback method of the returned future for registering a callback which will be called even when timeout exception is raised.

[afaideen]

oh my apology if it lacked of detail. I am guessing of how to translate a sync code like,

	try:
		r = requests.get(url, timeout=5)
	except requests.ConnectionError as e:
		print e 

to become something async like below,

    api_one = session.post(url, json=payload, headers=headers, timeout=5)
    api_two = session.get(url)
    try:
        r = api_one.result()
        r2 = api_two.result()
    except ConnectionError as e:
        print(e.message)
        return str(e.message)

I tried it tested but to no avail. I am also looking for detail doc something similar like requests.
Example above is for timeout but i also need case like ConnectionError and HttpError.
Thanks for the hint above it works but I need the case to be more specific. Would you help?

[ross]

Below are examples of the cases you're talking about. There's no difference between the standard requests behaviors and requests-futures other than things moving to the results() call. Your best bet is to look at the requests docs and questions/examples for it and move where the exception is thrown.

#!/usr/bin/env python

from pprint import pprint
from requests_futures.sessions import FuturesSession
from requests.exceptions import ConnectTimeout, HTTPError, ReadTimeout

session = FuturesSession()

future = session.get('http://httpbin.org/get', timeout=0.01)
try:
    # This will throw a ConnectionTimeout exception b/c 0.01 isn't long enough
    # to connect
    resp = future.result()
except ConnectTimeout as e:
    pprint(e)

future = session.get('http://httpbin.org/delay/5', timeout=1)
try:
    # this will throw a ReadTimeout exception b/c 1s is enough to connect, but
    # less time than we've asked httpbin to delay things
    resp = future.result()
except ReadTimeout as e:
    pprint(e)


future = session.get('http://httpbin.org/status/502')
# This will not throw an exception, b/c requests doesn't do that for http
# status codes (http errors)
resp = future.result()
try:
    # This will throw an exception for 4xx and 5xx statues, it'd be a bit weird
    # to call it and immediately catch it as I'm doing here, it'd probably be
    # cleaner to just check the status code directly, if you want it to throw,
    # but don't want to catch it right here then it'd make sense.
    resp.raise_for_status()
except HTTPError as e:
    pprint(e)

[afaideen]

thanks a lot really appreciate.

[dlparker]

The above helped me solve my problem, which is, I think, the same one the OP raised, namely how to do stuff to handle errors without blocking the caller. In my case the caller is a Kivy UI thread. So thanks for the clue.

The following may be a bit too verbose, but it demonstrates one way to get the job done. The whole business with re-raising the caught exception is just my way of visualizing the flow clearly in the context of the original post. In my real code I expect to map the exception classes to handlers and re-raise any unmapped exceptions in the caller's callback.

I also expect to add more context in the "pending" records so as to allow retry in an intelligent way (see #26 for more thoughts on that). My ultimate goal is to have reliable interaction with a server that might occasionally get balky, or when there is a short network issue (slightly unreliable wifi is involved in the application).

#!/usr/bin/env python
from pprint import pprint
import time
from requests_futures.sessions import FuturesSession
from requests.exceptions import ConnectTimeout, HTTPError, ReadTimeout

class Bar(object):
    def __init__(self):
        self.session = FuturesSession()
        self.pending = []

    def cb(self, future):
        index = 0
        callback = None
        for pend in self.pending:
            if pend['future'] == future:
                discard = self.pending.pop(index)
                callback = pend['callback']
                break
            index += 1
        if callback is None:
            raise Exception('logic error, no record of future')
        exc = future.exception()
        if exc is not None:
            try:
                print('got exception {}'.format(exc))
                raise exc
            except ConnectTimeout:
                print('got connect timeout, increasing timeout for retry')
                new_future = self.session.get('http://httpbin.org/get',
                                              timeout=2.0)

                new_future.add_done_callback(self.cb)
                pend = dict(future=new_future, callback=callback)
                self.pending.append(pend)
                return
            except ReadTimeout as e:
                print('got read timeout, increasing timeout for retry')
                new_future = self.session.get('http://httpbin.org/delay/1', timeout=3)
                new_future.add_done_callback(self.cb)
                pend = dict(future=new_future, callback=callback)
                self.pending.append(pend)
                return
            except Exception as e:
                callback(None, e)
        else:
            print('request got no exception')
            res = future.result()
            if res.status_code != 200:
                print('bad status code {}'.format(res.status_code))
                try:
                    res.raise_for_status()
                except Exception as e:
                    callback(None, e)
                    return
            callback(res)

    def get_connect_timeout(self, callback):
        future = self.session.get('http://httpbin.org/get',
                                  timeout=0.01,)
        future.add_done_callback(self.cb)
        pend = dict(future=future, callback=callback)
        self.pending.append(pend)

    def get_read_timeout(self, callback):
        future = self.session.get('http://httpbin.org/delay/5', timeout=1)
        future.add_done_callback(self.cb)
        pend = dict(future=future, callback=callback)
        self.pending.append(pend)

    def get_bad_status(self, callback):
        future = self.session.get('http://httpbin.org/status/502')
        future.add_done_callback(self.cb)
        pend = dict(future=future, callback=callback)
        self.pending.append(pend)

bar = Bar()
done = False

def cb(res, exc=None):
    global done
    if res:
        print('got callback with res {}'.format(res))
    else:
        print('got callback with exception {}'.format(exc))
    done = True

done = False
bar.get_connect_timeout(cb)
time.sleep(0.1)
while not done:
    time.sleep(1)
    print(time.time())

done = False
bar.get_read_timeout(cb)
time.sleep(0.1)
while not done:
    time.sleep(1)
    print(time.time())


done = False
bar.get_bad_status(cb)
time.sleep(0.1)
while not done:
    time.sleep(1)
    print(time.time())

Output:

$ python bar.py
got exception HTTPConnectionPool(host='httpbin.org', port=80): Max retries exceeded with url: /get (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x10a529ba8>, 'Connection to httpbin.org timed out. (connect timeout=0.01)'))
got connect timeout, increasing timeout for retry
request got no exception
got callback with res <Response [200]>
1520892929.684844
got exception HTTPConnectionPool(host='httpbin.org', port=80): Read timed out. (read timeout=1)
got read timeout, increasing timeout for retry
1520892930.7890759
1520892931.791595
request got no exception
got callback with res <Response [200]>
1520892932.794821
request got no exception
bad status code 502
got callback with exception 502 Server Error: BAD GATEWAY for url: http://httpbin.org/status/502
$