Comments (3)
something like the following should do the trick for something quick and dirty.
#!/usr/bin/env python
from datetime import datetime
from requests_futures.sessions import FuturesSession
session = FuturesSession()
times = {}
def time_it(sess, resp):
times[resp.url] = datetime.now() - times[resp.url]
def do_request(url):
times[url] = datetime.now()
return session.get(url, background_callback=time_it)
futures = []
for url in ('http://httpbin.org/get', 'http://httpbin.org/get?key=val',
'http://httpbin.org/get?key2=val2'):
futures.append(do_request(url))
for future in futures:
future.result() # you'd want to actually do something with the result here
# now that we've waited for everything times should be the durations
for url, delta in times.items():
dur = delta.days * 24 * 60 * 60 * 1000 + delta.seconds * 1000 + \
delta.microseconds / 1000.0
print '%4dms %s' % (dur, url)
if i were doing it in something real/perm i'd probably inherit from FuturesSession and build/add the timing logic in to the request method. something like.
#!/usr/bin/env python
from datetime import datetime
from requests_futures.sessions import FuturesSession
class TimingSession(FuturesSession):
def __init__(self, *args, **kwargs):
super(TimingSession, self).__init__(*args, **kwargs)
self.timing = {}
def request(self, method, url, *args, **kwargs):
background_callback = kwargs.pop('background_callback', None)
# start counting
self.timing[url] = datetime.now()
def time_it(sess, resp):
# here if you want to time the server stuff only
self.timing[resp.url] = datetime.now() - self.timing[resp.url]
if background_callback:
background_callback(sess, resp)
# here if you want to include any time in the callback
return super(TimingSession, self).request(method, url, *args,
background_callback=time_it,
**kwargs)
session = TimingSession()
futures = []
for url in ('http://httpbin.org/get', 'http://httpbin.org/get?key=val',
'http://httpbin.org/get?key2=val2'):
futures.append(session.get(url))
for future in futures:
future.result()
for url, delta in session.timing.items():
dur = delta.days * 24 * 60 * 60 * 1000 + delta.seconds * 1000 + \
delta.microseconds / 1000.0
print '%4dms %s' % (dur, url)
from requests-futures.
Wow! Thanks for the quick detailed answer. This will really help me out.
from requests-futures.
np
from requests-futures.
Related Issues (20)
- OSError Issues when used within celery task (gevent) HOT 2
- Adding attribute to response via hook (as seen in examples) fails HOT 1
- Cannot switch to a different thread HOT 3
- Possibility to accept old SSL versions? HOT 1
- Problem with data field in header request HOT 1
- Making a future request and letting callback work with result - some are missing HOT 2
- Error When there is a large parallel Requests go. HOT 2
- Provide wheels on PyPI HOT 1
- How do we pass arguments to the hooks? HOT 3
- AttributeError: 'Future' object has no attribute 'status_code' HOT 1
- 1.0.0: missing git tag HOT 5
- Response for batched requests HOT 6
- Add type annotations HOT 4
- Not showing as updated on PyPI HOT 3
- On exception it returns xx (Caused By None), whereas the requests module returns xx (Caused By xx) HOT 4
- The elapsed time hook example does not work HOT 3
- 1.0.0: pytest is failing in two units HOT 6
- Use pytest-httpbin? HOT 8
- Attribute set via response hooks does not work with ProcessPoolExecutor HOT 4
- RFE: preserve contextvars when performing the request HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from requests-futures.