Comments (2)
It looks like you're trying to use grequests
in combination with some kind of webserver. Fundamentally, your problem is compatibility with gevent
grequests
relies on gevent
and uses monkey-patching to enable concurrency. This often causes conflicts with other packages that either (1) also try to monkey-patch the same modules, (2) use the patched modules in a manner that is incompatible with the monkey-patched version of the module or (3) just plain don't play nice with gevent. Because webservers fundamentally will use the same underlying modules (threading, sockets, ssl, and so on) it is not uncommon for such problems to occur.
Based on the error you have, it's most likely that you have code somewhere that is running unpatched modules blocking gevent from switching threads. This can happen if a module imports a module that needs to be patched before the patching actually occurs, for example, if you import another module before grequests
.
You can also browse other gevent-related issues in this repo to find others with similar problems and workarounds.
Some common solutions that may fix your problem include:
- Ensuring that
grequests
is imported first before any other packages. This may require special consideration, depending how you are starting your server (this is the most likely the cause of your problem) - If you are using a debugger, use a gevent-compatible debugger (or disable the debugger)
- Upgrading/downgrading the version of
gevent
used - Configure the concurrency model of your WSGI server to be compatible with gevent (e.g., gevent vs eventlet backends)
- Use something else, like
requests-threads
orrequests-futures
instead, which don't rely on gevent/monkeypatching
Without more information about your code, dockerfile, etc., it's difficult to provide any further guidance.
from grequests.
I tried looking in all other relevant issues and still did not find my answer. Generally, I import monkey from gevent and patch it. This is the first thing to be imported in my project. I later import grequest in another module (it's the first import in that module). Generally I use flask server. Can it cause problems with grequests? I see other people use both of them without problems.
I also use Joblib and python's native threading lib for threading, can they cause problems with greenlet threads?
from grequests.
Related Issues (20)
- Grequest returns empty generators HOT 2
- How to convert response for a json HOT 1
- limit number of requests per second. HOT 1
- why size is no affect in map? HOT 1
- Add index in imap HOT 3
- Params of get/post functions HOT 3
- Import fails HOT 1
- Grequests warning and crash HOT 3
- Request for image HOT 2
- Append wait time or rate for every thread due to server limits HOT 1
- Python 3.6 the fastest environment?
- Python 3.6 the fastest environment? HOT 2
- gtimeout does not works with imap_enumerated HOT 1
- Flag to print the request being executed HOT 2
- How to ignore SSL cert verification failed in grequests? HOT 2
- grequests.map() is slow when passing requests with the data parameter HOT 2
- nose should be in `dev_requirements.txt`
- Process finished with exit code 132 (interrupted by signal 4: SIGILL) HOT 1
- how to parse single response when setting stream=True in map function?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grequests.