Comments (2)
Thanks for reporting this!
I'll look into the issue now and keep this thread updated.
from spotify.py.
Right, so I've investigated the issue and have a possible fix ready.
But first I'd like to address a couple points.
instead of waiting, justs tries requesting and exits with error code 439(Too many requests), when it should wait for the rate limit to pass.
The underlying http client does actually stall the request waiting for the rate limit to pass over. This issue here was that the stalling is performed only for that request and not all requests client wide resulting in a form of head of line blocking.
Additionally due to a large amount of requests made in a short period of time, which the library did not anticipate in the original design of the stalling mechanism, requests could essentially start "choking" each other out where one request finishes their rate limit stall but is then immediately rate limited again due to a sibling request taking its slot. The exception you received was a result of this following ten consecutive request failures all being rate limited.
When i do something like:
SNIP
And i get ratelimited
Unfortunately the API response does not let us know how many tries we have remaining in order to start performing client side throttling so the rate limit as a whole can be avoided. The response will only let us know if we've been rate limited or not and the library can only deal with that to the extent of waiting out the rate limit but not avoiding it.
If you do not want to hit the rate limit in the first place then you must throttle your own requests manually (this can be as simple as sleeping for a small amount of time between requests) or use multiple client applications as rate limits are applied on a per client app basis (although I do not know if this is endorsed by Spotify)
Right, so I've investigated the issue and have a possible fix ready.
I was a little reluctant to use the patch as it is right now since it feels inefficient (I performed 4096 search queries in four minutes with over 30 rate limits and I feel like that its possible to assume that either less time taken or less rate limits triggered but maybe not both) but I think its acceptable to release it under 0.8.6
temporarily and then succeed it with 0.8.7
as quickly as possible.
The fix that I've hacked together uses a barrier to stall all requests uniformly which gets controlled by the request that first gets rate limited. Unfortunately once the barrier gets lifted it's essentially a mad max free for all zerg rush for all stalled requests leading to more frequent rate limiting responses feeding a frequent stall, release, stall cycle until the amount of requests being made lowers enough to allow the API to recover from the clients madness.
Additionally that doesn't stop the possibility of choking out the entire pipeline leading to the same exception being raised, it merely delays it as much as possible. To combat this I've added a fallback stalling around the entire process, so when a request gets choked out the stalling falls back to an exponential backoff algorithm supplied by the new backoff
dependency.
Ideally the two systems would get merged into one to avoid unnecessary overhead and allow some sort of slow bleed of requests back into the http session, but I'm not sure of how to tackle this and am experimenting in the dark for the time being.
from spotify.py.
Related Issues (20)
- Impossible to play list of tracks HOT 2
- Player.progress_ms is only set once and not updated HOT 4
- 'Payload' referenced before assignment HOT 3
- Spotify Register Problem HOT 4
- Issue with session closing HOT 3
- Example with an async http server HOT 4
- OAuth needs "scopes" oauth2_url use "scope"
- edit_playlist issues when calling change_playlist_details
- Can't use parameters with user.top_artists() HOT 2
- Right, so I've investigated the issue and have a possible fix ready. HOT 2
- Playlist attribute `tracks` is always None HOT 8
- Suggestion: allow User.recently_played() to take same optional parameters as Http.recently_played() HOT 1
- Suggestion: add support for "Add an Item to the User's Playback Queue" endpoint
- When creating a Playlist using the result of HttpClient.get_playlist() an error occurs
- Todo: Support new "Podcasts API"
- When using album.get_all_tracks(), an error is produced. `'Nonetype' has no attribute 'images'` HOT 2
- TypeError: __await__() returned a coroutine HOT 2
- Receiving an error when trying to initialize player HOT 2
- Not possible to install spotify.py and pyspotify at the same time HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spotify.py.