Ability to limit concurrency? about unsync

[alex-sherman]

Functionality like that is usually left up to the application. I found a nice example on SO which I've adapted below into a handy decorator (that works with unsync).

asyncio.set_event_loop(unsync.loop)

def limit(n):
    def wrapper_gen(func):
        semaphore = asyncio.Semaphore(n)
        @wraps(func)
        async def limiter(*args, **kwargs):
            async with semaphore:
                return await func(*args, **kwargs)
        return limiter
    return wrapper_gen

@unsync
@limit(2)
async def request():
    await asyncio.sleep(1)
    print("Done")
    return "OK"

requests = [request() for i in range(10)]
print([request.result() for request in requests])

Something like this can be done with or without unsync, but IMO it's a bit easier with the single event loop. Thanks for the question!

[yan-hic]

@alex-sherman Could you please reopen ?
The decorator doesn't work anymore in Py 3.10 as Semaphore's loop has been deprecated.
But mainly, it doesn't work with a sync function returning something e.g. the json response from requests.get(). I think that's what the OP wanted to know.

One could (try to) implement aiohttp but then no need for unsync I believe.

[alex-sherman]

I'm not sure what I could do with reopening this, my point in closing it was that it can be accomplished outside of unsync. The solution I gave would work with any normal async method.

Since the deprecation of the loop parameter you can just use the unsync event loop with asyncio.set_event_loop(unsync.loop). The request method can also return a value. I updated my solution with these changes.

[yan-hic]

Thx @alex-sherman.
Semaphore didn't work out for me. I ended up using the backoff library - together with unsync - as the API I was targeting returned a specific message when over-the-limit. Not really what the OP was after as it doesn't limit the rate per se and rather retry but worked for my use case.