Coder Social home page Coder Social logo

django-celery-token-bucket's People

Contributors

esquire-gh avatar lennylip avatar lociii avatar regiohelden-dev avatar

Stargazers

avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

esquire-gh lociii monai knaperek lennylip

django-celery-token-bucket's Issues

multiple buckets for same task

Hi, I'm looking for a solution to have multiple buckets for the same task.

E.g. Im calling an api which has a 3600 per hour limit and a 100 request per second limit.

@rate_limit(token_bucket_name=['3600_bucket, 100_bucket'], countdown=2)  # the decorator has to make sure that both buckets have more than 1 token
def my_task(self):
#  3600_bucket --
# 100_bucket --

Is something like this possible to implement?

token_bucket_refill tasks starts to grow in exponents

Thanks for the app! I have this conf and some time it work good

CELERY_TOKEN_BUCKETS: Dict[str, TokenBucket] = {
    "api_client": TokenBucket(  # one per second
        name="api_client",
        schedule=timedelta(seconds=1),
        amount=1,
        maximum=1,
    )
}

But in some time pending tasks starts to grow in exponents. Any ideas?

Something like that:

/data # redis-cli llen celery
(integer) 1743
/data # redis-cli LINDEX celery 0
"{\"body\": \"W1siYXV0b190cmFkZV9hcGlfY2xpZW50Il0sIHt9LCB7ImNhbGxiYWNrcyI6IG51bGwsICJlcnJiYWNrcyI6IG51bGwsICJjaGFpbiI6IG51bGwsICJjaG9yZCI6IG51bGx9XQ==\", \"content-encoding\": \"utf-8\", \"content-type\": \"application/json\", \"headers\": {\"lang\": \"py\", \"task\": \"django_celery_token_bucket.tasks.token_bucket_refill\", \"id\": \"1dff1f2f-89d4-44eb-9239-344686f96c39\", \"shadow\": null, \"eta\": null, \"expires\": null, \"group\": null, \"group_index\": null, \"retries\": 0, \"timelimit\": [null, null], \"root_id\": \"1dff1f2f-89d4-44eb-9239-344686f96c39\", \"parent_id\": null, \"argsrepr\": \"['api_client']\", \"kwargsrepr\": \"{}\", \"origin\": \"gen1@00b655a3bc97\", \"ignore_result\": false}, \"properties\": {\"correlation_id\": \"1dff1f2f-89d4-44eb-9239-344686f96c39\", \"reply_to\": \"9850bc41-8ce5-3835-9620-3c1b81cfb719\", \"periodic_task_name\": \"api_client\", \"delivery_mode\": 2, \"delivery_info\": {\"exchange\": \"\", \"routing_key\": \"celery\"}, \"priority\": 0, \"body_encoding\": \"base64\", \"delivery_tag\": \"397e49ee-f9f0-4a9a-abe7-c1a69f5f6319\"}}"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.