API rate limits about xeroizer

[waynerobinson]

Hi Malc

I'm happy for this to go into the library. I've been thinking a little how to manage this myself as my app needs to do a rather extensive sync when the company is first linked (and heaven forbid Xero provides invoice line items in a complete invoice listing).

The API should already raise Xeroizer::OAuth::RateLimitExceeded exceptions when Xero does however, I would like to pre-emptively prevent these from occurring in the first place. I was going to implement this in my app as an evented, asynchronous queue that on receiving an exception, pauses until it is likely to be able to succeed again.

The biggest issue I see to implementing this solution in Xeroizer is the API would probably have to change significantly to allow for asynchronous-based messaging and callbacks and I'm not sure this would be the best thing for the most common use-cases of Xeroizer.

I'm happy to listen to other ideas though.

[malclocke]

Our current thoughts around 'the simplest thing that will work' is to cache 'last request', then microsleep until 1 second has expired from that time on the next call if required.

Would like to have this as a switch on the gateway object, e.g. @gateway.governor = true as it may not be required all the time, and default to false.

At the moment we're doing sleep 1 in our app for each call, which is, well ... less than satisfactory.

[waynerobinson]

Well, the rate limiter actually allows up to 60 requests per minute, so maybe implementing some type of sliding window that only allows 60 requests per second and then pauses for up to a second (if required) before allowing more requests. I don't think it is really worthwhile trying to keep track of the 1,000 requests per day.

From a simplicity point of view, maybe if we just catch the rate exceptions in the HTTP library and, if one is raised we wait for x and then auto-retry transparently to the client.

[malclocke]

Agree re daily limit.

Also agree re catching rate limit exceptions, will implement next week and send a push.

Malc

[malclocke]

No way to attach a pull request to an existing issue, so moving to #14