codeship / go-retro

Retryable errors in Golang

Retro - Retryable errors in Golang

⚠️ Note: This library is no longer maintained ⚠️

Retry allows you to wrap failure prone sections of code in retry blocks, and conditionally retry different types of errors. To retry an error within a block, simply return a custom error type matching the provided interface. By default, wrapping code in a retro block will have no effect.

Usage

// Wrap your code block with retro
finalErr := retro.DoWithRetry(func() error{
    // Hit an external API
    return DoMyFunc(args)
})

Any errors matching the RetryableError interface bubbled up will automatically retry the code block.

For more detailed examples see the examples folder.

Independent sections of code within a chain can be wrapped conditionally in retro blocks as needed. You should always try and keep retry blocks as small as possible to reduce code and requests being re-run.

Retrying

You can use two types of provided error retry patterns, or create your own:

Static retryable errors

These errors retry X times every Y seconds. To create a static retryable error use retro.NewStaticRetryableError(err error, maxAttempts, waitSeconds int). For generic errors you can reference a shared variable, for dynamic errors you can create an error with the relevant base error message each time. Error retrying state is stored in the retro loop, not the RetryableError.

Backoff retryable errors

These errors use quadrilateral equation based on Sidekiq retries to increasingly space out subsequent attempts. To create a backoff retryable error use retro.NewBackoffRetryableError(err error, maxAttempts int). For generic errors you can reference a shared variable, for dynamic errors you can create an error with the relevant base error message each time. Error retrying state is stored in the retro loop, not the RetryableError.

Retry attempts

The retro retry loop will keep track of how many times it has looped. Any time it gets an error it compares the state against the allowed retry conditions for the latest error. This means that should the loop initially retry with an error allowing ten retries, if the second error indicates only two retries or fewer are allowed, then the loop will no longer retry since it has already retried twice.

go-retro's Issues

Update Go-retro to support background workers, and goroutine handoff

Currently this works in the case where we have a synchronous set of goroutines processing work, such as with a web handler. With a limited worker pool the sleep methodology of this library makes it inefficient. A better approach would be to support some kind of handoff and a future execution time determine by the error type returned, e.g.

// Wrap your code block with retro
finalErr, retryAt := retro.DoWithAsyncRetry(func() error{
    // Hit an external API
    return DoMyFunc(args)
})

if finalErr != nil {
  logger.Error("my stuff failed :(")
  return finalErr
}

// code did not complete, need to retry
if retryAt != nil {
  // if Im using a worker queue I can specify that this task should not be handled until this time
  time.Sleep(*retryAt - time.Now())
  // try again, but in a more reusable way than this
  finalErr, retryAt := retro.DoWithAsyncRetry(func() error{
    // Hit an external API
    return DoMyFunc(args)
  })
} else {
  // error was nil, no retry needed
  return nil
}

maxAttempts at retro.NewStaticRetryableError doesn't behave as maxAttempts

The current implementation counts the number of attempts beginning from 0 instead of begin from 1.

infinite retries

I am suggesting add infinite retries feature.
For example if we use -1 in NewStaticRetryableError.

ErrNetwork = retro.NewStaticRetryableError(errors.New("error: Max retries attempts reached"), -1, 2)

Don’t just check errors, handle them gracefully

Compare strings to check errors is considered a very bad practice, like this example from the README:

if err.Error() == "EOF" {

Instead I recommend this reading http://dave.cheney.net/2016/04/27/dont-just-check-errors-handle-them-gracefully