Skip to content

Improve the retry strategy #155

@rhamzeh

Description

@rhamzeh

Checklist

Describe the problem you'd like to have solved

Retry-After is a standard header used by APIs to indicate when the SDK can retry.

The SDKs should:

  • Honor this header on 429s
  • Expose this header value in the error when received
  • Fallback to exponential retry when this header is not available (e.g. not sent by the server on 429s or on e.g. 500s)
  • Drop support for retrying based on X-Rate-Limit-Reset (currently only .NET SDK supports that), though still expose it in the logs

Current State

SDK Retries on Default Num Retries Max Num Retries State
Java 429s, 500s except for 501 3 15 Does not consider headers. Does not implement exponential backoff. Adds a fixed 100ms delay to each retry

Describe the ideal solution

Retry On

  • For all requests:
    • Retry on 429s, honor Retry-After when sent by the server, falling back to exponential backoff when it's not.
    • Retry on 5xxs (except 501 not implemented), honor Retry-After when sent by the server, falling back to exponential backoff when it's not
    • Retry on network errors using exponential backoff

Max Allowable Retries

15

Default Number of Retries

SDKs: 3

Retry Parameters

  1. If Retry-After header is found, use it

    1. if it is an integer, treat it as the number of seconds from now to retry, if it is <1 from now or >1800 from now (aka >30 min) - assume it is invalid and continue
    2. if it is a date, parse it but if it is <1 from now or >1800 from now (aka >30 min) - assume it is invalid and continue
  2. If neither header is found, use exponential backoff but we'll add some jitter, so the retry is a random number between

    1. 2^loopCount * 100ms and 2^(loopCount + 1) * 100ms
    2. if the result of (a) is > 120s, cap it at 120s which should happen between the 8th and 9th retry

That means:

  • if retry-after header was returned and is valid, we’ll use it - so if it says in 4 min all good
  • if retry-after header was not returned, we will retry at:
    • 100ms
    • 200ms
    • 400ms
    • 800ms
    • 1.6s
    • 3.2s
    • 6.4s
    • 12.8s
    • 25.6s
    • 51.2s
    • 102.4s
    • 120s ← at this point is is >4min since initial call
    • 120s
    • 120s
    • 120s

Alternatives and current workarounds

No response

References

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions