ericghara / redisratelimiter

A rate limiter using Redis

Redis Rate Limiter

A service which ensures that events occur only once in a period of time (event-duration).

Modes

Only Once

A typical sliding window limiter, allowing only the first of duplicated events during the event-duration.

Example

Event Duration: 3

Times and Events: {1, 🍎}, {2, 🌮}, {3, 🌮}, {4, 🌮}, {7, 🌮}

Allowed: {1, 🍎}, {2, 🌮}, {7, 🌮}

Strictly Once

A sliding window limiter, where if duplicate events occur during the event-duration, the later event(s) are rejected and the conflicting earlier event is invalidated. In short, at least event-duration time should separate events, otherwise no events are valid.

Example

Event Duration: 3

Times and Events: {1, 🍎}, {2, 🌮}, {3, 🌮}, {4, 🌮}, {7, 🌮}

Allowed: {1, 🍎}, {7, 🌮}

Event Lifecycle

• Submitted - Potentially publishable at the time of receipt. An event will not be submitted if it conflicts with another, previously submitted event. This is the entry point into the event lifecycle.
• Invalidated - A submitted event is invalidated when a subsequently received event creates a conflict. This is a terminal state transition, an invalidated event can never be published.
• Published - No rate limit rules were violated, for a submitted event, throughout the event duration. This is a terminal state transition, a published event cannot be invalidated.

Synchronization

Event status updates are pushed using STOMP over Websocket. Snapshots of incubating events (events submitted but not yet published) are periodically pushed. Combined, these allow real time synchronization with other services.

All status updates and snapshots are sent with a scalar clock. It is guaranteed that clock values are unique and an event with a lesser clock value happened-before an event with a greater clock value. The state of incubating events can therefore be modeled. A parallel can be drawn to video compression: snapshots can be considered key-frames and status updates, delta-frames.

The synchronization is not partition tolerant.

Build & Run

Requirements: Docker, Docker compose and Java 19 (uses preview features)

Start RateLimiter and Redis: gradle bootRun

Stop Redis: gradle stopRedis

redisratelimiter's Issues

Add Conditional Delete to EventMapService

A conditional delete that deletes events older than a certain timestamp would be useful for cleaning up the event map to remove old/expired events.

The EXPIRE command of Redis should not be used as in this way events could expire without being handled prior to expiration.

EventMapService should support delete event functionality

EventMapService should provide a higher level access to the new dao.EventMap#DeleteEvent feature implemented in PR #4.

Add a service to expire old events

Events in the EventMap are not cleared after expiration which could lead to high resource utilization over time. The EventQueue (see: #7) provides an ideal data structure to poll for expired events.

A service should be developed to expire old events by polling them from the EventQueue and removing them from the EventMap.