Rate limiter

Xu, vol. 1 (chapter 4)

sequenceDiagram
	participant c as Client
	participant m as Middleware
	participant r as Redis
	participant s as Rule store
	participant d as Downstream
m --) s: Fetch rules periodically
c -) m: HTTP request
m -) r: Prune and fetch sorted request set
m -) m: Decide
alt Throttled
	m -) c: Return error
else Accepted
	m --) r: Append request to request set
	m -) d: Propagate request
	d -) c: Return response	
end

Notes:

• When using multiple rate limiter middleware instances, storing a sorted set of user requests avoids race conditions inherent in storing a mere count.
• Redis’ native leader-based, asynchronous replication can handle scaling to arbitrary size. Given the use case, we can certainly tolerate the risk of missing some requests during a failover.