Consume with periodic sync to reduce number of requests

Consume with periodic sync to reduce number of requests

Note, this function sacrificing consistency to reduce calls to an external store.

const Redis = require('ioredis');
const redisClient = new Redis({ enableOfflineQueue: false });

const opts = {
  points: 100,
  duration: 60,
};

const rateLimiterMemory = new RateLimiterMemory(opts);
const rateLimiterRedis = new RateLimiterRedis({
  storeClient: redisClient,
  ...opts,
});

async function consumeWithPeriodicSync(key, syncEveryNRequests = 10) {
  let memoryRes = await rateLimiterMemory.consume(key);

  if (memoryRes.consumedPoints % syncEveryNRequests === 0) {
    let redisRes;
    try {
      redisRes = await rateLimiterRedis.consume(key, syncEveryNRequests);
    } catch (error) {
      if (!(error instanceof Error)) {
        // If it's not some Redis error (ref: https://github.com/animir/node-rate-limiter-flexible/wiki/Redis#usage)
        // we can assume it's insufficent points error, then update our local rate limiter
        await rateLimiterMemory.set(
          key,
          redisRes.consumedPoints,
          redisRes.msBeforeNext / 1000
        );
        memoryRes = redisRes;
      }

      // rethrow the error to handle it in the calling code
      throw error;
    }
  }

  return memoryRes;
}

Usage

consumeWithPeriodicSync(remoteAddress, 10)
  .then((rateLimiterRes) => {
    // ... Some app logic here ...
  })
  .catch((rejRes) => {
    if (rejRes instanceof Error) {
      // Some Redis error
    } else {
      // Can't consume
      // If there is no error, rateLimiterRedis promise rejected with number of ms before next request allowed
      const secs = Math.round(rejRes.msBeforeNext / 1000) || 1;
      res.set("Retry-After", String(secs));
      res.status(429).send("Too Many Requests");
    }
  });

In order to sync the rate limiter value between multiple servers running the rate limiter, we use 2 rate limiters in parallel in each server: Local (memory) and the global (Redis in this example) rate limiter. We are sacrificing some consistency to reduce calls to an external store, by only syncing the value on every n points consumed.

Why reduce calls to an external store? Mostly to reduce latency, because the server has to communicate with a remote store. Which is especially difficult (and costly?) if we were to have multiple servers, then the store also has to be replicated, etc. This "hack" makes it so we still benefit from the fast in memory rate limiting, but still being able to sync the rate limit across different nodes/servers.