Logs with configurable cooldown

[amab8901]

Is your feature request related to a problem? Please describe.

My program produces a variety of warn logs. Some warn logs are triggered frequently to the point of being spammy, thus making it hard to see if there's any other warn logs that give different insights about what went wrong. The spam is especially problematic when the program is running on a resource-limited device that's passively running in the background with only occasional checks by technicians.

Describe the solution you'd like

A rate-limited alternative to warn!() where each instance of warn!() has its own cooldown independent of other instances:

cooled_warn!("message")

In the tracing_subscriber, I'd like to have the ability to set the following configurations for the cooldown:

• cooldown duration (e.g. 1 min)
• what to do when the log gets triggered before cooldown:
  • silence
  • same log message but trace! level
  • same log message but debug! level
  • same log message but [insert log level]
• how long the cooldown must be repeatedly violated (i.e. attempted to get triggered before cooldown has passed) before escalating to a longer cooldown
• cooldown duration in escalated mode (e.g. 1h)
• how long the escalated mode needs to go without cooldown violations (i.e. attempts to trigger the log before cooldown has passed) in order to reset to normal mode (e.g. 1 min cooldown)

Describe alternatives you've considered

Alternative 1:

I'm currently doing the following on some parts of my program

1. write a function that extends the behavior of warn!()

pub(crate) fn warn_log_if_not_spammy_otherwise_info_log(
    replace_high_severity_logs_with_info: &mut bool,
    message: &str,
    start_time: &Instant,
    timestamp_of_last_high_severity_log: &mut Duration,
    pre_disconnection_timestamp: &Duration,
) {
    if *replace_high_severity_logs_with_info {
        info!(message);
        let timestamp_now = start_time.elapsed();
        let time_since_last_high_severity_log =
            timestamp_now - *timestamp_of_last_high_severity_log;

        let time_since_pre_disconnection = timestamp_now - *pre_disconnection_timestamp;

        if time_since_pre_disconnection.as_secs() < 3600 {
            if time_since_last_high_severity_log.as_secs() >= 60 {
                *replace_high_severity_logs_with_info = false;
            }
        } else if time_since_last_high_severity_log.as_secs() >= 3600 {
            *replace_high_severity_logs_with_info = false;
        }
    } else {
        warn!(message);
        *replace_high_severity_logs_with_info = true;
        *timestamp_of_last_high_severity_log = start_time.elapsed();
    }
}

2. use this function instead of warn!():

warn_log_if_not_spammy_otherwise_info_log(
    replace_high_severity_logs_with_info,
    message,
    start_time,
    timestamp_of_last_high_severity_log,
    pre_disconnection_timestamp,
);

Alternative 2:

cooled_warn!() implemented in tracing crate as a separate macro than warn()

Alternative 3:

Adding optimal cooling configurations to warn() via tracing_subscriber

Additional context

I applied the above solution to some selected parts of the program, but I can't really do this on all parts of the program because it would make the program a lot more verbose and it would require a lot of work to change each individual warn!() with the above function since I'd have to pass all those arguments all over the program. And everything I said about warn!() applies also to error!(). And while we're at it, might as well impl this feature for INFO, DEBUG, TRACING too.