flow.log: Rate-limit individual messages, possibly via macro magic.

[ygoldfeld]

Filed by @ygoldfeld pre-open-source:

The specific feature involves a coder, at a given log call site, specifying that that message should be rate-limited in a certain way. Internally some macro stuff happens that basically counts how much it has been logged lately (invisibly to coder) and doesn't log it beyond a certain point. It would probably log something like <...message logged 7,443 more times...>.

We have some code elsewhere, with notes in a related ticket at Akamai, that can be used as conceptual inspiration. Maybe we can do it more prettily than that, but from what I recall it was pretty clever actually.

According to certain devs and architect:

"did not require changes to it for 10+ years."

"the general idea is, that these macros add a number of static variables that do the rate limit calculation. It has only one downside: inside of a C++ block {} only a single of such rate limit macros can be used."

"The clever implementation detail of the is that the rate limit calculation happens before the printf() style log function is evaluated. If the rate limit says that the log should not be printed, then the printf() stuff doesn't happen and thus CPU cycles are not wasted."