Prune state cache based on size

[michaelsproul]

Once we can measure the size of the state cache in memory (see issue linked below) we can add some heuristics to prune the state cache to fit within a certain memory bound, e.g. 4 GB.

These heuristics might be quite subtle to get right, as the differential size of items in the cache depends on the order in which they're added. We might want to combine the existing heuristics that target certain kinds of states (e.g. old epoch boundary states) with the size heuristics, or just keep greedily pruning states one-at-a-time until we're below the bound (although this could result in very few states in the cache in the worst case).

Depends on:

• Measure state cache size in memory #7449

[michaelsproul]

Related to this:

• Improve & unify parallel de-duplication caches #5112