Ideas about forward sync improvements

[tbenr]

When we select a “target chain” to sync to, we consider our current “hot blocks” but only when they exceed by OPTIMISTIC_HISTORY_LENGTH (3000) the current firstNonFinalSlot. Thus in long period on no-finality we always redownload 3000 slots of block\blobs if the node goes offline for more than isCloseToInSync. This happens because, if we are not isCloseToInSync we don’t even start gossip and we don’t pretend to be able catch-up via blockByRoot lookups.
We may consider lowering that down OPTIMISTIC_HISTORY_LENGTH to reduce the overhead but the algorithm that selects the slot from which start redownloading is not that safe (see below).
3000 has been set pre-merge, when blocks where small and state transition was fast (small state too).

I think the algorithm of CommonAncestor::getCommonAncestor can be improved. The current implementation only reason about slot and doesn’t try to to check that the selected ancestor is REALLY common, by checking block roots: the remote block root at the selected slot must match the local block root. By Improving this check we could also significantly lower the “redownload overhead” . We could have an algorithm that, based on local heads, searches for remote common ancestors going backwards. So we can have higher security and lower overhead.
NOTE: the search of the exact common ancestor can be done on a single specific slot but rather inside a range (like 10?) because our head(s) can be on a fork compared to the remote canonical chain, so we need to check in the range of <ourCurrentBestHeadSlot-10,ourCurrentBestHeadSlot> and, if no match is found, retrying backwards by jumping like 100 slots back until we find a correspondence.