Optimize farm and cache identification

[nazar-pc]

Currently farming cluster operates on farms and caches without any knowledge of "farmer".

This works and is flexible, but results in a bit too many notifications from identification messages, which #2899 improve a little bit.

It might be beneficial to also associate farms with farmers (and similarly for caches), just some ephemeral identifier, such that we can reduce identification broadcast from one per farm to one per farmer most of the time.

Farm identification as such should likely be replaced with a stream request such that farmer can send a stream of responses back to controller at a sustainable rate rather than bursting potentially thousands of messages all at once that might be dropped by NATS server due to buffer limits.

[tediou5]

let me work on this. Its looks easy

[tediou5]

@nazar-pc I have an experimental implementation repo, but here i have a problem.

In the #2945 (comment), we mentioned support quick farmer restarts. But while coding, I found out that the farm's fingerprints include randomly generated ids each time. So how do we implement this feature?
(I've already finished the rest of farmer, so you can take a look at it in advance

[nazar-pc]

But while coding, I found out that the farm's fingerprints include randomly generated ids each time. So how do we implement this feature?

Check how we generate fingerprints right now. Essentially fingerprint of the farmer would be a generalization of the fingerprints of individual farms. The idea is to be able to tell whether it is the same farmer with the same set of farms in the same state or if some changes have happened to it during restart.

[tediou5]

@nazar-pc What do you think about me generating the fingerprint through the hash of plotted_sectors.

My current implementation is generate fingerprints from a randomly farmer ID for each startup and total_sectors_count for each farm , The concrete realisation is here.

Or I can support quick restarts in another pr?

[nazar-pc]

Have you see what is currently used as fingerprint? You can literally do the same thing and hash it together.