Implement crash recovery mechanism

[hidmic]

Feature description

Benchmarks with many parameters running against large datasets will invariably take a long time. Right now, if the benchmark is interrupted or crashes, there is no way to resume from where it was. We have to start over or make do with whatever we got before the crash. For a benchmark that needs 3 days to run, this is incredibly wasteful.

We need a mechanism to recover from crashes like this. Grabbing core dumps and system logs would also be useful for post-mortem analysis.

Implementation considerations

Perhaps we can take some ideas from filesystem journaling, logging benchmark runs' state.