pageserver: add time based image layer creation check

[VladLazar]

Problem

Assume a timeline with the following workload: very slow ingest of updates to a small number of keys
that fit within the same partition (as decided by KeySpace::partition). These tenants will create small
L0 layers since due to time based rolling, and, consequently, the L1 layers will also be small.

Currently, by default, we need to ingest 512 MiB of WAL before checking if an image layer is required.
This scheme works fine under the assumption that L1s are roughly of checkpoint distance size, but
as the first paragraph explained, that's not the case for all workloads.

Summary of changes

Check if new image layers are required at least once every checkpoint timeout interval.

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[koivunej]

This should work. Sadly I cannot see a way to test this, except in staging.

[VladLazar]

Sadly I cannot see a way to test this, except in staging.

That was my plan: get it staging asap and validate the metric goes down

[skyzh]

LGTM and I probably also need to modify the metadata image layer generation trigger.

[github-actions]

3111 tests run: 2984 passed, 0 failed, 127 skipped (full report)

---

Flaky tests (1)

Postgres 14

• test_basebackup_with_high_slru_count[github-actions-selfhosted-sequential-10-13-30]: release

Code coverage* (full report)

• functions: 32.6% (6931 of 21275 functions)
• lines: 50.0% (54495 of 108968 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
5c3cdff at 2024-07-05T12:28:31.364Z :recycle:}

Dismissing because I am unsure about the time+size based

[jcsp]

The major change here is that tiny tenants can generate image layers up to every 48 hours, whereas previously they'd never generate any at all: that's okay, but let's monitor it carefully to see how much extra work we're doing.

[VladLazar]

The major change here is that tiny tenants can generate image layers up to every 48 hours, whereas previously they'd never generate any at all: that's okay, but let's monitor it carefully to see how much extra work we're doing.

Will keep an eye on it