Add ability to scrub from last scrubbed txg #16301

oshogbo · 2024-06-26T15:01:17Z

Motivation and Context

Sometimes it is useful to only scrub a particular range of txg. Some users might want to scrub only new data because they would like to know if the new write wasn't corrupted. This PR adds possiblity scrub only newly flushed data.

Description

Users who would like to scrub only new data, a new zpool property was introduced, that stores last scrubbed txg, so users can run scrub only from the last saved point.
We use a scn_max_txg and scn_min_txg which are already built into scrub, to accomplish that.

How Has This Been Tested?

Via test cases.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Performance enhancement (non-breaking change which improves efficiency)
Code cleanup (non-breaking change which makes code smaller or more readable)
Breaking change (fix or feature that would cause existing functionality to change)
Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
Documentation (a change to man pages or other documentation)

Checklist:

My code follows the OpenZFS code style requirements.
I have updated the documentation accordingly.
I have read the contributing document.
I have added tests to cover my changes.
I have run the ZFS Test Suite with this change applied.
All commit messages are properly formatted and contain Signed-off-by.

man/man7/zpoolprops.7

man/man8/zpool-scrub.8

module/zfs/spa.c

module/zfs/dsl_scan.c

tonyhutter · 2024-08-27T23:05:45Z

I noticed a cannot scrub tank: delegated administration is disabled on pool error when trying to continue (-C) on a paused scrub:

$ sudo ./zpool scrub tank && sleep 1 && sudo ./zpool scrub -p tank
$ sudo ./zpool status
  pool: tank
 state: ONLINE
  scan: scrub paused since Tue Aug 27 16:02:26 2024
	scrub started on Tue Aug 27 16:02:25 2024
	14.7G / 14.7G scanned, 3.22G / 14.7G issued
	0B repaired, 21.95% done
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  vdb       ONLINE       0     0     0

errors: No known data errors

$ sudo ./zpool scrub -C tank
cannot scrub tank: delegated administration is disabled on pool

$ echo $?
1

$ sudo ./zpool status
  pool: tank
 state: ONLINE
  scan: scrub repaired 0B in 00:00:14 with 0 errors on Tue Aug 27 16:02:39 2024
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  vdb       ONLINE       0     0     0

errors: No known data errors

It returns an error code, but then the scrub seems to complete successfully. Any thoughts?

oshogbo · 2024-08-29T10:02:00Z

@tonyhutter This is after the applied patch, but without it, it works fine?

oshogbo · 2024-08-29T10:08:32Z

I have added another feature just to scrub "recent data".

cmd/zpool/zpool_main.c

man/man8/zpool-scrub.8

module/zfs/dsl_scan.c

amotin · 2024-10-31T01:48:44Z

@oshogbo Would you rebase and resolve the test failures?

oshogbo · 2024-10-31T12:05:49Z

@amotin Yes, let my try to fix that.

cmd/zpool/zpool_main.c

module/zfs/dsl_scan.c

amotin

Thank you. I have no objections for this variant.

allanjude

Reviewed-by: Allan Jude <[email protected]>

behlendorf

A few small comments but nothing major. This seems like a good first step for an interface and something we can build on.

man/man7/zpoolprops.7

include/sys/dmu.h

oshogbo · 2024-11-18T11:00:47Z

@behlendorf @allanjude @amotin Thank you for the review.

Sponsored-By: Wasabi Technology, Inc. Sponsored-By: Klara Inc. Signed-off-by: Mariusz Zaborski <[email protected]>

The `last_scrubbed_txg` property indicates the transaction group (TXG) up to which the most recent scrub operation has checked and repaired the dataset. This provides administrators with insight into the data integrity status of their pool at a specific point in time. Sponsored-By: Wasabi Technology, Inc. Sponsored-By: Klara Inc. Signed-off-by: Mariusz Zaborski <[email protected]>

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from 68edb6e to 42d967f Compare June 26, 2024 15:29

allanjude reviewed Jul 11, 2024

View reviewed changes

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from 42d967f to 8d59c2c Compare August 16, 2024 11:22

behlendorf added the Status: Code Review Needed Ready for review and testing label Aug 16, 2024

behlendorf self-requested a review August 16, 2024 23:53

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from 8d59c2c to cc72197 Compare August 29, 2024 10:07

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from cc72197 to 4823cc2 Compare August 29, 2024 10:22

amotin reviewed Sep 6, 2024

View reviewed changes

cmd/zpool/zpool_main.c Show resolved Hide resolved

cmd/zpool/zpool_main.c Outdated Show resolved Hide resolved

cmd/zpool/zpool_main.c Outdated Show resolved Hide resolved

man/man8/zpool-scrub.8 Outdated Show resolved Hide resolved

module/zfs/dsl_scan.c Outdated Show resolved Hide resolved

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch 2 times, most recently from 2a34659 to d9c7d38 Compare October 16, 2024 14:39

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from d9c7d38 to f108d90 Compare October 31, 2024 11:58

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from f108d90 to b08bd76 Compare October 31, 2024 15:18

amotin reviewed Nov 7, 2024

View reviewed changes

cmd/zpool/zpool_main.c Show resolved Hide resolved

module/zfs/dsl_scan.c Outdated Show resolved Hide resolved

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch 3 times, most recently from e5635df to 77f2265 Compare November 14, 2024 21:17

amotin approved these changes Nov 15, 2024

View reviewed changes

allanjude approved these changes Nov 15, 2024

View reviewed changes

behlendorf reviewed Nov 15, 2024

View reviewed changes

man/man7/zpoolprops.7 Outdated Show resolved Hide resolved

man/man7/zpoolprops.7 Outdated Show resolved Hide resolved

man/man7/zpoolprops.7 Outdated Show resolved Hide resolved

include/sys/dmu.h Show resolved Hide resolved

oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from 77f2265 to 0e0cd2a Compare November 18, 2024 11:00

oshogbo added 2 commits November 18, 2024 12:00

Redesign scan/scrub interfact to allow scrubing range of TXGs

4438e47

Sponsored-By: Wasabi Technology, Inc. Sponsored-By: Klara Inc. Signed-off-by: Mariusz Zaborski <[email protected]>

behlendorf approved these changes Nov 19, 2024

View reviewed changes

behlendorf removed the Status: Code Review Needed Ready for review and testing label Nov 19, 2024

behlendorf added the Status: Accepted Ready to integrate (reviewed, tested) label Nov 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to scrub from last scrubbed txg #16301

Add ability to scrub from last scrubbed txg #16301

oshogbo commented Jun 26, 2024

tonyhutter commented Aug 27, 2024

oshogbo commented Aug 29, 2024

oshogbo commented Aug 29, 2024

amotin commented Oct 31, 2024

oshogbo commented Oct 31, 2024

amotin left a comment

allanjude left a comment

behlendorf left a comment

oshogbo commented Nov 18, 2024

Add ability to scrub from last scrubbed txg #16301

Are you sure you want to change the base?

Add ability to scrub from last scrubbed txg #16301

Conversation

oshogbo commented Jun 26, 2024

Motivation and Context

Description

How Has This Been Tested?

Types of changes

Checklist:

tonyhutter commented Aug 27, 2024

oshogbo commented Aug 29, 2024

oshogbo commented Aug 29, 2024

amotin commented Oct 31, 2024

oshogbo commented Oct 31, 2024

amotin left a comment

Choose a reason for hiding this comment

allanjude left a comment

Choose a reason for hiding this comment

behlendorf left a comment

Choose a reason for hiding this comment

oshogbo commented Nov 18, 2024