-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use patch_size instead of chunk_size as base shape for sampling #4
Conversation
…nput image's chunk sizes
…maller than patch sizes
I'm resuming the conversation from PR #6 here @ClementCaporal. I'll be working in the next step to solving #3 and #5, by adding a way to extract overlapped patches. The following step would be to allow Padding is necessary because torch's |
…ow overlapping patches extraction
Hello @fercer. Thank you for the explanations. I will try this new implementation as soon as the overlap sample is ready. (I have my own small patch meanwhile) Have a good day, Clément |
Thanks for your contribution to improve ZarrDataset @ClementCaporal! |
patch_sampler = zds.PatchSampler(patch_size=patch_size, pad=pad, allow_incomplete_patches=True) | ||
``` | ||
|
||
Create a dataset from the list of filenames. All those files should be stored within their respective group "0". |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there is a typo here for group "0"
, should it be "4"
in this example?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for noticing this @ClementCaporal! I considered this change and added it to a recent PR #8 that addresses an incorrect sampling of masked regions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh Nice!
I was starting to use masked regions on friday and started noticing strange behavior so I just have to pull now thanks to you!
Have a good week,
Clément
This change allows
zarrdataset
to extract samples using as reference thepatch_size
instead of the input's chunk size.Basically, chunks are now considered as multiples of
patch_size
and therefore patches can be extracted without separation.This is helpful when using
zarrdataset
for inference on larger-than-memory inputs.No changes are needed
ImageBase
class since it can handle loading adjacent chunks. This will increase the memory usage if the patch size is not multiple of the chunk size due to multiple chunks being loaded.There is no impact on the multi-thread capability of
zarrdataset
to use multiple workers.That is because each worker has its own handler to access the zarr file, and chunks can be read safely without collisions.