Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Support slices like '3:10:mean(2)' #391

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

danielballan
Copy link
Member

Closes #32

As proposed in #32, this will support:

  • Reducing an N-dimensional array to an array of lower dimension by averaging over one or more dimensions. For example, average an image time series like ?slice=::mean
  • Downsamplig an N-dimensional array over some axes. For example, downsampling an image time series by a factor of 2 in x and y: ?slice=:,::mean(2),::mean(2)

The downsampling with employ local averaging as in https://scikit-image.org/docs/stable/api/skimage.transform.html#downscale-local-mean

Incidentally this removes the use of eval or client input, which I like. it was technically safe because it was guarded by regex, but it still made me nervous.

@EliotGann
Copy link

This would be extremely helpful for my igor Tiled client, where viewing downsampled images is really important - saving client memory and working with slower internet speeds. It also will allow for trivial data reduction for already resampled data. Essentially I will be able to construct a URL for users to pull valid 1D scattering curves from 2D datasets.

Copy link
Contributor

@gwbischof gwbischof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be rebased.
The test should be updated to test downsampling and averaging.
Client should be updated to support these updates.

@danielballan
Copy link
Member Author

This related project shows another way to spell this which I think I like better:

Leave the slice syntax as is, taking standard numpy-style slices. Add an optional query parameter downsample_method. Its default value can be stride but it can accept other values mean, max, min….

https://google.github.io/tensorstore/driver/downsample/index.html

@danielballan
Copy link
Member Author

It occurred to me later downsample_method as implemented there assumes we are using the same downsample method for all axes, whereas the original approach we considered here, ?slice=:,::mean(2),::mean(2), supports a more general case of different downsample methods on different axes.

I suppose we could combine the two and support:

?downsample_method=mean  # same on all axes
?downsample_method=mean,mean,max  # different for each axis

but that creates a dependency with the value of slice that seems a bit odd...I might be back to preferring our original plan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support reductions in slice notation, inspired by uhi
3 participants