Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arbitrary prediction time transforms #209

Merged
merged 4 commits into from
Dec 2, 2024
Merged

Arbitrary prediction time transforms #209

merged 4 commits into from
Dec 2, 2024

Conversation

ziw-liu
Copy link
Collaborator

@ziw-liu ziw-liu commented Nov 20, 2024

Use a standalone module to perform arbitrary forward and inverse transforms at prediction time. This is also the first step of moving to dependency injection base architecture configuration for our lightning modules.

Below is an example config field to perform a 2x XY zoom with linear interpolation before inference, and a 0.5 XY zoom with area interpolation (no aliasing) with afterwards to restore the image size. This is roughly what @tayllatheodoro has been doing manually for the virtual staining of OPS data (cc @ieivanov).

model:
  class_path: viscy.translation.engine.AugmentedPredictionVSUNet
  init_args:
    model:
      class_path: viscy.unet.networks.fcmae.FullyConvolutionalMAE
      init_args:
        in_channels: 1
        out_channels: 2
        encoder_blocks: [3, 3, 9, 3]
        dims: [96, 192, 384, 768]
        decoder_conv_blocks: 2
        stem_kernel_size: [5, 4, 4]
        in_stack_depth: 15
        pretraining: false
    forward_transforms:
      - class_path: viscy.transforms.BatchedZoom
        init_args:
          scale_factor: [1, 2, 2]
          mode: trilinear
    inverse_transforms:
      - class_path: viscy.transforms.BatchedZoom
        init_args:
          scale_factor: [1, 0.5, 0.5]
          mode: area

@ziw-liu ziw-liu requested a review from edyoshikun November 20, 2024 18:06
@ziw-liu ziw-liu added enhancement New feature or request translation Image translation (VS) labels Nov 20, 2024
@ziw-liu ziw-liu added this to the v0.4.0 milestone Nov 20, 2024
@ziw-liu ziw-liu changed the base branch from main to simple-cache November 20, 2024 18:07
@ziw-liu ziw-liu marked this pull request as ready for review November 20, 2024 18:08
@ziw-liu
Copy link
Collaborator Author

ziw-liu commented Dec 2, 2024

@edyoshikun I'm merging this to simplify workflows. Please review in #195.

@ziw-liu ziw-liu merged commit 96313fa into simple-cache Dec 2, 2024
4 checks passed
@ziw-liu ziw-liu deleted the predict-scale branch December 2, 2024 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request translation Image translation (VS)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant