Vector-optics birefringence reconstruction prototype (#170)

* Phase reconstruction is invariant to voxel-size (#164) * fix bug finding focus in stack with only one slice * refactor for clarify * formatting * print -> warnings.warn * test single-slice case * fix test bugs * z-scale-invariant test object * no rescaling on output * forward simulation takes a "brightness" - simulating real microscope * fix example script * add background parameter for fluorescence forward model * test voxel-size invariance * rename I_norm -> direct_intensity * refactor to clarify discretization factor * remove comment * fix fluorescence example bug * improved docsring --------- Co-authored-by: Ivan Ivanov <[email protected]> * poster scripting * may 14 poster draft * last-minute poster changes * quick clearning * checkpoint before svd refactor * SVD refactor * pass singular system * update visualization script * fix visualization script scaling * correct phase recon regression, legacy recon assumes axially even green's function * helper functions * fluorescence wrap safety * 3d phase wrap safety * fix axial nyquist bug * 2d phase wrap safety * fix interaction between padding and wrap safety * clean defaults * refactor singular system computation * remove accidental duplicate * refactor visuals * fix warnings from tensoring a tensor * match defaults * readme type * handle napari dependency * Revert "fix warnings from tensoring a tensor" This reverts commit ca4e7c5. * Reapply "fix warnings from tensoring a tensor" This reverts commit 790df57. * revive old version of greens tensor for backwards compatibility * fix tests that fail because of napari on github * wrap-safe vector transfer function * sampling tests * fourier-space oversampling * better visualizations for debugging * complex-valued napari visuals * complex utils * fix colormaps * debugging coherent tfs * formatting * fix rotation matrices * fix dc term * cleaning up * first-pass visuals checkpoint * matplotlib visuals * temporarily turn off rotations * debugging progress * example script to generate matplotlib tf figures * add rose asset * add all assets * plot all stokes * brighter plot for greens * visual improvements * revised rotation-symmetric Green's tensor * ignore * fix green's tensor units * modify circular anisotropy gellman matrices so that all transfer function are hermitian * improved matplotlib visuals * clean up plotting script * fix bug with 3x3 hardcoded shape * update tf components * minor reconstruction updates * refactor greens tensor spectrum * clean test script * clean models * simple memory reduction * clean debug statements * reorder svd for clean i/o * invert phase contrast * formatting * padding warning * revise visuals * visual cleanup * manage large reconstructions * Transfer function visuals (#178) * Bump torch to unpin numpy (#176) * bump torch to unpin numpy * add SPEC-0 conformant numpy requirement * Bump torch to unpin numpy (#176) * bump torch to unpin numpy * add SPEC-0 conformant numpy requirement * first-pass scripts * cleanup greens * clean transfer function support * fix naming issue * Wrap-safe transfer functions (#175) * helper functions * fluorescence wrap safety * 3d phase wrap safety * fix axial nyquist bug * 2d phase wrap safety * fix interaction between padding and wrap safety * green's tensor surfaces * dark theme default --------- Co-authored-by: Ziwen Liu <[email protected]> * matplotlib movie features * cheaper default viz script * move all visuals to /examples/visuals * update examples/README to table * typo * update badges * typo * revise main README * add 10-second slideshow * Clean installation instructions * cleanup vector model example * minor cleanup * remove matplotlib plot example * edits to README * clarify contrast mechanisms in README * simplify example installation w/ `pip install waveorder[examples]` * fix unintended diff * clarify `examples/README.md` * document 3d-to-2d transfer function * test orthonormal bases * fix accidental deletion * add deprecation warnings * remove plotting assets * label transfer function and simulation axes * guard type hint * matplotlib visuals docs and type hints * add type hints throughout models * revert TYPE_CHECKING to strings --------- Co-authored-by: Ivan Ivanov <[email protected]> Co-authored-by: Ziwen Liu <[email protected]> Co-authored-by: Shalin Mehta <[email protected]>
mehta-lab · Dec 22, 2024 · 1451bf1 · 1451bf1
1 parent cbda81d
commit 1451bf1
Show file tree

Hide file tree

Showing 31 changed files with 1,858 additions and 354 deletions.
diff --git a/.gitignore b/.gitignore
@@ -150,3 +150,4 @@ recOrder/_version.py
 *.npz
 *.png
 *.tif[f]
+*.pdf
diff --git a/README.md b/README.md
@@ -1,57 +1,94 @@
 # waveorder
 
-![PyPI - Python Version](https://img.shields.io/pypi/pyversions/waveorder)
-[![Downloads](https://pepy.tech/badge/waveorder)](https://pepy.tech/project/waveorder)
 [![Python package index](https://img.shields.io/pypi/v/waveorder.svg)](https://pypi.org/project/waveorder)
-[![Development Status](https://img.shields.io/pypi/status/napari.svg)](https://en.wikipedia.org/wiki/Software_release_life_cycle#Alpha)
+[![PyPI monthly downloads](https://img.shields.io/pypi/dm/waveorder.svg)](https://pypistats.org/packages/waveorder)
+[![Total downloads](https://pepy.tech/badge/waveorder)](https://pepy.tech/project/waveorder)
+[![GitHub contributors](https://img.shields.io/github/contributors-anon/mehta-lab/waveorder)](https://github.com/mehta-lab/waveorder/graphs/contributors)
+![GitHub Repo stars](https://img.shields.io/github/stars/mehta-lab/waveorder)
+![GitHub forks](https://img.shields.io/github/forks/mehta-lab/waveorder)
+![PyPI - Python Version](https://img.shields.io/pypi/pyversions/waveorder)
+
 
 This computational imaging library enables wave-optical simulation and reconstruction of optical properties that report microscopic architectural order.
 
-## Computational label-free imaging
+## Computational label-agnostic imaging
 
-This vectorial wave simulator and reconstructor enabled the development of a new label-free imaging method, __permittivity tensor imaging (PTI)__, that measures density and  3D orientation of biomolecules with diffraction-limited resolution. These measurements are reconstructed from polarization-resolved images acquired with a sequence of oblique illuminations.
+https://github.com/user-attachments/assets/4f9969e5-94ce-4e08-9f30-68314a905db6
 
-The acquisition, calibration, background correction, reconstruction, and applications of PTI are described in the following [preprint](https://doi.org/10.1101/2020.12.15.422951):
+`waveorder` enables simulations and reconstructions of label-agnostic microscopy data as described in the following [preprint](https://arxiv.org/abs/2412.09775)
+<details>	
+<summary> Chandler et al. 2024 </summary>
+<pre><code>
+@article{chandler_2024,
+    author = {Chandler, Talon and Hirata-Miyasaki, Eduardo and Ivanov, Ivan E. and Liu, Ziwen and Sundarraman, Deepika and Ryan, Allyson Quinn and Jacobo, Adrian and Balla, Keir and Mehta, Shalin B.},
+	title = {waveOrder: generalist framework for label-agnostic computational microscopy},
+	journal = {arXiv},
+	year = {2024},
+	month = dec,
+	eprint = {2412.09775},
+	doi = {10.48550/arXiv.2412.09775}
+}
+</code></pre>
+</details>
 
-```bibtex
- L.-H. Yeh, I. E. Ivanov, B. B. Chhun, S.-M. Guo, E. Hashemi, J. R. Byrum, J. A. Pérez-Bermejo, H. Wang, Y. Yu, P. G. Kazansky, B. R. Conklin, M. H. Han, and S. B. Mehta, "uPTI: uniaxial permittivity tensor imaging of intrinsic density and anisotropy," bioRxiv 2020.12.15.422951 (2020).
- ```
+Specifically, `waveorder` enables simulation and reconstruction of 2D or 3D:
 
-In addition to PTI, `waveorder` enables simulations and reconstructions of subsets of label-free measurements with subsets of the acquired data:
+1. __phase, projected retardance, and in-plane orientation__ from a polarization-diverse volumetric brightfield acquisition ([QLIPP](https://elifesciences.org/articles/55502)),
 
-1. Reconstruction of 2D or 3D phase, projected retardance, and in-plane orientation from a polarization-diverse volumetric brightfield acquisition ([QLIPP](https://elifesciences.org/articles/55502))
+2. __phase__ from a volumetric brightfield acquisition ([2D phase](https://www.osapublishing.org/ao/abstract.cfm?uri=ao-54-28-8566)/[3D phase](https://www.osapublishing.org/ao/abstract.cfm?uri=ao-57-1-a205)),
 
-2. Reconstruction of 2D or 3D phase from a volumetric brightfield acquisition ([2D](https://www.osapublishing.org/ao/abstract.cfm?uri=ao-54-28-8566)/[3D (PODT)](https://www.osapublishing.org/ao/abstract.cfm?uri=ao-57-1-a205) phase)
+3. __phase__ from an illumination-diverse volumetric acquisition ([2D](https://www.osapublishing.org/oe/fulltext.cfm?uri=oe-23-9-11394&id=315599)/[3D](https://www.osapublishing.org/boe/fulltext.cfm?uri=boe-7-10-3940&id=349951) differential phase contrast),
 
-3. Reconstruction of 2D or 3D phase from an illumination-diverse volumetric acquisition ([2D](https://www.osapublishing.org/oe/fulltext.cfm?uri=oe-23-9-11394&id=315599)/[3D](https://www.osapublishing.org/boe/fulltext.cfm?uri=boe-7-10-3940&id=349951) differential phase contrast)
+4. __fluorescence density__ from a widefield volumetric fluorescence acquisition (fluorescence deconvolution).  
 
-PTI provides volumetric reconstructions of mean permittivity ($\propto$ material density), differential permittivity ($\propto$ material anisotropy), 3D orientation, and optic sign. The following figure summarizes PTI acquisition and reconstruction with a small optical section of the mouse brain tissue:
+The [examples](https://github.com/mehta-lab/waveorder/tree/main/examples) demonstrate simulations and reconstruction for 2D QLIPP, 3D PODT, 3D fluorescence deconvolution, and 2D/3D PTI methods.
 
-![Data_flow](https://github.com/mehta-lab/waveorder/blob/main/readme.png?raw=true)
+If you are interested in deploying QLIPP, phase from brightfield, or fluorescence deconvolution for label-agnostic imaging at scale, checkout our [napari plugin](https://www.napari-hub.org/plugins/recOrder-napari),  [`recOrder-napari`](https://github.com/mehta-lab/recOrder).
 
+## Permittivity tensor imaging 
 
-The [examples](https://github.com/mehta-lab/waveorder/tree/main/examples) illustrate simulations and reconstruction for 2D QLIPP, 3D PODT, and 2D/3D PTI methods.
+Additionally, `waveorder` enabled the development of a new label-free imaging method, __permittivity tensor imaging (PTI)__, that measures density and  3D orientation of biomolecules with diffraction-limited resolution. These measurements are reconstructed from polarization-resolved images acquired with a sequence of oblique illuminations.
 
-If you are interested in deploying QLIPP or PODT for label-free imaging at scale, checkout our [napari plugin](https://www.napari-hub.org/plugins/recOrder-napari),  [`recOrder-napari`](https://github.com/mehta-lab/recOrder).
+The acquisition, calibration, background correction, reconstruction, and applications of PTI are described in the following [paper](https://doi.org/10.1101/2020.12.15.422951) published in Nature Methods:
 
-## Correlative imaging
+<details>
+<summary> Yeh et al. 2024 </summary>
+<pre><code>
+@article{yeh_2024,
+	author = {Yeh, Li-Hao and Ivanov, Ivan E. and Chandler, Talon and Byrum, Janie R. and Chhun, Bryant B. and Guo, Syuan-Ming and Foltz, Cameron and Hashemi, Ezzat and Perez-Bermejo, Juan A. and Wang, Huijun and Yu, Yanhao and Kazansky, Peter G. and Conklin, Bruce R. and Han, May H. and Mehta, Shalin B.},
+	title = {Permittivity tensor imaging: modular label-free imaging of 3D dry mass and 3D orientation at high resolution},
+	journal = {Nature Methods},
+	volume = {21},
+	number = {7},
+	pages = {1257--1274},
+	year = {2024},
+	month = jul,
+	issn = {1548-7105},
+	publisher = {Nature Publishing Group},
+	doi = {10.1038/s41592-024-02291-w}
+}
+</code></pre>
+</details>
 
-In addition to label-free reconstruction algorithms, `waveorder` also implements widefield fluorescence and fluorescence polarization reconstruction algorithms for correlative label-free and fluorescence imaging.
+PTI provides volumetric reconstructions of mean permittivity ($\propto$ material density), differential permittivity ($\propto$ material anisotropy), 3D orientation, and optic sign. The following figure summarizes PTI acquisition and reconstruction with a small optical section of the mouse brain tissue:
+
+![Data_flow](https://github.com/mehta-lab/waveorder/blob/main/readme.png?raw=true)
 
-1. Correlative measurements of biomolecular density and orientation from polarization-diverse widefield imaging ([multimodal Instant PolScope](https://opg.optica.org/boe/fulltext.cfm?uri=boe-13-5-3102&id=472350))
+## Examples
+The [examples](https://github.com/mehta-lab/waveorder/tree/main/examples) illustrate simulations and reconstruction for 2D QLIPP, 3D phase from brightfield, and 2D/3D PTI methods.
 
-We provide an [example notebook](https://github.com/mehta-lab/waveorder/blob/main/examples/documentation/fluorescence_deconvolution/fluorescence_deconv.ipynb) for widefield fluorescence deconvolution.
+If you are interested in deploying QLIPP or phase from brightbrield, or fluorescence deconvolution for label-agnostic imaging at scale, checkout our [napari plugin](https://www.napari-hub.org/plugins/recOrder-napari),  [`recOrder-napari`](https://github.com/mehta-lab/recOrder).
 
 ## Citation
 
 Please cite this repository, along with the relevant preprint or paper, if you use or adapt this code. The citation information can be found by clicking "Cite this repository" button in the About section in the right sidebar.
 
 ## Installation
 
-(Optional but recommended) install [anaconda](https://www.anaconda.com/products/distribution) and create a virtual environment:
+Create a virtual environment:
 
 ```sh
-conda create -y -n waveorder python=3.11
+conda create -y -n waveorder python=3.10
 conda activate waveorder
 ```
 
@@ -68,12 +105,11 @@ python
 >>> import waveorder
 ```
 
-(Optional) Download the repository, install `jupyter`, and experiment with the example notebooks
-
+(Optional) Install example dependencies, clone the repository, and run an example script:
 ```sh
+pip install waveorder[examples]
 git clone https://github.com/mehta-lab/waveorder.git
-pip install jupyter
-jupyter notebook ./waveorder/examples/
+python waveorder/examples/models/phase_thick_3d.py
 ```
 
 (M1 users) `pytorch` has [incomplete GPU support](https://github.com/pytorch/pytorch/issues/77764),

diff --git a/examples/README.md b/examples/README.md
@@ -1,7 +1,10 @@
 `waveorder` is undergoing a significant refactor, and this `examples/` folder serves as a good place to understand the current state of the repository.
 
-The `models/` folder demonstrates the latest functionality of `waveorder`. These scripts will run as is in an environment with `waveorder` and `napari` installed. Each script demonstrates a simulation and reconstruction with a **model**: a specific set of assumptions about the sample and the data being acquired. 
+Some examples require `pip install waveorder[examples]` for `napari` and `jupyter`. Visit the [napari installation guide](https://napari.org/dev/tutorials/fundamentals/installation.html) if napari installation fails.
 
-The `maintenance/` folder demonstrates the functionality of `waveorder` that we plan to move to `models/`. These scripts can be run as is, and they are being maintained with tests.
-
-The `documentation/` folder consists of examples that demonstrate reconstruction with real data. These examples require access to the complete datasets, so they are not being actively maintained and serve primarily as documentation.
+| Folder      | Requires                   | Description                                                                                           |
+|------------------|----------------------------|-------------------------------------------------------------------------------------------------------|
+| `models/`        | `pip install waveorder[examples]`      | Demonstrates the latest functionality of `waveorder` through simulations and reconstructions using various models. |
+| `maintenance/`   | `pip install waveorder`                | Examples of computational imaging methods enabled by functionality of waveorder; scripts are maintained with automated tests.               |
+| `visuals/`       | `pip install waveorder[examples]`      | Visualizations of transfer functions and Green's tensors.                                    |
+| `documentation/` | `pip install waveorder`, complete datasets | Provides examples of real-data reconstructions; serves as documentation and is not actively maintained. |
diff --git a/examples/documentation/PTI_experiment/PTI_Experiment_Recon3D_anisotropic_target_small.py b/examples/documentation/PTI_experiment/PTI_Experiment_Recon3D_anisotropic_target_small.py
@@ -6,11 +6,12 @@
 from numpy.fft import fftshift
 
 import waveorder as wo
-from waveorder import optics, waveorder_reconstructor, util, visual
+from waveorder import optics, waveorder_reconstructor, util
 
 import zarr
 from pathlib import Path
 from iohub import open_ome_zarr
+from waveorder.visuals import jupyter_visuals
 
 # %%
 # Initialization
@@ -110,7 +111,7 @@
     Source_PolState[i, 1] = E_in[1]
 
 
-visual.plot_multicolumn(
+jupyter_visuals.plot_multicolumn(
     fftshift(Source, axes=(1, 2)), origin="lower", num_col=5
 )
 
@@ -162,7 +163,7 @@
 
 # %%
 # browse raw intensity stacks (stack_idx_1: z index, stack_idx2: pattern index)
-visual.parallel_5D_viewer(
+jupyter_visuals.parallel_5D_viewer(
     np.transpose(I_meas[:, :, :, :, ::-1], (4, 1, 0, 2, 3)),
     num_col=4,
     size=10,
@@ -171,7 +172,7 @@
 
 # %%
 # browse uncorrected Stokes parameters (stack_idx_1: z index, stack_idx2: pattern index)
-visual.parallel_5D_viewer(
+jupyter_visuals.parallel_5D_viewer(
     np.transpose(S_image_recon, (4, 1, 0, 2, 3)),
     num_col=3,
     size=8,
@@ -182,7 +183,7 @@
 
 # %%
 # browse corrected Stokes parameters (stack_idx_1: z index, stack_idx2: pattern index)
-visual.parallel_5D_viewer(
+jupyter_visuals.parallel_5D_viewer(
     np.transpose(S_image_tm, (4, 1, 0, 2, 3)),
     num_col=3,
     size=8,
@@ -213,7 +214,7 @@
 
 # %%
 # browse the z-stack of components of scattering potential tensor
-visual.parallel_4D_viewer(
+jupyter_visuals.parallel_4D_viewer(
     np.transpose(f_tensor, (3, 0, 1, 2)),
     num_col=4,
     origin="lower",
@@ -278,7 +279,7 @@
 
 # %%
 # browse the reconstructed physical properties
-visual.parallel_4D_viewer(
+jupyter_visuals.parallel_4D_viewer(
     np.transpose(
         np.stack(
             [
@@ -546,7 +547,7 @@
 
 # %%
 # browse XY planes of the phase and differential permittivity
-visual.parallel_4D_viewer(
+jupyter_visuals.parallel_4D_viewer(
     np.transpose(
         [
             np.clip(phase_PT, phase_min, phase_max),
@@ -585,7 +586,7 @@
     ),
     (3, 1, 2, 0),
 )
-orientation_3D_image_RGB = visual.orientation_3D_to_rgb(
+orientation_3D_image_RGB = jupyter_visuals.orientation_3D_to_rgb(
     orientation_3D_image, interp_belt=20 / 180 * np.pi, sat_factor=1
 )
 
@@ -600,7 +601,7 @@
 
 # plot the top view of 3D orientation colorsphere
 plt.figure(figsize=(3, 3))
-visual.orientation_3D_colorwheel(
+jupyter_visuals.orientation_3D_colorwheel(
     wheelsize=256, circ_size=50, interp_belt=20 / 180 * np.pi, sat_factor=1
 )
 
@@ -639,7 +640,7 @@
     in_plane_orientation[:, y_layer], origin="lower", aspect=z_step / ps
 )
 plt.figure(figsize=(3, 3))
-visual.orientation_2D_colorwheel()
+jupyter_visuals.orientation_2D_colorwheel()
 
 # %%
 # out-of-plane tilt
@@ -686,7 +687,7 @@
 
 fig, ax = plt.subplots(1, 1, figsize=(15, 15))
 
-visual.plot3DVectorField(
+jupyter_visuals.plot3DVectorField(
     np.abs(differential_permittivity_PT[1, :, :, z_layer]),
     azimuth[1, :, :, z_layer],
     theta[1, :, :, z_layer],
@@ -722,7 +723,7 @@
 # %%
 # Angular histogram of 3D orientation
 
-visual.orientation_3D_hist(
+jupyter_visuals.orientation_3D_hist(
     azimuth[1].flatten(),
     theta[1].flatten(),
     ret_mask.flatten(),

diff --git a/examples/maintenance/PTI_simulation/PTI_Simulation_Forward_2D3D.py b/examples/maintenance/PTI_simulation/PTI_Simulation_Forward_2D3D.py
@@ -15,9 +15,9 @@
 from waveorder import (
     optics,
     waveorder_simulator,
-    visual,
     util,
 )
+from waveorder.visuals import jupyter_visuals
 
 #####################################################################
 # Initialization - imaging system and sample                        #
@@ -145,7 +145,7 @@
 ### Visualize sample properties
 
 #### XY sections
-visual.plot_multicolumn(
+jupyter_visuals.plot_multicolumn(
     [
         target[:, :, z_layer],
         azimuth[:, :, z_layer] % (2 * np.pi),
@@ -158,7 +158,7 @@
     set_title=True,
 )
 #### XZ sections
-visual.plot_multicolumn(
+jupyter_visuals.plot_multicolumn(
     [
         np.transpose(target[y_layer, :, :]),
         np.transpose(azimuth[y_layer, :, :]) % (2 * np.pi),
@@ -197,7 +197,7 @@
     ),
     (3, 1, 2, 0),
 )
-orientation_3D_image_RGB = visual.orientation_3D_to_rgb(
+orientation_3D_image_RGB = jupyter_visuals.orientation_3D_to_rgb(
     orientation_3D_image, interp_belt=20 / 180 * np.pi, sat_factor=1
 )
 
@@ -206,7 +206,7 @@
 plt.figure(figsize=(10, 10))
 plt.imshow(orientation_3D_image_RGB[:, y_layer], origin="lower")
 plt.figure(figsize=(3, 3))
-visual.orientation_3D_colorwheel(
+jupyter_visuals.orientation_3D_colorwheel(
     wheelsize=128,
     circ_size=50,
     interp_belt=20 / 180 * np.pi,
@@ -216,7 +216,7 @@
 plt.show()
 
 #### Angular histogram of 3D orientation
-visual.orientation_3D_hist(
+jupyter_visuals.orientation_3D_hist(
     azimuth.flatten(),
     inclination.flatten(),
     np.abs(target).flatten(),
@@ -258,7 +258,7 @@
 epsilon_tensor[2, 2] = epsilon_mean + epsilon_del * np.cos(2 * inclination)
 
 
-visual.plot_multicolumn(
+jupyter_visuals.plot_multicolumn(
     [
         epsilon_tensor[0, 0, :, :, z_layer],
         epsilon_tensor[0, 1, :, :, z_layer],
@@ -334,7 +334,7 @@
 )
 
 
-visual.plot_multicolumn(
+jupyter_visuals.plot_multicolumn(
     [
         del_f_component[0, :, :, z_layer],
         del_f_component[1, :, :, z_layer],
@@ -425,11 +425,11 @@
 
 #### Circularly polarized illumination patterns
 
-visual.plot_multicolumn(
+jupyter_visuals.plot_multicolumn(
     fftshift(Source_cont, axes=(1, 2)), origin="lower", num_col=5, size=5
 )
 # discretized illumination patterns used in simulation (faster forward model)
-visual.plot_multicolumn(
+jupyter_visuals.plot_multicolumn(
     fftshift(Source, axes=(1, 2)), origin="lower", num_col=5, size=5
 )
 print(Source_PolState)
-Original file line number
+Diff line change
@@ Expand Up / @@ -150,3 +150,4 @@ recOrder/_version.py @@
     *.npz
     *.png
     *.tif[f]
+    *.pdf