Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed improvements for saturation step #9011

Open
stscijgbot-jp opened this issue Dec 15, 2024 · 1 comment
Open

Speed improvements for saturation step #9011

stscijgbot-jp opened this issue Dec 15, 2024 · 1 comment

Comments

@stscijgbot-jp
Copy link
Collaborator

stscijgbot-jp commented Dec 15, 2024

Issue JP-3820 was created on JIRA by Timothy Brandt:

The saturation step is much slower than it needs to be.  This is of limited importance for jwst, but this step can take a significant fraction of the Stage 1 pipeline for Roman and I believe it is worth some effort at optimizing.  

There are a number of small changes that improve speed, but three stand out.  First, when a pixel saturates, that saturation is propagated into all subsequent reads.  This is done every group for a cost ~ngroup^2, but it could be handled with a running boolean tally of pixels that have saturated.  Second, ndimage.binary_dilation is very slow, an order of magnitude slower than writing the operations out explicitly.  Finally, if there is no averaging in groups, there is no point in the diluted saturation check; we can skip this check if the dilution factor is 1.

Making the three changes noted above, together with a number of other, very minor changes, improves runtime of the saturation step for jw01283001001_03101_00001_mirimage_uncal.fits (10 integrations, 100 groups) from 42s to 4.5s (a factor of nearly 10).  If forcing it to keep the diluted saturation step, the speed improvement is from 42s to 9s (a factor of 4.5).  The resulting 4D gdq array is identical in all cases.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Timothy Brandt on JIRA:

Suggested changes are here: https://github.com/t-brandt/stcal/tree/saturation_speedup

I can add a note to CHANGES.rst and initiate a PR if people agree.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant