JP-3997: Enable interpolation in pixel map creation for imaging data in resample by emolter · Pull Request #10137 · spacetelescope/jwst

emolter · 2026-01-13T21:56:50Z

This PR updates calls to gwcs_blot during resampling of imaging data to take advantage of interpolated computations made available by spacetelescope/stcal#488. Spectroscopic data are not affected.

I have confirmed that this change meaningfully improves runtime.

For a tiny (3-image) subset of the large mosaic program jw04111, the time spent in calc_pixmap went from 6.6 seconds on main to 0.8 seconds on the PR branch. In this case, calc_pixmap was not the rate-limiting step - overall runtime of calwebb_image3 went from 120s to 113s, a 5% decrease.

For a larger (~30-image) subset of the same program, the time spent in calc_pixmap went from 123 seconds on main to 14 seconds on the PR branch. The fractional runtime gain of calwebb_image3 was more substantial, going from 530s to 430s, a 20% decrease.

I'd anticipate that the fractional gain here will become larger for bigger mosaics, although I don't really feel like waiting for the full many-hundreds-of-images dataset to run through.

Tasks

…-3997

codecov · 2026-01-13T22:31:27Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.33%. Comparing base (bbf1e5f) to head (68ee77d).
⚠️ Report is 79 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10137      +/-   ##
==========================================
- Coverage   85.98%   85.33%   -0.65%     
==========================================
  Files         368      370       +2     
  Lines       38457    39132     +675     
==========================================
+ Hits        33066    33393     +327     
- Misses       5391     5739     +348

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…-3997

emolter · 2026-01-14T16:06:13Z

initial regtest round https://github.com/spacetelescope/RegressionTests/actions/runs/21000948898

The five failures all look very small.

It is unclear why test_nirspec_mos_spec3 and test_nirspec_fs_spec3_moving_target were affected. Although those differences are small, we did not expect to touch spectroscopic data - will investigate. These are also present in regression tests of Tim's stcal PR branch pointing to jwst main, so I'll investigate those on that ticket instead. It's not a bug with the way the jwst side of things is implemented.

test_miri_image3_i2d differences are only in the weight array, only in 4 pixels, and at the 0.2% level.
test_fgs_image3[i2d] differences are only in the weight array, only in 3 pixels, and at the 0.02% level.

…-3997

emolter · 2026-02-11T21:54:22Z

new round of regtests after merging and okifying the stcal PR: https://github.com/spacetelescope/RegressionTests/actions/runs/21924519427

braingram · 2026-02-11T21:56:56Z

Based off #9440 (comment) shouldn't these be optional step parameters that INS can test and then chose to enable via pars files?

emolter · 2026-02-11T22:03:00Z

As I stated at spacetelescope/romancal#2136 (comment) I'm not particularly happy with doing it that way. I had not realized that David had specifically requested this though, and I guess what he and @tapastro say goes.

mcara

I would also suggest updating stepsize and order to pixmap_stepsize and pixmap_order to keep this in agreement with changes in stcal and romancal.

mcara · 2026-02-12T00:18:05Z

jwst/resample/resample_spec.py

+            stepsize=1,
+            order=1,


Why is this hardcoded to use the full (not "sparse") pixel map?

same reason as above re spectroscopic data, although if we do indeed make all the parameters available at the step level then I suppose users will be able to decide if they want to turn it on

mcara · 2026-02-12T00:19:26Z

jwst/outlier_detection/utils.py

+            stepsize=1,
+            order=1,


Why is this hardcoded to stepsize=1?

This method is only used for outlier detection of spectroscopic data. In talking with Tim and others earlier we had initially decided not to attempt this for spectroscopic data. Has that changed? If so, I'd be happy to modify this.

I still think we probably shouldn't attempt the speed up for resampling spectral data without further study.

I thought it was disabled because when testing it with the original version of the stcal PR that did not account for the bounding boxes of spectral images it resulted in all NaN pixel maps - see my comment in spacetelescope/stcal#488 (comment) This, however, has been fixed.

jwst/outlier_detection/utils.py

changes/10137.resample.rst

emolter · 2026-02-12T18:41:26Z

https://github.com/spacetelescope/RegressionTests/actions/runs/21959536402

All passing as expected, since this is not on by default

…-3997

emolter · 2026-02-12T19:06:11Z

The last few commits should have addressed comments from @mcara and @braingram. Talking with @tapastro and @drlaw1558 , we are indeed going to make these parameters available at the step level for imaging data, and request for the INS teams to test them out for a build or two before turning this on by default.

mcara · 2026-02-12T19:52:49Z

jwst/resample/resample.py

+            Interpolation is only performed if ``pixmap_stepsize > 1``. Default is 1.
+
+        pixmap_order : int, optional
+            Order of the pixel map interpolation used. Default is 1.


Suggested change

Order of the pixel map interpolation used. Default is 1.

Interpolating spline order for pixel map computation. Default is 1.

Fixed in 68ee77d

mcara · 2026-02-12T19:57:51Z

jwst/resample/resample.py

+        pixmap_stepsize : int, optional
+            Step size for pixel map interpolation during resampling.
+            Larger step sizes result in faster performance at the cost of accuracy.
+            Interpolation is only performed if ``pixmap_stepsize > 1``. Default is 1.


stepsize is not required to be integer.

Suggested change

pixmap_stepsize : int, optional

Step size for pixel map interpolation during resampling.

Larger step sizes result in faster performance at the cost of accuracy.

Interpolation is only performed if ``pixmap_stepsize > 1``. Default is 1.

pixmap_stepsize : float, optional

Indicates the spacing at which WCS is evaluated when computing pixel map. WCS coordinates of the full pixel map is computed by interpolating over this sparse pixel map when ``pixmap_stepsize > 1``.

Larger step sizes result in faster performance at the cost of accuracy. Default is 1.

Fixed in 68ee77d

mcara · 2026-02-12T20:04:07Z

jwst/resample/resample_step.py

        enable_err = boolean(default=True)  # Compute and report the err array
        report_var = boolean(default=True)  # Report the variance array
+        pixmap_stepsize = integer(default=1)  # Interpolation step size for pixel map; interpolation is used for stepsize > 1
+        pixmap_order = integer(default=1)  # Order of the pixel mapping polynomial fit, must be 1 or 3


Suggested change

pixmap_order = integer(default=1) # Order of the pixel mapping polynomial fit, must be 1 or 3

pixmap_order = integer(default=1) # Spline order for pixel mapping, must be 1 or 3

First, it is not a "fit" - it is interpolation. Second, "polynomial" is too general, for example piece-wise polynomial interpolation is generally different from spline (which guaranties interpolating function smoothness when order>1).

Fixed in a6f5693

mcara · 2026-02-12T20:17:07Z

changes/10137.resample.rst

@@ -0,0 +1 @@
+Add ``pixmap_stepsize`` and ``pixmap_order`` parameters to allow interpolated calculation of pixel map for imaging mosaics; improves runtime of calwebb_image3 by 20 percent for a 30-image mosaic tested, with bigger improvements expected for larger mosaics.


with bigger improvements expected for larger mosaics.

I’m not fully convinced that larger output mosaics will translate into additional speedups. Since WCS evaluation cost mainly depends on the size of the input images, the practical improvement is still limited to about stepsize**2 (minus spline overhead) - and pixel map computation is only a fraction of the total resample step. Unless JWST cal images become larger, we may not see further gains. It might be helpful for each instrument team to identify the largest stepsize that preserves scientifically valid distortion behavior and include those recommendations in the calibration reference files.

That's fair, I think it's probably sufficient to just quote the single test I did here, without additional commentary

fixed in a6f5693

mcara · 2026-02-12T20:17:36Z

changes/10137.outlier_detection.rst

@@ -0,0 +1 @@
+Add pixmap_stepsize and pixmap_order parameters to allow interpolated calculation of pixel map for imaging mosaics when resampling


Suggested change

Add pixmap_stepsize and pixmap_order parameters to allow interpolated calculation of pixel map for imaging mosaics when resampling

Add ``pixmap_stepsize`` and ``pixmap_order`` parameters to allow interpolated calculation of pixel map for imaging mosaics when resampling

should be fixed in a6f5693

…-3997

emolter added 2 commits January 13, 2026 15:44

Propagate stepsize and order params up, set them for imaging modes

ac6d12b

Merge branch 'main' of https://github.com/spacetelescope/jwst into JP…

d7c5503

…-3997

github-actions bot added outlier_detection resample installation labels Jan 13, 2026

emolter mentioned this pull request Jan 13, 2026

Resample speedup spacetelescope/stcal#488

Merged

7 tasks

Merge branch 'main' of https://github.com/spacetelescope/jwst into JP…

65bec7d

…-3997

stscijirahub mentioned this pull request Jan 14, 2026

Support faster pixel coordinate computations for mosaics #9440

Open

Merge branch 'main' of https://github.com/spacetelescope/jwst into JP…

ec8b031

…-3997

emolter added this to the Build 12.3 milestone Jan 15, 2026

emolter added 2 commits February 11, 2026 09:46

Merge branch 'main' of https://github.com/spacetelescope/jwst into JP…

cd93473

…-3997

revert pyproject.toml changes

2e3d437

emolter removed the installation label Feb 11, 2026

Add changelog fragment

643c766

emolter requested review from t-brandt and tapastro February 11, 2026 16:16

emolter changed the title ~~Jp 3997~~ JP-3997: Enable interpolation in pixel map creation for imaging data in resample Feb 11, 2026

Merge branch 'main' of https://github.com/spacetelescope/jwst into JP…

476aa20

…-3997

emolter marked this pull request as ready for review February 11, 2026 21:55

emolter requested a review from a team February 11, 2026 21:55

emolter requested review from braingram, mcara and melanieclarke as code owners February 11, 2026 21:55

mcara reviewed Feb 12, 2026

View reviewed changes

rename params to go with stcal

7968de3

emolter added 2 commits February 12, 2026 12:42

Move pixmap_stepsize and pixmap_order to the step level

3afb3be

Also propagate through outlier detection, including test

3c1f68b

github-actions bot added the testing label Feb 12, 2026

emolter added 2 commits February 12, 2026 13:36

Add resample unit test

61a06a5

update changelogs

9d70d85

Merge branch 'main' of https://github.com/spacetelescope/jwst into JP…

0aba08f

…-3997

Relax tolerance a bit, a single pixel violated it and only on linux

0969513

mcara approved these changes Feb 12, 2026

View reviewed changes

emolter added 3 commits February 12, 2026 15:26

Merge branch 'main' of https://github.com/spacetelescope/jwst into JP…

360cc8f

…-3997

Improve documentation per Mihai suggestions

a6f5693

More wording improvements from Mihai

68ee77d

	Order of the pixel map interpolation used. Default is 1.
	Interpolating spline order for pixel map computation. Default is 1.

	pixmap_order = integer(default=1) # Order of the pixel mapping polynomial fit, must be 1 or 3
	pixmap_order = integer(default=1) # Spline order for pixel mapping, must be 1 or 3

		@@ -0,0 +1 @@
		Add ``pixmap_stepsize`` and ``pixmap_order`` parameters to allow interpolated calculation of pixel map for imaging mosaics; improves runtime of calwebb_image3 by 20 percent for a 30-image mosaic tested, with bigger improvements expected for larger mosaics.

		@@ -0,0 +1 @@
		Add pixmap_stepsize and pixmap_order parameters to allow interpolated calculation of pixel map for imaging mosaics when resampling

Conversation

emolter commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tasks

Uh oh!

codecov bot commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

emolter commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emolter commented Feb 11, 2026

Uh oh!

braingram commented Feb 11, 2026

Uh oh!

emolter commented Feb 11, 2026

Uh oh!

mcara left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

emolter Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcara Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

emolter commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emolter commented Feb 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcara Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

emolter commented Jan 13, 2026 •

edited

Loading

codecov bot commented Jan 13, 2026 •

edited

Loading

emolter commented Jan 14, 2026 •

edited

Loading

emolter Feb 12, 2026 •

edited

Loading

mcara Feb 12, 2026 •

edited

Loading

emolter commented Feb 12, 2026 •

edited

Loading

mcara Feb 12, 2026 •

edited

Loading