Skip to content

Commit 8317726

Browse files
desaidesai
authored andcommitted
responding to reviews by Andreas and Brigitta
1 parent a8ea38e commit 8317726

File tree

2 files changed

+26
-121
lines changed

2 files changed

+26
-121
lines changed

documentation/spherex_data_access.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Each of these data access layers is described in greater detail in the subsectio
1313

1414
## Browsable Directories
1515
SPHEREx data products are laid out in directories that can be navigated with standard web browsers.
16-
This is convenient for users to get a quick sense of the types of data products that are available, to quickly download some examples by clicking through the directory tree, and to script bulk downloads using wget or curl.
16+
This is convenient for users to get a quick sense of the types of data products that are available, to quickly download some examples by clicking through the directory tree, and to script bulk downloads using `wget` or `curl`.
1717

1818
The root of the SPHEREx data quick release data directories is:
1919
https://irsa.ipac.caltech.edu/ibe/data/spherex/qr
@@ -54,7 +54,7 @@ IRSA's generic SIA2 endpoint is:
5454
`https://irsa.ipac.caltech.edu/SIA?`
5555

5656
Users must add a `COLLECTION` parameter to this endpoint to specify which dataset to search.
57-
There are three SPHEREx-related SIA2 collections:
57+
There are three SPHEREx-related SIA2 collections:
5858

5959
* SPHEREx Quick Release Spectral Image MEFs that are part of the SPHEREx **Wide Survey** can be accessed with: `COLLECTION=spherex_qr`
6060

documentation/spherex_data_products.md

Lines changed: 24 additions & 119 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# SPHEREx Data Products Available at IRSA
1+
SPHEREx Data Products Available at IRSA
22

33
A detailed description of SPHEREx data products available to the public is provided in the [SPHEREx Explanatory Supplement](https://irsa.ipac.caltech.edu/data/SPHEREx/docs/SPHEREx_Expsupp_QR.pdf).
44
Here we provide a concise summary of the science, calibration, and additional data products available at IRSA.
@@ -7,11 +7,10 @@ This summary includes filenaming conventions, for which we adopt the following d
77
- `Planning Period` designates the survey plan uploaded to the spacecraft, e.g. `2025W18_2B`.
88
Each planning period covers approximately 3.5 days of operation.
99

10-
- `Observation ID` includes the survey planning period and the large
11-
and small slew counters, e.g. `2025W18_2B_0001_1`. Each large slew
12-
has a maximum of 4 small slews, so the allowed small slew counter
13-
range is 1 to 4. Some large slews will have less than 4 small
14-
slews.
10+
- `Observation ID` includes the survey planning period and the large and small slew counters.
11+
For example, `2025W18_2B_0001_1` contains the planning period (`2025W18_2B`), the large slew counter (`0001`), and the small slew counter (`1`).
12+
Each large slew has a maximum of 4 small slews, so the allowed small slew counter range is 1 to 4.
13+
Some large slews will have fewer than 4 small slews.
1514

1615
- `Detector` is an integer from 1 through 6.
1716

@@ -22,12 +21,11 @@ Each planning period covers approximately 3.5 days of operation.
2221

2322
## Main Science Data Product: Spectral Image Multi-Extension FITS Files (MEF)
2423

25-
The main Quick Release data product is the Level 2 Spectral Image MEF.
26-
There are 6 Spectral MEFs (one for each detector) for each sky
27-
pointing. Because data quality assessments are evaluated per spectral
28-
image band, some observations will not include all 6 bands in the
29-
archive. Each Spectral MEF is approximately 70 MB and contains 6
30-
extensions:
24+
The main Quick Release data product is the Level 2 Spectral Image MEF,as described in Section 2.1 of the [Explanatory Supplement](https://irsa.ipac.caltech.edu/data/SPHEREx/docs/SPHEREx_Expsupp_QR.pdf).
25+
26+
There are 6 Spectral MEFs (one for each detector) for each sky pointing.
27+
Because data quality assessments are evaluated per spectral image band, some observations will not include all 6 bands in the archive.
28+
Each Spectral MEF is approximately 70 MB and contains 6 extensions:
3129

3230
HDU 1: IMAGE
3331
: Calibrated surface brightness flux density in units of MJy/sr, stored as a 2040 x 2040 image.
@@ -87,121 +85,28 @@ HDU 6: WCS-WAVE
8785

8886
## Cutouts of Spectral Image MEFs
8987

90-
IRSA enables users to access rectangular cutouts of a SPHEREx Spectral
91-
Image MEF by simply appending a [query
92-
string](https://irsa.ipac.caltech.edu/ibe/cutouts.html) containing
93-
center and size parameters to the image URL. These cutout MEFs contain
94-
the same HDUs as the original Spectral Images (IMAGE, FLAGS, VARIANCE,
95-
ZODI, PSF, WCS-WAVE). However, the IMAGE, FLAGS, VARIANCE, AND ZODI
96-
HDUs have been modified to include only those pixels within the
97-
specified cutout size. The WCS-WAVE HDU has also modified to provide
98-
the correct mapping between the pixels in the cutout to
99-
wavelength. The PSF HDU from the original spectral image is included
100-
unmodified in the cutout MEF.
101-
102-
The spatially-varying PSF is represented as an image cube with 121
103-
planes representing different regions of the original, full Spectral
104-
Image. Users interested in performing photometry on a cutout using the
105-
information in the cutout PSF HDU will need to understand how to find
106-
the most applicable PSF cube plane for each pixel in the cutout. The
107-
steps are described below:
108-
109-
1. Determine the 0-based pixel coordinates of the position of interest
110-
in the IMAGE HDU of the cutout.
111-
112-
2. Determine the 0-based pixel coordinates of the position of interest
113-
in the IMAGE HDU of the original Spectral Image.
114-
115-
```python
116-
xpix_orig = 1 + xpix_cutout - CRPIX1A
117-
ypix_orig = 1 + ypix_cutout - CRPIX2A
118-
```
119-
120-
3. Examine the header of the PSF HDU of the cutout to determine the
121-
PSF zone and cube plane corresponding to the pixel of interest in the original
122-
Spectral Image.
123-
124-
The PSF HDU has a header containing the keywords `XCTR_*`, `YCTR_*`,
125-
`XWID_*`, and `YWID_*`, where * goes from [1 to 121]. To determine if
126-
a pixel in the original Spectral Image falls within a PSF zone, simply
127-
find the closest `XCTR_*` and `YCTR_*` to determine the cube plane
128-
that contains the corresponding PSF for this zone.
129-
130-
**Example:**
131-
132-
Consider this cutout:
133-
134-
`spectral_image_url = "https://irsa.ipac.caltech.edu/ibe/cutout?ra=305.59875000000005&dec=41.14888888888889&size=0.01&path=spherex/qr/level2/2025W18_1B/l2b-v13-2025-198/4/level2_2025W18_1B_0023_2D4_spx_l2b-v13-2025-198.fits"`
135-
136-
`cutout_hdulist = fits.open(spectral_image_url)`
137-
138-
1. Let's say that we are interested in the requested position:
139-
140-
```
141-
ra_deg = 305.59875000000005
142-
dec_deg = 41.14888888888889
143-
```
144-
145-
We can use the WCS header in the cutout to determine that these coordinates correspond to
88+
IRSA enables users to access rectangular cutouts of a SPHEREx Spectral Image MEF by simply appending a [query string](https://irsa.ipac.caltech.edu/ibe/cutouts.html) containing center and size parameters to the image URL.
89+
These cutout MEFs contain the same HDUs as the original Spectral Images (IMAGE, FLAGS, VARIANCE, ZODI, PSF, WCS-WAVE). However, the IMAGE, FLAGS, VARIANCE, AND ZODI HDUs have been modified to include only those pixels within the specified cutout size.
90+
The WCS-WAVE HDU has also modified to provide the correct mapping between the pixels in the cutout to wavelength.
91+
The PSF HDU from the original spectral image is included unmodified in the cutout MEF.
14692

147-
```
148-
cutout_header = cutout_hdulist[1].header
149-
150-
xpix_cutout, ypix_cutout = wcs.world_to_pixel(SkyCoord(ra=ra, dec=dec, unit="deg"))
151-
152-
print(xpix_cutout, ypix_cutout)
153-
154-
2.180467000669851 2.341315802377096
155-
```
93+
The spatially-varying PSF is represented as an image cube with 121 planes.
94+
Each plane is a 101x101 pixel image representing a PSF for a different region of the detector. Users interested in performing photometry on a cutout using the information in the cutout PSF HDU will need to understand how to find the most applicable PSF cube plane for each pixel in the cutout.
95+
The basic steps are described below, and a [https://caltech-ipac.github.io/irsa-tutorials/](Python notebook tutorial) is provided to help users get started with a simple implementation.
15696

157-
2. We can then use the values of CRPIX1A and CRPIX2A in the cutout header to
158-
calculate what pixels these coordinates correspond to in the original
159-
Spectral Image:
97+
1. Determine the 0-based pixel coordinates of the position of interest in the IMAGE HDU of the cutout.
16098

161-
```
162-
CRPIX1A = -220.0
163-
CRPIX2A = -2004.0
99+
2. Determine the 0-based pixel coordinates of the position of interest in the IMAGE HDU of the original Spectral Image.
164100

165-
xpix_orig = 1 + xpix_cutout - CRPIX1A = 1 + 2.180467000669851 - -220.0 = 223.180467001
166-
ypix_orig = 1 + xpix_cutout - CRPIX2A = 1 + 2.341315802377096 - -2004.0 = 2007.3413158
167101
```
168-
169-
3. Looking at the header of the cutout PSF HDU, we see that the XCTR_* values are:
170-
171-
```
172-
XCTR zone0 = 93.22727273
173-
XCTR zone1 = 278.68181818
174-
XCTR zone2 = 464.13636364
175-
XCTR zone3 = 649.59090909
176-
XCTR zone4 = 835.04545455
177-
XCTR zone5 = 1020.5
178-
XCTR zone6 = 1205.95454545
179-
XCTR zone7 = 1391.40909091
180-
XCTR zone8 = 81576.86363636
181-
XCTR zone9 = 1762.31818182
182-
XCTR zone10 = 1947.77272727
183-
```
184-
185-
and the `YCTR_*` values are:
186-
187-
```
188-
YCTR zone0 = 93.22727273
189-
YCTR zone1 = 278.68181818
190-
YCTR zone2 = 464.13636364
191-
YCTR zone3 = 649.59090909
192-
YCTR zone4 = 835.04545455
193-
YCTR zone5 = 1020.5
194-
YCTR zone6 = 1205.95454545
195-
YCTR zone7 = 1391.40909091
196-
YCTR zone8 = 1576.86363636
197-
YCTR zone9 = 1762.31818182
198-
YCTR zone10 = 1947.77272727
102+
xpix_orig = 1 + xpix_cutout - CRPIX1A
103+
ypix_orig = 1 + ypix_cutout - CRPIX2A
199104
```
200105

201-
Our original pixel coordates of 223.180467001, 2007.3413158 fall
202-
closest to `XCTR` zone1 and `YCTR` zone10. This PSF for this zone is
203-
stored in cube plane 22.
106+
3. Examine the header of the PSF HDU of the cutout to determine the PSF zone and cube plane corresponding to the pixel of interest in the original Spectral Image.
204107

108+
The PSF HDU has a header containing the keywords `XCTR_*`, `YCTR_*`, `XWID_*`, and `YWID_*`, where * goes from [1 to 121].
109+
To determine if a pixel in the original Spectral Image falls within a PSF zone, simply find the closest `XCTR_*` and `YCTR_*` to determine the cube plane that contains the corresponding PSF for this zone.
205110

206111
## Calibration Product: Absolute Gain Matrix
207112

0 commit comments

Comments
 (0)