You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: tutorials/parquet-catalog-demos/euclid-q1-hats/1-euclid-q1-hats-intro.md
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -334,7 +334,7 @@ external_flux_columns[:4]
334
334
+++
335
335
336
336
Euclid Q1 includes data from three Euclid Deep Fields: EDF-N (North), EDF-S (South), EDF-F (Fornax; also in the southern hemisphere).
337
-
There is also a small amount of data from a fourth field: LDN1641 (Lynds' Dark Nebula 1641), which was observed for technical reasons during Euclid's verification phase and mostly ignored here.
337
+
There is also a small amount of data from a fourth field: LDN1641 (Lynds' Dark Nebula 1641), which was observed for technical reasons during Euclid's verification phase.
338
338
The fields are described in [Euclid Collaboration: Aussel et al., 2025](https://arxiv.org/pdf/2503.15302) and can be seen on this [skymap](https://irsa.ipac.caltech.edu/data/download/parquet/euclid/q1/merged_objects/hats/euclid_q1_merged_objects-hats/skymap.png).
339
339
340
340
The regions are well separated, so we can distinguish them using a simple cone search without having to be too picky about the radius.
To demonstrate a basic query, we'll search for objects with a galaxy photometric redshift estimate of 6.0 (largest possible).
365
-
Other tutorials in this series will show more complex queries and describe the redshifts and other data in more detail.
365
+
Other tutorials in this series will show more complex queries, and describe the redshifts and other data in more detail.
366
+
PyArrow dataset filters are described at [Filtering by Expressions](https://arrow.apache.org/docs/python/compute.html#filtering-by-expressions), and the list of available functions is at [Compute Functions](https://arrow.apache.org/docs/python/api/compute.html).
**Authors:** Troy Raen, Vandana Desai, Andreas Faisst, Shoubaneh Hemmati, Jaladh Singhal, Brigitta Sipőcz, Jessica Krick, the IRSA Data Science Team, and the Euclid NASA Science Center at IPAC (ENSCI).
We'll convert the catalog fluxes to magnitudes following the [MER Photometry Cookbook](http://st-dm.pages.euclid-sgs.uk/data-product-doc/dmq1/merdpd/merphotometrycookbook.html).
110
-
For convenience, we'll have PyArrow do the conversion during the read operation and return only the magnitudes.
110
+
PyArrow can do the conversion during the read operation and return only the magnitudes.
111
+
To do this, we'll use the following function to define the magnitudes as `pyarrow.compute` (`pc`) functions, which are described at [Compute Functions](https://arrow.apache.org/docs/python/api/compute.html).
This needs to be a dictionary of PyArrow expressions (rather than a simple list of column names) because we're asking PyArrow to convert flux -> magnitude before returning the data.
155
+
This needs to be a dictionary (rather than a simple list of column names) because we're asking PyArrow to compute the magnitudes dynamically from the catalog fluxes.
156
+
The dictionary keys will be the column names in the resultant table.
157
+
The values must be `pyarrow.compute` expressions (described above).
155
158
156
159
```{code-cell} ipython3
157
160
I_MAG = "I (mag)"
@@ -215,6 +218,14 @@ Since the template-fit photometry is recommended for extended objects, we'll sep
215
218
[Euclid Collaboration: Tucci et al., 2025](https://arxiv.org/pdf/2503.15306) defines point-like objects as having `MUMAX_MINUS_MAG < -2.5`.
@@ -352,6 +373,6 @@ The offset is more pronounced for point-like objects (bottom row), likely due to
352
373
353
374
**Authors:** Troy Raen, Vandana Desai, Andreas Faisst, Shoubaneh Hemmati, Jaladh Singhal, Brigitta Sipőcz, Jessica Krick, the IRSA Data Science Team, and the Euclid NASA Science Center at IPAC (ENSCI).
0 commit comments