Merge pull request #56 from bsipocz/euclid_remove_irsadev

bsipocz · web-flow · commit 3e9f9c7f757f · 2025-03-18T19:14:53.000-07:00
Euclid: Remove irsadev urls and notes to testers
diff --git a/tutorials/euclid_access/1_Euclid_intro_MER_images.md b/tutorials/euclid_access/1_Euclid_intro_MER_images.md
@@ -19,7 +19,7 @@ kernelspec:
 
 +++
 
-By the end of this tutorial, you will: 
+By the end of this tutorial, you will:
 - Understand the basic characteristics of Euclid Q1 MER mosaics.
 - How do download full MER mosaics.
 - How to make smaller cutouts of MER mosaics.
@@ -32,11 +32,17 @@ By the end of this tutorial, you will:
 
 +++
 
-Euclid is a European Space Agency (ESA) space mission with NASA participation, to study the geometry and nature of the dark Universe. The Quick Data Release 1 (Q1) are the first data release from the Euclid mission after the Early Release Observations (ERO). On March 19, 2025 the data will be available on the ESA archive (https://easidr.esac.esa.int/sas/) and on the IRSA archive (https://irsa.ipac.caltech.edu).
+Euclid is a European Space Agency (ESA) space mission with NASA participation, to study the geometry and nature of the dark Universe.
+The Quick Data Release 1 (Q1) are the first data release from the Euclid mission after the Early Release Observations (ERO).
+On March 19, 2025 the data will be available on the [ESA archive](https://easidr.esac.esa.int/sas/) and on the [IRSA archive](https://irsa.ipac.caltech.edu).
 
-These notebooks focus on how to access, download, and process Euclid Q1 data from the IRSA archive. At the end of the notebook, we also include some information for how to access the Q1 data from the ESA archive. If you have any issues accessing data from the archives, please contact the helpdesk directly: IRSA (irsasupport@ipac.caltech.edu) and ESA (https://support.cosmos.esa.int/euclid).
+These Q1 notebooks focus on how to access, download, and process Euclid Q1 data from the IRSA archive.
+If you have any issues accessing data from the archives, please contact the helpdesk directly: [IRSA helpdesk](irsasupport@ipac.caltech.edu) and [ESA Euclid Helpdesk](https://support.cosmos.esa.int/euclid).
 
-MER mosaic images are all the images from Level 2 images in different filters mapped to a common pixel scale. This notebook provides an introduction to MER mosaics released as part of Euclid Q1. Other Euclid notebooks show how to use other data products released as part of Euclid Q1.
+MER mosaic images are all the images from Level 2 images in different filters mapped to a common pixel scale.
+
+This notebook provides an introduction to MER mosaics released as part of Euclid Q1.
+Other Euclid notebooks show how to use other data products released as part of Euclid Q1.
 
 +++
 
@@ -82,12 +88,12 @@ search_radius = 10 * u.arcsec
 coord = SkyCoord.from_name('HD 168151')
 ```
 
-Use IRSA to search for all Euclid data on this target. 
-This searches specifically in the euclid_DpdMerBksMosaic "collection" which is the MER images and catalogs. 
+Use IRSA to search for all Euclid data on this target.
+This searches specifically in the euclid_DpdMerBksMosaic "collection" which is the MER images and catalogs.
 This query will return any image with pixels that overlap the search region.
 
 ```{code-cell} ipython3
-irsa_service= vo.dal.sia2.SIA2Service('https://irsadev.ipac.caltech.edu/SIA')
+irsa_service= vo.dal.sia2.SIA2Service('https://irsa.ipac.caltech.edu/SIA')
 
 image_table = irsa_service.search(pos=(coord, search_radius), collection='euclid_DpdMerBksMosaic')
 ```
@@ -147,37 +153,22 @@ tileID=re.search(r'TILE\s*(\d{9})', filename).group(1)
 print('The MER tile ID for this object is :',tileID)
 ```
 
-```{code-cell} ipython3
-######### TEMP
-######## Note to testers, for now we need to replace the irsa.ipac.caltech.edu url with irsadev
-######## This will not be the same after the data are made public so this cell will be deleted at that time
-def add_dev_to_domain(domain):
-    parts = domain.split('.', 1)  # Split at the first dot
-    if len(parts) == 2:
-        return f"{parts[0]}dev.{parts[1]}"
-    return domain
-
-filename_dev = add_dev_to_domain(filename)
-print(filename_dev)  
-
-#####################
-```
-
 Download the MER image -- note this file is about 1.46 GB
 
 ```{code-cell} ipython3
-fname = download_file(filename_dev, cache=True)
+fname = download_file(filename, cache=True)
 hdu_mer_irsa = fits.open(fname)
 print(hdu_mer_irsa.info())
 
 head_mer_irsa = hdu_mer_irsa[0].header
 ```
 
-Now you've downloaded this large file, if you would like to save it to disk, uncomment the following cell
+Now you've downloaded this large file, if you would like to save it to disk, uncomment the following cell.
+Please also define a suitable download directory; by default it will be `data` at the same location as your notebook.
 
 ```{code-cell} ipython3
-# download_path='/Users/meshkat/data/Euclid/'
-# hdu_mer_irsa.writeto(download_path+'./MER_image_VIS.fits', overwrite=True)
+# download_path = 'data'
+# hdu_mer_irsa.writeto(os.path.join(download_path, 'MER_image_VIS.fits'), overwrite=True)
 ```
 
 Have a look at the header information for this image
@@ -215,17 +206,6 @@ urls = df_im_euclid['access_url'].to_list()
 urls
 ```
 
-```{code-cell} ipython3
-######### TEMP
-######## Note to testers, for now we need to replace the irsa.ipac.caltech.edu url with irsadev
-######## reusing the add_dev_to_domain function from before
-######## This will not be the same after the data are made public so this cell will be deleted at that time
-
-urls = list(map(add_dev_to_domain, urls))
-print(urls)
-#####################
-```
-
 Create an array with the instrument and filter name so we can add this to the plots.
 
 ```{code-cell} ipython3
@@ -245,8 +225,8 @@ filters
 ## How large do you want the image cutout to be?
 im_cutout= 1.0 * u.arcmin
 
-## What is the center of the cutout? 
-## For now choosing a random location on the image 
+## What is the center of the cutout?
+## For now choosing a random location on the image
 ## because the star itself is saturated
 ra = 273.8667
 dec =  64.525
@@ -269,14 +249,14 @@ for url in urls:
     print(f"Opened {url}")
 
     ## Store the header
-    header = hdu[0].header  
+    header = hdu[0].header
 
     ## Read in the cutout of the image that you want
     cutout_data = Cutout2D(hdu[0].section, position=coords_cutout, size=im_cutout, wcs=WCS(hdu[0].header))
 
     ## Close the file
     # hdu.close()
-    
+
     ## Define a new fits file based on this smaller cutout, with accurate WCS based on the cutout size
     new_hdu = fits.PrimaryHDU(data=cutout_data.data, header=header)
     new_hdu.header.update(cutout_data.wcs.to_header())
@@ -296,7 +276,7 @@ Need to determine the number of images for the grid layout, then we iterate thro
 ```{code-cell} ipython3
 num_images = len(final_hdulist)
 columns = 4
-rows = -(-num_images // columns)  
+rows = -(-num_images // columns)
 
 fig, axes = plt.subplots(rows, columns, figsize=(4 * columns, 4 * rows), subplot_kw={'projection': WCS(final_hdulist[0].header)})
 axes = axes.flatten()
@@ -326,7 +306,7 @@ filters
 ```
 
 ```{code-cell} ipython3
-filt_index = np.where(filters == 'VIS')[0][0]  
+filt_index = np.where(filters == 'VIS')[0][0]
 
 img1=final_hdulist[filt_index].data
 ```
@@ -340,7 +320,7 @@ Need to do some initial steps (swap byte order) with the cutout to prevent sep f
 
 ```{code-cell} ipython3
 img2 = img1.byteswap().view(img1.dtype.newbyteorder())
-c_contiguous_data = np.array(img2, dtype=np.float32)  
+c_contiguous_data = np.array(img2, dtype=np.float32)
 
 bkg = sep.Background(c_contiguous_data)
 
@@ -414,6 +394,6 @@ for i in range(len(sources_thr)):
 
 **Author**: Tiffany Meshkat (IPAC Scientist)
 
-**Updated**: March 19, 2025
+**Updated**: 2025-03-19
 
 **Contact:** [the IRSA Helpdesk](https://irsa.ipac.caltech.edu/docs/help_desk.html) with questions or reporting problems.
diff --git a/tutorials/euclid_access/2_Euclid_intro_MER_catalog.md b/tutorials/euclid_access/2_Euclid_intro_MER_catalog.md
@@ -19,7 +19,7 @@ kernelspec:
 
 +++
 
-By the end of this tutorial, you will: 
+By the end of this tutorial, you will:
 - Understand the basic characteristics of Euclid Q1 MER catalogs.
 - What columns are available in the MER catalog.
 - How to query with ADQL in the MER catalog.
@@ -31,11 +31,17 @@ By the end of this tutorial, you will:
 
 +++
 
-Euclid is a European Space Agency (ESA) space mission with NASA participation, to study the geometry and nature of the dark Universe. The Quick Data Release 1 (Q1) are the first data release from the Euclid mission after the Early Release Observations (ERO). On March 19, 2025 the data will be available on the ESA archive (https://easidr.esac.esa.int/sas/) and on the IRSA archive (https://irsa.ipac.caltech.edu).
+Euclid is a European Space Agency (ESA) space mission with NASA participation, to study the geometry and nature of the dark Universe.
+The Quick Data Release 1 (Q1) are the first data release from the Euclid mission after the Early Release Observations (ERO).
+On March 19, 2025 the data will be available on the [ESA archive](https://easidr.esac.esa.int/sas/) and on the [IRSA archive](https://irsa.ipac.caltech.edu).
 
-These notebooks focus on how to access, download, and process Euclid Q1 data from the IRSA archive. At the end of the notebook, we also include some information for how to access the Q1 data from the ESA archive. If you have any issues accessing data from the archives, please contact the helpdesk directly: IRSA (irsasupport@ipac.caltech.edu) and ESA (https://support.cosmos.esa.int/euclid).
+These Q1 notebooks focus on how to access, download, and process Euclid Q1 data from the IRSA archive.
+If you have any issues accessing data from the archives, please contact the helpdesk directly: [IRSA helpdesk](irsasupport@ipac.caltech.edu) and [ESA Euclid Helpdesk](https://support.cosmos.esa.int/euclid).
 
-Each entry in the MER catalog is a single source containing all its photometry from the MER Mosaics (VIS, Y, J, H and any accompanying external ground observations) along with other basic measurements, like size and shape. This notebook provides an introduction to the MER catalog released as part of Euclid Q1. Other Euclid notebooks show how to use other data products released as part of Euclid Q1.
+Each entry in the MER catalog is a single source containing all its photometry from the MER Mosaics (VIS, Y, J, H and any accompanying external ground observations) along with other basic measurements, like size and shape.
+
+This notebook provides an introduction to the MER catalog released as part of Euclid Q1.
+Other Euclid notebooks show how to use other data products released as part of Euclid Q1.
 
 +++
 
@@ -56,7 +62,7 @@ import pyvo as vo
 ## 1. Download MER catalog from IRSA directly to this notebook
 
 ```{code-cell} ipython3
-service = vo.dal.TAPService("https://irsadev.ipac.caltech.edu/TAP")
+service = vo.dal.TAPService("https://irsa.ipac.caltech.edu/TAP")
 ```
 
 ```{code-cell} ipython3
@@ -83,7 +89,7 @@ print(len(columns))
 
 ```{code-cell} ipython3
 for col in columns:
-    print(f'{f"{col.name}":30s}  {col.unit}  {col.description}') 
+    print(f'{f"{col.name}":30s}  {col.unit}  {col.description}')
 ```
 
 ### Define the following ADQL query to find the first 10k stars in the MER catalog
@@ -127,7 +133,7 @@ mag_h_s_irsa=-2.5*np.log10(df_s_irsa["flux_h_templfit"]) + 23.9 # H
 x_s_irsa = mag_y_s_irsa - mag_h_s_irsa # Y - H
 y_s_irsa = mag_y_s_irsa
 
-xerr_s_irsa= 2.5 / np.log(10) * np.sqrt((df_s_irsa["fluxerr_y_templfit"] / df_s_irsa["flux_y_templfit"])**2 
+xerr_s_irsa= 2.5 / np.log(10) * np.sqrt((df_s_irsa["fluxerr_y_templfit"] / df_s_irsa["flux_y_templfit"])**2
                                  + (df_s_irsa["fluxerr_h_templfit"] / df_s_irsa["flux_h_templfit"])**2)
 yerr_s_irsa= 2.5 / np.log(10) * (df_s_irsa["fluxerr_y_templfit"] / df_s_irsa["flux_y_templfit"])
 
@@ -144,6 +150,6 @@ plt.title('10k Stars in MER catalog -- IRSA')
 
 **Author**: Tiffany Meshkat (IPAC Scientist)
 
-**Updated**: March 19, 2025
+**Updated**: 2025-03-19
 
 **Contact:** [the IRSA Helpdesk](https://irsa.ipac.caltech.edu/docs/help_desk.html) with questions or reporting problems.
diff --git a/tutorials/euclid_access/3_Euclid_intro_1D_spectra.md b/tutorials/euclid_access/3_Euclid_intro_1D_spectra.md
@@ -19,7 +19,7 @@ kernelspec:
 
 +++
 
-By the end of this tutorial, you will: 
+By the end of this tutorial, you will:
 - Understand the basic characteristics of Euclid Q1 SIR 1D spectra.
 - What columns are available in the MER catalog.
 - How to query with ADQL in the MER catalog.
@@ -29,17 +29,27 @@ By the end of this tutorial, you will:
 
 ## Introduction
 
-+++
+++
 
-Euclid is a European Space Agency (ESA) space mission with NASA participation, to study the geometry and nature of the dark Universe. The Quick Data Release 1 (Q1) are the first data release from the Euclid mission after the Early Release Observations (ERO). On March 19, 2025 the data will be available on the ESA archive (https://easidr.esac.esa.int/sas/) and on the IRSA archive (https://irsa.ipac.caltech.edu).
+Euclid is a European Space Agency (ESA) space mission with NASA participation, to study the geometry and nature of the dark Universe.
+The Quick Data Release 1 (Q1) are the first data release from the Euclid mission after the Early Release Observations (ERO).
+On March 19, 2025 the data will be available on the [ESA archive](https://easidr.esac.esa.int/sas/) and on the [IRSA archive](https://irsa.ipac.caltech.edu).
 
-These notebooks focus on how to access, download, and process Euclid Q1 data from the IRSA archive. At the end of the notebook, we also include some information for how to access the Q1 data from the ESA archive. If you have any issues accessing data from the archives, please contact the helpdesk directly: IRSA (irsasupport@ipac.caltech.edu) and ESA (https://support.cosmos.esa.int/euclid).
+These Q1 notebooks focus on how to access, download, and process Euclid Q1 data from the IRSA archive.
+If you have any issues accessing data from the archives, please contact the helpdesk directly: [IRSA helpdesk](irsasupport@ipac.caltech.edu) and [ESA Euclid Helpdesk](https://support.cosmos.esa.int/euclid).
 
-For the Euclid Wide Survey standard operating mode, the telescope undertakes a 4-point dither pattern. At each position VIS and NISP each take a 570s exposure, consisting of a direct visible image and a red grism exposure. This is followed by further NISP exposures in the Y, J, and H band filters (112 seconds each). The telescope is then dithered, and the sequence is repeated starting with a different grism position angle. There are actually two operational grisms oriented 180 degrees from each other. Each grism which will be used twice in this sequence, but with slight angular offsets (+/- 4 degrees), effectively creating the four different grism angles (Scaramella et al. 2022, A&A 662, A112).	
+For the Euclid Wide Survey standard operating mode, the telescope undertakes a 4-point dither pattern.
+At each position VIS and NISP each take a 570s exposure, consisting of a direct visible image and a red grism exposure.
+This is followed by further NISP exposures in the Y, J, and H band filters (112 seconds each).
+The telescope is then dithered, and the sequence is repeated starting with a different grism position angle.
+There are actually two operational grisms oriented 180 degrees from each other. Each grism which will be used twice in this sequence, but with slight angular offsets (+/- 4 degrees), effectively creating the four different grism angles (Scaramella et al. 2022, A&A 662, A112).
 
-Data which can be obtained for SIR include: SIR "images", which effectively show the full image of objects with the spectral traces overlapping, and SIR 1D spectra for individual objects. Below we will describe how to access and process the 1D spectra products. For most users, simply accessing th 1D spectra is probably the preferred option, unless they would like to extract the spectrum again, or inspect the images to see if there is any artifact which might add noise to the spectrum.
+Data which can be obtained for SIR include: SIR "images", which effectively show the full image of objects with the spectral traces overlapping, and SIR 1D spectra for individual objects.
+In this notebook we describe how to access and process the 1D spectra products.
+For most users, simply accessing th 1D spectra is probably the preferred option, unless they would like to extract the spectrum again, or inspect the images to see if there is any artifact which might add noise to the spectrum.
 
-This notebook provides an introduction to the SIR 1D spectra released as part of Euclid Q1. Other Euclid notebooks show how to use other data products released as part of Euclid Q1.
+This notebook provides an introduction to the SIR 1D spectra released as part of Euclid Q1.
+Other Euclid notebooks show how to use other data products released as part of Euclid Q1.
 
 +++
 
@@ -68,7 +78,7 @@ import pyvo as vo
 Search for all tables in IRSA labeled as euclid
 
 ```{code-cell} ipython3
-service = vo.dal.TAPService("https://irsadev.ipac.caltech.edu/TAP")
+service = vo.dal.TAPService("https://irsa.ipac.caltech.edu/TAP")
 
 tables = service.tables
 for tablename in tables.keys():
@@ -84,7 +94,7 @@ table_galaxy_candidates= 'euclid_q1_spectro_zcatalog_spe_galaxy_candidates'
 ```
 
 ```{code-cell} ipython3
-## Change the settings so we can see all the columns in the dataframe and the full column width 
+## Change the settings so we can see all the columns in the dataframe and the full column width
 ## (to see the full long URL)
 pd.set_option('display.max_columns', None)
 pd.set_option('display.max_colwidth', None)
@@ -100,7 +110,7 @@ pd.set_option('display.max_colwidth', None)
 ```{code-cell} ipython3
 obj_id=2739401293646823742
 
-## Pull the data on these objects 
+## Pull the data on these objects
 adql_object = f"SELECT * \
 FROM {table_1dspectra} \
 WHERE objectid = {obj_id} \
@@ -115,7 +125,7 @@ df2
 ### Create the full filename/url
 
 ```{code-cell} ipython3
-irsa_url='https://irsadev.ipac.caltech.edu/'
+irsa_url='https://irsa.ipac.caltech.edu/'
 
 file_url=irsa_url+df2['uri'].iloc[0]
 file_url
@@ -130,7 +140,7 @@ Currently IRSA has the spectra stored in very large files containing multiple (1
 #### Currently commented out
 
 # ## Complete file url with the irsa url at the start
-# url = file_url  
+# url = file_url
 # response = requests.get(url)
 
 # hdul = fits.open(BytesIO(response.content))  # Open FITS file from memory
@@ -167,6 +177,6 @@ plt.title(obj_id)
 
 **Author**: Tiffany Meshkat (IPAC Scientist)
 
-**Updated**: March 19, 2025
+**Updated**: 2025-03-19
 
 **Contact:** [the IRSA Helpdesk](https://irsa.ipac.caltech.edu/docs/help_desk.html) with questions or reporting problems.
diff --git a/tutorials/euclid_access/5_Euclid_intro_SPE_catalog.md b/tutorials/euclid_access/5_Euclid_intro_SPE_catalog.md
