Addresses Issue #3138 - ESO - Enable Table Access Protocol (TAP) queries #3339

juanmcloaiza · 2025-06-05T12:32:44Z

Closes #3138

Changes:

Switch querying interface from WDB to TAP in querying functions:

Deprecate arguments:

open_form
cache

New functionality:

Allow plain ADQL queries via query_tap_service (with authentication as well)
Cone search using cone_ra, cone_dec, cone_radius arguments
Retrieve record count before querying the archive, via count_only argument
Ask query functions to print the underlying ADQL queries without issuing them

codecov · 2025-06-05T12:59:07Z

Codecov Report

❌ Patch coverage is 82.56228% with 49 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.67%. Comparing base (b81cecb) to head (685c807).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
astroquery/eso/core.py	75.00%	45 Missing ⚠️
astroquery/eso/utils.py	96.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3339      +/-   ##
==========================================
+ Coverage   70.07%   70.67%   +0.59%     
==========================================
  Files         232      233       +1     
  Lines       19893    19839      -54     
==========================================
+ Hits        13940    14021      +81     
+ Misses       5953     5818     -135

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

juanmcloaiza · 2025-07-11T09:26:01Z

Hi @bsipocz , could you please give us a time estimate of the review process? No pressure at all though, we only need to plan our activities internally. Thanks a lot in advance.

bsipocz · 2025-07-21T18:08:01Z

@juanmcloaiza - sorry for the delay. I'll try to get to the review this week.

bsipocz

I'm sorry to have this many comments and thank you for your patience while waiting for this review.

I acknowledge that this seems to a lot, but most of them are minor and/or helping API consistency with the other modules and/or making the code cleaner and easier to read and follow; which will be helpful for the future maintenance.

bsipocz · 2025-07-31T01:24:03Z

astroquery/eso/core.py


+import astropy.table


L28 explicitly imports from here, remove this one add to that list as necessary

bsipocz · 2025-07-31T01:25:14Z

astroquery/eso/core.py

 import astropy.utils.data
 import keyring
 import requests.exceptions
 from astropy.table import Table, Column
 from astropy.utils.decorators import deprecated_renamed_argument
 from bs4 import BeautifulSoup
+from pyvo.dal import TAPService
+import pyvo.dal.exceptions as pde


can you directly import the classes here? It's quite obvious reading the code that those are coming from pyvo (and this alias is not helping to make it more obvious anyway)

bsipocz · 2025-07-31T01:25:35Z

astroquery/eso/core.py

-                                 "problem; the service may be offline.")
-    elif b"# No data returned !" not in content:
-        return True
+__doctest_skip__ = ['EsoClass.*']


why do we skip doctesting?

I know this is not a valid answer, but "because it was so in the main branch, L30", so I left it like that. I'll address this comment later, when I have a valid answer.

OK, I removed the flag __doctest_skip__.

bsipocz · 2025-07-31T01:27:45Z

astroquery/eso/core.py

-                                 "problem; the service may be offline.")
-    elif b"# No data returned !" not in content:
-        return True
+__doctest_skip__ = ['EsoClass.*']




have an explicit __all__ with listing the public API

__all__ is present in eso/__init__.py (L28):

__all__ = ['Eso', 'EsoClass', 'Conf', 'conf', ]

Is this what you're referring to? Perhaps I'm missing the point.

bsipocz · 2025-07-31T01:29:39Z

astroquery/eso/core.py

+    @property
+    def maxrec(self):


Why does maxrec need to be a class properly rather than a parameter for the query functions? After all it should be specific to individual queries rather than being set on the instance. (See the other modules)

We also discussed this point internally. While both maxrec and top limit the number of results, their intent is different. top is part of the ADQL query, used limit the result size from a logical/scientific standpoint. maxrec, on the other hand, is a TAP server parameter used as a safety cap regardless of the query’s intent. For example, I'd use maxrec to avoid system overload when I'm on my laptop, and for quickly testing the notebook’s logic and plots before the "real-thing-run" that could last some hours.

Since maxrec reflects user's environment rather than query logic, it makes more sense to have it as a configurable property, rather than a per-query argument (as top). Regarding other modules: if they don’t yet distinguish between maxrec and top, I’d warmly suggest they do, since it’s an important conceptual separation.

Please let me know your thoughts on this, before I proceed to make any changes. I'd very much appreciate if we can leave maxrec as an instance property, though :)

bsipocz · 2025-07-31T02:16:26Z

astroquery/eso/core.py

+    @deprecated_renamed_argument(('request_all_objects', 'request_id'), (None, None),
+                                 since=['0.4.7', '0.4.7'])


This is a bit unrelated so no need to do it in this PR, but 0.4.7 was a while ago, so these deprecations can be now removed.

bsipocz · 2025-07-31T02:18:26Z

astroquery/eso/utils.py

+    return where_constraints
+
+
+def reorder_columns(table: Table,


I have multiple questions here:

is this really necessary? Why the default column order is not suffucient?

but if this is necessary, then please just go by suggested way for table column reordering from the astropy docs, "Select or reorder columns" on this page: https://docs.astropy.org/en/stable/table/modify_table.html

Also, minor nitpicking, but all the other functions here are private, why is this one not?

Hmmm... but the function is already doing exactly what the docs suggest (isn't it?)...

From the docs:

new_order = ['a', 'c', 'b'] # List or tuple t_acb = t[new_order]

From the ESO module:

[...] last_cols = first_cols + last_cols # this is a list of column names table = table[last_cols]

The code above those two lines is just computing the list of column names.

Other than that,

the function is now private, good point.

yes, the reordering is necessary, because the default is alphabetical order and the first fields are long urls and other info that is not legible and take much space. The reorder was suggested by several beta testers, which were expecting the columns in the order that this function generates.

bsipocz · 2025-07-31T02:18:40Z

astroquery/eso/utils.py

+    if not isinstance(table, Table):
+        return table


Is this an expected case?

yes, table may be a string or an int, depending on the user parameters query_str_only only and count_only.

I admit this is not the best software design at this point, but a complete refactoring would have taken more time from both developers and reviewers. We will come with better encapsulation, separation of concerns, etc in the next pull request. Our aim for now was to switch to TAP with as little impact as possible to the API and to the overall code.

bsipocz · 2025-07-31T02:37:48Z

astroquery/eso/tests/test_eso_remote.py

I haven't reviewed the tests, but running them returned 77 failures, so please have a look.

pytest -P eso -R should work if you have all the test dependencies installed.

All tests were passing originally, but now I updated the fork from upstream and indeed, they are failing. I'll have a look.

tests are passing again ✅

bsipocz · 2025-07-31T02:39:05Z

docs/eso/eso.rst

@@ -4,6 +4,33 @@
 ESO Queries (`astroquery.eso`)
 ******************************

+.. warning::


This is very good! Thanks for adding.

juanmcloaiza · 2025-08-05T13:32:51Z

@bsipocz, thanks a lot for your comments, I have addressed most of them (the easy ones); for the rest, I'd still need some feedback from you. If you can give me feedback today, I can work immediately on it and push and update by tomorrow. Afterwards, I won't be able to work on Astroquery for the next couple of weeks. Just so you know, in case you can squeeze this review in your day 🤓 Thanks again! 👍

bsipocz · 2025-08-05T17:54:38Z

@juanmcloaiza - I'm sorry, it's very unlikely that I can get back to this today, but I'll try to do it soon so you can pick it up again in a few weeks time.

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch from 720fbd9 to 22d0c30 Compare June 5, 2025 12:55

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch 2 times, most recently from 76675d2 to c32f348 Compare June 5, 2025 13:18

juanmcloaiza added 8 commits June 5, 2025 15:45

Define tap_url in config; maxrec=1000

31bf997

Define helper functions in utils.py

7357a9c

Update and expand test suite

ccdef09

Update data needed by test suite

fbe4341

Update test data paths

2d663ee

Write TAP-enabled query functions

2deaa07

Update documentation to be consistent with code

f8a28d6

Update CHANGES.rst

4b775f8

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch from a987745 to 4b775f8 Compare June 5, 2025 13:49

juanmcloaiza mentioned this pull request Jun 5, 2025

Enable TAP for ESO module #3141

Closed

8 tasks

bsipocz added the eso label Jul 31, 2025

bsipocz added this to the v0.4.11 milestone Jul 31, 2025

bsipocz reviewed Jul 31, 2025

View reviewed changes

juanmcloaiza and others added 2 commits August 4, 2025 10:39

Merge branch 'astropy:main' into feature/ISSUE-3138-TAP

ca7acc1

Address review from maintainers - mostly renamings

d3be2af

juanmcloaiza force-pushed the feature/ISSUE-3138-TAP branch from 3742bb9 to d3be2af Compare August 5, 2025 12:15

Remove __doctest_skip__

685c807

		@deprecated_renamed_argument(('request_all_objects', 'request_id'), (None, None),
		since=['0.4.7', '0.4.7'])

Uh oh!

Addresses Issue #3138 - ESO - Enable Table Access Protocol (TAP) queries #3339

Are you sure you want to change the base?

Addresses Issue #3138 - ESO - Enable Table Access Protocol (TAP) queries #3339

Uh oh!

Conversation

juanmcloaiza commented Jun 5, 2025 • edited by bsipocz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes:

Uh oh!

codecov bot commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

juanmcloaiza commented Jul 11, 2025

Uh oh!

bsipocz commented Jul 21, 2025

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juanmcloaiza Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juanmcloaiza Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juanmcloaiza commented Aug 5, 2025

Uh oh!

bsipocz commented Aug 5, 2025

Uh oh!

Uh oh!

juanmcloaiza commented Jun 5, 2025 •

edited by bsipocz

Loading

codecov bot commented Jun 5, 2025 •

edited

Loading

juanmcloaiza Aug 4, 2025 •

edited

Loading

juanmcloaiza Aug 4, 2025 •

edited

Loading