Adds enhanced search #287

slowe · 2025-06-02T14:13:40Z

This adds a first draft of an enhanced search interface that uses ReadTheDocs's API to search across multiple projects as requested by @Cadair.

This uses the existing search feature but then adapts the interface to add inline results rather than redirect to the search results page. If CORS is not enabled (with ReadTheDocs) it will redirect to the standard site search page. Alternatively, you can add ?debug to the URL to use https://corsproxy.io/ to get around CORS issues.

The inline search results will be arranged into tabs showing "All" results and results broken down by project. By default, the script will attempt to find projects from the page. It will look for elements with class="nav-link" from a menu item with id="Documentation". However, you can manually over-ride this behaviour by using the Javascript variable set_search_config e.g.

<script>
const set_search_config = {
  "no-results":{
   "label":"Nothing to show here."
  },
  "load-more":{
    "label": "Load more results",
    "class": "btn btn-lg btn-sunpy btn-sunpy1"
  },
  "projects":[
    {"name":"sunpy","link":"https://docs.sunpy.org/"},
    {"name":"ndcube","link":"https://docs.sunpy.org/projects/ndcube/"},
    {"name":"sunraster","link":"https://docs.sunpy.org/projects/sunraster/"},
    {"name":"aiapy","link":"https://aiapy.readthedocs.io/"}
  ]
};
</script>
<script src="path/to/sunpy_enhanced_search.js"></script>

where:

no-results - an optional object to set the label that gets displayed when there are no results;
load-more - an optional object to set the class/label of the "load more" button;
projects - an ordered array of projects to include (if empty this will be constructed from the ".nav-link" items found within #Documentation). The order of this array sets the order of the search results tabs.

Cadair · 2025-06-02T15:39:09Z

src/sunpy_sphinx_theme/theme/sunpy/static/sunpy_enhanced_search.js

+			let path = (page || ("api/v3/search/?q=" + projstr + "+" + encodeURIComponent(str)));
+			if(debug) {
+				url = "https://corsproxy.io/?https://readthedocs.org/" + path;
+			}else{
+				url = "/_/" + path;
+			};


@slowe I did bad things to try and get the RTD preview working.

and I broke "load more" in the process 😭

I think I fixed it for load more, but I made it much more ugly in the process!

Cadair · 2025-06-02T16:01:52Z

@slowe here's some things I noticed while testing with the RTD preview:

Using the keyboard I appear to only be able to move between the first two tabs (pressing left and right)
I'm getting the Load More button when I don't really expect it to appear, also using the keyboard right shortcut seems to cause it to pop up (maybe related to above issue?) - oh it seems to always show? Is there any way to only show it if there are hidden results?
The styling on the load more button could be nicer.

slowe · 2025-06-13T09:47:59Z

First thing to note: I'm only able to test on a local file system so I may not have the same situation as you.

Using the keyboard I appear to only be able to move between the first two tabs (pressing left and right)

This is probably down to the change to hide empty tabs. I'll have a look.

I'm getting the Load More button when I don't really expect it to appear

Can you give an example of when? I am not seeing the same thing so it is probably down to a difference in expectation.

Is there any way to only show it if there are hidden results?

By "hidden" do you mean "not loaded"?

The styling on the load more button could be nicer.

Please can you advise on what that should be?

slowe · 2025-06-13T10:10:41Z

To clarify what I think is happening around the "Load more" button... I make one request to the ReadTheDocs API that includes all the projects. ReadTheDocs then returns up to 50 results at a time e.g. https://readthedocs.org/api/v3/search/?q=project:sunpy+project:ndcube+project:sunraster+project:aiapy+fits

I'm then doing the totalling up by project. It is only possible to know the count by project for the results that have been loaded so far. So there may only be, say, "4" results loaded in a particular project but that doesn't mean there couldn't be more in the next batch of 50 results. The "Load more" button should keep appearing until all the results have been loaded.

An alternative would be to make n requests to ReadTheDocs, one for each project every time the typeahead activates. This is inefficient and, potentially, acts more like a denial-of-service attack. Is it worth the number of requests to remove a slight, potential, confusion? I would also hope that most people have found what they were looking for within the first 50.

slowe · 2025-06-13T10:17:06Z

The styling on the "Load more" button is the same as the styling on the buttons on the front of sunpy.org as it uses the classes load-more btn btn-lg btn-sunpy btn-sunpy1. So it should inherit the style from the theme. The load-more class should allow you to specifically style that button in your theme style (the JS code doesn't apply any styling of its own to it).

slowe · 2025-06-13T10:28:34Z

Ah ha. This is how it looks for me locally:

And this is on the "website preview" link (set to debug so that results load) https://output.circle-artifacts.com/output/job/00b5f97b-26f5-4665-9602-9ff4a13b7b61/artifacts/0/.tmp/website/sunpy.org/_build/html/index.html?debug=true

But what I said about the styles is because the classes are set in set_search_config i.e.

const set_search_config = {
  "no-results":{
   "label":"Nothing to show here."
  },
  "load-more":{
    "label": "Load more results",
    "class": "btn btn-lg btn-sunpy btn-sunpy1"
  },
  "projects":[
    {"name":"sunpy","link":"https://docs.sunpy.org/"},
    {"name":"ndcube","link":"https://docs.sunpy.org/projects/ndcube/"},
    {"name":"sunraster","link":"https://docs.sunpy.org/projects/sunraster/"},
    {"name":"aiapy","link":"https://aiapy.readthedocs.io/"}
  ]
};

I didn't add btn btn-lg btn-sunpy btn-sunpy1 by default (only load-more) because I wasn't clear if they were part of the overall theme or project-specific.

Cadair · 2025-06-18T06:29:52Z

It is only possible to know the count by project for the results that have been loaded so far. So there may only be, say, "4" results loaded in a particular project but that doesn't mean there couldn't be more in the next batch of 50 results. The "Load more" button should keep appearing until all the results have been loaded.

I would also hope that most people have found what they were looking for within the first 50.

Ah ok, that makes sense. It just surprised me that I could see a (4) for a project followed by load more. I assume that if there are no more pages of results to load for any project the load more wont show?

I didn't add btn btn-lg btn-sunpy btn-sunpy1 by default (only load-more) because I wasn't clear if they were part of the overall theme or project-specific.

Ah I see, I think we can safely add them by default they are defined in this theme. (Which I see you have done.)

@slowe Could you have a look over the changes I made to the cors proxy code if you haven't already?

I think there's a couple of things left to do before we merge this, but I think they are mostly on me:

Add python / sphinx side config to set projects and general config from the html_theme_config variable.
Figure out the CSS theming on the button which isn't working on RTD but is working locally.

A more compact version of the URL building. Remove/define some variables.

Switching to classes `sd-btn sd-bg-primary sd-bg-text-primary` from https://sunpy--287.org.readthedocs.build/projects/sunpy-sphinx-theme/287/web-components.html

slowe · 2025-06-18T09:12:33Z

I've changed the default "Load more" button styles to load-more btn sd-btn sd-bg-primary sd-bg-text-primary. The sd- prefixed styles are taken from https://sunpy--287.org.readthedocs.build/projects/sunpy-sphinx-theme/287/web-components.html

I assume that if there are no more pages of results to load for any project the load more wont show?

That's the case, yes. It basically depends on the existence (or not) of a next page in the ReadTheDocs API result.

Would it help if the icon in each project tab was a filter icon (e.g. https://icons.getbootstrap.com/icons/funnel-fill/) to visually distinguish those tabs from the "All" tab?

Cadair · 2025-06-18T09:28:39Z

Would it help if the icon in each project tab was a filter icon

Yeah, I think that's a good idea.

slowe · 2025-06-24T08:07:30Z

@Cadair How about this? https://output.circle-artifacts.com/output/job/b4dcff59-b622-479c-8eeb-641acf71eda0/artifacts/0/.tmp/website/sunpy.org/_build/html/index.html?debug=true

slowe and others added 3 commits June 2, 2025 14:54

Add enhanced search

0e4f98a

Include search js in sphinx build

ca20a4f

Use /_/ proxy on RTD

2cf5674

Cadair reviewed Jun 2, 2025

View reviewed changes

Cadair added 2 commits June 2, 2025 16:59

Use /_/ proxy on RTD

ad1d0d4

Second attempt at using _ RTD api

eee8538

Fix bug with keyboard navigation on project tabs

fbae5cd

Add some default classes

7379115

slowe added 2 commits June 18, 2025 09:46

Tidy up code

8c4d6f1

A more compact version of the URL building. Remove/define some variables.

Change button style

55c4ce8

Switching to classes `sd-btn sd-bg-primary sd-bg-text-primary` from https://sunpy--287.org.readthedocs.build/projects/sunpy-sphinx-theme/287/web-components.html

Add filter icon

3882c00

Cadair mentioned this pull request Jul 8, 2025

List astropy coordinated affiliated packages on main documentation page astropy/astropy#8221

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Adds enhanced search #287

Adds enhanced search #287

Uh oh!

slowe commented Jun 2, 2025

Uh oh!

Cadair Jun 2, 2025

Uh oh!

Cadair Jun 2, 2025

Uh oh!

Cadair Jun 2, 2025

Uh oh!

Cadair commented Jun 2, 2025 •

edited

Loading

Uh oh!

slowe commented Jun 13, 2025

Uh oh!

slowe commented Jun 13, 2025

Uh oh!

slowe commented Jun 13, 2025

Uh oh!

slowe commented Jun 13, 2025

Uh oh!

Cadair commented Jun 18, 2025

Uh oh!

slowe commented Jun 18, 2025

Uh oh!

Cadair commented Jun 18, 2025

Uh oh!

slowe commented Jun 24, 2025

Uh oh!

Uh oh!

Uh oh!

Adds enhanced search #287

Are you sure you want to change the base?

Adds enhanced search #287

Uh oh!

Conversation

slowe commented Jun 2, 2025

Uh oh!

Cadair Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

Cadair Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

Cadair Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

Cadair commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

slowe commented Jun 13, 2025

Uh oh!

slowe commented Jun 13, 2025

Uh oh!

slowe commented Jun 13, 2025

Uh oh!

slowe commented Jun 13, 2025

Uh oh!

Cadair commented Jun 18, 2025

Uh oh!

slowe commented Jun 18, 2025

Uh oh!

Cadair commented Jun 18, 2025

Uh oh!

slowe commented Jun 24, 2025

Uh oh!

Uh oh!

Cadair commented Jun 2, 2025 •

edited

Loading