Plan for Scientific Python lecture notes?

[drammock]

Various questions here:

1. There was some talk at the 2024 summit about updating the Lecture Notes to use MystMD MyST instead of rST.

   • is there still interest in that?
   • who (if anyone) was leading that effort?
   • were there any barriers / capability limitations of MystMD MyST that prevented that work from starting?

2. The current styling of the site... could be improved. Is there an appetite for / an existing decision on whether to stick with the scientific_python_lectures theme versus adopting a new theme?

3. I recall at the 2024 summit some talk of incorporating pyodide into the lecture notes.

   • are there barriers / blockers?

4. are there any sequencing considerations to items 1-3 above? I.e., is there a compelling reason to do e.g. pyodide first (or last)?

[mdhaber]

Re: 1

• I am still interested in doing grunt work. I cannot set things up, but if someone can pave the way, I can work on the translation.
• I don't know that anyone was leading an effort.
• I'm not sure about the limitations question.

Re: 4, I would imagine that 2 and 3 (both of which sound great) are independent of one another but that both would come after at least making a decision about 1. I'm not an expert, though!

[jarrodmillman]

See scipy-lectures/scientific-python-lectures#781

[drammock]

OK great! So a tentative plan for MystMD MyST conversion might be:

1. test out rst-to-myst on the whole codebase, see which files error (if any).
2. If result looks reasonable, try to build the site from resulting markdown files using sphinx.
3. If that works, QC the built site (might be possible to do an automated diff on the HTML files, between the live site and the built-from-myst one?)
4. Fix any bugs / add missing features to rst-to-myst, and/or tweak the source files as needed, to get it all looking correct.

[rossbar]

There was some talk at the 2024 summit about updating the Lecture Notes to use MystMD instead of rST.

Yes, @melissawm and I and others looked at this a bit! Unfortunately it's generally a little more involved than rst-to-myst can handle as the existing infrastructure makes use of quite a few sphinx-specific features. Some of these are captured by the auto-conversion, but many are not.

The current styling of the site... could be improved. Is there an appetite for / an existing decision on whether to stick with the scientific_python_lectures theme versus adopting a new theme?

+1 for this, though I suspect it will have some of the same hangups as crop up in the above bullet-point. FWIW I'd personally try sphinx-book-theme, which is an extension of pydata-sphinx-theme for narrative documentation. It's what we've chosen for numpy-tutorials and nx-guides, though I'm interested to hear if folks have other theming suggestions.

To me there are two separate issues: 1) the infrastructure (e.g. rst vs. myst, sphinx, executablebooks, etc.) and 2) the content.

To me, the content is much more critical - the information itself needs some serious love and there are tons of things throughout the entire lecture notes that are out of date. I think both infrastructure and content are important, but to me, updating the content would be a better activity for the summit as there will be multiple experts co-located enabling a tight write/review cycle!

If there were interest in infrastructure work, I'd personally advocate for it to be done before the summit, then presented there so interested folks can focus on the rough edges!

[bsipocz]

I am also interested in this topic, and especially in figuring out the infrastructure.

Also, I would just add it to the list, that it's also an option to go with the sphinx-based myst-nb/notebook based approach first, practically what we use for numpy-tutorials, and the rest; and then switch to the newer mystmd and interactive JB2 stack once we're sure everything we would need is already available. So the infrastructure is not blocking any of the content revitalisation.

[rossbar]

Also, I would just add it to the list, that it's also an option to go with the sphinx-based myst-nb/notebook based approach first, practically what we use for numpy-tutorials, and the rest; and then switch to the newer mystmd and interactive JB2 stack once we're sure everything we would need is already available. So the infrastructure is not blocking any of the content revitalisation.

This would be my approach as well, if for no other reason than I personally have a lot more experience with this stack (and sphinx/Python in general) than the JS side of things!

[bsipocz]

This would be my approach as well, if for no other reason than I personally have a lot more experience with this stack (and sphinx/Python in general) than the JS side of things!

Same here, and I also have maintainer rights to a couple of components, so we can fix and release any issues we run into (as the main myst/JB team already focuses on the JS side of things). Ultimately I would love to have many of the features of the JS stack, but if we take this intermediate step we at least land with infrastructure we know works and in which we already maintain several tutorial/lecture repos so the bus factor the the SP lecture notes would already improve a lot.

[drammock]

Unfortunately it's generally a little more involved than rst-to-myst can handle as the existing infrastructure makes use of quite a few sphinx-specific features. Some of these are captured by the auto-conversion, but many are not.

Yeah I'm assuming that if it "just worked" it would be already done by now. Assembling a list of which features are and are not correctly translated by rst-to-myst seems like a good first step, on the assumption that we can patch rst-to-myst to handle (some of) the missed ones, and then manually handle the ones that aren't easy to automate. But LMK if I'm missing some additional complication here.

I'd personally try sphinx-book-theme

seems like a good choice.

it's also an option to go with the sphinx-based myst-nb/notebook based approach first [...] and then switch to the newer mystmd and interactive JB2 stack once we're sure everything we would need is already available.

another +1 for sticking with sphinx first. I realize I was saying "MystMD" above when what I really meant was "MyST" --- i.e., I wasn't meaning to take a position on going straight to the npm-based tool. Will edit prior posts to correct.

[rootsmusic]

Sorry @drammock, I think the latest comment more relevantly addresses the lectures infrastructure.