Skip to content

Conversation

@jameskerr
Copy link
Contributor

@jameskerr jameskerr commented Nov 19, 2024

Summary of changes.

In the front matter:

• sidebar_position becomes weight.
• sidebar_title becomes title.
• README becomes _index
• h1's have been removed from the markdown and rendered in the template with .Title or .Heading

If you want a different heading that what the title is, you can use the heading: key in the front matter.

@philrz
Copy link
Contributor

philrz commented Jan 29, 2025

tl;dr

I've pushed many more commits to this branch, primarily focused on converting as many examples as possible to the new mdtest-spq syntax (introduced in #5594) so they can be rendered in the interactive Playground while the remaining examples can stay as mdtest-command syntax for now. At this point, I think this PR is ready for merge, with some additional "Next Steps" pointed out at the bottom of this comment.

Details

This branch has been wired up for months now to have its changes published to https://superdb.org/docs/, so you can see the rendering of everything touched by this PR already "live".

On the whole, the mdtest-spq conversions have been quite successful. I was certain going in that there would be corner cases that would need to be excluded, and this has been indeed been true, e.g.:

  1. Examples I've converted to mdtest-spq that could still benefit from additional enhancements to improve readability, or,
  2. Examples that don't lend themselves to mdtest-spq conversion immediately or ever.

Once this PR gets merged, I plan to open some issues to point out the corner cases that we could perhaps address with some additional investment of time, and then we can come to some group "now/later/never" consensus on if we want to take action. But here's a quick overview of corner cases you may spot in this branch:

  1. Docs that are explicitly intended to show command line operations (e.g., the super command doc and tutorial) can justifiably remain mostly/all mdtest-command.

  2. Examples that call out to additional shell commands (e.g., jq, seq, etc.) have had to remain as mdtest-command.

  3. Examples that show lake operations have had to remain as mdtest-command.

  4. Examples that make heavy use of mdtest-input (e.g, the join tutorial in particular relies on this to show multiple named join inputs) have had to remain as mdtest-command.

  5. Some examples that used output formats other than compact ZSON (most of which used pretty printed -Z for readability reasons) are now using -z compact ZSON in mdtest-spq, but this is not ideal.

  6. The Zeek integration docs in particular hit many of the above points and their future seems overall suspect, so I've left them as they are.

Since I effectively had to visit every example in the docs I took care of some other things along the way. Specifically I reverted much of the use of |> pipe syntax back to | (I got clearance from @mccanne on this change some time ago) and also tweaked some of our "how we talk about the language" terminology to be in line with our current trends. That said, I know there's still more passes of de-Zed-ification needed in the docs so this was in many ways a "progress, not perfection" part of the effort. Therefore I'd ask reviewers to not spend time pointing out every time you see "SuperPipe" or "Zed" still mentioned somewhere in this branch. I'm aware it all still needs to go, and it'll be an ongoing effort. This PR is already plenty huge.

Next Steps

To me, this PR has been the main blocker on the new docs site going fully "live" (i.e., pointing to it from the top-level super repo README) since up until now tons of the examples were busted and I didn't want users seeing that. Now that's been addressed thanks to mdtest-spq, it seems we're ready to proceed. Next steps:

  1. Before this PR merges, Stop showing Next and add an announcement bar about SuperDB docs zed-docs-site#87 needs to be approved and merged. That PR has more detail, but most importantly, it won't make sense to push any docs changes from the super repo to that old Docusaurus-based site anymore because it's incompatible with the new Hugo stuff that's in this PR.

  2. Before this PR merges, I'll push the final change referenced in the comment above so the superdb-website repo will start to be notified when super's main branch changes. I'm holding it pointing at this branch for now just in case there's final commits during review that we want pushed to the site.

  3. Merge this PR!

@philrz philrz requested a review from nwt January 30, 2025 16:08
@philrz philrz merged commit 24492d9 into main Jan 30, 2025
5 checks passed
@philrz philrz deleted the hugo-docs branch January 30, 2025 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants