Improving the tooling/framework used for the doc (Jekyll currently)

[bjohansebas]

I’ve been talking with @crandmck about migrating the website to a newer technology that can save us from unnecessary work (for example, copying and pasting content to make it available in other languages, improving accessibility, JavaScript logic that doesn’t add much value to the documentation content, styles like text line spacing, etc.). This is work that can very likely be handled by the framework we choose to use.

While the focus so far has been on the content rather than the technical aspects, I think it’s time to make this change so we can finally focus solely on the content.

The important thing here is to choose a tool that is strong in several aspects, such as:

• i18n
• Accessibility
• Ease of creating/modifying content
• Preferably good Markdown support

Possible technologies that could be used for this change:

• Docusaurus
• Astro Starlight
• Hugo
• Eleventy
• VitePress

With all this in mind, I’d like to invite @expressjs/express-tc, @expressjs/docs-wg, and all members of this community to help us choose a good tool for the documentation.

[UlisesGascon]

This is a bold move, and I think it's a good moment to do it 👏 👏.

I have only a small amount of experience with Hugo and Eleventy. I’ve never used Astro, Starlight, or VitePress... but I am a great fan of Docusaurus, mainly because it is widely adopted by other projects in the foundation, such as Jest. We can learn a lot from their base code (https://github.com/jestjs/jest/tree/main/website), as they also support multiple versions of their product. This is something we should consider, like express@3, express@4, and express@5... as a requirement when doing the decision.

That said, I’m not very opinionated on the framework used.

[carlosstenzel]

I like the idea of using Docusaurus too

[chrisdel101]

I honestly don’t have any experience with these documentation generators (if that’s the word for them) but they all look good. I think Jekyll I works okay, but is quite inefficient and i also dislike the copy paste ideology and would love to get rid of it!
I guess I wonder how we even start this? Would we work on it along side the prod site until it’s ready?
It’s a good and exciting discussion to have

[joeyguerra]

I feel like whoever is managing the website should just pick one and use it. So often we discuss tech when most of the time it's a matter of preference. And I feel like the person who's doing the work should be able to say what they want to use for the work. Sure, collaborate, think out loud, and give people an opportunity to discuss. But make sure to value making progress.

+1 fo Docusaurus. ;)

[bjohansebas]

The only thing I don’t like about Docusaurus is that it doesn’t provide a fallback for content when it’s not available in another language, which would mean we’d end up copying and pasting (I don’t want that). Astro Starlight does handle that fallback, I need to take a closer look at the others.

Examples of how they behave in i18n when content doesn’t exist yet.

• Astro: https://biomejs.dev/uk/guides/getting-started/
• Docusaurus: https://www.electronjs.org/es/docs/latest/tutorial/navigation-history
• Please, if you have examples of how i18n works in the other technologies, share them.

I don’t want you to take this comment as me preferring one technology over another, I’m just valuing something that I think is important, which is i18n support.

[bjohansebas]

I have been researching and creating a list of the advantages of some of them:

Docusaurus:

Pros:

• It is widely used within the foundation.
• Design/Accessibility: It has a predefined design (saves us time).
• Supports Markdown/MDX.
• Fast to compile.
• Can be deployed on GitHub Pages (no need for new infrastructure).
• It has i18n support with Crowdin (if we end up needing it).

Cons:

• It is very React-centric.
• It doesn't do fallback for content in other languages when it doesn't exist.

---

Astro Starlight:

Pros:

• Design/Accessibility: It has a predefined design (saves us time).
• Supports Markdown/MDX.
• Can be deployed on GitHub Pages (no need for new infrastructure).
• Fast to compile.
• I18n: Good support, does fallback for content when it doesn't exist in other languages (we wouldn't have to copy and paste to complete the i18n).

Cons:

• It uses Astro, which is relatively new.

---

Eleventy:

Pros:

• Fast to compile.
• Supports Markdown/MDX, Liquid, EJS, Pug, etc.
• Can be deployed on GitHub Pages (no need for new infrastructure).
• Has i18n support (Not sure if it does fallback for content).

Cons:

• It doesn't have a predefined design (We want to focus on content, not accessibility).

If I had to choose, it would be Starlight or Docusaurus

[wesleytodd]

I do not consider myself an expert in this space, so if in y'alls opinion another direction is better I am all good with that, but my vote goes to Docusaurus. I dislike the coupling to React in a general sense, but it is certainly the highest likelihood of having contributors easily on boarded to contribute.

[bjohansebas]

A brief summary of how this migration will be done, there is no start date yet. If anyone wants to expand on it, for me that's fine, but I think this is the necessary information.

1. Choose the new tool (The website captains will choose it, we still need some time)(@expressjs/docs-captains)
2. We will create a list of tasks to be done to achieve this migration, so we can divide the tasks. (@expressjs/docs-wg)
3. We will create a new separate branch (@bjohansebas)
4. Once there are changes to review, we will open a draft PR. (Anyone with write permissions will be able to do it)

Note: We will follow the same process of creating PRs, not pushing directly to the branch.

[jonchurch]

Thanks for kicking off this discussion and being dedicated to improving the docs site! I wanted to share some thoughts, not as a pushback but to make sure we're solving the right problems in the best way.

What's the core problem we are solving?

From what I understand and am reading here, translation management is the biggest issue. Specifically needing to sync translations with the english version as the english updates.

Docusaurus alone doesnt fix that, but does provide a paved path to Crowdin integration which provides a much nicer interface for translators. The real win there is Crowdin integration, which we can implement regardless of whether we use Jekyll, Docusaurus, or another framework.

Should we migrate? Or first fix the biggest issue?

Beyond translation, other raised concerns are accessibility, styling, JavaScript overhead, authoring experience. These are important, but let's make sure a migration would meaningfully improve these areas rather than shifting the work elsewhere.

For consideration:

• Jekyll is not inherently inaccessible. We can make incremental improvements without migrating. What specific accessibility issues are we experiencing now?
• Styling can be improved by switching to a better jekyll theme rather than switching to a new platform entirely. (I'll talk more about this in another section)
• Reducing unnecessary JavaScript is a good goal, but it would be helpful to clarify what specific issues we're looking to solve here. A single page application moves us further away from reducing JS on page than what we have now, so are we talking about JS scripts we have written ourselves?
• Authoring experience for jekyll and another framework would be largely the same, if we are sticking to mostly writing markdown.

For those who have worked with Docusaurus (or other suggested tools), do you feel the trade offs justify a migration, or are there other specific benefits that haven’t been captured yet?

Are there other issues not captured in this discussion?

I said Id touch on styling again. I think the background to this conversation is that our site is currently ugly and clunky.

Since we own the theme, we own fixing that. We own the accessibility story, the look and feel of the site. And the local dev experience of Jekyll is clunky and outdated to JS devs.

So is the root of this discussion, the big problem we want to solve, that no one wants to improve the theme of our Jekyll site or work with Jekyll tooling?

The biggest benefit I see to switching to a new framework is that we get a new theme and don't have to invest in accessibility or design time. We can piggyback on well maintained themes from someone else. As soon as we deviate from that, by writing custom components and styling, we lose that benefit and are back to maintaining the theme ourselves. We also have built custom tooling into our site, to grab external readmes as an example, which increases maintenance burden and complexity.

If the root issue is not wanting to maintain our own theme, should we be framing this discussion around finding a good out of the box solution rather than focusing on translation or accessibility? If so, what existing themes in Docusaurus, Hugo, or other options stand out as strong contenders?

The above is a more compelling reason to switch to me than the other things outlined previously. But with that comes unknown unknowns, we can't shake all those out before making a decision either way, so being crystal clear first on what we are solving for and how we think a migration can improve things is crucial.

I think it would be really helpful to get more clarity on what people see as the biggest blockers with Jekyll beyond i18n. If we can clearly define what’s painful, we’ll have a stronger case for either fixing it incrementally or migrating with confidence.

[bjohansebas]

Thank you for joining the discussion

translation management is the biggest issue. Specifically needing to sync translations with the english version as the english updates.

That is part of one of the current problems. Indeed, Docusaurus does not solve this issue, but I believe Starlight partially addresses it by bringing in the English content, as I mentioned in my comparison above. A complete solution is difficult to achieve. Crowdin is used in Node.js, and I know they struggle to get people to help with translations.

Should we migrate? Or first fix the biggest issue?

Here, you’ll address several points, and I will try to respond as organized as possible:

The main goal of this migration is to improve our experience when editing documents, freeing us from having to modify styles to maintain or improve the site's accessibility. Instead, we should focus on keeping the documentation up to date.

Authoring experience for jekyll and another framework would be largely the same, if we are sticking to mostly writing markdown.

Actually, these tools provide better use of markdown. Currently, to display alerts on the page, I have to create a 'component' in Liquid, and to use that component, the syntax is very ugly and hard to read, which makes it more difficult for new people to join in and help maintain the documentation.

https://github.com/expressjs/expressjs.com/blob/gh-pages/en/guide/routing.md?plain=1#L72-L87

For those who have worked with Docusaurus (or other suggested tools), do you feel the trade offs justify a migration, or are there other specific benefits that haven’t been captured yet?

I believe so, I'm not sure what others think, and I would appreciate it if they could share their thoughts on this as well. When I created this site, I didn’t have to worry about the site's styles, just adapting the site to only use markdown, since we currently use Liquid. It didn’t take me long because Starlight and Docusaurus already have all the styling and accessibility covered with components they’ve already tested.

We can piggyback on well maintained themes from someone else. As soon as we deviate from that, by writing custom components and styling, we lose that benefit and are back to maintaining the theme ourselves

While i maintain the documentation, i would block all those PRs, unless there’s a good reason to edit that style. But in reality, that’s unnecessary, because Starlight and Docusaurus already bring everything needed in terms of styles. We wouldn’t have a CSS file with over 1,000 lines, which has become increasingly difficult to maintain.

If the root issue is not wanting to maintain our own theme

Exactly, that is the main problem (which comes with accessibility issues). Until recently, no one had shown interest in improving the design. With the documentation team, we’ve tried to improve it, but is this really how we want to spend our time instead of improving the content? A long time ago, StrongLoop and IBM were behind supporting the improvement of the documentation, but now Express is fully open-source, and no one pays us to maintain the design or the content. So, if we can reduce maintenance in terms of design, that would be the best option.

If so, what existing themes in Docusaurus, Hugo, or other options stand out as strong contenders?

• Docusaurus only has one theme
• Hugo I think we have to create our own theme
• Starlight only has one theme,
• Eleventy, we need to create our own theme.

And btw, thanks to you creating the Docker container to develop the page, I was able to contribute, as installing Ruby is very complicated compared to Node.js (that’s my experience).

[crandmck]

@jonchurch makes some excellent points: We should definitely consider what we want to accomplish; i.e. what our real requirements are. Ultimately, it comes down to fixing what we have or "starting from scratch" (as it were) with a new static site gen. tool and migrating the content.

It's a complex issue with lots of factors to consider. Let me say up front that I only have experience with Jekyll and Docusaurus. I know the others are popular but I don't have any direct experience with them.

Also, re localization. In previous TC discussions the subject came up, and the possibility of using machine translation seems more realistic than it has before. AI language models have come a long way recently! That may obviate the need for extensive contributions of translations (but likely not eliminate the need entirely, e.g. corrections and improvements will likely still be required).

The bottom line is that no matter what SSG we use, if we want a distinctive look and feel (and I assume we do; not something generic) then we will always have to do a certain amount of HTML/CSS hacking and maintenance. It's true the current Jekyll setup is rather unwieldy, but I don't think that's a fundamental limitation of Jekyll. @hacksparrow set it up many years ago and he wasn't very familiar with it at that time, so it uses a lot of HTML vs. markdown and doesn't do a lot of stuff that would be table-stakes today (e.g. page TOC, programmatic menu generation from YAML or JSON, etc.)

Actually, these tools provide better use of markdown. Currently, to display alerts on the page, I have to create a 'component' in Liquid...

This is due to the way that the site was set up (see above), but isn't a fundamental limitation of Jekyll. For example, it's possible to use something like "Font Awesome" to make alerts as simple as this:

Here's some useful information.
{: .info }

You can see this in action here: https://crandmck.github.io/jekyll-demo/ (source in https://github.com/crandmck/jekyll-demo).

I think it would be a relatively small amount of work to clean up the current Jekyll setup and adopt some of the features in (for example) Tom Johnson's https://idratherbewriting.com/documentation-theme-jekyll/.

Would that be better than starting fresh with an entirely new SSG? Unclear. Converting all the current content would be a lot of work, and would likely need to be scripted and then manually reviewed. I've done that kind of thing before and it's time consuming and error prone. But ultimately, it might be better than trying to fix what we have now....

I suggest we identify the key requirements and "nice to have" features, and then compare how each candidate SSG and the current setup would fulfill them. Then we can make a more informed decision on how to move forward.

[bjohansebas]

Each of you has good points, which is why I started this discussion before doing any major work (whether with the current setup or a migration). I don't have much experience with Jekyll, but with that article, we could try to clean up the Jekyll configuration, though it will be labor-intensive.

This is due to the way that the site was set up (see above), but isn't a fundamental limitation of Jekyll. For example, it's possible to use something like "Font Awesome" to make alerts as simple as this:

I've been trying to remove Font Awesome because we're loading all of its CSS. I'm sure we're not using more than 20% of Font Awesome. But i could accept it as long as it provides a better experience on the page.

[crandmck]

I've been trying to remove Font Awesome because we're loading all of its CSS.

I think you're right that we're not using much of it...

Do we have any idea how much this affects performance? I haven't looked into it closely, but in the minimal demo I posted before it loads a single CSS file that's about 100 KB in size. I honestly don't know if that's large enough to have a significant impact on performance...? If so, we could look into alternatives.

[bjohansebas]

It makes the webpage load a bit slower, although it's not that critical since it's not a 10MB file. That's why I could accept it, because it's just milliseconds/seconds. I'm not sure how much it would affect slow connections, but it shouldn't make the page load too slowly.

[bjohansebas]

@expressjs/docs-wg, what do you think? Should we stick with Jekyll?

I don't want to block the maintenance of the page either, that's why I want to know, so we can either close this or keep discussing it. (I say this because, with this issue, you probably don't know whether to keep working on something or wait to make a decision)

[clicktodev]

Docusaurus is the most feature complete documentation generator i've worked with and it's fast and customizable

Examples:

• https://trpc.io/
• https://reactnative.dev/

supports everything you've mentioned and more

• i18n
• Accessibility
• Ease of creating/modifying content
• Preferably good Markdown support MDX

I can offer my help as i have worked extensively with it

+1 Docusaurus

[bjohansebas]

Probably, we need to see the support GitHub wants to give to Jekyll. I mean, GitHub hasn't updated pages-gem in a while, Jekyll 4 (which will likely make my build faster) was released over 5 years ago, and GitHub wants to use version 3.10.0. They have several open PRs updating the tools, but there isn't much movement on them wanting to maintain it

[AugustinMauroy]

Without hesitation, you should use Docausorus. It's a well-maintained technology that supports MDX and react. And above all, it's widespread.

-1 astro and vitepress because they are too new and we need something stable

-1 for non-js tools (Hugo, Zola ...)

-1 for tools that are just framwork. I don't think there's any point in doing everything from scratch like on node.

[clicktodev]

Pretty sure the team will land on docusaurus eventually

[bjohansebas]

For now, we are going to stick with Jekyll since we will be using Crowdin for i18n (which means we no longer have to copy and paste), and we have been separating the content from the layout, making the page more maintainable. It also seems that the documentation team is fine with this since they haven’t responded to this message.

We can always reopen this issue and discuss it again if maintaining the page truly becomes difficult. I have opened several issues to improve the current configuration, which are currently being worked on.

I wouldn't want to maintain the current page design, but I can live with it and still improve it, as my main goal is to enhance the experience of using the documentation.

Thanks to everyone who shared their comments—I’m glad to see that the community supports changes like this.

[clicktodev]

Docusaurus supports Crowndin btw

[chrisdel101]

Im OK staying with Jekyll since it’s not react based and it’s simple. And no copy paste solves biggest issue. Would be nice to have something that loads faster in dev in the future.
Docusurus looks very nice though. Maybe a we can run a staging env to try it out?

[bjohansebas]

For those who aren't aware, express is planning a complete redesign. That's why I'm reopening this discussion, we need to clearly define the stack we're going to use for the redesign. There have been suggestions to use web components (as mentioned in the last meeting). It also needs to be easy to maintain

[clicktodev]

Docusaurus (React and MDX) imo is the best option in terms of features, customizability, maintainability, and popularity.
I see no reason to go with anything else, if anyone has a convincing argument against docusaurus then i'm all ears