Fix trailing slash issues (depend on hosting provider)

[slorber]

Edit: as part of the analysis to solve this issue, I'm writing a guide to describe factually the behavior of various static site generators and static hosting providers: https://github.com/slorber/trailing-slash-guide

💥 Proposal

Fix trailing host slash issues: write /myDoc.html instead of /myDoc/index.html

We have a few issues that are related to docs trailing slashes: #2654, #2394, #2134, #3351, #3380

I suspect these issues are related to the Docusaurus output, as for a doc at /docs/myDoc we write /myDoc/index.html instead of /myDoc.html

Always creating a subfolder is probably a signal for the hosting providers (Netlify, Github Pages...) to add a trailing slash automatically to URLs, which can be annoying.

It can break relative links as now the link resolution is different due to the trailing slash, and we end up with broken links in production, not catched at build time (see #3380)

This behavior is provided by a dependency: https://github.com/markdalgleish/static-site-generator-webpack-plugin/blob/master/index.js#L165

We already run on a fork of this dependency. I guess we could as well include a way to write /myDoc.html directly if a route path does not end with a trailing slash.

As this is a risky change, I suggest making this optional. We can make it the default behavior (breaking change) later if it works fine for early adopters.

See also #2394

[slorber]

Seems related to markdalgleish/static-site-generator-webpack-plugin#53

[lunelson]

Always creating a subfolder is probably a signal for the hosting providers (Netlify, Github Pages...) to add a trailing slash automatically to URLs, which can be annoying.

@slorber fwiw this is fixable on Netlify by changing your post-processing settings. They have a UI bug, where the master checkbox to disable post-processing does not disable the pretty URL setting, and that is what causes e.g. https://v2.docusaurus.io/docs/2.0.0-alpha.63/installation to 301 redirect to https://v2.docusaurus.io/docs/2.0.0-alpha.63/installation/. You have to just turn them all off individually.

[slorber]

Thanks @lunelson , didn't know about this bug, do you have a link to track this?

Also, I tested these options on my own website, but still have a trailing slash issue 😅 https://sebastienlorber.com/records-and-tuples-for-react

[lunelson]

@slorber If you changed the options and tested your site right away, it might not have worked: the changes seem to take a few minutes to propagate. For me the URL above now works without redirecting to a trailing slash.

And no I don't know of an issue that tracks it at Netlify but I've seen it come up in discussions here and on Gatsby about trailing slash problems.

[lunelson]

@slorber Having pointed that out however, I must say that in my company's websites (including 3 Docusaurus v2 sites and potentially soon more), we will probably turn this Netlify feature back on.

This is because we have had the strong recommendation from an SEO agency that—while neither format (trailing-slash or not) is "better"—there should be 301 redirects to make sure that only one of the two formats is getting crawled and indexed. And because Netlify treats these URL formats identically, and its redirects system therefore doesn't allow you to 301 from one to the other (it would create an infinite loop), our only alternative is to use this pretty URLs feature (which does create a 301) and to accept the decision which is made for us, namely that trailing-slash will be the final format.

This creates a challenge for me right now because currently Docusaurus generates all URL patterns in router/anchor links, sitemap and canonical link tags without trailing slashes 😬

[mpsq]

The sitemap can be generated with trailing slashes, that's what the option trailingSlash is for: https://v2.docusaurus.io/docs/using-plugins#docusaurusplugin-sitemap.

[lunelson]

@mpsq well, that's one down at least 😅

[mpsq]

Yes :) On your list, "URL patterns in router/anchor links" and "canonical link tags" are actually the same thing, they depend on how the permalink prop is generated. All you need is to enable/allow that prop to have a trailing slash.

EDIT:
Maybe this could be a more general setting, rather than a granular one specific to the sitemap plugin.

[lunelson]

Maybe this could be a more general setting, rather than a granular one specific to the sitemap plugin.

YES. This would be ideal.

@mpsq is there any chance you could give me an idea of where I need to intervene in my custom theme, to modify that permalink property in a way that will affect the router, and generated anchor and link tags all together?