Fix broken documentation links after migration to MkDocs

[rashidakanchwala]

Description

After migrating our documentation to MkDocs and restructuring the information architecture, several links are now broken — both:

• In the README files across repositories (e.g. kedro-datasets, kedro-plugins, etc.), and
• Within the MkDocs site itself (internal page references).

For example, the “How to create a custom dataset” tutorial link now returns a 404:
https://docs.kedro.org/en/1.0.0/data/how_to_create_a_custom_dataset.html
The correct URL is:
https://docs.kedro.org/en/stable/extend/how_to_create_a_custom_dataset/

Tasks:

• [] Audit and fix broken links in all READMEs and documentation files.
• [] Enable a strict link checker for MkDocs builds (e.g. mkdocs build --strict or a link-checking plugin). This will help identify broken internal links quickly and make the audit more efficient.
• [] - Audit and fix internal cross-links within the MkDocs site using the checker’s output.
• [] Integrate the link checker into CI so that 404s and broken references fail the build.

Notes:

Coordinate with @jo-stitchbury MkDocs’ strict mode is currently disabled by default.
The goal is to ensure all links across READMEs and published docs remain valid and automatically verified in future releases.

Possible Alternatives

[ravi-kumar-pilla]

Related issue: #5159

[lrcouto]

As of now, I have written a GitHub action to check for broken links. I'd like opinions on when we should run it.

Option 1 - Every time we push something.
Option 2 - Every time we change docs.
Option 3 - On a schedule.

I personally think option 2 is not ideal, because external links can change without our input. It'd be better to run it more frequently so we catch this kind of thing early.

On push would yield the most frequent checks, but it's another check that we have to wait for when pushing something, on our already lengthy list of CI jobs.

On schedule has the risk of something being uncaught for a little while, but might be the best solution. I think maybe once a day would be sufficient.

What does everyone think?

[lrcouto]

Also, using the lychee CLI tool to find broken links found:

• The McKinsey website timeouts if you try to access it through this tool.
• A lot of external sites 403 when you try to access them this way.
• Didn't find any actual 404s.

I tested all of the external links that showed issues manually, and they're all reachable. So I'm not gonna run lychee on CI together with the MKDocs link check, maybe put it on a separate job that we can run once in a while.

[lrcouto]

And without running lychee, just the mkdocs strict build is a pretty quick check, so it can stay on regular CI jobs that run on push IMO.

[lrcouto]

Here are the PRs for this task:
Kedro - #5173
Kedro Viz - kedro-org/kedro-viz#2509
Kedro Datasets - kedro-org/kedro-plugins#1222

[merelcht]

The link check job looks really fast, so I think it's fine to run on every push. Thanks @lrcouto !