Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ export default defineConfig({
"recipes/caching",
"recipes/excluding-links",
"recipes/excluding-paths",
"recipes/local-folder",
"recipes/migration",
"recipes/base-url",
"recipes/root-dir",
Expand Down
112 changes: 112 additions & 0 deletions src/content/docs/recipes/local-folder.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
---
title: Checking a Local Folder with URL Remapping
description: Checking a local folder of HTML files which will be uploaded to a particular URL.
---

lychee can be used to check a directory of local HTML files before they're
uploaded to a website. Even though your site isn't deployed yet, lychee can
verify that all links will work when the site is uploaded to its online
location.

This works by mapping the content's future URLs to local files on your
computer, using lychee's URL remapping feature. For links to these URLs, lychee
will check that the corresponding files exist inside the local directory,
rather than checking the online website.

This means you can use fully-qualified links in your HTML files and the links
will be checked correctly, even if those pages aren't online yet. This is
especially useful as part of a CI pipeline which builds a static website.

## Do You Need URL Remapping?

In simple cases, you don't!

By default, lychee can already resolve relative links to adjacent local files.
By adding [`--root-dir`][root-dir], lychee can also resolve root-relative links
(beginning with `/`) to the given root directory. In simple cases, this is all
you need.

Continue reading if:
- you have fully-qualified links to files which exist locally but aren't online yet, or
- your local folder will be uploaded to a _subdirectory_ of the website domain.

[root-dir]: /recipes/root-dir/

## Mapping Remote Domain to a Local Folder

Suppose you have a local directory `out` and this will be uploaded to
the domain root at `https://docs.example.com`.

You can map URLs beginning with this domain into the local directory:
```bash
lychee ./out --root-dir ./out --remap "https://docs\.example\.com file://$(pwd)/out"
```
This will remap URLs so `https://docs.example.com/page.html` becomes
`./out/page.html`, for example.

:::caution[Caveats when using remaps]
- For each URL, remaps are tried in order and the *first* matching
remap will be applied. If you are using remaps for multiple purposes, be aware
of potential conflicts between them.

- Remaps are applied textually. For instance, `--remap 'https://example\.com/a
file://tmp` would also apply to a URL of `https://example.com/ab`. This is
undesirable but safely excluding this is very hard. Using regex's `$` would
conflict with `#fragments` or `?query` parameters, and appending a `/` would
cause the URL without a slash to be missed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While reading this, I did wonder if we should mention a workaround. For example, we could suggest the following regular expression either here or in a footnote.

^https://example\.com/a($|[/?#])

It's not flawless, but still helpful, perhaps.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's a good idea and, in practice, it's probably good enough for 99% of cases. I shouldn't be so pessimistic about things like this :-)

Copy link
Member

@MichaIng MichaIng Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also skip the technical explanation why this is hard to avoid. I would replace that with Matthias' example which should cover most cases properly, as you said.

Suggested change
- Remaps are applied textually. For instance, `--remap 'https://example\.com/a
file://tmp` would also apply to a URL of `https://example.com/ab`. This is
undesirable but safely excluding this is very hard. Using regex's `$` would
conflict with `#fragments` or `?query` parameters, and appending a `/` would
cause the URL without a slash to be missed.
- Remaps are applied textually. For instance, `--remap 'https://example\.com/foo'
'file://tmp'` would also apply to a URL of `https://example.com/foobar`.
To avoid this, a regex pattern can be used to match the end of a path element,
e.g `'^https://example\.com/foo($|[/?#])'`.

EDIT: Ah, that regex is interpreted is mentioned next. Maybe it should be reordered somehow. Though I have an idea how to avoid it ... see below

Copy link
Contributor Author

@katrinafyi katrinafyi Jan 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As of https://github.com/lycheeverse/lycheeverse.github.io/blob/aaebd37af4652e170bd98da8bb847fb1222cc581/src/content/docs/recipes/local-folder.mdx, this dot point looks like this. Thanks to @mre for the workaround suggestion.

  • Remaps are applied textually. As an example, the remap

    --remap "^https://example\.com/docs file://$(pwd)/out"
    

    applies to any URL beginning with that string even if it's inside a different subfolder.
    For instance, it would also apply to a URL of https://example.com/docs-2/page.

    If you need to guard against this, you can change the regex to end with
    ([?#/]|$) and add $1 to the replacement, like so:

    --remap "^https://example\.com/docs([?#/]|$) file://$(pwd)/out$1"
    

edit: need escape $1 in bash

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. And yes, $1 would need to become \$1 here, not only in bash, but in any UNIX shell within double-quotes or no quotation. Single-quotes could be used to avoid this, but then $(pwd) wouldn't expand either.

But does $1 really add the (...) match to the replacement string here? Typically, regex match + replacement features use \1 (resp. \2 etc) for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's mixed. sed uses \1, but rust and javascript use $1. The PR adding tests includes this at https://github.com/rina-forks/lychee/blob/2c4fb5c4c60aec74b5c5f59ccd7c1b00752eef2d/lychee-bin/tests/cli.rs#L3268


- Remap patterns are regular expressions, so `.` in domain names should be
escaped to avoid matching more than you expect.
:::

## Mapping a Remote Subfolder to a Local Folder

If, instead, your local folder will be uploaded to a _subdirectory_ of the website
(rather than the domain root), you will need some more set up.

Suppose that the local directory `out` will be uploaded to a subfolder at
`https://example.com/docs/`.

To make `--root-dir` work in this context, your local folder structure
has to mimic the structure of the remote website. For example, you can set up
the following folder structure by using symbolic links:
```
├── out
│ └── page.html
└── temp-root-dir
└── docs -> ../out
```
```bash
mkdir temp-root-dir
ln -s ../out temp-root-dir/docs
```

Additionally, since the local folder is only a subset of the website, certain
relative links should be treated as links to the online website (for example,
the root link `/`). In effect, this means that paths inside `./temp-root-dir`
but outside of `./temp-root-dir/docs` must be redirected to the online website.

Putting it all together, the lychee command looks like this:
```bash
lychee ./out \
--root-dir ./temp-root-dir \
--remap "https://example\.com/docs file://$(pwd)/out" \
--remap "file://$(pwd)/temp-root-dir/docs file://$(pwd)/temp-root-dir/docs" \
--remap "file://$(pwd)/temp-root-dir https://example\.com"
```
Note that the `temp-root-dir` is only needed for `--root-dir`, and that the
order of remaps is significant—earlier remaps are tried first and have
priority over later ones.

## See Also

If your URLs make use of automatic index files or automatic file extensions, see
[Pretty URLs](/recipes/pretty-urls/) to enable the same features for local
files.

This documentation page was motivated by certain issue reports
([#1918](https://github.com/lycheeverse/lychee/issues/1918),
[#1594](https://github.com/lycheeverse/lychee/issues/1594)).
In particular, the UX/documentation issue was discussed in
[#1718](https://github.com/lycheeverse/lychee/issues/1718).