-
Notifications
You must be signed in to change notification settings - Fork 18
Add page for "Checking a Local Folder with URL Remapping" #140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
842a44a
5974081
c5c7cfc
37990f3
b87caad
aaebd37
6b183b4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,112 @@ | ||||||||||||||||||||
| --- | ||||||||||||||||||||
| title: Checking a Local Folder with URL Remapping | ||||||||||||||||||||
| description: Checking a local folder of HTML files which will be uploaded to a particular URL. | ||||||||||||||||||||
| --- | ||||||||||||||||||||
|
|
||||||||||||||||||||
| lychee can be used to check a directory of local HTML files before they're | ||||||||||||||||||||
| uploaded to a website. Even though your site isn't deployed yet, lychee can | ||||||||||||||||||||
| verify that all links will work when the site is uploaded to its online | ||||||||||||||||||||
| location. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| This works by mapping the content's future URLs to local files on your | ||||||||||||||||||||
| computer, using lychee's URL remapping feature. For links to these URLs, lychee | ||||||||||||||||||||
| will check that the corresponding files exist inside the local directory, | ||||||||||||||||||||
| rather than checking the online website. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| This means you can use fully-qualified links in your HTML files and the links | ||||||||||||||||||||
mre marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||
| will be checked correctly, even if those pages aren't online yet. This is | ||||||||||||||||||||
| especially useful as part of a CI pipeline which builds a static website. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ## Do You Need URL Remapping? | ||||||||||||||||||||
|
|
||||||||||||||||||||
| In simple cases, you don't! | ||||||||||||||||||||
|
|
||||||||||||||||||||
| By default, lychee can already resolve relative links to adjacent local files. | ||||||||||||||||||||
| By adding [`--root-dir`][root-dir], lychee can also resolve root-relative links | ||||||||||||||||||||
| (beginning with `/`) to the given root directory. In simple cases, this is all | ||||||||||||||||||||
| you need. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| Continue reading if: | ||||||||||||||||||||
| - you have fully-qualified links to files which exist locally but aren't online yet, or | ||||||||||||||||||||
| - your local folder will be uploaded to a _subdirectory_ of the website domain. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| [root-dir]: /recipes/root-dir/ | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ## Mapping Remote Domain to a Local Folder | ||||||||||||||||||||
|
|
||||||||||||||||||||
| Suppose you have a local directory `out` and this will be uploaded to | ||||||||||||||||||||
| the domain root at `https://docs.example.com`. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| You can map URLs beginning with this domain into the local directory: | ||||||||||||||||||||
| ```bash | ||||||||||||||||||||
| lychee ./out --root-dir ./out --remap "https://docs\.example\.com file://$(pwd)/out" | ||||||||||||||||||||
| ``` | ||||||||||||||||||||
| This will remap URLs so `https://docs.example.com/page.html` becomes | ||||||||||||||||||||
| `./out/page.html`, for example. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| :::caution[Caveats when using remaps] | ||||||||||||||||||||
| - For each URL, remaps are tried in order and the *first* matching | ||||||||||||||||||||
| remap will be applied. If you are using remaps for multiple purposes, be aware | ||||||||||||||||||||
| of potential conflicts between them. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| - Remaps are applied textually. For instance, `--remap 'https://example\.com/a | ||||||||||||||||||||
| file://tmp` would also apply to a URL of `https://example.com/ab`. This is | ||||||||||||||||||||
| undesirable but safely excluding this is very hard. Using regex's `$` would | ||||||||||||||||||||
| conflict with `#fragments` or `?query` parameters, and appending a `/` would | ||||||||||||||||||||
| cause the URL without a slash to be missed. | ||||||||||||||||||||
|
||||||||||||||||||||
| - Remaps are applied textually. For instance, `--remap 'https://example\.com/a | |
| file://tmp` would also apply to a URL of `https://example.com/ab`. This is | |
| undesirable but safely excluding this is very hard. Using regex's `$` would | |
| conflict with `#fragments` or `?query` parameters, and appending a `/` would | |
| cause the URL without a slash to be missed. | |
| - Remaps are applied textually. For instance, `--remap 'https://example\.com/foo' | |
| 'file://tmp'` would also apply to a URL of `https://example.com/foobar`. | |
| To avoid this, a regex pattern can be used to match the end of a path element, | |
| e.g `'^https://example\.com/foo($|[/?#])'`. |
EDIT: Ah, that regex is interpreted is mentioned next. Maybe it should be reordered somehow. Though I have an idea how to avoid it ... see below
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As of https://github.com/lycheeverse/lycheeverse.github.io/blob/aaebd37af4652e170bd98da8bb847fb1222cc581/src/content/docs/recipes/local-folder.mdx, this dot point looks like this. Thanks to @mre for the workaround suggestion.
-
Remaps are applied textually. As an example, the remap
--remap "^https://example\.com/docs file://$(pwd)/out"applies to any URL beginning with that string even if it's inside a different subfolder.
For instance, it would also apply to a URL ofhttps://example.com/docs-2/page.If you need to guard against this, you can change the regex to end with
([?#/]|$)and add$1to the replacement, like so:--remap "^https://example\.com/docs([?#/]|$) file://$(pwd)/out$1"
edit: need escape $1 in bash
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good. And yes, $1 would need to become \$1 here, not only in bash, but in any UNIX shell within double-quotes or no quotation. Single-quotes could be used to avoid this, but then $(pwd) wouldn't expand either.
But does $1 really add the (...) match to the replacement string here? Typically, regex match + replacement features use \1 (resp. \2 etc) for this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's mixed. sed uses \1, but rust and javascript use $1. The PR adding tests includes this at https://github.com/rina-forks/lychee/blob/2c4fb5c4c60aec74b5c5f59ccd7c1b00752eef2d/lychee-bin/tests/cli.rs#L3268
Uh oh!
There was an error while loading. Please reload this page.