WebHost, Multiple Worlds: fix images not showing in guides (#5576)

* Multiple: resize FR RA network commands screenshot

This is now more in line with the text (and the english version).

* Multiple: optimize EN RA network commands screenshot

The URL has changed, so it's a good time to optimize.

* WebHost, Worlds: fix retroarch images not showing

Implements a src/url replacement for relative paths.
Moves the RA screenshots to worlds/generic since they are shared.
Also now uses the FR version in ffmq.
Also fixes the formatting that resultet in the list breaking.
Also moves imports in render_markdown.

Guides now also properly render on Github.

* Factorio: optimize screenshots

The URL has changed, so it's a good time to optimize.

* Factorio: change guide screenshots to use relative URL

* Test: markdown: fix tests on Windows

We also can't use delete=True, delete_on_close=False
because that's not supported in Py3.11.

* Test: markdown: fix typo

I hope that's it now. *sigh*

* Landstalker: fix doc images not showing

Change to relative img urls.

* Landstalker: optimize doc PNGs

The URL has changed, so it's a good time to optimize.

Author: black-sliver

WebHostLib/markdown.py

@@ -0,0 +1,90 @@
+import re
+from collections import Counter
+
+import mistune
+from werkzeug.utils import secure_filename
+
+
+__all__ = [
+    "ImgUrlRewriteInlineParser",
+    'render_markdown',
+]
+
+
+class ImgUrlRewriteInlineParser(mistune.InlineParser):
+    relative_url_base: str
+
+    def __init__(self, relative_url_base: str, hard_wrap: bool = False) -> None:
+        super().__init__(hard_wrap)
+        self.relative_url_base = relative_url_base
+
+    @staticmethod
+    def _find_game_name_by_folder_name(name: str) -> str | None:
+        from worlds.AutoWorld import AutoWorldRegister
+
+        for world_name, world_type in AutoWorldRegister.world_types.items():
+            if world_type.__module__ == f"worlds.{name}":
+                return world_name
+        return None
+
+    def parse_link(self, m: re.Match[str], state: mistune.InlineState) -> int | None:
+        res = super().parse_link(m, state)
+        if res is not None and state.tokens and state.tokens[-1]["type"] == "image":
+            image_token = state.tokens[-1]
+            url: str = image_token["attrs"]["url"]
+            if not url.startswith("/") and not "://" in url:
+                # replace relative URL to another world's doc folder with the webhost folder layout
+                if url.startswith("../../") and "/docs/" in self.relative_url_base:
+                    parts = url.split("/", 4)
+                    if parts[2] != ".." and parts[3] == "docs":
+                        game_name = self._find_game_name_by_folder_name(parts[2])
+                        if game_name is not None:
+                            url = "/".join(parts[1:2] + [secure_filename(game_name)] + parts[4:])
+                # change relative URL to point to deployment folder
+                url = f"{self.relative_url_base}/{url}"
+                image_token['attrs']['url'] = url
+        return res
+
+
+def render_markdown(path: str, img_url_base: str | None = None) -> str:
+    markdown = mistune.create_markdown(
+        escape=False,
+        plugins=[
+            "strikethrough",
+            "footnotes",
+            "table",
+            "speedup",
+        ],
+    )
+
+    heading_id_count: Counter[str] = Counter()
+
+    def heading_id(text: str) -> str:
+        nonlocal heading_id_count
+
+        # there is no good way to do this without regex
+        s = re.sub(r"[^\w\- ]", "", text.lower()).replace(" ", "-").strip("-")
+        n = heading_id_count[s]
+        heading_id_count[s] += 1
+        if n > 0:
+            s += f"-{n}"
+        return s
+
+    def id_hook(_: mistune.Markdown, state: mistune.BlockState) -> None:
+        for tok in state.tokens:
+            if tok["type"] == "heading" and tok["attrs"]["level"] < 4:
+                text = tok["text"]
+                assert isinstance(text, str)
+                unique_id = heading_id(text)
+                tok["attrs"]["id"] = unique_id
+                tok["text"] = f"<a href=\"#{unique_id}\">{text}</a>"  # make header link to itself
+
+    markdown.before_render_hooks.append(id_hook)
+    if img_url_base:
+        markdown.inline = ImgUrlRewriteInlineParser(img_url_base)
+
+    with open(path, encoding="utf-8-sig") as f:
+        document = f.read()
+    html = markdown(document)
+    assert isinstance(html, str), "Unexpected mistune renderer in render_markdown"
+    return html

WebHostLib/misc.py

@@ -9,6 +9,7 @@
 
 from worlds.AutoWorld import AutoWorldRegister, World
 from . import app, cache
+from .markdown import render_markdown
 from .models import Seed, Room, Command, UUID, uuid4
 from Utils import title_sorted
 
@@ -27,49 +28,6 @@ def get_visible_worlds() -> dict[str, type(World)]:
     return worlds
 
 
-def render_markdown(path: str) -> str:
-    import mistune
-    from collections import Counter
-
-    markdown = mistune.create_markdown(
-        escape=False,
-        plugins=[
-            "strikethrough",
-            "footnotes",
-            "table",
-            "speedup",
-        ],
-    )
-
-    heading_id_count: Counter[str] = Counter()
-
-    def heading_id(text: str) -> str:
-        nonlocal heading_id_count
-        import re  # there is no good way to do this without regex
-
-        s = re.sub(r"[^\w\- ]", "", text.lower()).replace(" ", "-").strip("-")
-        n = heading_id_count[s]
-        heading_id_count[s] += 1
-        if n > 0:
-            s += f"-{n}"
-        return s
-
-    def id_hook(_: mistune.Markdown, state: mistune.BlockState) -> None:
-        for tok in state.tokens:
-            if tok["type"] == "heading" and tok["attrs"]["level"] < 4:
-                text = tok["text"]
-                assert isinstance(text, str)
-                unique_id = heading_id(text)
-                tok["attrs"]["id"] = unique_id
-                tok["text"] = f"<a href=\"#{unique_id}\">{text}</a>"  # make header link to itself
-
-    markdown.before_render_hooks.append(id_hook)
-
-    with open(path, encoding="utf-8-sig") as f:
-        document = f.read()
-    return markdown(document)
-
-
 @app.errorhandler(404)
 @app.errorhandler(jinja2.exceptions.TemplateNotFound)
 def page_not_found(err):
@@ -91,10 +49,9 @@ def game_info(game, lang):
         theme = get_world_theme(game)
         secure_game_name = secure_filename(game)
         lang = secure_filename(lang)
-        document = render_markdown(os.path.join(
-            app.static_folder, "generated", "docs",
-            secure_game_name, f"{lang}_{secure_game_name}.md"
-        ))
+        file_dir = os.path.join(app.static_folder, "generated", "docs", secure_game_name)
+        file_dir_url = url_for("static", filename=f"generated/docs/{secure_game_name}")
+        document = render_markdown(os.path.join(file_dir, f"{lang}_{secure_game_name}.md"), file_dir_url)
         return render_template(
             "markdown_document.html",
             title=f"{game} Guide",
@@ -119,10 +76,9 @@ def tutorial(game: str, file: str):
         theme = get_world_theme(game)
         secure_game_name = secure_filename(game)
         file = secure_filename(file)
-        document = render_markdown(os.path.join(
-            app.static_folder, "generated", "docs",
-            secure_game_name, file+".md"
-        ))
+        file_dir = os.path.join(app.static_folder, "generated", "docs", secure_game_name)
+        file_dir_url = url_for("static", filename=f"generated/docs/{secure_game_name}")
+        document = render_markdown(os.path.join(file_dir, f"{file}.md"), file_dir_url)
         return render_template(
             "markdown_document.html",
             title=f"{game} Guide",

test/webhost/test_markdown.py

@@ -0,0 +1,78 @@
+import os
+import unittest
+from tempfile import NamedTemporaryFile
+
+from mistune import HTMLRenderer, Markdown
+
+from WebHostLib.markdown import ImgUrlRewriteInlineParser, render_markdown
+
+
+class ImgUrlRewriteTest(unittest.TestCase):
+    markdown: Markdown
+    base_url = "/static/generated/docs/some_game"
+
+    def setUp(self) -> None:
+        self.markdown = Markdown(
+            renderer=HTMLRenderer(escape=False),
+            inline=ImgUrlRewriteInlineParser(self.base_url),
+        )
+
+    def test_relative_img_rewrite(self) -> None:
+        html = self.markdown("![Image](image.png)")
+        self.assertIn(f'src="{self.base_url}/image.png"', html)
+
+    def test_absolute_img_no_rewrite(self) -> None:
+        html = self.markdown("![Image](/image.png)")
+        self.assertIn(f'src="/image.png"', html)
+        self.assertNotIn(self.base_url, html)
+
+    def test_remote_img_no_rewrite(self) -> None:
+        html = self.markdown("![Image](https://example.com/image.png)")
+        self.assertIn(f'src="https://example.com/image.png"', html)
+        self.assertNotIn(self.base_url, html)
+
+    def test_relative_link_no_rewrite(self) -> None:
+        # The parser is only supposed to update images, not links.
+        html = self.markdown("[Link](image.png)")
+        self.assertIn(f'href="image.png"', html)
+        self.assertNotIn(self.base_url, html)
+
+    def test_absolute_link_no_rewrite(self) -> None:
+        html = self.markdown("[Link](/image.png)")
+        self.assertIn(f'href="/image.png"', html)
+        self.assertNotIn(self.base_url, html)
+
+    def test_auto_link_no_rewrite(self) -> None:
+        html = self.markdown("<https://example.com/image.png>")
+        self.assertIn(f'href="https://example.com/image.png"', html)
+        self.assertNotIn(self.base_url, html)
+
+    def test_relative_img_to_other_game(self) -> None:
+        html = self.markdown("![Image](../../generic/docs/image.png)")
+        self.assertIn(f'src="{self.base_url}/../Archipelago/image.png"', html)
+
+
+class RenderMarkdownTest(unittest.TestCase):
+    """Tests that render_markdown does the right thing."""
+    base_url = "/static/generated/docs/some_game"
+
+    def test_relative_img_rewrite(self) -> None:
+        f = NamedTemporaryFile(delete=False)
+        try:
+            f.write("![Image](image.png)".encode("utf-8"))
+            f.close()
+            html = render_markdown(f.name, self.base_url)
+            self.assertIn(f'src="{self.base_url}/image.png"', html)
+        finally:
+            os.unlink(f.name)
+
+    def test_no_img_rewrite(self) -> None:
+        f = NamedTemporaryFile(delete=False)
+        try:
+            f.write("![Image](image.png)".encode("utf-8"))
+            f.close()
+            html = render_markdown(f.name)
+            self.assertIn(f'src="image.png"', html)
+            self.assertNotIn(self.base_url, html)
+        finally:
+            os.unlink(f.name)

worlds/alttp/docs/multiworld_en.md

@@ -88,9 +88,8 @@ You only have to do these steps once.
 1. Enter the RetroArch main menu screen.
 2. Go to Settings --> User Interface. Set "Show Advanced Settings" to ON.
 3. Go to Settings --> Network. Set "Network Commands" to ON. (It is found below Request Device 16.) Leave the default
-   Network Command Port at 55355.
-
-![Screenshot of Network Commands setting](/static/generated/docs/A%20Link%20to%20the%20Past/retroarch-network-commands-en.png)
+   Network Command Port at 55355. \
+   ![Screenshot of Network Commands setting](../../generic/docs/retroarch-network-commands-en.png)
 4. Go to Main Menu --> Online Updater --> Core Downloader. Scroll down and select "Nintendo - SNES / SFC (bsnes-mercury
    Performance)".
 

worlds/alttp/docs/multiworld_es.md

@@ -88,9 +88,8 @@ Sólo hay que seguir estos pasos una vez.
 1. Comienza en la pantalla del menú principal de RetroArch.
 2. Ve a Ajustes --> Interfaz de usario. Configura "Mostrar ajustes avanzados" en ON.
 3. Ve a Ajustes --> Red. Pon "Comandos de red" en ON. (Se encuentra bajo Request Device 16.) Deja en 55355 el valor por defecto,
- el Puerto de comandos de red.
-
-![Captura de pantalla del ajuste Comandos de red](/static/generated/docs/A%20Link%20to%20the%20Past/retroarch-network-commands-en.png)
+   el Puerto de comandos de red. \
+   ![Captura de pantalla del ajuste Comandos de red](../../generic/docs/retroarch-network-commands-en.png)
 4. Ve a Menú principal --> Actualizador en línea --> Descargador de núcleos. Desplázate y selecciona "Nintendo - SNES /
    SFC (bsnes-mercury Performance)".
 

worlds/alttp/docs/multiworld_fr.md

@@ -89,9 +89,8 @@ Vous n'avez qu'à faire ces étapes qu'une fois.
 1. Entrez dans le menu principal RetroArch
 2. Allez dans Réglages --> Interface utilisateur. Mettez "Afficher les réglages avancés" sur ON.
 3. Allez dans Réglages --> Réseau. Mettez "Commandes Réseau" sur ON. (trouvé sous Request Device 16.) Laissez le 
-Port des commandes réseau à 555355.
-
-![Screenshot of Network Commands setting](/static/generated/docs/A%20Link%20to%20the%20Past/retroarch-network-commands-fr.png)
+   Port des commandes réseau à 555355. \
+   ![Screenshot of Network Commands setting](../../generic/docs/retroarch-network-commands-fr.png)
 4. Allez dans Menu Principal --> Mise à jour en ligne --> Téléchargement de cœurs. Descendez jusqu'a"Nintendo - SNES / SFC (bsnes-mercury Performance)" et 
    sélectionnez le.
 

worlds/alttp/docs/retroarch-network-commands-en.png

@@ -111,9 +111,8 @@ You only have to do these steps once. Note, RetroArch 1.9.x will not work as it
 1. Enter the RetroArch main menu screen.
 2. Go to Settings --> User Interface. Set "Show Advanced Settings" to ON.
 3. Go to Settings --> Network. Set "Network Commands" to ON. (It is found below Request Device 16.) Leave the default
-   Network Command Port at 55355.
-
-![Screenshot of Network Commands setting](/static/generated/docs/A%20Link%20to%20the%20Past/retroarch-network-commands-en.png)
+   Network Command Port at 55355. \
+  ![Screenshot of Network Commands setting](../../generic/docs/retroarch-network-commands-en.png)
 4. Go to Main Menu --> Online Updater --> Core Downloader. Scroll down and select "Nintendo - SNES / SFC (bsnes-mercury
    Performance)".