Support highlight language fallbacks in blocks

[cmarqu]

Some documentation, like Python's Logging Cookbook use Sphinx's Literal Blocks (i.e. just ::) instead of the .. code-block:: directive.

It would be nice to support these docs too. The extension would need to support the highlight directive and the highlight_language configuration option (the latter being used in the Python docs).

[cmarqu]

Oh, I see that the extension already supports this.

I thought it's probably just that the example I was using (the docs for Python itself) sets highlight_language to python3 which only becomes supported with #163, but this alone is not sufficient... I'll investigate more.

[cmarqu]

I think the extension is missing fallback handling to highlight and highlight_language,
like in https://github.com/sphinx-doc/sphinx/blob/c49d925b4d861d28cc11356846f687c64069887d/sphinx/transforms/post_transforms/code.py#L30

To support highlight_language which is enough to support most of the Python docs, I crudely hacked this incomplete solution:

diff --git a/src/sphinx_codeautolink/extension/__init__.py b/src/sphinx_codeautolink/extension/__init__.py
index 1ccf33f..29d1620 100644
--- a/src/sphinx_codeautolink/extension/__init__.py
+++ b/src/sphinx_codeautolink/extension/__init__.py
@@ -138,6 +138,7 @@ def parse_blocks(self, app, doctree) -> None:
             global_preface=self.global_preface,
             custom_blocks=self.custom_blocks,
             concat_default=self.concat_default,
+            highlight_language=self.highlight_language,
         )
         doctree.walkabout(visitor)
         self.cache.transforms[visitor.current_document] = visitor.source_transforms
diff --git a/src/sphinx_codeautolink/extension/block.py b/src/sphinx_codeautolink/extension/block.py
index 3e0fe49..454c805 100644
--- a/src/sphinx_codeautolink/extension/block.py
+++ b/src/sphinx_codeautolink/extension/block.py
@@ -113,6 +113,7 @@ def __init__(
         global_preface: list[str],
         custom_blocks: dict[str, Callable[[str], str]],
         concat_default: bool,
+        highlight_language: str,
         **kwargs,
     ) -> None:
         super().__init__(*args, **kwargs)
@@ -129,6 +130,7 @@ def __init__(
         self.concat_global = concat_default
         self.concat_section = False
         self.concat_sources = []
+        self.highlight_language = highlight_language
         self.skip = None
 
     def unknown_visit(self, node) -> None:
@@ -185,7 +187,7 @@ def visit_doctest_block(self, node):
 
     def visit_literal_block(self, node: nodes.literal_block):
         """Visit a generic literal block."""
-        return self.parse_source(node, node.get("language", None))
+        return self.parse_source(node, node.get("language", self.highlight_language))
 
     def parse_source(
         self, node: nodes.literal_block | nodes.doctest_block, language: str | None

[felix-hilden]

Thanks for the suggestion and info! Thanks to that I was able to figure out the whole chain pretty easily 👌 After the linked PR is merged, we'll support:

• the default lexer (sphinx assumes them to be python, we ignored them previously)
• literal blocks
• the highlight directive
• highlight_language configuration

I'll still have to make sure we don't fail on an invalid default block (which is also Sphinx's assumption), but other than that I would appreciate a quick review on it to make sure it matches what you had in mind, just in case!

[cmarqu]

Awesome! I'll review in a bit.
Related, I am testing the waters over here: https://discuss.python.org/t/add-the-sphinx-codeautolink-extension-to-doc-build-process/80814, I hope that's okay with you.

[cmarqu]

Related, I am testing the waters over here: https://discuss.python.org/t/add-the-sphinx-codeautolink-extension-to-doc-build-process/80814, I hope that's okay with you.

(Obviously, the honors of opening such a PR would belong to you first.)

[cmarqu]

It seems that Sphinx is automatically handling pycon syntax for Literal Blocks, even if highlight_language is set to python3 and no other selection is made - see e.g. the code examples in the https://docs.python.org/3/howto/unicode.html#the-string-type section.

EDIT: it's likely this function: https://github.com/sphinx-doc/sphinx/blob/2364f169ade55916164dbeecbee656afb8fd549f/sphinx/transforms/post_transforms/code.py#L115-L132
used in
https://github.com/sphinx-doc/sphinx/blob/2364f169ade55916164dbeecbee656afb8fd549f/sphinx/transforms/post_transforms/code.py#L99-L100

sphinx-codeautolink instead is requiring explicit language selection for its parser to be happy.

[felix-hilden]

Oh, that's awesome! Thank you for proposing it 🙏

Also thanks for the info on the pycon support 👌 I'll see what makes sense for us to do!

[felix-hilden]

I created another issue to track that, it wasn't obvious to me on the first look but I'll explore more later. I want to get this in independently 👍