Testable code snippets

[petemounce]

Code snippets easily go out of date.

In a different $job we acutely felt this pain in tutorials and reduced it by writing a Hugo short code that let us include the snippet from "real" source code files.

The code files blocks were annotated with comments to begin-and-identify the block, and end the block.

This meant that the code snippets that were exposed in docs:

• could be subject to automated testing coverage, with minimal changes to anyone's workflow.
• were actually representative vs contrived, because they could be part of an actual fully illustrative use-case. This second-order effect improved our tutorial-type content; the authors were implicitly dogfooding now instead of writing contrived hand-wavey examples.

Is this something that can be achieved inside this ecosystem too? Where would I start to look for it?

[facelessuser]

Is this something that can be achieved inside this ecosystem too? Where would I start to look for it?

I think something like this is achievable but is there already a canned what to do this? Probably not.

I'm not 100% certain as to what you envision your solution to look like, but there are ways to include code from snippets as code content: https://facelessuser.github.io/pymdown-extensions/extensions/snippets/. Is this extension powerful enough to use in the way you wish to? 🤷🏻 There are plans to extend the aforementioned plugin to some extent, but I have no idea as to the complexity you wish to employ and whether that exceeds the scope of the plugin. But, could something be constructed with such a plugin as it exists currently, I believe so, yes.

We also have custom-fenced code blocks. Theoretically, a user could construct a special fence that overrides the default. It could run some tests on a given code block and then pass it on to be highlighted by the underlying default highlighter. There is a lot that can be done with custom fences, one example is what you see in this example. Our code block source has code that is executed to create the examples seen in the documentation. They are then able to edit the original source and execute it live.

You could possibly parse the exported HTML, find code blocks, and do some sort of validation on the code as well.

I have no idea as to what your actual requirements are, but with proper motivation and time, I am sure some solution could be crafted.

Now, is there a canned solution that can already achieve what you'd like? Probably not, but I think it is possible.

[facelessuser]

Yes, such things could be possible. Currently, the Snippets extension does not support arbitrary sections to extract specific lines as you've demonstrated. There have been requests, but I have not yet decided if that is a direction I want to go. With that said, it could be done with a plugin. Are there other include plugins that might be able to do this already? Maybe.

As far as executing the code, I see no reason why a custom fence could not be created to accomplish such things as I believe it is perfectly doable. As I demonstrated in my earlier post, I've done an extremely complex example with Python already, I believe a go solution could be much simpler as it only needs to execute the code at documentation build time.

As I noted before though, I am unaware of any pre-packaged solution. I think it could be done, assuming someone was ambitious enough to tackle it.

[petemounce]

I don't think the markdown extension needs to have any awareness of the "execute code inside snippet"; I had expected the test automation for the code to be taken care of by simply writing entirely standard (no awareness of markdown) test code.

I haven't thought of any coupling there at all. Am I missing something?

[facelessuser]

"execute code inside snippet"

I have no idea what your envisioned plan is, so I'm speaking generally. If you have a way to accomplish part of this, that is fine. I'm not sure specifically what you are asking as the question seems pretty broad. It may be that I incorrectly assumed you wanted to not just find the code example, but also provide the results of the example by executing it. If I am wrong, just ignore that part.

---

I had expected the test automation for the code to be taken care of by simply writing entirely standard (no awareness of markdown) test code.

So, it sounds like you want just want a testable file that you can then just grab a very specific piece from via comments.

I think that concept would work in pretty much every language that supports comments.

From this, I get the impression you also want it to be some kind of general solution.

Languages are very specific, and while some share some similarities in how comments are done, not all languages do. Editors provide general solutions by implementing language server protocols where specific languages can be plugged in. I'm certain you aren't looking to plug into LSPs.

There are many ways to preprocess and inject the code you want, though I am doubtful there are pre-packaged ways to do it exactly as you want to do it, and with generalized knowledge of all commenting styles and their associated language, but maybe the generalized part isn't actually a strong requirement.

Regardless of how exactly you want to do this, I imagine you are going to need to write some code for a proper solution.

If all you simply want an extension that recognizes things like {% include_snippet file="../relative-to-markdown/go_code.go" snippet_id="demo-1" %} and finds desired content between comments of any programming language, you may have to write your own Python Markdown extension (or MkDocs plugin).

If you'd like the solution to specifically look between fenced code blocks, you could piggyback off of SuperFences, this could be done with custom fences. You could override a specific language like go or all languages if you want to try a general solution. The content between the fences would be sent to the custom fence function to be preprocessed so you could parse your special include info, find the requested text, extract content between comments, and then you could send it through the highlighter.

If you want to generalize it, you may just have to provide a mapping of comment styles and language names and implement logic for finding code between each supported comment style. I guess you could feed the entire file through something like Pygments, which when highlighted, would provide comment context, and then extract the desired HTML between the comment scopes using some HTML scraping tools. The overhead would kind of suck, so maybe caching the processed files would be advantageous.

My point is that it is possible to do it within the current Markdown infrastructure. And the end feel could be very much like what you describe. The intelligence has to be inserted somewhere though. Either a pre-processor that can modify the Markdown before it is parsed, extending something like SuperFences via custom fences to specifically preprocess fence content before highlighting it, or even some postprocessor that works on the HTML results after Markdown rendering. How that is specifically approached is up to you, but there is not currently a pre-packaged way to accomplish this.

One thing I'm certain of is this is 100% doable, but you'll have to decide exactly how you want to accomplish it.

[facelessuser]

Okay, I have a prototype for something similar to this here: facelessuser/pymdown-extensions#1825

The idea is that it a section can be specified via:

```
---8<--- "somefile.js:section"
```

And then we look for ---8<--- [start: section] and ---8<--- [end: section]. We ignore everything on the lines where these markers occur, so you can embed them in a comment on its own line. Everything between these markers would be included.

// ---8<--- [start: section]
var test = 3
console.log(test)
// ---8<--- [end: section]

This would get you something like this:

<pre><code>var test = 3
console.log(test)</code></pre>

This is completely experimental and just a prototype, but I thought I'd link it.

[squidfunk]

Love it!