Skip to content

Add example of content processing flow in developer documentation #2777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Harjun751 wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Add example of content processing flow in developer documentation #2777

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
567e7d2
Add small example on Content Processing in DG
Harjun751 Jan 15, 2026
4e87523
Add content processing flow example to DG
Harjun751 Jan 15, 2026
50dc3cc
Push buttons to bottom, rename header
Harjun751 Jan 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 56 additions & 1 deletion docs/devGuide/design/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,5 +60,60 @@ Its syntax is also the most compatible and independent of the other stages.

3. Having processed possibly conflicting Nunjucks and Markdown syntax, HTML is then processed last.

### Demonstrating the content processing flow
To demonstrate the content processing flow, let's take a look at a small toy MarkBind file:
```markdown
{% raw %}{% set myVariable = "Item" %}

# A basic level 1 header
There will be 5 items here:
<ul>
{% for item in [1, 2, 3, 4] %}
<li>{{ myVariable }} #{{ item }}</li>
{% endfor %}
</ul>

A link that gets [converted](contents/topic1.md)

<include src="contents/topic1.md" />{% endraw %}
```

At the first step of the processing flow, the `VariableProcessor` converts Nunjucks template code into HTML:
```markdown
{% raw %}# A basic level 1 header
There will be 5 items here:
<ul>
<li>Item #1</li>
<li>Item #2</li>
<li>Item #3</li>
<li>Item #4</li>
</ul>

A link that gets [converted](contents/topic1.md)

<include src="contents/topic1.md" />{% endraw %}
```
Notice that `myVariable` is consumed and that the unordered list is expanded.

Next, the NodeProcessor converts Markdown to HTML:
Copy link

@yihao03 yihao03 Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps can include more detail on how NodeProcessor works, e.g. The use of markdown-it and the custom syntaxes defined
can refer to packages/core/src/html/NodeProcessor.ts or this section on deepwiki

Copy link
Author

@Harjun751 Harjun751 Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I think I don't want to go too low-level, the section right after roughly shows how it's done (the switch cases match the custom syntax). Maybe I should just mention it uses markdown-it? Wdyt

Copy link

@yihao03 yihao03 Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya ig mentioning we are using markdown-it with some custom logic would be great. Personally I was most interested in finding out how the '.md' source files are transformed into html pages.

Copy link

@MoshiMoshiMochi MoshiMoshiMochi Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the small incremental steps for this example @Harjun751 has provided already follows the architecture diagram which does already state that it uses markdown-it, but yea ig it wouldnt hurt to mention it again at the specific step to reiterate this point. Perhaps we can create another issue in the future which goes into greater detail about the lower level implementationr egarding what's going on at each stage by referencing this example provided 😃.

```markdown
{% raw %}<h1 id="a-basic-level-1-header">A basic level 1 header<a class="fa fa-anchor" href="#a-basic-level-1-header" onclick="event.stopPropagation()"></a></h1>
<p>There will be 5 items here:</p>
<ul>
<li>Item #1</li>
<li>Item #2</li>
<li>Item #3</li>
<li>Item #4</li></ul>
<p>A link that gets <a href="/contents/topic1.html">converted</a></p>
<div>
<br>
<h1 id="topic-1">Topic 1<a class="fa fa-anchor" href="#topic-1" onclick="event.stopPropagation()"></a></h1>
<blockquote>
<p>This is a placeholder page - more content to be added.</p></blockquote></div>{% endraw %}
```
It does this by traversing the node graph and matching node titles to their HTML equivalents. `include` nodes are recursively traversed and converted.

After this, the content is embedded into the layout and the .html file is written to.

{% from "njk/common.njk" import previous_next %}
{{ previous_next('projectStructure', 'serverSideRendering') }}
{{ previous_next('projectStructure', 'serverSideRendering') }}