Skip to content

Convert adoc tables to html tables #138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
james-ball-qualcomm merged 1 commit into from
Nov 27, 2025
Merged

Convert adoc tables to html tables #138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions converters/tags.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,8 +144,8 @@ def node_text_content(node)
cell.content.join("\n")
end
end
# Separate cells by |.
end.join("|")
# Prefix cells with "|"
end.map { |s| '|' + s }.join
# Separate rows by newlines.
end.join("\n")
# Separate table sections by ===.
Expand Down
34 changes: 17 additions & 17 deletions tests/norm-rule/expected/test-norm-rules.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,15 +89,15 @@ It's got 2 lines.

.1+| table1
| ===
WITH anchor
WITHOUT anchor
|WITH anchor
|WITHOUT anchor
=== a| link:test.html#norm:table:anchors-in-cells:entire-table[norm:table:anchors-in-cells:entire-table]

.1+| table2
| Header 1|Header 2
| |Header 1|Header 2
===
Cell in column 1, row 1|Cell in column 2, row 1
Cell in column 1, row 2|Cell in column 2, row 2
|Cell in column 1, row 1|Cell in column 2, row 1
|Cell in column 1, row 2|Cell in column 2, row 2
=== a| link:test.html#norm:table:no-anchors-in-cells:entire-table[norm:table:no-anchors-in-cells:entire-table]

.1+| table3
Expand All @@ -107,19 +107,19 @@ Cell in column 1, row 2|Cell in column 2, row 2
| ABC DEF a| link:test.html#norm:table:anchors-in-cells:entire-table-not-tagged:cell[norm:table:anchors-in-cells:entire-table-not-tagged:cell]

.1+| table5
| Name|Color
| |Name|Color
===
Roses|Red
Violets|Blue
Name1|Color1
Name2|Color2
Name3|Color3
Name4|Color4
Name5|Color5
Name6|Color6
Name7|Color8
Name9|Color9
Name10|Color10
|Roses|Red
|Violets|Blue
|Name1|Color1
|Name2|Color2
|Name3|Color3
|Name4|Color4
|Name5|Color5
|Name6|Color6
|Name7|Color7
|Name8|Color8
|Name9|Color9
... a| link:test.html#norm:table:many-rows[norm:table:many-rows]

.1+| unordered1
Expand Down
6 changes: 3 additions & 3 deletions tests/norm-rule/expected/test-norm-rules.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,12 +255,12 @@ <h3>my-chapter_name</h3>
</tr>
<tr>
<td rowspan=1 id="table1">table1</td>
<td>===<br>WITH anchor<br>WITHOUT anchor<br>===</td>
<td><table><tbody><tr><td>WITH anchor</td></tr><tr><td>WITHOUT anchor</td></tr></tbody></table></td>
<td><a href="test.html#norm:table:anchors-in-cells:entire-table">norm:table:anchors-in-cells:entire-table</a></td>
</tr>
<tr>
<td rowspan=1 id="table2">table2</td>
<td>Header 1|Header 2<br>===<br>Cell in column 1, row 1|Cell in column 2, row 1<br>Cell in column 1, row 2|Cell in column 2, row 2<br>===</td>
<td><table><thead><tr><th>Header 1</th><th>Header 2</th></tr></thead><tbody><tr><td>Cell in column 1, row 1</td><td>Cell in column 2, row 1</td></tr><tr><td>Cell in column 1, row 2</td><td>Cell in column 2, row 2</td></tr></tbody></table></td>
<td><a href="test.html#norm:table:no-anchors-in-cells:entire-table">norm:table:no-anchors-in-cells:entire-table</a></td>
</tr>
<tr>
Expand All @@ -275,7 +275,7 @@ <h3>my-chapter_name</h3>
</tr>
<tr>
<td rowspan=1 id="table5">table5</td>
<td>Name|Color<br>===<br>Roses|Red<br>Violets|Blue<br>Name1|Color1<br>Name2|Color2<br>Name3|Color3<br>Name4|Color4<br>Name5|Color5<br>Name6|Color6<br>Name7|Color8<br>Name9|Color9<br>Name10|Color10<br>...</td>
<td><table><thead><tr><th>Name</th><th>Color</th></tr></thead><tbody><tr><td>Roses</td><td>Red</td></tr><tr><td>Violets</td><td>Blue</td></tr><tr><td>Name1</td><td>Color1</td></tr><tr><td>Name2</td><td>Color2</td></tr><tr><td>Name3</td><td>Color3</td></tr><tr><td>Name4</td><td>Color4</td></tr><tr><td>Name5</td><td>Color5</td></tr><tr><td>Name6</td><td>Color6</td></tr><tr><td>Name7</td><td>Color7</td></tr><tr><td>Name8</td><td>Color8</td></tr><tr><td>Name9</td><td>Color9</td></tr><tr><td>Name10</td><td>Color10</td></tr><tr><td>...</td><td>...</td></tr></tbody></table></td>
<td><a href="test.html#norm:table:many-rows">norm:table:many-rows</a></td>
</tr>
<tr>
Expand Down
6 changes: 3 additions & 3 deletions tests/norm-rule/expected/test-norm-rules.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,7 @@
"tags": [
{
"name": "norm:table:anchors-in-cells:entire-table",
"text": "===\nWITH anchor\nWITHOUT anchor\n===",
"text": "===\n|WITH anchor\n|WITHOUT anchor\n===",
"tag_filename": "/build/test-norm-tags.json",
"stds_doc_url": "test.html"
}
Expand All @@ -346,7 +346,7 @@
"tags": [
{
"name": "norm:table:no-anchors-in-cells:entire-table",
"text": "Header 1|Header 2\n===\nCell in column 1, row 1|Cell in column 2, row 1\nCell in column 1, row 2|Cell in column 2, row 2\n===",
"text": "|Header 1|Header 2\n===\n|Cell in column 1, row 1|Cell in column 2, row 1\n|Cell in column 1, row 2|Cell in column 2, row 2\n===",
"tag_filename": "/build/test-norm-tags.json",
"stds_doc_url": "test.html"
}
Expand Down Expand Up @@ -385,7 +385,7 @@
"tags": [
{
"name": "norm:table:many-rows",
"text": "Name|Color\n===\nRoses|Red\nViolets|Blue\nName1|Color1\nName2|Color2\nName3|Color3\nName4|Color4\nName5|Color5\nName6|Color6\nName7|Color8\nName9|Color9\nName10|Color10\nName11|Color11\nName12|Color12\nName13|Color13\nName14|Color14\n===",
"text": "|Name|Color\n===\n|Roses|Red\n|Violets|Blue\n|Name1|Color1\n|Name2|Color2\n|Name3|Color3\n|Name4|Color4\n|Name5|Color5\n|Name6|Color6\n|Name7|Color7\n|Name8|Color8\n|Name9|Color9\n|Name10|Color10\n|Name11|Color11\n|Name12|Color12\n|Name13|Color13\n|Name14|Color14\n===",
"tag_filename": "/build/test-norm-tags.json",
"stds_doc_url": "test.html"
}
Expand Down
Binary file modified tests/norm-rule/expected/test-norm-rules.xlsx
View file
Open in desktop
Binary file not shown.
6 changes: 3 additions & 3 deletions tests/norm-rule/expected/test-norm-tags.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@
"norm:hyperlink4": "DEF &lt;&lt;non-norm-anchor,custom text&gt;&gt; GHI",
"norm:hyperlink5": "GHI &lt;&lt;norm:superscript&gt;&gt; and &lt;&lt;norm:subscript&gt;&gt; JKL",
"norm:hyperlink6": "JKL &lt;&lt;norm:superscript,hello&gt;&gt; and &lt;&lt;norm:subscript,goodbye&gt;&gt; MNO",
"norm:table:no-anchors-in-cells:entire-table": "Header 1|Header 2\n===\nCell in column 1, row 1|Cell in column 2, row 1\nCell in column 1, row 2|Cell in column 2, row 2\n===",
"norm:table:no-anchors-in-cells:entire-table": "|Header 1|Header 2\n===\n|Cell in column 1, row 1|Cell in column 2, row 1\n|Cell in column 1, row 2|Cell in column 2, row 2\n===",
"norm:table:anchors-in-cells:entire-table-tagged:cell": "WITH anchor",
"norm:table:anchors-in-cells:entire-table": "===\nWITH anchor\nWITHOUT anchor\n===",
"norm:table:anchors-in-cells:entire-table": "===\n|WITH anchor\n|WITHOUT anchor\n===",
"norm:table:anchors-in-cells:entire-table-not-tagged:cell": "ABC DEF",
"norm:table:many-rows": "Name|Color\n===\nRoses|Red\nViolets|Blue\nName1|Color1\nName2|Color2\nName3|Color3\nName4|Color4\nName5|Color5\nName6|Color6\nName7|Color8\nName9|Color9\nName10|Color10\nName11|Color11\nName12|Color12\nName13|Color13\nName14|Color14\n===",
"norm:table:many-rows": "|Name|Color\n===\n|Roses|Red\n|Violets|Blue\n|Name1|Color1\n|Name2|Color2\n|Name3|Color3\n|Name4|Color4\n|Name5|Color5\n|Name6|Color6\n|Name7|Color7\n|Name8|Color8\n|Name9|Color9\n|Name10|Color10\n|Name11|Color11\n|Name12|Color12\n|Name13|Color13\n|Name14|Color14\n===",
"norm:unordered-list:no-anchors-in-items:entire-list": "Item A\nItem B\nItem C",
"norm:unordered-list:anchors-in-items:item1": "Item 1",
"norm:unordered-list:anchors-in-items:item2": "Item 2",
Expand Down
5 changes: 3 additions & 2 deletions tests/norm-rule/test.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ Here's some text that isn't normative but the normatively tagged text has an exi
[[norm:table:anchors-in-cells:entire-table]]
|===

// FAILS - Want tagged text to be "cell with anchor"
// FAILS - Want tagged text to be "WITH anchor"
| [#norm:table:anchors-in-cells:entire-table-tagged:cell]#WITH anchor#
| WITHOUT anchor
|===
Expand All @@ -145,7 +145,8 @@ a| Cell is adoc
|Name4|Color4
|Name5|Color5
|Name6|Color6
|Name7|Color8
|Name7|Color7
|Name8|Color8
|Name9|Color9
|Name10|Color10
|Name11|Color11
Expand Down
101 changes: 96 additions & 5 deletions tools/create_normative_rules.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@

NORM_PREFIX = "norm:"

MAX_TABLE_ROWS = 12 # Max rows of a table displayed in a cell.

###################################
# Classes for Normative Rule Tags #
###################################
Expand Down Expand Up @@ -1016,7 +1018,7 @@ def html_chapter_table(f, table_num, chapter_name, nr_defs, tags, tag_fname2url)

unless nr.description.nil?
f.puts(%Q{ <tr>}) unless row_started
f.puts(%Q{ <td>#{html_convert_newlines(nr.description)}</td>})
f.puts(%Q{ <td>#{convert_newlines_to_html(nr.description)}</td>})
f.puts(%Q{ <td>Rule's "description" property</td>})
f.puts(%Q{ </tr>})
row_started = false
Expand Down Expand Up @@ -1052,8 +1054,7 @@ def html_chapter_table(f, table_num, chapter_name, nr_defs, tags, tag_fname2url)
html_fname = tag_fname2url[tag.tag_filename]
fatal("No fname tag to HTML mapping (-tag2url cmd line arg) for tag fname #{tag.tag_filename} for tag name #{tag.name}") if html_fname.nil?

tag_text = html_convert_newlines(limit_table_rows(Adoc2HTML::convert(tag.text)))

tag_text = convert_newlines_to_html(convert_tags_tables_to_html(Adoc2HTML::convert(tag.text)))

# Convert adoc links to normative text in tag text to html links.
#
Expand Down Expand Up @@ -1139,14 +1140,104 @@ def html_script(f)
)
end

# Convert the tagged text containing entire tables. Uses format created by "tags" Asciidoctor backend.
#
# Two possible formats:
#
# Without heading:
#
# ===
# | ABC | DEF
# |GHI |JKL
# ===
#
# Actual string from tags: "===\n| ABC | DEF\n|GHI |JKL\n==="
#
# With heading:
#
# | H1 | H2
# ===
# | GHI | JKL
# ===
#
# Actual string from tags: "| H1 | H2\n===\n| GHI | JKL\n==="

def convert_tags_tables_to_html(text)
raise ArgumentError, "Expected String for text but was passed a #{text}.class" unless text.is_a?(String)

ret = text # Default to input

text.gsub(/(.*?)===\n(.+)\n===/m) do
# Found a "tags" formatted table
heading = $1.chomp # Remove trailing newline
rows = $2.split("\n") # Split into array of rows

ret = "<table>".dup # Start html table

# Add heading if present
heading_cells = extract_tags_table_cells(heading)
unless heading_cells.empty?
ret << "<thead>"
ret << "<tr>"
ret << heading_cells.map { |cell| "<th>#{cell}</th>" }.join("")
ret << "</tr>"
ret << "</thead>"
end

# Add each row
ret << "<tbody>"
rows.each_with_index do |row,index|
if index < MAX_TABLE_ROWS
ret << "<tr>"
row_cells = extract_tags_table_cells(row)
ret << row_cells.map { |cell| "<td>#{cell}</td>" }.join("")
ret << "</tr>"
elsif index == MAX_TABLE_ROWS
ret << "<tr>"
row_cells = extract_tags_table_cells(row)
ret << row_cells.map { |cell| "<td>...</td>" }.join("")
Copy link

Copilot AI Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On line 1197, row_cells is extracted but then not used. Instead, line 1198 creates ... for every cell, which means the number of ... placeholders will match the column count. However, this defeats the purpose of extracting row_cells on line 1197.

The code should either:

  1. Remove line 1197 entirely since row_cells is not used, OR
  2. Use row_cells.size to determine how many ... cells to create

Currently, the unused variable may cause confusion about the code's intent.

Suggested change
ret << row_cells.map { |cell| "<td>...</td>" }.join("")
ret << row_cells.size.times.map { "<td>...</td>" }.join("")

Copilot uses AI. Check for mistakes.
ret << "</tr>"
end
end

ret << "</tbody>"
ret << "</table>" # End html table
Comment on lines +1168 to +1204
Copy link

Copilot AI Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function does not return the result of gsub. The variable ret is assigned inside the gsub block (line 1175), but this assignment is scoped to the block. The function returns the original ret (initialized to text on line 1168), not the transformed HTML table.

To fix this, assign the result of gsub to ret:

ret = text.gsub(/(.*?)===\n(.+)\n===/m) do
  html = "<table>".dup
  # ... build up html instead of ret ...
  html
end
return ret
Suggested change
ret = text # Default to input
text.gsub(/(.*?)===\n(.+)\n===/m) do
# Found a "tags" formatted table
heading = $1.chomp # Remove trailing newline
rows = $2.split("\n") # Split into array of rows
ret = "<table>".dup # Start html table
# Add heading if present
heading_cells = extract_tags_table_cells(heading)
unless heading_cells.empty?
ret << "<thead>"
ret << "<tr>"
ret << heading_cells.map { |cell| "<th>#{cell}</th>" }.join("")
ret << "</tr>"
ret << "</thead>"
end
# Add each row
ret << "<tbody>"
rows.each_with_index do |row,index|
if index < MAX_TABLE_ROWS
ret << "<tr>"
row_cells = extract_tags_table_cells(row)
ret << row_cells.map { |cell| "<td>#{cell}</td>" }.join("")
ret << "</tr>"
elsif index == MAX_TABLE_ROWS
ret << "<tr>"
row_cells = extract_tags_table_cells(row)
ret << row_cells.map { |cell| "<td>...</td>" }.join("")
ret << "</tr>"
end
end
ret << "</tbody>"
ret << "</table>" # End html table
ret = text.gsub(/(.*?)===\n(.+)\n===/m) do
# Found a "tags" formatted table
heading = $1.chomp # Remove trailing newline
rows = $2.split("\n") # Split into array of rows
html = "<table>".dup # Start html table
# Add heading if present
heading_cells = extract_tags_table_cells(heading)
unless heading_cells.empty?
html << "<thead>"
html << "<tr>"
html << heading_cells.map { |cell| "<th>#{cell}</th>" }.join("")
html << "</tr>"
html << "</thead>"
end
# Add each row
html << "<tbody>"
rows.each_with_index do |row,index|
if index < MAX_TABLE_ROWS
html << "<tr>"
row_cells = extract_tags_table_cells(row)
html << row_cells.map { |cell| "<td>#{cell}</td>" }.join("")
html << "</tr>"
elsif index == MAX_TABLE_ROWS
html << "<tr>"
row_cells = extract_tags_table_cells(row)
html << row_cells.map { |cell| "<td>...</td>" }.join("")
html << "</tr>"
end
end
html << "</tbody>"
html << "</table>" # End html table
html

Copilot uses AI. Check for mistakes.
end

return ret
end

# Return array of table columns from one row/header of a table.
def extract_tags_table_cells(text)
raise ArgumentError, "Expected String for text but was passed a #{text}.class" unless text.is_a?(String)

# This pattern matches strings that:
# - Start with a non-pipe, non-whitespace character
# - Then contain zero or more non-pipe characters (can include internal spaces)
# - End with a non-pipe, non-whitespace character
#
# All leading/trailing whitespace is removed.
#
# Examples:
# "| H1 | H2".scan(/[^|\s][^|]*[^|\s]/)
# => ["H1", "H2"]
#
# "| ABC | DEF GHI |".scan(/[^|\s][^|]*[^|\s]/)
# => ["ABC", "DEF GHI"] # Note: internal space preserved
#
# "| Name | Value |".scan(/[^|\s][^|]*[^|\s]/)
# => ["Name", "Value"] # Leading/trailing spaces removed
text.scan(/[^|\s][^|]*[^|\s]/)
Comment on lines +1214 to +1230
Copy link

Copilot AI Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern /[^|\s][^|]*[^|\s]/ requires at least two non-whitespace, non-pipe characters, which means it will fail to match single-character cell content. For example, a cell containing just "A" or "1" would not be extracted.

To fix this, the pattern should be changed to /[^|\s]([^|]*[^|\s])?/ to match either:

  • A single non-pipe, non-whitespace character, OR
  • A non-pipe, non-whitespace character followed by zero or more non-pipe characters, ending with a non-pipe, non-whitespace character

Alternatively, use a simpler approach: /\|([^|]*)/ and then trim whitespace from each captured group.

Suggested change
# This pattern matches strings that:
# - Start with a non-pipe, non-whitespace character
# - Then contain zero or more non-pipe characters (can include internal spaces)
# - End with a non-pipe, non-whitespace character
#
# All leading/trailing whitespace is removed.
#
# Examples:
# "| H1 | H2".scan(/[^|\s][^|]*[^|\s]/)
# => ["H1", "H2"]
#
# "| ABC | DEF GHI |".scan(/[^|\s][^|]*[^|\s]/)
# => ["ABC", "DEF GHI"] # Note: internal space preserved
#
# "| Name | Value |".scan(/[^|\s][^|]*[^|\s]/)
# => ["Name", "Value"] # Leading/trailing spaces removed
text.scan(/[^|\s][^|]*[^|\s]/)
# This pattern matches everything between pipes, including single-character and empty cells.
# All leading/trailing whitespace is removed from each cell.
#
# Examples:
# "| H1 | H2".scan(/\|([^|]*)/)
# => [["H1 "], [" H2"]]
# .map(&:strip) => ["H1", "H2"]
#
# "| A | 1 |".scan(/\|([^|]*)/)
# => [["A "], [" 1"]]
# .map(&:strip) => ["A", "1"]
#
# "| Name | Value |".scan(/\|([^|]*)/)
# => [[" Name "], [" Value "]]
# .map(&:strip) => ["Name", "Value"]
text.scan(/\|([^|]*)/).map { |m| m[0].strip }

Copilot uses AI. Check for mistakes.
end

# Cleanup the tag text to be suitably displayed.
def limit_table_rows(text)
raise ArgumentError, "Expected String for text but was passed a #{text}.class" unless text.is_a?(String)

# This is the detection pattern for an entire table being tagged from the "tags.rb" AsciiDoctor backend.
if text.end_with?("\n===")
# Limit table size displayed.
truncate_after_newlines(text, 12)
truncate_after_newlines(text, MAX_TABLE_ROWS)
else
text
end
Expand Down Expand Up @@ -1181,7 +1272,7 @@ def count_parameters(defs)
end

# Convert newlines to <br>.
def html_convert_newlines(text)
def convert_newlines_to_html(text)
raise ArgumentError, "Expected String for text but was passed a #{text}.class" unless text.is_a?(String)

text.gsub(/\n/, '<br>')
Expand Down
Loading