Skip to content

Commit 7cf63fd

Browse files
authored
feat(md, config): Implement streaming markdown rendering (#399)
Introduce the `jp_md` crate to handle progressive rendering of CommonMark content from LLM streams. This implementation solves the latency issues associated with waiting for a full response before displaying formatted output. The core of this change is the `Buffer` utility, which segments incoming text chunks into complete markdown blocks based on CommonMark grammar. This allows for immediate rendering of paragraphs, headers, and lists as they are completed. Fenced code blocks are handled with line-by-line granularity to enable real-time syntax highlighting. A custom `TerminalFormatter` replaces standard rendering to ensure ANSI-aware line wrapping. Unlike generic formatters, it correctly calculates visual column widths by excluding escape sequences and manages SGR state across line breaks to prevent background color bleed. Users can now configure their rendering preferences via the new `style.markdown` configuration section: ```toml [style.markdown] wrap_width = 80 table_max_column_width = 40 theme = "gruvbox-dark" hr_style = "line" ``` The architectural decisions and trade-offs for this implementation are documented in RFD 004. NOTE: This change has NOT been wired up to the CLI yet. This will happen in a follow-up PR. --------- Signed-off-by: Jean Mertz <git@jeanmertz.com>
1 parent f090bfa commit 7cf63fd

File tree

119 files changed

+6892
-27
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

119 files changed

+6892
-27
lines changed

.clippy.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ disallowed-methods = []
33
disallowed-types = []
44

55
too-many-arguments-threshold = 10
6-
doc-valid-idents = ["OpenAI", "OpenRouter", ".."]
6+
doc-valid-idents = ["OpenAI", "OpenRouter", "CommonMark", ".."]
77
allow-unwrap-in-tests = true
88
allow-print-in-tests = true
99
allowed-duplicate-crates = [
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,26 @@
11

22
# cargo-vet audits file
33

4+
[[audits.comrak]]
5+
who = "Jean Mertz <git@jeanmertz.com>"
6+
criteria = "safe-to-deploy"
7+
delta = "0.49.0 -> 0.50.0"
8+
9+
[[audits.rand_xorshift]]
10+
who = "Jean Mertz <git@jeanmertz.com>"
11+
criteria = "safe-to-deploy"
12+
delta = "0.3.0 -> 0.4.0"
13+
14+
[[audits.rand_xorshift]]
15+
who = "Jean Mertz <git@jeanmertz.com>"
16+
criteria = "safe-to-deploy"
17+
delta = "0.3.0 -> 0.4.0"
18+
19+
[[audits.siphasher]]
20+
who = "Jean Mertz <git@jeanmertz.com>"
21+
criteria = "safe-to-deploy"
22+
version = "1.0.2"
23+
424
[[audits.syntect]]
525
who = "rgrant <rgrant@contract.design>"
626
criteria = "safe-to-deploy"
@@ -11,6 +31,16 @@ who = "rgrant <rgrant@contract.design>"
1131
criteria = "safe-to-deploy"
1232
delta = "0.1.6 -> 0.1.8"
1333

34+
[[audits.two-face]]
35+
who = "Jean Mertz <git@jeanmertz.com>"
36+
criteria = "safe-to-deploy"
37+
version = "0.4.5"
38+
39+
[[audits.unarray]]
40+
who = "Jean Mertz <git@jeanmertz.com>"
41+
criteria = "safe-to-deploy"
42+
version = "0.1.4"
43+
1444
[[trusted.aho-corasick]]
1545
criteria = "safe-to-deploy"
1646
user-id = 189 # Andrew Gallant (BurntSushi)
@@ -365,6 +395,30 @@ user-id = 3618 # David Tolnay (dtolnay)
365395
start = "2019-03-19"
366396
end = "2027-02-13"
367397

398+
[[trusted.phf]]
399+
criteria = "safe-to-deploy"
400+
user-id = 51017 # Yuki Okushi (JohnTitor)
401+
start = "2021-06-17"
402+
end = "2027-02-20"
403+
404+
[[trusted.phf_codegen]]
405+
criteria = "safe-to-deploy"
406+
user-id = 51017 # Yuki Okushi (JohnTitor)
407+
start = "2021-06-17"
408+
end = "2027-02-20"
409+
410+
[[trusted.phf_generator]]
411+
criteria = "safe-to-deploy"
412+
user-id = 51017 # Yuki Okushi (JohnTitor)
413+
start = "2021-06-17"
414+
end = "2027-02-20"
415+
416+
[[trusted.phf_shared]]
417+
criteria = "safe-to-deploy"
418+
user-id = 51017 # Yuki Okushi (JohnTitor)
419+
start = "2021-06-17"
420+
end = "2027-02-20"
421+
368422
[[trusted.potential_utf]]
369423
criteria = "safe-to-deploy"
370424
user-id = 1139 # Manish Goregaokar (Manishearth)
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -590,6 +590,10 @@ criteria = "safe-to-deploy"
590590
version = "1.4.1"
591591
criteria = "safe-to-run"
592592

593+
[[exemptions.proptest]]
594+
version = "1.9.0"
595+
criteria = "safe-to-deploy"
596+
593597
[[exemptions.quick-xml]]
594598
version = "0.38.4"
595599
criteria = "safe-to-deploy"
File renamed without changes.
Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -389,6 +389,34 @@ user-id = 3618
389389
user-login = "dtolnay"
390390
user-name = "David Tolnay"
391391

392+
[[publisher.phf]]
393+
version = "0.13.1"
394+
when = "2025-08-23"
395+
user-id = 51017
396+
user-login = "JohnTitor"
397+
user-name = "Yuki Okushi"
398+
399+
[[publisher.phf_codegen]]
400+
version = "0.13.1"
401+
when = "2025-08-23"
402+
user-id = 51017
403+
user-login = "JohnTitor"
404+
user-name = "Yuki Okushi"
405+
406+
[[publisher.phf_generator]]
407+
version = "0.13.1"
408+
when = "2025-08-23"
409+
user-id = 51017
410+
user-login = "JohnTitor"
411+
user-name = "Yuki Okushi"
412+
413+
[[publisher.phf_shared]]
414+
version = "0.13.1"
415+
when = "2025-08-23"
416+
user-id = 51017
417+
user-login = "JohnTitor"
418+
user-name = "Yuki Okushi"
419+
392420
[[publisher.potential_utf]]
393421
version = "0.1.3"
394422
when = "2025-08-28"
@@ -670,8 +698,8 @@ user-login = "Manishearth"
670698
user-name = "Manish Goregaokar"
671699

672700
[[publisher.unicode-width]]
673-
version = "0.2.1"
674-
when = "2025-06-09"
701+
version = "0.2.2"
702+
when = "2025-10-06"
675703
user-id = 1139
676704
user-login = "Manishearth"
677705
user-name = "Manish Goregaokar"
@@ -3536,6 +3564,12 @@ criteria = "safe-to-deploy"
35363564
delta = "0.5.13 -> 0.5.14"
35373565
aggregated-from = "https://raw.githubusercontent.com/zcash/wallet/main/supply-chain/audits.toml"
35383566

3567+
[[audits.zcash.audits.rand_xorshift]]
3568+
who = "Sean Bowe <ewillbefull@gmail.com>"
3569+
criteria = "safe-to-deploy"
3570+
version = "0.3.0"
3571+
aggregated-from = "https://raw.githubusercontent.com/zcash/zcash/master/qa/supply-chain/audits.toml"
3572+
35393573
[[audits.zcash.audits.rustc-demangle]]
35403574
who = "Jack Grigg <jack@electriccoin.co>"
35413575
criteria = "safe-to-deploy"

0 commit comments

Comments
 (0)