Skip to content

Update codeblock parsing rule #865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chiragsalian merged 3 commits into from
Jul 8, 2025
Merged

Update codeblock parsing rule #865

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5b13dd9
Update codeblock parsing rule
Skalakid Jul 4, 2025
edebc32
Update tests
Skalakid Jul 4, 2025
06c328c
Merge branch 'main' into @Skalakid/update-codeblock-parsing-rules
Skalakid Jul 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 36 additions & 33 deletions __tests__/ExpensiMark-HTML-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -468,10 +468,10 @@ test('Test code fencing with spaces and new lines', () => {
expect(parser.replace(codeFenceExample)).toBe('<pre>#&#32;test<br /></pre>');

codeFenceExample = '``` \r\n# test\r\n```';
expect(parser.replace(codeFenceExample)).toBe('<pre>#&#32;test<br /></pre>');
expect(parser.replace(codeFenceExample)).toBe('&#x60;&#x60;&#x60; <br /><h1>test</h1>&#x60;&#x60;&#x60;');

codeFenceExample = '```test\r\n# test\r\n```';
expect(parser.replace(codeFenceExample)).toBe('<pre>#&#32;test<br /></pre>');
expect(parser.replace(codeFenceExample)).toBe('&#x60;&#x60;&#x60;test<br /><h1>test</h1>&#x60;&#x60;&#x60;');

codeFenceExample = '```\r\n\r\n# test\r\n\r\n```';
expect(parser.replace(codeFenceExample)).toBe('<pre><br />#&#32;test<br /><br /></pre>');
Expand Down Expand Up @@ -535,24 +535,24 @@ test('Test code fencing with ExpensiMark syntax inside', () => {

test('Test code fencing with ExpensiMark syntax outside', () => {
let codeFenceExample = '# Test1 ```\ncode\n``` Test2';
expect(parser.replace(codeFenceExample)).toBe('<h1>Test1 </h1><pre>code<br /></pre> Test2');
expect(parser.replace(codeFenceExample)).toBe('<h1>Test1 &#x60;&#x60;&#x60;</h1>code<br />&#x60;&#x60;&#x60; Test2');

codeFenceExample = '*Test1 ```\ncode\n``` Test2*';
expect(parser.replace(codeFenceExample)).toBe('*Test1 <pre>code<br /></pre> Test2*');
expect(parser.replace(codeFenceExample, {shouldKeepRawInput: true})).toBe('*Test1 <pre>\ncode\n</pre> Test2*');
codeFenceExample = '*Test1\n```\ncode\n```\nTest2*';
expect(parser.replace(codeFenceExample)).toBe('*Test1<br /><pre>code<br /></pre>Test2*');
expect(parser.replace(codeFenceExample, {shouldKeepRawInput: true})).toBe('*Test1\n<pre>\ncode\n</pre>\nTest2*');

codeFenceExample = '_Test1 ```\ncode\n``` Test2_';
expect(parser.replace(codeFenceExample)).toBe('_Test1 <pre>code<br /></pre> Test2_');
expect(parser.replace(codeFenceExample, {shouldKeepRawInput: true})).toBe('_Test1 <pre>\ncode\n</pre> Test2_');
codeFenceExample = '_Test1\n```\ncode\n```\nTest2_';
expect(parser.replace(codeFenceExample)).toBe('_Test1<br /><pre>code<br /></pre>Test2_');
expect(parser.replace(codeFenceExample, {shouldKeepRawInput: true})).toBe('_Test1\n<pre>\ncode\n</pre>\nTest2_');

codeFenceExample = '~Test1 ```\ncode\n``` Test2~';
expect(parser.replace(codeFenceExample)).toBe('~Test1 <pre>code<br /></pre> Test2~');
expect(parser.replace(codeFenceExample, {shouldKeepRawInput: true})).toBe('~Test1 <pre>\ncode\n</pre> Test2~');
codeFenceExample = '~Test1\n```\ncode\n```\nTest2~';
Copy link

@jmusial jmusial Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NAB, or part of this issue, but IMO the tests would benefit from making

Test code fencing with ExpensiMark syntax outside

a describe and having test cases as test with separate descriptions

Copy link
Contributor Author

@Skalakid Skalakid Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think it applies to all other markdown tests. They are heavily mixed, so reorganizing them into describe sections would be beneficial

expect(parser.replace(codeFenceExample)).toBe('~Test1<br /><pre>code<br /></pre>Test2~');
expect(parser.replace(codeFenceExample, {shouldKeepRawInput: true})).toBe('~Test1\n<pre>\ncode\n</pre>\nTest2~');

codeFenceExample = '[```\ncode\n```](google.com)';
expect(parser.replace(codeFenceExample)).toBe('[<pre>code<br /></pre>](<a href="https://google.com" target="_blank" rel="noreferrer noopener">google.com</a>)');
codeFenceExample = '[\n```\ncode\n```\n](google.com)';
expect(parser.replace(codeFenceExample)).toBe('[<br /><pre>code<br /></pre>](<a href="https://google.com" target="_blank" rel="noreferrer noopener">google.com</a>)');
expect(parser.replace(codeFenceExample, {shouldKeepRawInput: true})).toBe(
'[<pre>\ncode\n</pre>](<a href="https://google.com" data-raw-href="google.com" data-link-variant="auto" target="_blank" rel="noreferrer noopener">google.com</a>)',
'[\n<pre>\ncode\n</pre>\n](<a href="https://google.com" data-raw-href="google.com" data-link-variant="auto" target="_blank" rel="noreferrer noopener">google.com</a>)',
);
});

Expand Down Expand Up @@ -1753,11 +1753,11 @@ test('Test here mention with @here@here', () => {
});

test('Test link with code fence inside the alias text part', () => {
const testString = '[```\ncode\n```](google.com) ' + '[test ```\ncode\n``` test](google.com)';
const testString = '[\n```\ncode\n```](google.com) ' + '[test\n```\ncode\n``` test](google.com)';

const resultString =
'[<pre>code<br /></pre>](<a href="https://google.com" target="_blank" rel="noreferrer noopener">google.com</a>) ' +
'[test <pre>code<br /></pre> test](<a href="https://google.com" target="_blank" rel="noreferrer noopener">google.com</a>)';
'[<br /><pre>code<br /></pre>](<a href="https://google.com" target="_blank" rel="noreferrer noopener">google.com</a>) ' +
'[test<br /><pre>code<br /></pre> test](<a href="https://google.com" target="_blank" rel="noreferrer noopener">google.com</a>)';
expect(parser.replace(testString)).toBe(resultString);
});

Expand Down Expand Up @@ -1820,11 +1820,11 @@ test('Linebreak between end of text and start of code block should be remained',
},
{
testString: 'text1```\ncode\n```text2',
resultString: 'text1<pre>code<br /></pre>text2',
resultString: 'text1&#x60;&#x60;&#x60;<br />code<br />&#x60;&#x60;&#x60;text2',
},
{
testString: 'text1 ```\n code \n``` text2',
resultString: 'text1 <pre>&#32;code&#32;<br /></pre> text2',
resultString: 'text1 &#x60;&#x60;&#x60;<br /> code <br />&#x60;&#x60;&#x60; text2',
},
{
testString: 'text1\n```\ncode\n```\ntext2',
Expand Down Expand Up @@ -1858,26 +1858,26 @@ test('Linebreak between end of text and start of code block should be remained',

test('Test autoEmail with markdown of <pre>, <code>, <a>, <mention-user> and <em> tag', () => {
const testString =
'`code`[email protected] ' +
'```\ncode block\n```[email protected] ' +
'`code`[email protected]\n' +
'```\ncode block\n``` [email protected] ' +
'[Google](https://google.com)[email protected] ' +
'[email protected]_ ' +
'_test\n\[email protected]_ ' +
'`[email protected]` ' +
'```\[email protected]\n``` ' +
'`[email protected]`\n' +
'```\[email protected]\n```\n' +
'@[email protected] ' +
'_@[email protected]_ ' +
'[https://staging.new.expensify.com/details/[email protected]](https://staging.new.expensify.com/details/[email protected]) ' +
'[test italic style wrap email [email protected]_ inside a link](https://google.com) ';

const resultString =
'<code>code</code><a href="mailto:[email protected]">[email protected]</a> ' +
'<pre>code&#32;block<br /></pre><a href="mailto:[email protected]">[email protected]</a> ' +
'<code>code</code><a href=\"mailto:[email protected]\">[email protected]</a><br />' +
'<pre>code&#32;block<br /></pre> <a href="mailto:[email protected]">[email protected]</a> ' +
'<a href="https://google.com" target="_blank" rel="noreferrer noopener">Google</a><a href="mailto:[email protected]">[email protected]</a> ' +
'<em><a href="mailto:[email protected]">[email protected]</a></em> ' +
'<em>test<br /><br /><a href="mailto:[email protected]">[email protected]</a></em> ' +
'<code>[email protected]</code> ' +
'<pre>[email protected]<br /></pre> ' +
'<code>[email protected]</code><br />' +
'<pre>[email protected]<br /></pre>' +
'<mention-user>@[email protected]</mention-user> ' +
'<em><mention-user>@[email protected]</mention-user></em> ' +
'<a href="https://staging.new.expensify.com/details/[email protected]" target="_blank" rel="noreferrer noopener">https://staging.new.expensify.com/details/[email protected]</a> ' +
Expand Down Expand Up @@ -2074,19 +2074,22 @@ describe('when should keep raw input flag is enabled', () => {

test('Test code fence within inline code', () => {
let testString = 'Hello world `(```\ntest\n```)` Hello world';
expect(parser.replace(testString)).toBe('Hello world &#x60;(<pre>test<br /></pre>)&#x60; Hello world');
expect(parser.replace(testString)).toBe('Hello world <code>(</code>&#x60;&#x60;<br />test<br />&#x60;&#x60;<code>)</code> Hello world');

testString = 'Hello world `(```\ntest\ntest\n```)` Hello world';
expect(parser.replace(testString)).toBe('Hello world &#x60;(<pre>test<br />test<br /></pre>)&#x60; Hello world');
expect(parser.replace(testString)).toBe('Hello world <code>(</code>&#x60;&#x60;<br />test<br />test<br />&#x60;&#x60;<code>)</code> Hello world');

testString = 'Hello world ```\n(`test`)\n``` Hello world';
expect(parser.replace(testString)).toBe('Hello world <pre>(&#x60;test&#x60;)<br /></pre> Hello world');
expect(parser.replace(testString)).toBe('Hello world &#x60;&#x60;&#x60;<br />(<code>test</code>)<br />&#x60;&#x60;&#x60; Hello world');

testString = 'Hello world `test`space```\nblock\n``` Hello world';
expect(parser.replace(testString)).toBe('Hello world <code>test</code>space<pre>block<br /></pre> Hello world');
expect(parser.replace(testString)).toBe('Hello world <code>test</code>space&#x60;&#x60;&#x60;<br />block<br />&#x60;&#x60;&#x60; Hello world');

testString = 'Hello world ```\nblock\n```space`test` Hello world';
expect(parser.replace(testString)).toBe('Hello world <pre>block<br /></pre>space<code>test</code> Hello world');
expect(parser.replace(testString)).toBe('Hello world &#x60;&#x60;&#x60;<br />block<br />&#x60;&#x60;<code>space</code>test&#x60; Hello world');

testString = 'Hello world `\n```\nblock\n```\n` space`test` Hello world';
expect(parser.replace(testString)).toBe('Hello world &#x60;<br /><pre>block<br /></pre><code> space</code>test&#x60; Hello world');
});

test('Test italic/bold/strikethrough markdown to keep consistency', () => {
Expand Down
2 changes: 1 addition & 1 deletion lib/ExpensiMark.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ export default class ExpensiMark {
name: 'codeFence',

// &#x60; is a backtick symbol we are matching on three of them before then after a new line character
regex: /(&#x60;&#x60;&#x60;.*?(\r\n|\n))((?:\s*?(?!(?:\r\n|\n)?&#x60;&#x60;&#x60;(?!&#x60;))[\S])+\s*?(?:\r\n|\n))(&#x60;&#x60;&#x60;)/g,
regex: /(?<![^^\r\n])(&#x60;&#x60;&#x60;(\r\n|\n))((?:\s*?(?!(?:\r\n|\n)?&#x60;&#x60;&#x60;(?!&#x60;))[\S])+\s*?(?:\r\n|\n))(&#x60;&#x60;&#x60;)/g,

// We're using a function here to perform an additional replace on the content
// inside the backticks because Android is not able to use <pre> tags and does
Expand Down