Skip to content

Conversation

yeonjuan
Copy link
Owner

@yeonjuan yeonjuan requested a review from Copilot July 20, 2025 14:21
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds customTags options to the HTML tokenizer, allowing users to specify custom HTML tags that should have their content treated as raw text (similar to script/style tags). This feature addresses issue #381 by enabling custom tag handling with configurable behavior.

Key changes:

  • Introduces CustomTagOptions and CustomTagOptionsConfig types for configuration
  • Adds custom tag raw content handling with a new tokenizer context
  • Updates the tokenize function API to accept options object instead of separate parameters

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/types/parse.ts Defines new types for custom tag configuration options
src/types/tokenizer-state.ts Adds custom tag support to tokenizer state and context parameters
src/constants/tokenizer-context-types.ts Adds new CustomTagRawContent context type enum
src/tokenizer/tokenize.ts Updates tokenize function signature to accept options object
src/tokenizer/handlers/open-tag-end.ts Implements custom tag detection and context switching logic
src/tokenizer/handlers/custom-tag-raw-content.ts New handler for processing custom tag raw content
src/tokenizer/handlers/index.ts Exports the new custom tag raw content handler
src/parser/parse.ts Updates parse function to pass through options object
src/tokenizer/tests/tokenize.spec.ts Updates test cases to use new options object format
src/tokenizer/tests/input/custom-tag-raw-content.html Test input file for custom tag functionality
src/tokenizer/tests/output/custom-tag-raw-content.ts Expected test output for custom tag functionality

state.sourceCode.next();
return;
}
const regex = new RegExp(
Copy link
Preview

Copilot AI Jul 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating a new RegExp instance on every parse call is inefficient. Consider creating the regex once and reusing it, or use a more efficient string matching approach for the closing tag detection.

Copilot uses AI. Check for mistakes.

Comment on lines +19 to +23
const regex = new RegExp(
"</" +
state.contextParams[TokenizerContextTypes.CustomTagRawContent]?.tagName +
"\\s*>"
);
Copy link
Preview

Copilot AI Jul 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tag name is directly interpolated into a regex without escaping. This could lead to regex injection if the tag name contains special regex characters. Use a regex escape function or string-based matching instead.

Suggested change
const regex = new RegExp(
"</" +
state.contextParams[TokenizerContextTypes.CustomTagRawContent]?.tagName +
"\\s*>"
);
const escapeRegExp = (string: string) =>
string.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const tagName = state.contextParams[TokenizerContextTypes.CustomTagRawContent]?.tagName;
const regex = new RegExp("</" + escapeRegExp(tagName) + "\\s*>");

Copilot uses AI. Check for mistakes.

@yeonjuan yeonjuan marked this pull request as ready for review July 20, 2025 14:29
@yeonjuan yeonjuan changed the title feat: add customTags options feat: add rawContentTags options Jul 21, 2025
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.05%. Comparing base (9f327bb) to head (22887d6).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #32      +/-   ##
==========================================
+ Coverage   94.91%   95.05%   +0.13%     
==========================================
  Files          56       57       +1     
  Lines        1140     1172      +32     
  Branches      163      172       +9     
==========================================
+ Hits         1082     1114      +32     
  Misses         58       58              
Flag Coverage Δ
unittest 95.05% <100.00%> (+0.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/parser/parse.ts 100.00% <100.00%> (ø)
src/tokenizer/handlers/custom-tag-raw-content.ts 100.00% <100.00%> (ø)
src/tokenizer/handlers/open-tag-end.ts 100.00% <100.00%> (ø)
src/tokenizer/tokenize.ts 96.29% <100.00%> (+0.14%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@yeonjuan yeonjuan merged commit ab87f37 into main Jul 21, 2025
3 checks passed
@yeonjuan yeonjuan deleted the implement-custom-tags branch July 21, 2025 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants