Skip to content

Conversation

sy-records
Copy link
Member

Summary

Background

While generating slugs, we noticed two issues affecting consistency:

  1. Invisible FE0F (Variation Selector-16)

    • Certain symbols can appear either as text or emoji, depending on whether they are followed by U+FE0F.

      • Text style: (U+270F)
      • Emoji style: ✏️ (U+270F U+FE0F)
    • If FE0F is not explicitly removed, the slug may contain invisible characters, leading to inconsistent URLs.

  2. NFKD decomposition side effects

    • Using normalize("NFKD") decomposes precomposed characters like é (U+00E9) into e + U+0301.
    • This results in visually identical strings with different underlying byte sequences, which can cause issues in comparisons and storage.

Changes

  • Remove U+FE0F explicitly

    • Ensures emoji and text variants of the same symbol produce identical slugs.

    • Example:

      • "✏ note""note"
      • "✏️ note""note"
  • Switch normalization from NFKD → NFC

    • Guarantees precomposed characters (like é) remain stable instead of being split into combining sequences.
    • Keeps accented characters intact, avoiding invisible combining marks in slugs.

Examples

Input Old (NFKD, FE0F not removed) New (NFC, FE0F removed)
café café (e + ◌́) café (U+00E9)
✏ note note note
✏️ note (with FE0F) note (contained FE0F) note

Related issue, if any:

What kind of change does this PR introduce?

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

For any code change,

  • Related documentation has been updated, if needed
  • Related tests have been added or updated, if needed

Does this PR introduce a breaking change?

  • Yes
  • No

Tested in the following browsers:

  • Chrome
  • Firefox
  • Safari
  • Edge

Copy link

vercel bot commented Sep 1, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
docsify-preview Ready Ready Preview Comment Sep 2, 2025 3:26am

@sy-records sy-records requested review from paulhibbitts and a team September 1, 2025 10:27
@sy-records sy-records changed the title fix: remove emoji variation selector from slug generation fix: normalize slugs to NFC and remove emoji variation selector Sep 1, 2025
@paulhibbitts
Copy link
Collaborator

Hi again @sy-records , please ignore my earlier reply as I misunderstood the issue being addressed, hopefully now I understand it and have verified the fix with my own Docsify v5 test site.

https://paulhibbitts.github.io/docsify-v5-emoji-test/#/emoji

Does this help confirm your fix?

@sy-records
Copy link
Member Author

That's right.

Legacy Style ⚠️ should be legacy-style-, while the previous version was legacy-style-%ef%b8%8f

café

before:

U+0063 : LATIN SMALL LETTER C
U+0061 : LATIN SMALL LETTER A
U+0066 : LATIN SMALL LETTER F
U+0065 : LATIN SMALL LETTER E
U+0301 : COMBINING ACUTE ACCENT {stress mark; Greek oxia, tonos}

current:
U+0063 : LATIN SMALL LETTER C
U+0061 : LATIN SMALL LETTER A
U+0066 : LATIN SMALL LETTER F
U+00E9 : LATIN SMALL LETTER E WITH ACUTE

@sy-records sy-records merged commit 5999f09 into docsifyjs:develop Sep 2, 2025
8 checks passed
@sy-records sy-records deleted the fix/slugify branch September 2, 2025 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants