fix: case fold reference link labels - #4077
Merged
Merged
Conversation
|
@gyanu2507 is attempting to deploy a commit to the MarkedJS Team on Vercel. A member of the Team first needs to authorize it. |
Labels were matched with toLowerCase(), which is not the Unicode case fold
CommonMark asks for, so labels that only agree once folded never matched:
[ẞ] [Straße]
[SS]: /url [STRASSE]: /url
'ẞ'.toLowerCase() is 'ß', never 'ss'. Round-tripping through upper case
reaches the fold, the way commonmark.js normalizeReference() does; lowercasing
afterwards keeps def.tag and the tokens.links keys in the lower case form
they have always had.
The two link-map lookups in Lexer used the label verbatim, so they went
through the same helper: without it a folded key would stop the reflink
masking from finding definitions it used to find.
This turns on CommonMark example 540 (Links 81/90 -> 82/90).
gyanu2507
force-pushed
the
fix/reflink-unicode-case-fold
branch
from
August 31, 2026 10:48
b845cfc to
397dabb
Compare
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
UziTech
approved these changes
Aug 31, 2026
styfle
approved these changes
Sep 8, 2026
github-actions Bot
pushed a commit
that referenced
this pull request
Sep 12, 2026
## [18.0.13](v18.0.12...v18.0.13) (2026-09-12) ### Bug Fixes * allow tabs in the thematic break that ends a list item ([#4087](#4087)) ([afbb27c](afbb27c)) * avoid O(n^2) scanning in reflinkSearch ([#4090](#4090)) ([c6a25bb](c6a25bb)) * case fold reference link labels ([#4077](#4077)) ([aed9336](aed9336)) * drop the leading whitespace after a hard line break ([#4075](#4075)) ([123ce04](123ce04)) * match html block start conditions when ending a list item ([#4072](#4072)) ([c2facac](c2facac)) * respect raw tokens when closing link labels ([#4066](#4066)) ([ef394f7](ef394f7)) * strip a tab that follows spaces in an indented code block ([#4080](#4080)) ([dbb393d](dbb393d))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Reference link labels are matched with
toLowerCase(), but CommonMark asks for a Unicode case fold:toLowerCase()doesn't get there, so labels that only agree once folded never match:renders as literal text today, because
'ẞ'.toLowerCase()is'ß'— never'ss'. Round-tripping through upper case reaches the fold, which is what commonmark.js does innormalizeReference():I lowercase once more afterwards so
def.tagand thetokens.linkskeys stay in the lower case form they've always had — an uppercase key would be a visible change for anyone readinglexer.tokens.linksor adeftoken, and it isn't needed to get the fold.The two link-map lookups in
Lexer(linkInTextand the reflink mask) used the label verbatim rather than normalized, so they go through the same helper. Without that they'd start missing definitions they used to find as soon as the stored keys are folded. As a side effect the mask now also handles labels whose case differs from the definition.This enables CommonMark example 540, taking Links from 81/90 to 82/90.
Contributor
Verification notes:
shouldFailflag from example 540 in bothtest/specs/commonmarkandtest/specs/gfm.test/specs/new/reflink_case_folding.{md,html}coveringẞ/SS,Straße/STRASSE,Dž/DŽandfi/FI. I generated the expected HTML from commonmark.js 0.32 and it is byte-identical to marked's output with this change; the fixture fails onmaster.npm testis green: specs 1789 -> 1791 passing, unit 191 passing, 0 failures, lint clean. No existing test needed adjusting.