Skip to content

fix: case fold reference link labels - #4077

Merged
UziTech merged 1 commit into
markedjs:masterfrom
gyanu2507:fix/reflink-unicode-case-fold
Sep 8, 2026
Merged

UziTech merged 1 commit into
markedjs:masterfrom
gyanu2507:fix/reflink-unicode-case-fold

Conversation

@gyanu2507

@gyanu2507 gyanu2507 commented Aug 31, 2026

Copy link
Copy Markdown

Description

Reference link labels are matched with toLowerCase(), but CommonMark asks for a Unicode case fold:

To normalize a label, strip off the opening and closing brackets, perform the Unicode case fold, strip leading and trailing spaces, tabs, and line endings, and collapse consecutive internal spaces, tabs, and line endings to a single space.

toLowerCase() doesn't get there, so labels that only agree once folded never match:

[] and [Straße].

[SS]: /sharp-s
[STRASSE]: /street

renders as literal text today, because 'ẞ'.toLowerCase() is 'ß' — never 'ss'. Round-tripping through upper case reaches the fold, which is what commonmark.js does in normalizeReference():

.toLowerCase()
.toUpperCase();

I lowercase once more afterwards so def.tag and the tokens.links keys stay in the lower case form they've always had — an uppercase key would be a visible change for anyone reading lexer.tokens.links or a def token, and it isn't needed to get the fold.

The two link-map lookups in Lexer (linkInText and the reflink mask) used the label verbatim rather than normalized, so they go through the same helper. Without that they'd start missing definitions they used to find as soon as the stored keys are folded. As a side effect the mask now also handles labels whose case differs from the definition.

This enables CommonMark example 540, taking Links from 81/90 to 82/90.

Contributor

  • Test(s) exist to ensure functionality and minimize regression (if no tests added, list tests covering this PR)
  • Docs have been added/updated (for bug fixes/features)

Verification notes:

  • Removed the shouldFail flag from example 540 in both test/specs/commonmark and test/specs/gfm.
  • Added test/specs/new/reflink_case_folding.{md,html} covering /SS, Straße/STRASSE, Dž/DŽ and /FI. I generated the expected HTML from commonmark.js 0.32 and it is byte-identical to marked's output with this change; the fixture fails on master.
  • npm test is green: specs 1789 -> 1791 passing, unit 191 passing, 0 failures, lint clean. No existing test needed adjusting.

@vercel

vercel Bot commented Aug 31, 2026

Copy link
Copy Markdown

@gyanu2507 is attempting to deploy a commit to the MarkedJS Team on Vercel.

A member of the Team first needs to authorize it.

Labels were matched with toLowerCase(), which is not the Unicode case fold
CommonMark asks for, so labels that only agree once folded never matched:

    [ẞ]            [Straße]
    [SS]: /url     [STRASSE]: /url

'ẞ'.toLowerCase() is 'ß', never 'ss'. Round-tripping through upper case
reaches the fold, the way commonmark.js normalizeReference() does; lowercasing
afterwards keeps def.tag and the tokens.links keys in the lower case form
they have always had.

The two link-map lookups in Lexer used the label verbatim, so they went
through the same helper: without it a folded key would stop the reflink
masking from finding definitions it used to find.

This turns on CommonMark example 540 (Links 81/90 -> 82/90).
@gyanu2507
gyanu2507 force-pushed the fix/reflink-unicode-case-fold branch from b845cfc to 397dabb Compare August 31, 2026 10:48
@vercel

vercel Bot commented Aug 31, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marked-website Ready Ready Preview Aug 31, 2026 2:05pm

Request Review

@UziTech UziTech left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! 💯

@gyanu2507
gyanu2507 requested a review from UziTech September 1, 2026 18:03
@UziTech
UziTech merged commit aed9336 into markedjs:master Sep 8, 2026
8 checks passed
github-actions Bot pushed a commit that referenced this pull request Sep 12, 2026
## [18.0.13](v18.0.12...v18.0.13) (2026-09-12)

### Bug Fixes

* allow tabs in the thematic break that ends a list item ([#4087](#4087)) ([afbb27c](afbb27c))
* avoid O(n^2) scanning in reflinkSearch ([#4090](#4090)) ([c6a25bb](c6a25bb))
* case fold reference link labels ([#4077](#4077)) ([aed9336](aed9336))
* drop the leading whitespace after a hard line break ([#4075](#4075)) ([123ce04](123ce04))
* match html block start conditions when ending a list item ([#4072](#4072)) ([c2facac](c2facac))
* respect raw tokens when closing link labels ([#4066](#4066)) ([ef394f7](ef394f7))
* strip a tab that follows spaces in an indented code block ([#4080](#4080)) ([dbb393d](dbb393d))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants