Backport to Java 17 - #9

The zero-width non-joiner and joiner are how Unicode lets an author
override contextual joining — U+200C forbids a join the letters would
otherwise make, U+200D forces one — and both were deleted before the
shaper, the only thing that reads them, ever ran. They are category C
like any other control, so the sanitizer that runs first removed them
along with the rest, and a document could not express the distinction
at all.

The shaper needed almost nothing for the non-joiner: it is an opaque
non-joining character, so the lookups that walk letters already came to
the right answer once it survived. The joiner is the one that had to be
taught, since it stands in for a letter that is not there. Both are
consumed during shaping, because past that point they are code points no
font can encode — measured as one thing and drawn as a substitution
mark. Text with no Arabic in it never reaches the shaper, so the glyph
seam drops them there instead.

The joining controls get their own predicate rather than joining the
bidirectional ones: these say nothing about direction, only about
whether two letters connect.

Also:

- Auto-size measured a highlight run unshaped while the token it builds
  from that run is shaped, so a chip was sized against a string the
  layout never uses. Both paths shape now. This is not currently visible
  end to end, because auto-size does not shrink a paragraph holding a
  chip at all for any script, which is its own defect.
- The degraded-shaping warning is keyed on the font rather than on
  (font, code point). A font that carries no presentation forms carries
  none of them, so the second letter said nothing the first had not, and
  a document turned one fact about the font into a warning per letter.
- The changelog claimed mirroring per UAX #9 L4 without the scope the
  implementation has always documented: the punctuation that occurs in
  documents, not the whole mirroring table.

Which way a paragraph runs was worked out twice from the same text. The
builder scanned for the first character of strong directionality to
decide which edge an unaligned paragraph sits at; the layout asked the
bidirectional algorithm. Two readings of one rule, and they did not
agree.

They part company on isolates. UAX #9 rule P2 skips everything between
an isolate initiator and its matching PDI when looking for the first
strong character — that is what an isolate is for — and a plain scan
reads straight into it. A paragraph opening with an isolated Hebrew
quotation therefore aligned to the right edge and laid out from the
left. This is the release that started carrying isolates through control
sanitizing, so the disagreement was reachable by exactly the author who
took the documentation's advice on steering a neutral stretch of text.

ParagraphDirection is now the single answer, and it lives in the layout
package because that is the sanctioned bridge from the canonical surface
to the engine: the builder may call there, and only there does the call
reach the resolver.

Word is the third caller. AUTO used to reach the DOCX export unresolved,
on the reasoning that it had already become a concrete alignment — but
alignment is not direction, and what Word actually received was a
paragraph with no base direction at all, left to infer one. That is the
thing w:bidi exists to prevent, and it came out the other way round for
a line opening with a digit or a parenthesis. The export is now marked
from the same answer the page used, and the test that pinned the old
behaviour asserts the new one in three cases instead.

Shaping runs before measurement — on the logical lines of the plain and
markdown paths, at token construction on the inline path — because the
contextual forms have their own advance widths and the engine's contract is
that what is measured is what is drawn. Text without an Arabic letter passes
through as the same instances.

Each backend again does only what its medium requires. The PDF draws the
forms, since its content stream will never run the font's own shaping.
PowerPoint gets the base letters back at its seam: it shapes Arabic itself,
and frozen presentation forms would end up in a file users search and copy
from. Word was never handed forms at all. Mirroring lives with reversal at
the PDF seam and nowhere else, for the same reason reversal does — both
PowerPoint and Word apply UAX #9 L4 themselves, and a mirror baked into the
span would come out double-mirrored there.

A font that carries the Arabic letters but not the forms — the GSUB-only
families — now degrades at the glyph seam to unjoined base letters instead
of '?', which costs the joining rather than the text; a lam-alef ligature
decomposes back into its two letters.

The proofs read the content stream: every Arabic glyph on the page is a
presentation form and none a base letter, a parenthesis before Hebrew is
drawn facing what it encloses, and the slide text carries base letters with
no form leaked into it.

The zero-width non-joiner and joiner are how Unicode lets an author
override contextual joining — U+200C forbids a join the letters would
otherwise make, U+200D forces one — and both were deleted before the
shaper, the only thing that reads them, ever ran. They are category C
like any other control, so the sanitizer that runs first removed them
along with the rest, and a document could not express the distinction
at all.

The shaper needed almost nothing for the non-joiner: it is an opaque
non-joining character, so the lookups that walk letters already came to
the right answer once it survived. The joiner is the one that had to be
taught, since it stands in for a letter that is not there. Both are
consumed during shaping, because past that point they are code points no
font can encode — measured as one thing and drawn as a substitution
mark. Text with no Arabic in it never reaches the shaper, so the glyph
seam drops them there instead.

The joining controls get their own predicate rather than joining the
bidirectional ones: these say nothing about direction, only about
whether two letters connect.

Also:

- Auto-size measured a highlight run unshaped while the token it builds
  from that run is shaped, so a chip was sized against a string the
  layout never uses. Both paths shape now. This is not currently visible
  end to end, because auto-size does not shrink a paragraph holding a
  chip at all for any script, which is its own defect.
- The degraded-shaping warning is keyed on the font rather than on
  (font, code point). A font that carries no presentation forms carries
  none of them, so the second letter said nothing the first had not, and
  a document turned one fact about the font into a warning per letter.
- The changelog claimed mirroring per UAX #9 L4 without the scope the
  implementation has always documented: the punctuation that occurs in
  documents, not the whole mirroring table.

Shaping runs before measurement — on the logical lines of the plain and
markdown paths, at token construction on the inline path — because the
contextual forms have their own advance widths and the engine's contract is
that what is measured is what is drawn. Text without an Arabic letter passes
through as the same instances.

Each backend again does only what its medium requires. The PDF draws the
forms, since its content stream will never run the font's own shaping.
PowerPoint gets the base letters back at its seam: it shapes Arabic itself,
and frozen presentation forms would end up in a file users search and copy
from. Word was never handed forms at all. Mirroring lives with reversal at
the PDF seam and nowhere else, for the same reason reversal does — both
PowerPoint and Word apply UAX #9 L4 themselves, and a mirror baked into the
span would come out double-mirrored there.

A font that carries the Arabic letters but not the forms — the GSUB-only
families — now degrades at the glyph seam to unjoined base letters instead
of '?', which costs the joining rather than the text; a lam-alef ligature
decomposes back into its two letters.

The proofs read the content stream: every Arabic glyph on the page is a
presentation form and none a base letter, a parenthesis before Hebrew is
drawn facing what it encloses, and the slide text carries base letters with
no form leaked into it.

The zero-width non-joiner and joiner are how Unicode lets an author
override contextual joining — U+200C forbids a join the letters would
otherwise make, U+200D forces one — and both were deleted before the
shaper, the only thing that reads them, ever ran. They are category C
like any other control, so the sanitizer that runs first removed them
along with the rest, and a document could not express the distinction
at all.

The shaper needed almost nothing for the non-joiner: it is an opaque
non-joining character, so the lookups that walk letters already came to
the right answer once it survived. The joiner is the one that had to be
taught, since it stands in for a letter that is not there. Both are
consumed during shaping, because past that point they are code points no
font can encode — measured as one thing and drawn as a substitution
mark. Text with no Arabic in it never reaches the shaper, so the glyph
seam drops them there instead.

The joining controls get their own predicate rather than joining the
bidirectional ones: these say nothing about direction, only about
whether two letters connect.

Also:

- Auto-size measured a highlight run unshaped while the token it builds
  from that run is shaped, so a chip was sized against a string the
  layout never uses. Both paths shape now. This is not currently visible
  end to end, because auto-size does not shrink a paragraph holding a
  chip at all for any script, which is its own defect.
- The degraded-shaping warning is keyed on the font rather than on
  (font, code point). A font that carries no presentation forms carries
  none of them, so the second letter said nothing the first had not, and
  a document turned one fact about the font into a warning per letter.
- The changelog claimed mirroring per UAX #9 L4 without the scope the
  implementation has always documented: the punctuation that occurs in
  documents, not the whole mirroring table.