Editor since 2005; WMF developer since 2013. I work on Parsoid and OCG, and dabble with VE, real-time collaboration, and OOjs.
On github: https://github.com/cscott
See https://en.wikipedia.org/wiki/User:cscott for more.
User Details
User Details
- User Since
- Oct 21 2014, 6:47 PM (558 w, 5 d)
- Availability
- Available
- IRC Nick
- cscott
- LDAP User
- C. Scott Ananian
- MediaWiki User
- Cscott [ Global Accounts ]
Fri, Jul 4
Fri, Jul 4
cscott added a comment to T398655: mediawiki-phan-config minimum php target version for libraries is broken.
I'm still getting PhanCompatibleTrailingCommaParameterList on https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/1166266 for some reason (PS).
Thu, Jul 3
Thu, Jul 3
cscott updated subscribers of T398655: mediawiki-phan-config minimum php target version for libraries is broken.
Sorry for wasting your time! I assumed that wikipeg got a bump to mediawiki-phan-config at the same time @Jdforrester-WMF bumped the min version in the composer.json.
cscott renamed T398656: PageEditStash (and Content) should use JSON serialization from PageEditStash should use JSON serialization to PageEditStash (and Content) should use JSON serialization.
cscott updated the task description for T398656: PageEditStash (and Content) should use JSON serialization.
cscott added a comment to T398402: Remaining feature parity issues between the two Cite parsers.
It appears like Parsoid allows much more complex nesting of <ref> tags. This was intentionally blocked in the classic parser.
Wed, Jul 2
Wed, Jul 2
cscott added a comment to T397789: Define stable interface policy and coding conventions for named arguments.
We don't have to manually review 4000 method signatures. We can assume that the original authors gave their method reasonable names, and review the fewer cases in the Make method parameter names consistent for PHP 8 named arguments patch where there seem to be done disagreement. If even that is too hard, we can add @no-named-arguments only to those classes where these is an existing disagreement, as highlighted by that patch.
cscott added a comment to T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading!.
This is still causing some production errors due to bad content stuck in the ParserCache, but that should resolve itself with time.
cscott merged T398489: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading! Node id: 63 Stored data: {"parsoid":{"stx":"piped","a":{"href":"./jour"},"sa":{"href":"jour"},"dsr":{"start":779,"end":793,"openWidth":7,"closeWi into T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading!.
cscott merged task T398489: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading! Node id: 63 Stored data: {"parsoid":{"stx":"piped","a":{"href":"./jour"},"sa":{"href":"jour"},"dsr":{"start":779,"end":793,"openWidth":7,"closeWi into T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading!.
cscott added a comment to T398489: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading! Node id: 63 Stored data: {"parsoid":{"stx":"piped","a":{"href":"./jour"},"sa":{"href":"jour"},"dsr":{"start":779,"end":793,"openWidth":7,"closeWi.
Yeah, almost certainly fixed, the bad content (html with content outside of the <body>) just takes a while to leave the parser cache.
cscott added a comment to T390628: Wikimedia\Assert\InvariantException: Invariant failed: Bogus nodeId given!.
Yeah, we probably fixed it, there's just bad content in the cache that will take a while to expire.
cscott added a comment to T390628: Wikimedia\Assert\InvariantException: Invariant failed: Bogus nodeId given!.
Almost certainly a dup of T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading! which I thought we'd fixed.
cscott added a comment to T398489: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading! Node id: 63 Stored data: {"parsoid":{"stx":"piped","a":{"href":"./jour"},"sa":{"href":"jour"},"dsr":{"start":779,"end":793,"openWidth":7,"closeWi.
Almost certainly the same problem as T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading!, which I thought we'd fixed.
cscott added a comment to T398488: PHP Warning: Undefined array key 4.
This is a dup of T390344: v3 parserfunction serialization doesn't properly support named arguments and is caused by using named arguments with wikifunctions, which is not (yet) supported.
cscott added a comment to T397789: Define stable interface policy and coding conventions for named arguments.
To be clear, under current conventions PHP (and phan) will allow named argument calls unless there is an explicit @no-named-arguments annotation. There's a lot of work whichever way -- either someone needs to write a new phan plugin to enforce a new "no named arguments unless ..." and a new annotation type (unique to mediawiki) for permitting named arguments ... or we can use the existing tooling to sync up our argument names in short order with a small number of explicit @no-named-arguments annotations where appropriate (@Tacsipacsi makes the case that SpecialPage::execute() may be one such exception).
Tue, Jul 1
Tue, Jul 1
cscott added a comment to T379525: Add a magic word that returns a page's DISPLAYTITLE.
One option would be to make displaytitle a separate metadata property independent of the page wikitext, like language links are now. Then the parser function could echo the metadata without causing circularity issues with wikitext parsing.
cscott added a comment to T379525: Add a magic word that returns a page's DISPLAYTITLE.
cscott moved T373472: Make ParserMigration indicator optional from Q4 FY24-25 to In Progress on the Content-Transform-Team (Work In Progress) board.
cscott moved T365371: ParserMigration: Add "report visual bug" link from Q4 FY24-25 to In Progress on the Content-Transform-Team (Work In Progress) board.
cscott moved T395946: PEG backtracking causes OOMs in some cases from Q4 FY24-25 to Code Review on the Content-Transform-Team (Work In Progress) board.
cscott moved T393726: Cache WikiLink processing in WikiLinkHandler from Code Review to To Verify on the Content-Transform-Team (Work In Progress) board.
Mon, Jun 30
Mon, Jun 30
cscott added a comment to T363484: Update ParserMigration notice.
Updated the patchdemo with the render information as a separate sentence: https://patchdemo.wmcloud.org/wikis/d6bfc596bf/wiki/New_York_City
cscott added a comment to T398175: AbuseFilterConsequencesTest broken when $wgParsoidSettings['linting'] is set to true.
My wild guess would be that Parsoid is logging something (probably something inconsequential) when linting is turned on, and that's interfering with the naive way that the AF test is looking for its log entries:
cscott added a comment to T397789: Define stable interface policy and coding conventions for named arguments.
cscott updated subscribers of T397999: Have a way to include wikicode in a redirect that will be displayed at the top of the target page when the redirect is followed.
Discussed this task at the Content-Transform-Team tech forum today. A few notes:
- @ihurbain was wondering how you planned to handle self links to redirects.
- @ssastry noted that, if you wanted this to be a uniform feature of articles on toki wikipedia, you don't need the explicit __keeptitle__ page property, you could just make this a configuration variable. That would shrink the already-small Extension:MultiTitle by quite a bit, since a lot of what's going on at the moment is just boilerplate related to the __keeptitle__ magic word.
cscott added a comment to T397789: Define stable interface policy and coding conventions for named arguments.
It's also worth noting briefly that by default psalm propagates the name of the base class/interface to all of its subclasses/implementations. It's relatively easy to do a bulk rename by renaming the parameter on the base class/interface and then re-running psalm --alter --issues=ParamNameMismatch; you can cherry-pick the base psalm configuration from DNM: add psalm checker. So we can have a discussion about what 'good' parameter names are during the 1.45 cycle and make updates fairly easily before we fix the names in place going forward.
cscott added a comment to T397789: Define stable interface policy and coding conventions for named arguments.
Sun, Jun 29
Sun, Jun 29
cscott added a comment to T397789: Define stable interface policy and coding conventions for named arguments.
Let's also consider how you would safely rename a parameter.
function foo($oldName) { ... }first new add the new name, making it available for invocation as a named parameter, while preserving argument order:
function foo($newName, $oldName=null) {
if ($oldName!==null) {
$newName = $oldName;
wfDeprecated(__METHOD__ . ' with $oldName', '1.xx');
}
}then after the appropriate amount of time, the trailing $oldName parameter can be removed.
cscott added a comment to T310511: Metadata comparison testing between Parsoid and the legacy parser.
This can be based on the code we added to the Linter to do real time performance comparsions: T393399.
cscott added a comment to T397789: Define stable interface policy and coding conventions for named arguments.
Just as a straw proposal: why not do the opposite? Method names should be considered part of the API unless you have an explicit @no-named-arguments annotation?
cscott added a comment to T396813: Make use of PHP 8.0 and 8.1 features in existing code.
cscott added a comment to T363484: Update ParserMigration notice.
Patch demo at https://patchdemo.wmcloud.org/wikis/e5a16a7d5c/wiki/New_York_(magazine) -- no back end yet, but this is a good place to check the UX layout of the dialog.
Fri, Jun 27
Fri, Jun 27
cscott updated subscribers of T363484: Update ParserMigration notice.
In his review, @Arlolra pointed out that the footer text is not used in the MinervaNeue skin. (I tested every other skin available on patchdemo and they looked fine).
cscott added a comment to T397999: Have a way to include wikicode in a redirect that will be displayed at the top of the target page when the redirect is followed.
(I've also looked at the code for Extension:KeepTitle which actually seems like a relatively small and efficient way to do what you want, using existing Hooks in completely appropriate ways. I'm inclined to think it would be easier to have WMF adopt that extension than to have WMF resource a new #redirect functionality, but ¯\_(ツ)_/¯
cscott added a comment to T397999: Have a way to include wikicode in a redirect that will be displayed at the top of the target page when the redirect is followed.
For the record, I'm on a personal quest (T204370) to eventually replace all the "magic words" in wikitext with a uniform alternative {{#....}} syntax. This would make it easier to add additional arguments, as was noted. So:
{{#redirect|NewTitle|special message}}is a somewhat reasonable straw-proposal. That would put 'old title' => 'special message' in a db table (maybe a new column of the existing redirects table), and then the existing code which puts the redirect header at the top (OutputTransform/Stages/AddRedirectHeader.php and the post-cache part of WikitextContentHandler::fillParserOutput which calls setRedirectHeader) could swap in a custom redirect header where appropriate. Since this modification was done post-cache it wouldn't require splitting the parser cache.
cscott added a comment to T397999: Have a way to include wikicode in a redirect that will be displayed at the top of the target page when the redirect is followed.
As I think @bvibber probably notes, implementing this literally as the bug description states would be A Bad Idea, since it would mean the each redirected title would have to be cached as an entirely separate object, since the content of each page could depend on arbitrary ways on the "original title". That's a nonstarter, and I suspect the task description should be changed to describe /what/ is wanted, ie "text at the top that depends on the original redirect title" not the /how/ (a specific magic word).
Thu, Jun 26
Thu, Jun 26
cscott added a comment to T303788: InvalidArgumentException: Asked for code outside of range (55296).
This was probably related to the fix I landed in 4cd16aa25622b2f63ba57dc8fef9c344ae2e4e5b?
cscott added a comment to T380516: Introduce ParserOptions type for "propagated to ParserOutput but not otherwise affecting the parse".
This is partially done: we have ParserOutput::setFromParserOptions(ParserOptions $po) now which transfers a number of things: wrapper class (T303615), section edit link flag, the collapsible sections flag (T363496) and the preview flag (T341010). The missing piece is to add a list of these in ParserOptions akin to the existing $initialCacheVaryingOptionsHash, $initialLazyOptions, etc lists so that these "only for ParserOutput" options aren't included in the cache key -- right now collapsibleSections and suppressSectionEditLinks are included in the cache key, and all of these are included in ParserOptions::matches() etc.
cscott added a comment to T303615: There's no reason to split the ParserCache on wrapclass.
Wrapclass was removed from the parser cache key in 8b0e7298ac25ccbb83d382bcb81f6c9433d053ca in 2017.
cscott closed T385609: Wikimedia\Assert\InvariantException: Invariant failed: Bad UTF-8 at start of string as Resolved.
Likely an extension returning bad UTF-8 data. The stack trace seems to indicate that the bad UTF-8 was wrapped as a strip marker. https://he.wikipedia.org/wiki/%D7%95%D7%99%D7%A7%D7%99%D7%A4%D7%93%D7%99%D7%94%3A%D7%97%D7%93%D7%A9%D7%95%D7%AA doesn't display any errors at the present, and during February we were still actively working on the fragment/strip marker code.
cscott added a comment to T385753: Temporary data lost when serializing dom to html.
I suspect this has been fixed by our continued work in this area. In particular, we don't round-trip through HTML strings much (if at all) any more, so the issue roundtripping through the string form should have been fixed.
cscott closed T373256: Define ParserCache strategy for pages with content placeholder slots (as with wikifunctions), a subtask of T392133: Async content needs !misermode, as Resolved.
cscott closed T373256: Define ParserCache strategy for pages with content placeholder slots (as with wikifunctions) as Resolved.
We've implemented a reasonable strategy for now; see patches above. We also have metrics in place (part of the wikifunctions SLO/SLA) which will allow us to determine if our caching strategy works/backfires/etc. So we can resolve this for now, and open a new task if the metrics show that we need to revisit our initial caching strategy.
cscott closed T396886: TokenUtils: PHP Warning: foreach() argument must be of type array|object, null given as Resolved.
Verified that the last instance of this exception was on Jun 19.
cscott added a comment to T316050: Sunset Reading List browser Extension on Web.
The Content-Transform-Team in theory owns Reading Lists, but none of us have actually touched that code AFAIK. I think clearly communicating the sunset is preferable to maintaining an illusion of support.
Wed, Jun 25
Wed, Jun 25
cscott added a comment to T396835: Interwiki links with double underscore get rendered as single underscore.
Space normalization (replace all spaces with _, compress runs of spaces) is part of the wiki title normalization and shouldn't be changed.
Tue, Jun 24
Tue, Jun 24
cscott added a comment to T393308: Charts with errors are not editable in VE.
<p><wiki-chart><div>......</div></wiki-chart></p> is parsed by every browser as <p><wiki-chart></wiki-chart></p><div>......</div><p></p> so that's not a parsoid bug per se. It's one of the dangers of using custom elements, which are always parsed as inline AFAIK.
cscott added a comment to T396813: Make use of PHP 8.0 and 8.1 features in existing code.
Named arguments require that all implementations/overrides of a method use the same name for the method arguments. psalm has a sniff for this and an automatic fixer, I don't know if phan/phpcs do? This should be turned on IMO as it is a prerequisite to using named arguments in our codebase. (I've already merged patches to fix Parsoid's argument naming, written by psalm.)
cscott added a comment to T383328: Kartographer map overlay not visible in Parsoid rendering.
I like the second idea (add useparsoid=1 to ApiQueryMapData).
Mon, Jun 23
Mon, Jun 23
cscott moved T363484: Update ParserMigration notice from Blocked to In Progress on the Content-Transform-Team (Work In Progress) board.
Sat, Jun 21
Sat, Jun 21
cscott added a comment to T392775: Add link color for temporary usernames.
It is stuck awaiting review. It got to a C+1 and never managed to get merged. It's not clear which team is ultimately responsible for the review.
Thu, Jun 19
Thu, Jun 19
cscott added a comment to T363484: Update ParserMigration notice.
Sorry, you'll need the parser migration extension installed, and to have parsoid read views enabled for your account:
https://www.mediawiki.org/wiki/Help:Extension%3AParserMigration
That is installed on beta and testwiki, I believe, so it should just be a matter of enabling parsoid read views in your user preferences.
cscott created T397389: Test Task.
Mon, Jun 16
Mon, Jun 16
cscott updated the task description for T397073: CTT Tasks week of 2025-06-13.
cscott updated the task description for T397073: CTT Tasks week of 2025-06-13.
cscott updated the task description for T397073: CTT Tasks week of 2025-06-13.
cscott renamed T396208: CTT tasks week of 2025-06-06 from CTT tasks week of 2025-06-05 to CTT tasks week of 2025-06-06.
cscott renamed T396208: CTT tasks week of 2025-06-06 from CTT tasks week of 2025-05-06 to CTT tasks week of 2025-06-05.
cscott added a comment to T396615: ImageMap fails to render when useParsoid=1 if links are providing using the "pipetrick".
{T4700: Pre-save transform skips extensions using wikitext (gallery, references, footnotes, Cite, status indicators, pipe trick, subst, signatures)}, which has a patch https://gerrit.wikimedia.org/r/c/mediawiki/core/+/981193 but https://gerrit.wikimedia.org/r/c/mediawiki/core/+/607367 is the better patch I think.
Sat, Jun 14
Sat, Jun 14
cscott added a comment to T396913: Possible parsercache corruption in Charts localization.
Because the legacy parser is pre-cache, we might already be splitting the parser cache by user variant, but parsoid expects to do language conversion post cache. It is more technically correct to use user language here because user language strings are also pre-converted, excluded from language converter, etc. That said, your content might be hidden behind a strip marker and hidden from language converter anyway.
Thu, Jun 12
Thu, Jun 12
cscott updated subscribers of T396656: TypeError: MediaWiki\Parser\ParserOutput::appendJsConfigVar(): Argument #2 ($value) must be of type string, int given.
Welcome to my favorite misfeature of PHP: numeric strings used as keys to associative arrays are silently converted to ints. I count 16 (string) casts in ParserOutput alone which are all due to this same misfeature. phan doesn't flag this, likely because the false positive rate would be high. I've changed a number of APIs to make the problem less likely to occur for clients, for example ParserOutput::getCategoryNames() returns a list<string> which in connection with ParserOutput::getCategorySortKey() avoiding the problems of the previous interface (::getCategoryMap()) which exported a array<string,string> directly which was really/secretly an array<string|int,string> trapping the unwary (::getCategoryMap() at least documents the correct type now).
cscott added a comment to T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading!.
This orphan tag shows up locally using the API as well, but you have to use body_only=false:
$ php bin/parse.php --domain da.wiktionary.org --page mercredi --wrapSections --body_only=false ... </section></section></body>
cscott added a comment to T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading!.
Something is adding bogus HTML to the bottom of the document before the OutputTransform stages get started:
<!DOCTYPE html> <html prefix="dc: http://purl.org/dc/terms/ mw: http://mediawiki.org/rdf/"><head prefix="mwr: https://da.wiktionary.org/wiki/Special:Redirect/"><meta charset="utf-8"/><meta property="mw:pageId" content="0"/><meta property="mw:pageNamespace" content="0"/> ... </section></li></section></section></body>
cscott added a comment to T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading!.
I can reproduce on parsoidtest1001:
$ sudo -u www-data php /srv/mediawiki/multiversion/MWScript.php view.php --wiki=dawiktionary mercredi > mercredi.wt $ sudo -u www-data php /srv/mediawiki/multiversion/MWScript.php parse.php --wiki=dawiktionary -p < mercredi.wt
seems like I should add a --page option to core's parse.php to save a step here. regardless, now I have something to look at...
cscott added a comment to T391031: UnreachableException should be Normalized.
Unreachable exceptions shouldn't be reached. Not sure it makes sense to try to normalize this, as opposed to fixing the bug.
cscott added a comment to T390629: Wikimedia\Assert\UnreachableException: Trying to fetch node data without loading!.
Reproducible: https://da.wiktionary.org/wiki/mercredi?useparsoid=1 ; note that https://da.wiktionary.org/wiki/mercredi?useparsoid=0 is fine. Also running via the API of integrated modes on parsoidtest1001 is fine; this is a bug in the OutputTransform pipeline after Parsoid is done parsing.
cscott added a comment to T391487: Category sort key set incorrectly.
Indeed, it looks like we're using some information in data-parsoid to indicate whether an explicit sort key should be provided, and the id attribute is telling selser to use the data-parsoid from the previous element. We should be flagging this as modified (and thus not using the old data-parsoid) but apparently are not. Yep, probably a bug in Parsoid.
cscott added a comment to T372387: Wrong section numbering if Parsoid is used and wikitext is invalid.
Ok, Bug fix for 'Rule reference variables should not be affected by failed rules' and the patch it follows up, Rule reference variables should not be affected by failed rules, will probably fix this once we move Parsoid to the new wikipeg.
Mon, Jun 9
Mon, Jun 9
cscott added a comment to T393306: Chart output makes VE support difficult.
To briefly update: part of the issue here is how we sanitize content cut-and-pasted into VisualEditor. VE uses a somewhat restrictive sanitizer. Supporting custom components naively would require updating the sanitizer every time to include each custom tag name defined by an extension.
Jun 5 2025
Jun 5 2025
cscott added a comment to T363484: Update ParserMigration notice.
@mwilliams anything learned from the "bit of time next week to explore a couple other possible solutions", or is this design ready to implement?
cscott closed T391702: Document v3 parser function DOM, a subtask of T388786: Follow up from Parsoid Fragment support, as Resolved.
cscott added a comment to T395979: Provide means to extract the parameters and values of a template invocation in wikitext.
The proposed summary fails to mention a couple of additional implementations, including in Parsoid. This strikes me as a https://xkcd.com/927/ issue.
Jun 4 2025
Jun 4 2025
cscott updated the task description for T367616: Rename data-mw.attrs to extAttrs to avoid confusion with data-mw.attribs.
cscott added a comment to T395968: Semantic template information in TemplateData.
https://en.wikipedia.org/wiki/User:Cscott/Opportunities_for_Content_Transform_Team mentions both Subbu's Typed Templates proposal as well as a number of other possible initial clients/use cases.
Jun 3 2025
Jun 3 2025
cscott added a comment to T394836: Use refactored grammar for `{{....}}` constructs.
An effective reproduction of the backtracking is this snippet:
* a {{bar|1
* b