Page MenuHomePhabricator
Create Task
Maniphest T354908

evaluate and migrate in-use parsoid metrics to statslib
Open, Needs TriagePublic
Actions

Description

See More

For the following paranoid metrics:

  • MediaWiki.Parsoid.html2wt.setup (html2wt_setup_seconds)
  • MediaWiki.Parsoid.html2wt.selser.serialize (html2wt_serialize_seconds)
  • MediaWiki.Parsoid.html2wt.selser.domDiff (html2wt_domDiff_seconds)
  • MediaWiki.Parsoid.html2wt.init (deprecated May 23)
  • MediaWiki.Parsoid.html2wt.total (deprecated May23)
  • MediaWiki.Parsoid.html2wt.timePerInputKB (deprecated May 23)

Please follow the migration process as outlined below:

  • Secure/Conduct a code review.
  • Deploy the changes to production via the train (https://wikitech.wikimedia.org/wiki/Deployments/Train).
  • Verify that the changes have been successfully implemented.
  • Update the dashboard by replacing the old Graphite metric with the new Prometheus metric.

Please follow the guidelines and standards outlined in the provided documentation:

https://www.mediawiki.org/wiki/Manual:Stats for detailed guidance on the conversion process.
https://drive.google.com/file/d/12yQEuOapkML1vb9MgCaX1QzbLBdXE6X2/view for a video tutorial on the conversion process.
https://docs.google.com/presentation/d/1SZWf_D3mWNX-XHN8PHYI84LDZr6GUQC2AMhZ9mQXCI0/edit#slide=id.g2795460c956_0_23 for slides on the best practices for converting metrics to statslib.

Details

SubjectRepoBranchLines +/-
Bump wikimedia/parsoid to 0.20.0-a24mediawiki/vendormaster+367 -183
stats: Use units as metrics suffixmediawiki/services/parsoidmaster+72 -23
Use statslib for metrics emitted by parsoidmediawiki/services/parsoidmaster+160 -40
Bump wikimedia/parsoid to 0.20.0-a20mediawiki/vendormaster+324 -134
Parsoid SiteConfig: allow stat counters incremented by arbitrary valuesmediawiki/coremaster+10 -2
Make Utils\Timing helper aware of Wikimedia\Stats libmediawiki/services/parsoidmaster+10 -3
Extract StatsFactory methods in parsoid SiteConfigmediawiki/coremaster+105 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

lmata created this task.Jan 11 2024, 9:39 PM
lmata updated the task description. (Show Details)
lmata renamed this task from migrate MediaWiki.Parsoid.html2wt.setup to statslib to migrate top used parsoid to statslib.Jan 11 2024, 9:41 PM
lmata updated the task description. (Show Details)
lmata updated the task description. (Show Details)
lmata mentioned this in T350592: EPIC: migrate in use metrics and dashboards to statslib.Jan 11 2024, 9:44 PM
lmata edited projects, added SRE Observability (FY2023/2024-Q3); removed SRE Observability (FY2023/2024-Q2).Jan 12 2024, 5:36 PM
lmata edited projects, added SRE Observability (FY2023/2024-Q4); removed SRE Observability (FY2023/2024-Q3).Mar 27 2024, 6:03 PM
MSantos added a project: Content-Transform-Team.Apr 10 2024, 5:17 PM
MSantos edited projects, added Content-Transform-Team-WIP; removed Content-Transform-Team.Apr 11 2024, 2:05 PM
lmata renamed this task from migrate top used parsoid to statslib to migrate top used parsoid metrics to statslib.Apr 11 2024, 4:15 PM
lmata renamed this task from migrate top used parsoid metrics to statslib to evaluate and migrate in-use parsoid metrics to statslib.
MSantos added a project: OKR-Work.Apr 15 2024, 1:46 PM
MSantos added a project: Content-Transform-Team.Apr 15 2024, 1:53 PM
MSantos moved this task from Backlog to External Request on the Content-Transform-Team board.
MSantos assigned this task to Jgiannelos.May 6 2024, 3:23 PM
Jgiannelos moved this task from Backlog to In Progress on the Content-Transform-Team-WIP board.Jun 18 2024, 2:34 PM
colewhite removed a project: SRE Observability (FY2023/2024-Q4).Jul 2 2024, 5:58 PM
gerritbot added a comment.Jul 5 2024, 1:47 PM

Change #1052308 had a related patch set uploaded (by Jgiannelos; author: Arlolra):

[mediawiki/services/parsoid@master] Make Utils\Timing helper aware of Wikimedia\Stats lib

https://gerrit.wikimedia.org/r/1052308

gerritbot added a project: Patch-For-Review.Jul 5 2024, 1:47 PM
gerritbot added a comment.Jul 10 2024, 10:07 AM

Change #1053267 had a related patch set uploaded (by Jgiannelos; author: Jgiannelos):

[mediawiki/core@master] Abstract StatsFactory metrics in parsoid SiteConfig

https://gerrit.wikimedia.org/r/1053267

Jgiannelos mentioned this in T369719: Decide how to use use StatsFactory inside Parsoid.Jul 10 2024, 1:48 PM
Krinkle added a subtask: T369719: Decide how to use use StatsFactory inside Parsoid.Jul 17 2024, 1:37 PM
Krinkle updated the task description. (Show Details)
gerritbot added a comment.Jul 19 2024, 10:20 PM

Change #1053267 merged by jenkins-bot:

[mediawiki/core@master] Extract StatsFactory methods in parsoid SiteConfig

https://gerrit.wikimedia.org/r/1053267

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.15; 2024-07-23).Jul 19 2024, 11:00 PM
gerritbot added a comment.Aug 2 2024, 7:29 PM

Change #1059421 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@master] Parsoid SiteConfig: allow stat counters incremented by arbitrary values

https://gerrit.wikimedia.org/r/1059421

gerritbot added a comment.Aug 5 2024, 10:02 AM

Change #1052308 abandoned by Jgiannelos:

[mediawiki/services/parsoid@master] Make Utils\Timing helper aware of Wikimedia\Stats lib

https://gerrit.wikimedia.org/r/1052308

gerritbot added a comment.Aug 13 2024, 12:49 PM

Change #1062375 had a related patch set uploaded (by Jgiannelos; author: Jgiannelos):

[mediawiki/services/parsoid@master] Use statslib for metrics emitted by parsoid

https://gerrit.wikimedia.org/r/1062375

Jgiannelos moved this task from In Progress to Blocked on the Content-Transform-Team-WIP board.Aug 16 2024, 10:21 AM
Jgiannelos moved this task from Blocked to Code Review on the Content-Transform-Team-WIP board.Aug 16 2024, 12:47 PM
gerritbot added a comment.Aug 19 2024, 9:57 PM

Change #1059421 merged by jenkins-bot:

[mediawiki/core@master] Parsoid SiteConfig: allow stat counters incremented by arbitrary values

https://gerrit.wikimedia.org/r/1059421

Jgiannelos moved this task from Code Review to To Deploy on the Content-Transform-Team-WIP board.Sep 9 2024, 2:10 PM
gerritbot added a comment.Sep 9 2024, 2:36 PM

Change #1062375 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Use statslib for metrics emitted by parsoid

https://gerrit.wikimedia.org/r/1062375

Maintenance_bot removed a project: Patch-For-Review.Sep 9 2024, 3:30 PM
gerritbot added a comment.Sep 9 2024, 8:12 PM

Change #1071697 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.20.0-a20

https://gerrit.wikimedia.org/r/1071697

gerritbot added a project: Patch-For-Review.Sep 9 2024, 8:12 PM
gerritbot added a comment.Sep 9 2024, 11:47 PM

Change #1071697 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.20.0-a20

https://gerrit.wikimedia.org/r/1071697

Maintenance_bot removed a project: Patch-For-Review.Sep 10 2024, 12:30 AM
Jgiannelos moved this task from To Deploy to To Verify on the Content-Transform-Team-WIP board.Sep 11 2024, 1:01 PM
gerritbot added a comment.Sep 23 2024, 10:27 AM

Change #1074967 had a related patch set uploaded (by Jgiannelos; author: Jgiannelos):

[mediawiki/services/parsoid@master] stats: Use base units as metrics suffix

https://gerrit.wikimedia.org/r/1074967

gerritbot added a project: Patch-For-Review.Sep 23 2024, 10:27 AM
gerritbot added a comment.Oct 4 2024, 11:53 PM

Change #1074967 merged by jenkins-bot:

[mediawiki/services/parsoid@master] stats: Use units as metrics suffix

https://gerrit.wikimedia.org/r/1074967

Maintenance_bot removed a project: Patch-For-Review.Oct 5 2024, 12:30 AM
gerritbot added a comment.Oct 7 2024, 4:29 PM

Change #1078448 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.20.0-a24

https://gerrit.wikimedia.org/r/1078448

gerritbot added a project: Patch-For-Review.Oct 7 2024, 4:29 PM
gerritbot added a comment.Oct 7 2024, 5:43 PM

Change #1078448 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.20.0-a24

https://gerrit.wikimedia.org/r/1078448

Maintenance_bot removed a project: Patch-For-Review.Oct 7 2024, 6:30 PM
cscott closed subtask T369719: Decide how to use use StatsFactory inside Parsoid as Declined.Nov 9 2024, 8:10 AM
tappof updated the task description. (Show Details)Apr 7 2025, 2:31 PM
tappof updated the task description. (Show Details)
Jgiannelos closed this task as Resolved.Apr 7 2025, 2:34 PM
tappof subscribed.Apr 7 2025, 2:39 PM

image.png (525×777 px, 42 KB)

image.png (505×666 px, 35 KB)

The 'init', 'total', and 'timePerInputKB' metrics haven't received data since 27/10/2023.

cscott reopened this task as Open.Apr 15 2025, 7:47 PM
cscott subscribed.

Re-opening this task, as several histogram-based Parsoid metrics are still not recording correct data in Prometheus, which is blocking us from moving away from the Graphite dashboards.

cscott added a subtask: T348796: MediaWiki: Define new metric type - Histogram.Apr 15 2025, 7:48 PM
colewhite changed the status of subtask T348796: MediaWiki: Define new metric type - Histogram from In Progress to Stalled.Apr 15 2025, 7:49 PM
colewhite closed subtask T348796: MediaWiki: Define new metric type - Histogram as Resolved.Apr 17 2025, 8:48 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits