Skip to content

macOS FileProviderExt: MaterializedEnumerationObserver re-marks the same items "dataless" in a non-converging loop, starving all sync operations on large accounts (34.0.1) #10558

Description

@potanin

⚠️ Before submitting, please verify the following: ⚠️

Environment

  • Nextcloud desktop client 34.0.1 (macOS, File Provider / virtual files backend)
  • macOS (Darwin 25.5.0), APFS
  • Account: ~426 GB, >150k files (server: Nextcloud 34.0.2, snap)
  • Client ↔ server over Tailscale (~37 ms RTT); network is not the bottleneck

Symptom

FileProviderExt enters a loop where MaterializedEnumerationObserver logs
Updating item state to dataless. continuously (~14–19 lines/sec), pegging
1–2 cores, growing to 2.5–4.4 GB RSS, and starving everything else the
extension does: content fetches drop to zero, directory enumerations hang
(even ls of an unvisited folder blocks), uploads crawl at ~5 MB/h, and the
domain's Realm DB bloats (347 MB → 877 MB in ~1 day).

Evidence it is a loop, not a large one-time pass

Within a single process lifetime (no restarts in between):

  • Migrated domain: 81,563 dataless-marks covering only 17,473 distinct items
    in 87 min — 99.9% of items marked more than once, avg 4.7×, cycle ≈ 15–19 min.
  • After a full domain rebuild (fresh Realm DB and fresh domain UUID): loop
    returns immediately, scaled to the new (small) materialized set — 104,874
    marks over 718 distinct items, max 438 repeats per item, in ~85 min.
    Loop cost appears proportional to materialized-set size.

What does NOT fix it

  • Killing/restarting the appex (fresh process re-enters the loop in seconds;
    note the respawned appex also has no credentials until the GUI app restarts)
  • Restarting the GUI app; restarting macOS fileproviderd
  • Waiting (2+ h undisturbed: zero convergence, same items re-marked)
  • v34.0.1 already contains the fix: Refactor visitedDirectory handling in MaterializedEnumerationObserver #10011 visitedDirectory refactor
  • Removing the account's fileProviderDomainIdentifier to mint a fresh
    domain + fresh Realm DB — loop returns, just smaller until the
    materialized set grows again

Observed side effects

  • After each fresh appex start there is a "fast window" (tens of MB of Realm
    growth in ~20 min) before the loop's overhead dominates again
  • With a stale DB the observer's flood coincided with mass placeholder
    dismantling by fileproviderd (local replica shrank 121 GB → 87 GB in 1 h)

Possibly related

Logs

JSONL logs available (hundreds of MB across several process lifetimes) under
File Provider Domains/<uuid>/Logs/; happy to attach excerpts, fileproviderctl
output, or run instrumented builds.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions