Skip to content

fix(subscription): don't panic when subscribing during an emit - #1105

Open
revl wants to merge 1 commit into
loro-dev:mainfrom
revl:fix/subscribe-during-emit
Open

revl wants to merge 1 commit into
loro-dev:mainfrom
revl:fix/subscribe-during-emit

Conversation

@revl

@revl revl commented Sep 15, 2026

Copy link
Copy Markdown

The bug

SubscriberSet::retain checks an emitter's subscriber map out and leaves a ThreadId marker in its place while it invokes callbacks. SubscriberSet::insert unwraps that slot as if it always held the map:

lock.subscribers
    .entry(emitter_key_1)
    .or_insert_with(|| Either::Left(BTreeMap::new()))
    .as_mut()
    .unwrap_left()   // panics when the slot holds Right(ThreadId)

So subscribing to an emitter that is mid-emit panics:

called `Either::unwrap_left()` on a `Right` value: ThreadId(7)

This is reachable whenever one thread subscribes while another emits for the same key, or a callback subscribes re-entrantly. retain itself handles contention (it sees Right, drops the lock, sleeps, retries) — only insert assumes the map is always there.

We hit this in a server that holds a LoroDoc per document: a subscribe_local_update racing an import panicked the tokio worker. The process stayed up and the socket stayed open, so clients waited on a response that never came — one occurrence wedged a test run for eight hours before we noticed.

The fix

The marker can't simply be replaced with a fresh map: it is what makes a concurrent retain wait rather than emit the same emitter twice, and what lets is_recursive_calling detect re-entrancy.

So subscribers added during an emit are parked in a new pending_subscribers map and folded back in exactly where retain already merges the ones added while the callback ran:

// Add any new subscribers that were added while invoking the callback.
if let Some(Either::Left(new_subscribers)) = lock.subscribers.remove(emitter) {
    subscribers.extend(new_subscribers);
}
if let Some(parked) = lock.pending_subscribers.remove(emitter) {
    subscribers.extend(parked);
}

Merged before the dropped sweep, so a subscriber unsubscribed again mid-emit is still dropped. remove drops parked subscribers along with the emitter they belong to.

Semantics: a subscriber added mid-emit does not receive the event already being delivered, and is registered for later ones. That matches what the existing merge does for the case it already handled.

Tests

Two, both in crates/loro-internal/src/utils/subscription.rs:

  • insert_during_emit_does_not_panic — a callback subscribes re-entrantly; asserts the new subscriber misses the in-flight event and receives the next one.
  • concurrent_insert_and_emit_do_not_panic — one thread parks inside a callback (emitter checked out) while another subscribes to the same key.

Both reproduce the original panic verbatim when the insert change is reverted. The rest of loro-internal's suite (380 tests) passes.

`retain` checks an emitter's subscriber map out and leaves a
`ThreadId` marker in its place while it invokes callbacks. An
`insert` landing in that window unwrapped the marker as if it were
the map:

    called `Either::unwrap_left()` on a `Right` value: ThreadId(7)

which is reachable whenever one thread subscribes while another emits
for the same key, or a callback subscribes re-entrantly. In a server
holding a LoroDoc this takes down the worker thread.

The marker cannot simply be replaced with a fresh map: it is what
makes a concurrent `retain` wait rather than emit the same emitter
twice, and what lets `is_recursive_calling` detect re-entrancy. So
subscribers added during an emit are parked in `pending_subscribers`
and folded back in where `retain` already merges the ones added while
the callback ran — before the dropped sweep, so one unsubscribed again
mid-emit is still dropped. `remove` drops parked subscribers with the
emitter they belong to.

A subscriber added mid-emit does not receive the event already being
delivered, and is registered for later ones.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 15, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-15T13:13:37.167458Z 3f7d353 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant