Skip to content

feat(asr): 火山引擎说话人分离,过滤非主要说话人 - #760

Merged
appergb merged 1 commit into
Open-Less:betafrom
Shicongcoding:feat/speaker-diarization
Jul 5, 2026
Merged

feat(asr): 火山引擎说话人分离,过滤非主要说话人#760
appergb merged 1 commit into
Open-Less:betafrom
Shicongcoding:feat/speaker-diarization

Conversation

@Shicongcoding

@Shicongcoding Shicongcoding commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

User description

摘要

开启火山引擎流式 ASR 的 enable_speaker_info 选项,让返回的每个 utterance 带说话人标签。然后按说话人分组,取说话时长最长的作为「主要说话人」,只拼接其文本,丢弃其他人(比如听写时旁边同事插话)的片段。

这是「只识别我自己的声音」需求的 Quick Win,不是真正的声纹注册验证——后者需要本地 embedding 模型,后续另做。

改动(仅 volcengine.rs,无前端 / IPC / 配置改动)

  • build_first_frame_payload:请求参数加 enable_speaker_info: true
  • 结果组装:统计每个 speakerend_time - start_time 总和 → 取最大者为主要说话人 → 只拼接该 speaker 的 utterance 文本
  • 无 speaker 标签时行为不变(全量拼接),向后兼容
  • 加日志 [asr] speaker filter: primary=..., kept=..., filtered=...,过滤生效时可见

不做什么

  • 不做声纹注册 / 本地 embedding 验证(第二步,后续另做)
  • 不动其他 ASR provider(bailian / whisper / local 等暂不跟进)
  • 不改 RawTranscript 结构(过滤后的 text 直接体现在 text 字段,无需新增字段)
  • 不加前端 UI / 开关(如需开关后续再加)

测试

  • 编译通过(cargo check + tauri build,既有 101 warnings,无新增错误)
  • macOS 1.3.13-Beta.1 实测:自己说话时旁边有人插话,非主要说话人片段被过滤
  • 单人说话场景:行为与改动前一致(只有一个 speaker,无过滤)

局限

  • 「主要说话人」= 说话时长最长者。若你说话比旁边人短,会被误判为次要而被过滤(真正的声纹验证可解决,见「不做什么」)
  • 火山返回的 speaker 是匿名标签(speaker_0 / speaker_1),不等于「已注册用户」

PR Type

Enhancement


Description

  • Enable speaker diarization for Volcengine streaming ASR

  • Filter utterances to keep only the primary speaker (longest speaking time)

  • Fall back to all utterances if no speaker labels present

  • Add logging for speaker filter statistics


Diagram Walkthrough

flowchart LR
  A[Enable speaker_info] --> B[Collect speaker durations]
  B --> C[Select primary speaker]
  C --> D[Filter utterances]
  D --> E[Concatenate text]
Loading

File Walkthrough

Relevant files
Enhancement
volcengine.rs
Add speaker diarization filtering logic                                   

openless-all/app/src-tauri/src/asr/volcengine.rs

  • Added enable_speaker_info: true to first frame payload
  • Implemented speaker duration tracking and primary speaker selection
  • Filtered utterances to keep only primary speaker's text
  • Added logging for filtered utterances count
+51/-4   

Open volcengine streaming ASR's `enable_speaker_info` flag so each
returned utterance carries a speaker label. Then in the result
assembly, group utterances by speaker, pick the one with the longest
total speaking time as the "primary speaker", and only concatenate
their text — dropping utterances from other speakers (e.g. a
colleague interjecting while you dictate).

This is a heuristic quick-win for the "only transcribe my voice"
request; it does not yet do enrolled voiceprint verification. If no
speaker labels are present, behavior is unchanged (all utterances
concatenated).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ No major issues detected

@appergb

appergb commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

@claude 帮我审核一下这个更改和改动,实际的改动是否正确、会不会有影响,以及这个 PR 的质量如何?对于其他模型和其他平台会有影响吗?

@appergb

appergb commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

! 谁又改我 @ 逻辑了?我要跟他爆了,你信不信?

@appergb

appergb commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

@jiangmuran @claude 自动审核已被移除、不再响应,这个 PR 还没人审。麻烦你有空看一下 @Shicongcoding 这个改动:实现是否正确、会不会有副作用、整体质量如何,以及对其他 ASR provider / 其他平台有没有影响?辛苦。

@H-Chris233

Copy link
Copy Markdown
Collaborator

! 谁又改我 @ 逻辑了?我要跟他爆了,你信不信?

hhh,我来审一遍吧

@H-Chris233 H-Chris233 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我已完成审核,以下是我的两点建议:
只有 speaker >=2 时才启用过滤。(防止把逻辑搞坏)
主要 speaker 必须占比超过 60%~70%,否则不开启过滤(防止在声音极度混乱情况下造成丢失)

@appergb
appergb merged commit f77bc8f into Open-Less:beta Jul 5, 2026
1 check passed
@H-Chris233

Copy link
Copy Markdown
Collaborator

@appergb 我要跟你爆了💢我刚审查完请求更改的,你怎么直接合并了?

@Shicongcoding

Copy link
Copy Markdown
Contributor Author

我草这么暴躁是AI吧?

@H-Chris233

Copy link
Copy Markdown
Collaborator

我草这么暴躁是AI吧?

睁大你眼睛看看我@谁呢?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants