Skip to content

feat(lyric): 支持杜比全景声(6声道)TTML歌词偏移#989

Open
kid141252010 wants to merge 1 commit intoimsyy:devfrom
kid141252010:feat/dolby-atmos-lyric-offset-imsyy
Open

feat(lyric): 支持杜比全景声(6声道)TTML歌词偏移#989
kid141252010 wants to merge 1 commit intoimsyy:devfrom
kid141252010:feat/dolby-atmos-lyric-offset-imsyy

Conversation

@kid141252010
Copy link
Copy Markdown

  • parseTTML: 新增 extractTtmlLyricOffsetMs 和 applyLyricOffsetToLines
  • IPlaybackEngine: 新增 getChannels() 可选方法
  • FFmpegAudioPlayer/AudioElementPlayer/MpvPlayer: 实现 getChannels()
  • MpvPlayer: 监听 audio-params/channel-count 获取声道数
  • MpvService: 观察 audio-params/channel-count 属性
  • AudioManager: 暴露 getChannels() 方法
  • status store: 新增 currentAudioChannels 状态
  • LyricManager: TTML 偏移仅在6声道时应用,修复预加载缓存时序
  • PlayerController: canplay 时更新声道数,声道变化时自动重载歌词

- parseTTML: 新增 extractTtmlLyricOffsetMs 和 applyLyricOffsetToLines
- IPlaybackEngine: 新增 getChannels() 可选方法
- FFmpegAudioPlayer/AudioElementPlayer/MpvPlayer: 实现 getChannels()
- MpvPlayer: 监听 audio-params/channel-count 获取声道数
- MpvService: 观察 audio-params/channel-count 属性
- AudioManager: 暴露 getChannels() 方法
- status store: 新增 currentAudioChannels 状态
- LyricManager: TTML 偏移仅在6声道时应用,修复预加载缓存时序
- PlayerController: canplay 时更新声道数,声道变化时自动重载歌词
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on improving the lyric display accuracy for Dolby Atmos (6-channel) audio by implementing TTML lyric offset support. It involves modifications across multiple components, including playback engines, the AudioManager, and the LyricManager, to ensure correct lyric synchronization based on the audio channel configuration. The changes also address potential issues with pre-loaded lyric caches and ensure that the lyric display is updated dynamically when the audio channel configuration changes.

Highlights

  • TTML Lyric Offset: This PR introduces support for Dolby Atmos (6-channel) TTML lyric offset, enhancing the lyric synchronization for spatial audio.
  • Channel Count Handling: The changes include adding methods to retrieve audio channel counts from different playback engines and updating the lyric display based on the channel configuration.
  • Lyric Manager Updates: The LyricManager is updated to apply TTML offsets only when 6 channels are detected and to handle pre-loaded lyric caches correctly.
Changelog
  • electron/main/services/MpvService.ts
    • Observes the 'audio-params/channel-count' property to track audio channel changes.
  • src/core/audio-player/AudioElementPlayer.ts
    • Added channel count tracking and a method to retrieve the number of channels.
  • src/core/audio-player/IPlaybackEngine.ts
    • Introduced an optional 'getChannels' method to retrieve the audio channel count.
  • src/core/audio-player/MpvPlayer.ts
    • Implemented channel count tracking and a method to retrieve the number of channels, resetting channels on new source.
  • src/core/audio-player/ffmpeg-engine/FFmpegAudioPlayer.ts
    • Added a method to retrieve the audio channel count from metadata.
  • src/core/player/AudioManager.ts
    • Exposed a method to retrieve the audio channel count.
  • src/core/player/LyricManager.ts
    • Implemented TTML lyric offset application based on channel count and fixed pre-load cache timing issues.
  • src/core/player/PlayerController.ts
    • Updated channel count on 'canplay' event and reloads lyrics when channels change.
  • src/stores/status.ts
    • Added a state property to store the current audio channel count.
  • src/utils/lyric/parseTTML.ts
    • Added functions to extract TTML lyric offset and apply it to lyric lines.
Activity
  • Added extractTtmlLyricOffsetMs and applyLyricOffsetToLines in parseTTML.
  • Implemented getChannels() in FFmpegAudioPlayer, AudioElementPlayer and MpvPlayer.
  • Observed audio-params/channel-count in MpvService and MpvPlayer.
  • Exposed getChannels() method in AudioManager.
  • Added currentAudioChannels state in status store.
  • Applied TTML offset only in 6-channel mode and fixed pre-load cache timing in LyricManager.
  • Updated channel count on canplay and reloaded lyrics on channel change in PlayerController.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

本次 PR 主要是为了支持杜比全景声(6声道)TTML歌词的偏移功能。整体实现思路清晰,通过在各个播放引擎中获取声道数,并在 LyricManager 中根据声道数判断是否应用 TTML 文件中定义的偏移,逻辑正确。代码在多个层面进行了修改,包括播放器引擎接口、具体实现、状态管理和播放控制器,覆盖了所需的功能点。

我发现了一些可以改进的地方:

  • PlayerController 中使用 setTimeout 轮询检查声道数,这种方式不够健壮,建议改为事件驱动的模式。
  • LyricManager 中存在几处重复的 TTML 解析逻辑,可以提取为辅助函数以提高代码可维护性。
  • LyricManager 中有一处空的 catch 块,建议添加日志以方便问题排查。

具体的修改建议请见代码评论。

Comment on lines +689 to +694
setTimeout(() => {
syncChannelsAndRefreshLyric();
}, 200);
setTimeout(() => {
syncChannelsAndRefreshLyric();
}, 600);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

使用多个 setTimeout 来轮询获取音频声道数的方式不够健壮。它依赖于固定的延迟,如果引擎获取声道数的时间超过 600ms,可能会导致逻辑失败。

建议采用事件驱动的模式来代替轮询:

  1. IPlaybackEngine 接口中增加一个可选的 channelschanged 事件。
  2. 在各个播放器引擎实现中(MpvPlayer, AudioElementPlayer, FFmpegAudioPlayer),当获取到声道数或声道数发生变化时,派发 channelschanged 事件。
  3. PlayerController 中,监听 audioManagerchannelschanged 事件,并在事件回调中调用 syncChannelsAndRefreshLyric

这种方式可以更及时、可靠地响应声道数的变化,并移除当前的 setTimeout 写法。

Comment on lines +339 to +344
const lyricOffsetMs = this.shouldApplyTtmlOffset()
? extractTtmlLyricOffsetMs(ttmlContent)
: 0;
const sorted = cleanTTMLTranslations(ttmlContent);
const parsed = parseTTML(sorted);
const lines = parsed?.lines || [];
const lines = applyLyricOffsetToLines(parsed?.lines || [], lyricOffsetMs);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

这部分解析和应用TTML歌词偏移的逻辑在 adoptTTML (此处), fetchLocalLyric (479-482行), 和 fetchLocalOverrideLyric (567-572行) 中存在重复。为了提高代码的可维护性,建议将此逻辑提取到一个私有的辅助函数中,例如 parseAndOffsetTtml(ttmlContent: string): LyricLine[]。这样可以统一处理TTML的解析、清理和偏移应用,使代码更简洁。

Comment on lines +872 to +874
} catch {
// 回退使用预加载结果
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

空的 catch 块会静默地隐藏潜在的错误。当 fetchLyric 失败时,虽然逻辑会回退到使用预加载的歌词,但开发者无法从日志中得知重获取操作失败了。建议在此处添加一条警告日志,以便于调试。

} catch (e) {
          console.warn("TTML歌词重获取失败,回退使用预加载版本。", e);
          // 回退使用预加载结果
        }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant