Release video frames against the audio clock

With the codec fixed the picture stopped stuttering but ran ahead of the
sound, and for a structural reason: the audio takes a longer road to the
call — our PCM goes through revoice's own ffmpeg and its buffer — while
frames went out the moment they were decoded.

Each frame now carries its position in the stream and waits until the
audio that belongs with it has actually played, using the player's own
playback position as the shared clock. Frames that fall more than 250 ms
behind are dropped rather than shown late, so the picture recovers by
itself instead of accumulating drift.

The queue is bounded by bytes and never by pausing the stream: one ffmpeg
feeds both outputs, so blocking the video pipe would stop the audio whose
clock we are waiting on — a deadlock.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Leonid Pershin
2026-09-09 02:56:04 +03:00
co-authored by Claude Opus 5
parent ec50e4c4df
commit bc567e953e
3 changed files with 235 additions and 147 deletions
+9 -1
View File
@@ -379,7 +379,15 @@ export class GuildPlayer extends EventEmitter<GuildPlayerEvents> {
private async startScreenShare(video: NonNullable<PlaybackInput["video"]>): Promise<void> {
const room = this.connection?.room;
if (!room) return;
const publisher = new VideoPublisher({ width: video.width, height: video.height });
const publisher = new VideoPublisher({
width: video.width,
height: video.height,
fps: video.fps,
// Frames are released against the sound that has actually played, which is
// the only clock both sides share: the audio path runs through revoice's
// own ffmpeg and buffer, so it always trails the raw frames.
audioClock: () => this.media?.seconds ?? 0,
});
try {
await publisher.start(room, video.stream, (reason) => {
this.notify(`📺 Видео отключено: ${reason}.`);