Three problems visible at once on a running instance: the picture
stuttered and drifted behind the sound, it was letterboxed oddly, and it
appeared as a separate tile instead of coming from the bot.
- YouTube handed us AV1 (format 398). Software-decoding AV1 at 720p does
not sustain real time on a small server, which explains both the
stutter and the drift; H.264 is now requested first, VP9 second.
- The frame was padded into a fixed box, so a clip whose proportions
differed got black bars baked in and then more from the client. Size is
now a bounding box and the frame keeps the clip's own proportions.
- Video was published as a screen share, which every client renders as
its own tile. VIDEO_SOURCE=camera (the new default) puts it inside the
bot's tile; "screen" keeps the old behaviour.
VIDEO_SYNC_OFFSET_MS is there for the residual drift, since audio and
video travel as two separately published tracks. Measured loudnorm first
to rule it out as the cause of the desync: it adds 0 ms.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Leonid PershinandClaude Opus 5
0b17b880d6