Fix video codec, aspect and placement
Three problems visible at once on a running instance: the picture stuttered and drifted behind the sound, it was letterboxed oddly, and it appeared as a separate tile instead of coming from the bot. - YouTube handed us AV1 (format 398). Software-decoding AV1 at 720p does not sustain real time on a small server, which explains both the stutter and the drift; H.264 is now requested first, VP9 second. - The frame was padded into a fixed box, so a clip whose proportions differed got black bars baked in and then more from the client. Size is now a bounding box and the frame keeps the clip's own proportions. - Video was published as a screen share, which every client renders as its own tile. VIDEO_SOURCE=camera (the new default) puts it inside the bot's tile; "screen" keeps the old behaviour. VIDEO_SYNC_OFFSET_MS is there for the residual drift, since audio and video travel as two separately published tracks. Measured loudnorm first to rule it out as the cause of the desync: it adds 0 ms. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
9285fd7fbd
commit
0b17b880d6
@@ -69,9 +69,14 @@ const schema = z.object({
|
||||
.transform((value) => value === "true"),
|
||||
// YouTube only serves 360p as a single progressive file, and only such files
|
||||
// can be streamed, so that is the realistic default.
|
||||
// Treated as a bounding box: the frame keeps the clip's own proportions.
|
||||
VIDEO_WIDTH: z.coerce.number().int().positive().default(640),
|
||||
VIDEO_HEIGHT: z.coerce.number().int().positive().default(360),
|
||||
VIDEO_FPS: z.coerce.number().int().min(1).max(60).default(24),
|
||||
/** "camera" shows up inside the bot's tile, "screen" as a separate one. */
|
||||
VIDEO_SOURCE: z.enum(["camera", "screen"]).default("camera"),
|
||||
/** Nudge the picture against the sound, in milliseconds; positive delays video. */
|
||||
VIDEO_SYNC_OFFSET_MS: z.coerce.number().int().default(0),
|
||||
|
||||
DEFAULT_VOLUME: z.coerce.number().min(0).max(200).default(60),
|
||||
MAX_QUEUE_SIZE: z.coerce.number().int().positive().default(500),
|
||||
|
||||
Reference in New Issue
Block a user