mrleo1nid/stoat-mbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fall back to separate video and audio downloads

Browse Source 
Playing with video only worked for clips YouTube still serves as a single
progressive file, and it mostly does not: the log showed "Requested
format is not available" and every track fell back to sound only.

A progressive format is still preferred when offered — one download, one
decode — but when there is none, video and audio are fetched in parallel
and handed to one ffmpeg over separate pipes, which pairs them by
timestamp. This also lifts the 360p ceiling: the height is now a CPU
choice rather than a limit.

The extra-descriptor plumbing (two inputs, two outputs on one ffmpeg) was
verified against synthetic media: exactly 5.0s of PCM and 120 whole
frames at 24 fps, no partial tail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Leonid Pershin
2026-09-09 02:21:14 +03:00
co-authored by Claude Opus 5
parent a9b680c418
commit 9285fd7fbd
5 changed files with 370 additions and 313 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+3 -2
View File
@@ -77,8 +77,9 @@ YTDLP_JS_RUNTIME=node
# Показывать клип в голосовом канале как демонстрацию экрана (только YouTube). # Показывать клип в голосовом канале как демонстрацию экрана (только YouTube).
# Требует прав Video у бота и включённого видео в конфигурации инстанса. # Требует прав Video у бота и включённого видео в конфигурации инстанса.
# YouTube отдаёт одним файлом только 360p, а стримить можно лишь такой формат — # 360p по умолчанию из осторожности: кодирование видео заметно грузит CPU
# отсюда размер по умолчанию. Кодирование видео заметно грузит CPU сервера. # сервера. Можно поднять до 720 (VIDEO_WIDTH=1280, VIDEO_HEIGHT=720), если
# машина позволяет.
# Это лишь разрешение: сам показ включается тумблером в панели или командой # Это лишь разрешение: сам показ включается тумблером в панели или командой
# !video, и по умолчанию выключен. При false тумблер в панели не показывается. # !video, и по умолчанию выключен. При false тумблер в панели не показывается.
VIDEO_ENABLED=false VIDEO_ENABLED=false
README.md
+9 -5
View File
@@ -249,15 +249,19 @@ VIDEO_FPS=24
Как это устроено и почему так: Как это устроено и почему так:
- **Только YouTube и только 360p.** Стримить можно лишь прогрессивный формат — один файл со - **Только YouTube.** Для остальных источников картинки нет, трек играет звуком.
звуком и картинкой. Раздельные дорожки (720p и выше) yt-dlp обязан сначала скачать целиком - **Два способа получить картинку.** Если YouTube отдаёт прогрессивный формат (один файл со
и лишь потом склеить, то есть воспроизведение началось бы после полной загрузки. Отдавать звуком и видео) — используется он: одна загрузка, один декодер. Такие форматы встречаются всё
ffmpeg прямые ссылки на CDN тоже нельзя: YouTube их для сторонних клиентов подвешивает. реже, поэтому есть запасной путь: видео и звук качаются двумя процессами параллельно и
сводятся одним ffmpeg по таймкодам. Склейка средствами yt-dlp не годится — он скачивает оба
потока целиком, прежде чем выдать первый байт; прямые ссылки на CDN тоже: YouTube подвешивает
их для сторонних клиентов.
- **Один процесс ffmpeg, два выхода:** PCM для голосовой дорожки и сырые I420-кадры для видео. - **Один процесс ffmpeg, два выхода:** PCM для голосовой дорожки и сырые I420-кадры для видео.
Темп задаёт `-re`, иначе кадры улетали бы вперёд звука и съедали память — распакованный кадр Темп задаёт `-re`, иначе кадры улетали бы вперёд звука и съедали память — распакованный кадр
720p весит 1.4 МБ. 720p весит 1.4 МБ.
- **Цена.** Кодирование видео ложится на CPU сервера и держится всё время трека, в отличие от - **Цена.** Кодирование видео ложится на CPU сервера и держится всё время трека, в отличие от
почти бесплатного звука. Ставьте `VIDEO_FPS` пониже, если нагрузка мешает. почти бесплатного звука. 360p по умолчанию выбран из осторожности; поднимайте до 720p, если
машина тянет, и снижайте `VIDEO_FPS`, если нагрузка мешает.
- Живые трансляции и не-YouTube источники играют звуком, как раньше. - Живые трансляции и не-YouTube источники играют звуком, как раньше.
## Прокси, когда YouTube недоступен ## Прокси, когда YouTube недоступен
src/sources/index.ts
+250 -250
View File
@@ -1,250 +1,250 @@
import type { Readable } from "node:stream"; import type { Readable } from "node:stream";
import { config } from "../config.js"; import { config } from "../config.js";
import { logger } from "../logger.js"; import { logger } from "../logger.js";
import { UserFacingError, type Requester, type SearchResult, type Track } from "../types.js"; import { UserFacingError, type Requester, type SearchResult, type Track } from "../types.js";
import * as direct from "./direct.js"; import * as direct from "./direct.js";
import * as local from "./local.js"; import * as local from "./local.js";
import { openVideoPipeline } from "./video-pipeline.js"; import { openVideoPipeline } from "./video-pipeline.js";
import * as ytdlp from "./ytdlp.js"; import * as ytdlp from "./ytdlp.js";
export { checkAvailable as checkYtDlp, checkCookies, checkProxy, describeProxy } from "./ytdlp.js"; export { checkAvailable as checkYtDlp, checkCookies, checkProxy, describeProxy } from "./ytdlp.js";
export { isEnabled as isLocalLibraryEnabled, listFiles as listLocalFiles } from "./local.js"; export { isEnabled as isLocalLibraryEnabled, listFiles as listLocalFiles } from "./local.js";
const log = logger.child({ mod: "sources" }); const log = logger.child({ mod: "sources" });
const YOUTUBE_HOSTS = ["youtube.com", "youtu.be", "music.youtube.com", "m.youtube.com"]; const YOUTUBE_HOSTS = ["youtube.com", "youtu.be", "music.youtube.com", "m.youtube.com"];
const SOUNDCLOUD_HOSTS = ["soundcloud.com", "on.soundcloud.com", "m.soundcloud.com"]; const SOUNDCLOUD_HOSTS = ["soundcloud.com", "on.soundcloud.com", "m.soundcloud.com"];
function asUrl(value: string): URL | null { function asUrl(value: string): URL | null {
if (!/^https?:\/\//i.test(value)) return null; if (!/^https?:\/\//i.test(value)) return null;
try { try {
return new URL(value); return new URL(value);
} catch { } catch {
return null; return null;
} }
} }
function hostMatches(url: URL, hosts: string[]): boolean { function hostMatches(url: URL, hosts: string[]): boolean {
const host = url.hostname.replace(/^www\./, ""); const host = url.hostname.replace(/^www\./, "");
return hosts.some((candidate) => host === candidate || host.endsWith(`.${candidate}`)); return hosts.some((candidate) => host === candidate || host.endsWith(`.${candidate}`));
} }
/** /**
* A YouTube link copied while a mix or playlist is open carries both `v` and * A YouTube link copied while a mix or playlist is open carries both `v` and
* `list`. People mean the track they were listening to — expanding the list * `list`. People mean the track they were listening to — expanding the list
* would dump a whole radio station into the queue. The rule stays deliberately * would dump a whole radio station into the queue. The rule stays deliberately
* blunt (any `v=` means one track) so pasting a link is predictable; a * blunt (any `v=` means one track) so pasting a link is predictable; a
* `/playlist?list=…` URL is how you ask for the whole list. * `/playlist?list=…` URL is how you ask for the whole list.
*/ */
function isTrackInsidePlaylist(url: URL): boolean { function isTrackInsidePlaylist(url: URL): boolean {
return Boolean(url.searchParams.get("v") && url.searchParams.get("list")); return Boolean(url.searchParams.get("v") && url.searchParams.get("list"));
} }
interface ParsedQuery { interface ParsedQuery {
text: string; text: string;
forced: "youtube" | "soundcloud" | "local" | null; forced: "youtube" | "soundcloud" | "local" | null;
} }
function parsePrefix(raw: string): ParsedQuery { function parsePrefix(raw: string): ParsedQuery {
const trimmed = raw.trim(); const trimmed = raw.trim();
const match = /^(yt|youtube|sc|soundcloud|local|file):\s*(.+)$/is.exec(trimmed); const match = /^(yt|youtube|sc|soundcloud|local|file):\s*(.+)$/is.exec(trimmed);
if (!match) return { text: trimmed, forced: null }; if (!match) return { text: trimmed, forced: null };
const [, prefix, rest] = match as unknown as [string, string, string]; const [, prefix, rest] = match as unknown as [string, string, string];
const key = prefix.toLowerCase(); const key = prefix.toLowerCase();
if (key === "sc" || key === "soundcloud") return { text: rest.trim(), forced: "soundcloud" }; if (key === "sc" || key === "soundcloud") return { text: rest.trim(), forced: "soundcloud" };
if (key === "local" || key === "file") return { text: rest.trim(), forced: "local" }; if (key === "local" || key === "file") return { text: rest.trim(), forced: "local" };
return { text: rest.trim(), forced: "youtube" }; return { text: rest.trim(), forced: "youtube" };
} }
/** Turns whatever a user typed into a playable set of tracks. */ /** Turns whatever a user typed into a playable set of tracks. */
export async function resolveQuery( export async function resolveQuery(
rawQuery: string, rawQuery: string,
requestedBy: Requester, requestedBy: Requester,
maxTracks = config.MAX_QUEUE_SIZE, maxTracks = config.MAX_QUEUE_SIZE,
): Promise<SearchResult> { ): Promise<SearchResult> {
const { text, forced } = parsePrefix(rawQuery); const { text, forced } = parsePrefix(rawQuery);
if (!text) throw new UserFacingError("Укажите название трека или ссылку"); if (!text) throw new UserFacingError("Укажите название трека или ссылку");
if (forced === "local") { if (forced === "local") {
const tracks = await local.search(text, config.SEARCH_RESULT_LIMIT, requestedBy); const tracks = await local.search(text, config.SEARCH_RESULT_LIMIT, requestedBy);
if (tracks.length === 0) throw new UserFacingError("В медиатеке ничего не найдено"); if (tracks.length === 0) throw new UserFacingError("В медиатеке ничего не найдено");
return { tracks: tracks.slice(0, 1), playlist: null }; return { tracks: tracks.slice(0, 1), playlist: null };
} }
const url = asUrl(text); const url = asUrl(text);
if (url) { if (url) {
const limit = Math.min(maxTracks, config.MAX_PLAYLIST_TRACKS); const limit = Math.min(maxTracks, config.MAX_PLAYLIST_TRACKS);
const singleTrack = isTrackInsidePlaylist(url); const singleTrack = isTrackInsidePlaylist(url);
if (hostMatches(url, YOUTUBE_HOSTS) || hostMatches(url, SOUNDCLOUD_HOSTS)) { if (hostMatches(url, YOUTUBE_HOSTS) || hostMatches(url, SOUNDCLOUD_HOSTS)) {
return ytdlp.resolveUrl(text, requestedBy, limit, { singleTrack }); return ytdlp.resolveUrl(text, requestedBy, limit, { singleTrack });
} }
const probed = await direct.probe(text); const probed = await direct.probe(text);
if (probed.isMedia) { if (probed.isMedia) {
return { tracks: [direct.toTrack(text, requestedBy, probed)], playlist: null }; return { tracks: [direct.toTrack(text, requestedBy, probed)], playlist: null };
} }
// Not a raw media URL — let yt-dlp try its extractors (Bandcamp, Vimeo, ...). // Not a raw media URL — let yt-dlp try its extractors (Bandcamp, Vimeo, ...).
return ytdlp.resolveUrl(text, requestedBy, limit, { singleTrack }); return ytdlp.resolveUrl(text, requestedBy, limit, { singleTrack });
} }
if (local.isEnabled() && forced === null) { if (local.isEnabled() && forced === null) {
const localHits = await local.search(text, 1, requestedBy); const localHits = await local.search(text, 1, requestedBy);
if (localHits.length > 0 && localHits[0]) return { tracks: [localHits[0]], playlist: null }; if (localHits.length > 0 && localHits[0]) return { tracks: [localHits[0]], playlist: null };
} }
const tracks = await searchWithFallback(text, forced, 1, requestedBy); const tracks = await searchWithFallback(text, forced, 1, requestedBy);
if (tracks.length === 0) throw new UserFacingError(NOTHING_FOUND); if (tracks.length === 0) throw new UserFacingError(NOTHING_FOUND);
return { tracks: tracks.slice(0, 1), playlist: null }; return { tracks: tracks.slice(0, 1), playlist: null };
} }
export type SearchSource = "all" | "youtube" | "soundcloud" | "local"; export type SearchSource = "all" | "youtube" | "soundcloud" | "local";
/** Round-robins two result lists so neither source buries the other. */ /** Round-robins two result lists so neither source buries the other. */
function interleave(a: Track[], b: Track[]): Track[] { function interleave(a: Track[], b: Track[]): Track[] {
const merged: Track[] = []; const merged: Track[] = [];
for (let i = 0; i < Math.max(a.length, b.length); i += 1) { for (let i = 0; i < Math.max(a.length, b.length); i += 1) {
const first = a[i]; const first = a[i];
const second = b[i]; const second = b[i];
if (first) merged.push(first); if (first) merged.push(first);
if (second) merged.push(second); if (second) merged.push(second);
} }
return merged; return merged;
} }
/** Multi-result search used by the `search` command and the web panel. */ /** Multi-result search used by the `search` command and the web panel. */
export async function searchTracks( export async function searchTracks(
rawQuery: string, rawQuery: string,
requestedBy: Requester, requestedBy: Requester,
limit = config.SEARCH_RESULT_LIMIT, limit = config.SEARCH_RESULT_LIMIT,
source: SearchSource = "all", source: SearchSource = "all",
): Promise<Track[]> { ): Promise<Track[]> {
const { text, forced } = parsePrefix(rawQuery); const { text, forced } = parsePrefix(rawQuery);
if (!text) return []; if (!text) return [];
const url = asUrl(text); const url = asUrl(text);
if (url) { if (url) {
const result = await resolveQuery(text, requestedBy); const result = await resolveQuery(text, requestedBy);
return result.tracks; return result.tracks;
} }
// A prefix inside the query is an explicit instruction and outranks the picker. // A prefix inside the query is an explicit instruction and outranks the picker.
const target: SearchSource = forced ?? source; const target: SearchSource = forced ?? source;
if (target === "local") return local.search(text, limit, requestedBy); if (target === "local") return local.search(text, limit, requestedBy);
if (target === "youtube" || target === "soundcloud") { if (target === "youtube" || target === "soundcloud") {
return ytdlp.search(text, target, limit, requestedBy); return ytdlp.search(text, target, limit, requestedBy);
} }
const [youtube, soundcloud, localHits] = await Promise.all([ const [youtube, soundcloud, localHits] = await Promise.all([
ytdlp.search(text, "youtube", limit, requestedBy).catch(() => []), ytdlp.search(text, "youtube", limit, requestedBy).catch(() => []),
ytdlp.search(text, "soundcloud", limit, requestedBy).catch(() => []), ytdlp.search(text, "soundcloud", limit, requestedBy).catch(() => []),
local.isEnabled() ? local.search(text, 3, requestedBy).catch(() => []) : Promise.resolve([]), local.isEnabled() ? local.search(text, 3, requestedBy).catch(() => []) : Promise.resolve([]),
]); ]);
return [...localHits, ...interleave(youtube, soundcloud)].slice(0, limit); return [...localHits, ...interleave(youtube, soundcloud)].slice(0, limit);
} }
/** /**
* YouTube silently returns nothing for queries its restricted mode dislikes — * YouTube silently returns nothing for queries its restricted mode dislikes —
* same query, same words, results in a browser but an empty list over the API. * same query, same words, results in a browser but an empty list over the API.
* Falling back to SoundCloud rescues a good share of those. * Falling back to SoundCloud rescues a good share of those.
*/ */
async function searchWithFallback( async function searchWithFallback(
text: string, text: string,
forced: "youtube" | "soundcloud" | "local" | null, forced: "youtube" | "soundcloud" | "local" | null,
limit: number, limit: number,
requestedBy: Requester, requestedBy: Requester,
): Promise<Track[]> { ): Promise<Track[]> {
const primary = forced === "soundcloud" ? "soundcloud" : "youtube"; const primary = forced === "soundcloud" ? "soundcloud" : "youtube";
const tracks = await ytdlp.search(text, primary, limit, requestedBy); const tracks = await ytdlp.search(text, primary, limit, requestedBy);
if (tracks.length > 0 || forced !== null) return tracks; if (tracks.length > 0 || forced !== null) return tracks;
return ytdlp.search(text, "soundcloud", limit, requestedBy).catch(() => []); return ytdlp.search(text, "soundcloud", limit, requestedBy).catch(() => []);
} }
export const NOTHING_FOUND = export const NOTHING_FOUND =
"Ничего не найдено. YouTube иногда прячет результаты от ботов (например, из-за ограниченного режима) — попробуйте другие слова или вставьте ссылку на трек."; "Ничего не найдено. YouTube иногда прячет результаты от ботов (например, из-за ограниченного режима) — попробуйте другие слова или вставьте ссылку на трек.";
export interface PlaybackInput { export interface PlaybackInput {
/** Either a file path / URL for ffmpeg, or a piped stream. */ /** Either a file path / URL for ffmpeg, or a piped stream. */
input: string | Readable; input: string | Readable;
inputOptions: string[]; inputOptions: string[];
cleanup(): void; cleanup(): void;
/** Resolves with a reason if the downloader died on its own, for reporting. */ /** Resolves with a reason if the downloader died on its own, for reporting. */
failure?: Promise<string | null>; failure?: Promise<string | null>;
/** Raw I420 frames to publish as a screen share, when video is on. */ /** Raw I420 frames to publish as a screen share, when video is on. */
video?: { stream: Readable; width: number; height: number }; video?: { stream: Readable; width: number; height: number };
} }
export interface PlaybackOptions { export interface PlaybackOptions {
/** Publish the picture too; the server-wide switch still has to allow it. */ /** Publish the picture too; the server-wide switch still has to allow it. */
video?: boolean; video?: boolean;
} }
/** Only YouTube reliably carries a picture worth showing next to the audio. */ /** Only YouTube reliably carries a picture worth showing next to the audio. */
function canShowVideo(track: Track, wanted: boolean): boolean { function canShowVideo(track: Track, wanted: boolean): boolean {
return wanted && config.VIDEO_ENABLED && track.source === "youtube" && !track.isLive; return wanted && config.VIDEO_ENABLED && track.source === "youtube" && !track.isLive;
} }
const HTTP_RESILIENCE = [ const HTTP_RESILIENCE = [
"-reconnect", "1", "-reconnect", "1",
"-reconnect_streamed", "1", "-reconnect_streamed", "1",
"-reconnect_delay_max", "5", "-reconnect_delay_max", "5",
]; ];
/** /**
* ffmpeg speaks HTTP proxies only, and just for http(s) inputs — it has no SOCKS * ffmpeg speaks HTTP proxies only, and just for http(s) inputs — it has no SOCKS
* support. So an http(s) proxy is handed to ffmpeg for direct links, while * support. So an http(s) proxy is handed to ffmpeg for direct links, while
* anything else keeps ffmpeg off the network entirely (see openPlayback). * anything else keeps ffmpeg off the network entirely (see openPlayback).
*/ */
function ffmpegProxyOptions(): string[] { function ffmpegProxyOptions(): string[] {
const proxy = config.YTDLP_PROXY; const proxy = config.YTDLP_PROXY;
if (!proxy || !/^https?:\/\//i.test(proxy)) return []; if (!proxy || !/^https?:\/\//i.test(proxy)) return [];
return ["-http_proxy", proxy]; return ["-http_proxy", proxy];
} }
/** Opens an ffmpeg-compatible input for a track, optionally starting at an offset. */ /** Opens an ffmpeg-compatible input for a track, optionally starting at an offset. */
export async function openPlayback( export async function openPlayback(
track: Track, track: Track,
seekSeconds = 0, seekSeconds = 0,
options: PlaybackOptions = {}, options: PlaybackOptions = {},
): Promise<PlaybackInput> { ): Promise<PlaybackInput> {
const seekOptions = seekSeconds > 0 ? ["-ss", seekSeconds.toFixed(2)] : []; const seekOptions = seekSeconds > 0 ? ["-ss", seekSeconds.toFixed(2)] : [];
if (track.source === "local") { if (track.source === "local") {
const filePath = await local.assertInsideLibrary(track.url); const filePath = await local.assertInsideLibrary(track.url);
return { input: filePath, inputOptions: seekOptions, cleanup: () => {} }; return { input: filePath, inputOptions: seekOptions, cleanup: () => {} };
} }
if (track.source === "direct") { if (track.source === "direct") {
return { return {
input: track.url, input: track.url,
inputOptions: [...HTTP_RESILIENCE, ...ffmpegProxyOptions(), ...seekOptions], inputOptions: [...HTTP_RESILIENCE, ...ffmpegProxyOptions(), ...seekOptions],
cleanup: () => {}, cleanup: () => {},
}; };
} }
// The format is checked before committing to the video pipeline: audio comes // The format is checked before committing to the video pipeline: audio comes
// out of that same pipeline, so falling back afterwards would kill the sound. // out of that same pipeline, so falling back afterwards would kill the sound.
if ( const videoMode = canShowVideo(track, options.video ?? false)
canShowVideo(track, options.video ?? false) && ? await ytdlp.selectVideoMode(track.url, config.VIDEO_HEIGHT)
(await ytdlp.hasProgressiveVideo(track.url, config.VIDEO_HEIGHT)) : null;
) { if (videoMode) {
log.info({ title: track.title }, "playing with video"); log.info({ title: track.title, mode: videoMode }, "playing with video");
const pipeline = openVideoPipeline(track.url, seekSeconds); const pipeline = openVideoPipeline(track.url, seekSeconds, videoMode);
return { return {
input: pipeline.audio, input: pipeline.audio,
inputOptions: pipeline.audioInputOptions, inputOptions: pipeline.audioInputOptions,
cleanup: () => pipeline.kill(), cleanup: () => pipeline.kill(),
video: { stream: pipeline.video, width: pipeline.width, height: pipeline.height }, video: { stream: pipeline.video, width: pipeline.width, height: pipeline.height },
}; };
} }
// Everything goes through yt-dlp: YouTube stalls direct CDN URLs fetched by // Everything goes through yt-dlp: YouTube stalls direct CDN URLs fetched by
// anything else, and ffmpeg would bypass a SOCKS proxy anyway. Seeking // anything else, and ffmpeg would bypass a SOCKS proxy anyway. Seeking
// therefore costs a decode up to the offset instead of an HTTP range request. // therefore costs a decode up to the offset instead of an HTTP range request.
const proc = ytdlp.openAudioStream(track.url); const proc = ytdlp.openAudioStream(track.url);
return { return {
input: proc.stream, input: proc.stream,
inputOptions: seekOptions, inputOptions: seekOptions,
cleanup: () => proc.kill(), cleanup: () => proc.kill(),
failure: proc.failure, failure: proc.failure,
}; };
} }
src/sources/video-pipeline.ts
+78 -40
View File
@@ -1,9 +1,9 @@
import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process"; import { spawn, type ChildProcess, type ChildProcessWithoutNullStreams } from "node:child_process";
import { createRequire } from "node:module"; import { createRequire } from "node:module";
import type { Readable } from "node:stream"; import type { Duplex, Readable, Writable } from "node:stream";
import { config } from "../config.js"; import { config } from "../config.js";
import { logger } from "../logger.js"; import { logger } from "../logger.js";
import { downloadArgs, progressiveFormat } from "./ytdlp.js"; import { AUDIO_FORMAT, downloadArgs, progressiveFormat, videoFormat, type VideoMode } from "./ytdlp.js";
const require = createRequire(import.meta.url); const require = createRequire(import.meta.url);
/** ffmpeg-static ships the binary revoice.js already relies on. */ /** ffmpeg-static ships the binary revoice.js already relies on. */
@@ -23,33 +23,35 @@ export interface VideoPipeline {
kill(): void; kill(): void;
} }
/** function spawnDownload(pageUrl: string, format: string) {
* One download, one decode, two outputs: PCM for the voice track and I420 frames return spawn(
* for the screen share.
*
* yt-dlp does the fetching (so cookies and the proxy apply as everywhere else)
* and hands ffmpeg a progressive stream. `-re` paces the decode at playback
* speed: it keeps picture and sound together and stops the video from racing
* ahead — a decoded 720p frame is 1.4 MB, so an unpaced stream would eat memory
* in seconds.
*/
export function openVideoPipeline(pageUrl: string, seekSeconds = 0): VideoPipeline {
const { VIDEO_WIDTH: width, VIDEO_HEIGHT: height, VIDEO_FPS: fps } = config;
const ytdlp = spawn(
config.YTDLP_PATH, config.YTDLP_PATH,
[ [...downloadArgs(), "-f", format, "--no-playlist", "--quiet", "-o", "-", pageUrl],
...downloadArgs(),
"-f",
progressiveFormat(height),
"--no-playlist",
"--quiet",
"-o",
"-",
pageUrl,
],
{ stdio: ["ignore", "pipe", "pipe"], windowsHide: true }, { stdio: ["ignore", "pipe", "pipe"], windowsHide: true },
); );
}
/**
* Decodes a clip into the two things a call needs: PCM for the voice track and
* raw I420 frames for the screen share, both out of a single ffmpeg.
*
* A progressive format (one file with both streams) is the cheap path — one
* download, one decode. When YouTube offers no such format, which is now the
* common case, picture and sound are downloaded in parallel and fed to ffmpeg
* over separate pipes, and it pairs them by timestamp.
*
* Everything is fetched by yt-dlp, so cookies and the proxy apply as everywhere
* else — handing CDN URLs to ffmpeg instead does not work, YouTube stalls them.
* `-re` paces the decode at playback speed, which keeps the two in step and
* stops the video from racing ahead: a decoded 720p frame is 1.4 MB, so an
* unpaced stream would eat memory in seconds.
*/
export function openVideoPipeline(pageUrl: string, seekSeconds = 0, mode: VideoMode = "split"): VideoPipeline {
const { VIDEO_WIDTH: width, VIDEO_HEIGHT: height, VIDEO_FPS: fps } = config;
const split = mode === "split";
const videoSource = spawnDownload(pageUrl, split ? videoFormat(height) : progressiveFormat(height));
const audioSource = split ? spawnDownload(pageUrl, AUDIO_FORMAT) : null;
const filters = [ const filters = [
`scale=${width}:${height}:force_original_aspect_ratio=decrease`, `scale=${width}:${height}:force_original_aspect_ratio=decrease`,
@@ -58,19 +60,25 @@ export function openVideoPipeline(pageUrl: string, seekSeconds = 0): VideoPipeli
"format=yuv420p", "format=yuv420p",
].join(","); ].join(",");
const seek = seekSeconds > 0 ? ["-ss", seekSeconds.toFixed(2)] : [];
// Split: fd 0 video in, fd 3 audio in, fd 4 frames out.
// Progressive: fd 0 everything in, fd 3 frames out.
const videoOutFd = split ? 4 : 3;
const ffmpeg = spawn( const ffmpeg = spawn(
FFMPEG_PATH, FFMPEG_PATH,
[ [
"-hide_banner", "-hide_banner",
"-loglevel", "-loglevel",
"error", "error",
...(seekSeconds > 0 ? ["-ss", seekSeconds.toFixed(2)] : []), ...seek,
"-re", "-re",
"-i", "-i",
"pipe:0", "pipe:0",
...(split ? [...seek, "-re", "-i", "pipe:3"] : []),
// Audio branch → stdout. // Audio branch → stdout.
"-map", "-map",
"0:a:0?", split ? "1:a:0" : "0:a:0",
"-f", "-f",
"s16le", "s16le",
"-ar", "-ar",
@@ -78,27 +86,56 @@ export function openVideoPipeline(pageUrl: string, seekSeconds = 0): VideoPipeli
"-ac", "-ac",
"2", "2",
"pipe:1", "pipe:1",
// Video branch → the extra pipe on fd 3. // Video branch → its own pipe.
"-map", "-map",
"0:v:0", "0:v:0",
"-vf", "-vf",
filters, filters,
"-f", "-f",
"rawvideo", "rawvideo",
"pipe:3", `pipe:${videoOutFd}`,
], ],
{ stdio: ["pipe", "pipe", "pipe", "pipe"], windowsHide: true }, {
) as ChildProcessWithoutNullStreams & { stdio: Readable[] }; stdio: split
? ["pipe", "pipe", "pipe", "pipe", "pipe"]
: ["pipe", "pipe", "pipe", "pipe"],
windowsHide: true,
},
) as ChildProcessWithoutNullStreams & { stdio: Array<Duplex & Readable> };
ytdlp.stdout.pipe(ffmpeg.stdin); const videoOut = ffmpeg.stdio[videoOutFd] as Readable;
videoSource.stdout.pipe(ffmpeg.stdin);
ytdlp.stderr.setEncoding("utf8"); const pipes: Array<Readable | Writable | Duplex> = [
ytdlp.stderr.on("data", (chunk: string) => log.warn({ ytdlp: chunk.trim() }, "video download")); videoSource.stdout,
ffmpeg.stdin,
ffmpeg.stdout,
videoOut,
];
if (audioSource) {
const audioIn = ffmpeg.stdio[3] as Duplex;
audioSource.stdout.pipe(audioIn);
pipes.push(audioSource.stdout, audioIn);
}
const children: ChildProcess[] = [videoSource, ffmpeg];
if (audioSource) children.push(audioSource);
for (const [name, child] of [
["video", videoSource],
["audio", audioSource],
] as const) {
if (!child) continue;
child.stderr.setEncoding("utf8");
child.stderr.on("data", (chunk: string) =>
log.warn({ stream: name, ytdlp: chunk.trim() }, "video pipeline download"),
);
}
ffmpeg.stderr.setEncoding("utf8"); ffmpeg.stderr.setEncoding("utf8");
ffmpeg.stderr.on("data", (chunk: string) => log.warn({ ffmpeg: chunk.trim() }, "ffmpeg")); ffmpeg.stderr.on("data", (chunk: string) => log.warn({ ffmpeg: chunk.trim() }, "ffmpeg"));
// A closed pipe when the other side goes away is expected, not an error. // A closed pipe when the other side goes away is expected, not an error.
for (const stream of [ytdlp.stdout, ffmpeg.stdin, ffmpeg.stdout]) { for (const stream of pipes) {
stream.on("error", (err: NodeJS.ErrnoException) => { stream.on("error", (err: NodeJS.ErrnoException) => {
if (err.code !== "EPIPE") log.debug({ err }, "pipeline stream error"); if (err.code !== "EPIPE") log.debug({ err }, "pipeline stream error");
}); });
@@ -108,14 +145,15 @@ export function openVideoPipeline(pageUrl: string, seekSeconds = 0): VideoPipeli
return { return {
audio: ffmpeg.stdout, audio: ffmpeg.stdout,
audioInputOptions: ["-f", "s16le", "-ar", "48000", "-ac", "2"], audioInputOptions: ["-f", "s16le", "-ar", "48000", "-ac", "2"],
video: ffmpeg.stdio[3] as Readable, video: videoOut,
width, width,
height, height,
kill: () => { kill: () => {
if (killed) return; if (killed) return;
killed = true; killed = true;
if (ytdlp.exitCode === null) ytdlp.kill("SIGKILL"); for (const child of children) {
if (ffmpeg.exitCode === null) ffmpeg.kill("SIGKILL"); if (child.exitCode === null) child.kill("SIGKILL");
}
}, },
}; };
} }
src/sources/ytdlp.ts
+30 -16
View File
@@ -346,43 +346,57 @@ export function downloadArgs(): string[] {
return baseArgs({ download: true }); return baseArgs({ download: true });
} }
export type VideoMode = "progressive" | "split";
/** /**
* Checks whether YouTube still offers a progressive format (video and audio in * Decides how a clip can be played with picture, in one lookup:
* one file) at this size. Only those can be streamed: anything that needs *
* merging makes yt-dlp download both streams in full before writing a byte, and * - "progressive" — one file carries both streams. Cheapest: a single download
* direct CDN URLs are no longer fetchable by ffmpeg — YouTube stalls them. * and a single decode.
* - "split" — video and audio come separately, so they are downloaded in
* parallel and paired by ffmpeg. YouTube increasingly offers only this.
* - null — no usable picture; the track plays as audio.
*/ */
export async function hasProgressiveVideo(pageUrl: string, maxHeight: number): Promise<boolean> { export async function selectVideoMode(pageUrl: string, maxHeight: number): Promise<VideoMode | null> {
try { try {
const { stdout } = await runYtDlp([ const { stdout } = await runYtDlp([
...baseArgs({ download: true }), ...baseArgs({ download: true }),
"-f", "-f",
progressiveFormat(maxHeight), `b[height<=${maxHeight}]/bv*[height<=${maxHeight}]/b`,
"--no-playlist", "--no-playlist",
"--print", "--print",
"%(format_id)s", "%(format_id)s|%(acodec)s|%(vcodec)s",
pageUrl, pageUrl,
]); ]);
const formatId = stdout.trim(); const [formatId, acodec, vcodec] = stdout.trim().split("|");
if (!formatId) { if (!formatId || !vcodec || vcodec === "none") {
log.warn({ pageUrl }, "no progressive format offered, playing audio only"); log.warn({ pageUrl }, "no video format offered, playing audio only");
return false; return null;
} }
log.info({ pageUrl, formatId }, "progressive format for video playback"); const mode: VideoMode = acodec && acodec !== "none" ? "progressive" : "split";
return true; log.info({ pageUrl, formatId, mode }, "video format selected");
return mode;
} catch (err) { } catch (err) {
// Worth seeing: this is the difference between "clip plays" and "sound only". // Worth seeing: this is the difference between "clip plays" and "sound only".
const reason = err instanceof Error ? err.message : String(err); const reason = err instanceof Error ? err.message : String(err);
log.warn({ pageUrl, reason }, "format lookup failed, playing audio only"); log.warn({ pageUrl, reason }, "format lookup failed, playing audio only");
return false; return null;
} }
} }
/** Best single-file format within the height budget. */ /** Single file with both streams, when YouTube still offers one. */
export function progressiveFormat(maxHeight: number): string { export function progressiveFormat(maxHeight: number): string {
return `b[height<=${maxHeight}]/b`; return `b[height<=${maxHeight}]`;
} }
/** Video-only stream, paired with AUDIO_FORMAT by ffmpeg. */
export function videoFormat(maxHeight: number): string {
return `bv*[height<=${maxHeight}]/b[height<=${maxHeight}]/b`;
}
/** Audio stream to pair with the picture. */
export const AUDIO_FORMAT = "ba/b";
/** Private, disposable copy of the cookie jar for concurrent reads. */ /** Private, disposable copy of the cookie jar for concurrent reads. */
async function copyCookies(): Promise<string | null> { async function copyCookies(): Promise<string | null> {
if (!config.YTDLP_COOKIES) return null; if (!config.YTDLP_COOKIES) return null;