mrleo1nid/stoat-mbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fall back to separate video and audio downloads

Browse Source 
Playing with video only worked for clips YouTube still serves as a single
progressive file, and it mostly does not: the log showed "Requested
format is not available" and every track fell back to sound only.

A progressive format is still preferred when offered — one download, one
decode — but when there is none, video and audio are fetched in parallel
and handed to one ffmpeg over separate pipes, which pairs them by
timestamp. This also lifts the 360p ceiling: the height is now a CPU
choice rather than a limit.

The extra-descriptor plumbing (two inputs, two outputs on one ffmpeg) was
verified against synthetic media: exactly 5.0s of PCM and 120 whole
frames at 24 fps, no partial tail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Leonid Pershin
2026-09-09 02:21:14 +03:00
co-authored by Claude Opus 5
parent a9b680c418
commit 9285fd7fbd
5 changed files with 370 additions and 313 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+3 -2
View File
@@ -77,8 +77,9 @@ YTDLP_JS_RUNTIME=node
# Показывать клип в голосовом канале как демонстрацию экрана (только YouTube).
# Требует прав Video у бота и включённого видео в конфигурации инстанса.
# YouTube отдаёт одним файлом только 360p, а стримить можно лишь такой формат —
# отсюда размер по умолчанию. Кодирование видео заметно грузит CPU сервера.
# 360p по умолчанию из осторожности: кодирование видео заметно грузит CPU
# сервера. Можно поднять до 720 (VIDEO_WIDTH=1280, VIDEO_HEIGHT=720), если
# машина позволяет.
# Это лишь разрешение: сам показ включается тумблером в панели или командой
# !video, и по умолчанию выключен. При false тумблер в панели не показывается.
VIDEO_ENABLED=false
README.md
+9 -5
View File
@@ -249,15 +249,19 @@ VIDEO_FPS=24
Как это устроено и почему так:
- **Только YouTube и только 360p.** Стримить можно лишь прогрессивный формат — один файл со
звуком и картинкой. Раздельные дорожки (720p и выше) yt-dlp обязан сначала скачать целиком
и лишь потом склеить, то есть воспроизведение началось бы после полной загрузки. Отдавать
ffmpeg прямые ссылки на CDN тоже нельзя: YouTube их для сторонних клиентов подвешивает.
- **Только YouTube.** Для остальных источников картинки нет, трек играет звуком.
- **Два способа получить картинку.** Если YouTube отдаёт прогрессивный формат (один файл со
звуком и видео) — используется он: одна загрузка, один декодер. Такие форматы встречаются всё
реже, поэтому есть запасной путь: видео и звук качаются двумя процессами параллельно и
сводятся одним ffmpeg по таймкодам. Склейка средствами yt-dlp не годится — он скачивает оба
потока целиком, прежде чем выдать первый байт; прямые ссылки на CDN тоже: YouTube подвешивает
их для сторонних клиентов.
- **Один процесс ffmpeg, два выхода:** PCM для голосовой дорожки и сырые I420-кадры для видео.
Темп задаёт `-re`, иначе кадры улетали бы вперёд звука и съедали память — распакованный кадр
720p весит 1.4 МБ.
- **Цена.** Кодирование видео ложится на CPU сервера и держится всё время трека, в отличие от
почти бесплатного звука. Ставьте `VIDEO_FPS` пониже, если нагрузка мешает.
почти бесплатного звука. 360p по умолчанию выбран из осторожности; поднимайте до 720p, если
машина тянет, и снижайте `VIDEO_FPS`, если нагрузка мешает.
- Живые трансляции и не-YouTube источники играют звуком, как раньше.
## Прокси, когда YouTube недоступен
src/sources/index.ts
+250 -250
View File
@@ -1,250 +1,250 @@
import type { Readable } from "node:stream";
import { config } from "../config.js";
import { logger } from "../logger.js";
import { UserFacingError, type Requester, type SearchResult, type Track } from "../types.js";
import * as direct from "./direct.js";
import * as local from "./local.js";
import { openVideoPipeline } from "./video-pipeline.js";
import * as ytdlp from "./ytdlp.js";
export { checkAvailable as checkYtDlp, checkCookies, checkProxy, describeProxy } from "./ytdlp.js";
export { isEnabled as isLocalLibraryEnabled, listFiles as listLocalFiles } from "./local.js";
const log = logger.child({ mod: "sources" });
const YOUTUBE_HOSTS = ["youtube.com", "youtu.be", "music.youtube.com", "m.youtube.com"];
const SOUNDCLOUD_HOSTS = ["soundcloud.com", "on.soundcloud.com", "m.soundcloud.com"];
function asUrl(value: string): URL | null {
if (!/^https?:\/\//i.test(value)) return null;
try {
return new URL(value);
} catch {
return null;
}
}
function hostMatches(url: URL, hosts: string[]): boolean {
const host = url.hostname.replace(/^www\./, "");
return hosts.some((candidate) => host === candidate || host.endsWith(`.${candidate}`));
}
/**
* A YouTube link copied while a mix or playlist is open carries both `v` and
* `list`. People mean the track they were listening to — expanding the list
* would dump a whole radio station into the queue. The rule stays deliberately
* blunt (any `v=` means one track) so pasting a link is predictable; a
* `/playlist?list=…` URL is how you ask for the whole list.
*/
function isTrackInsidePlaylist(url: URL): boolean {
return Boolean(url.searchParams.get("v") && url.searchParams.get("list"));
}
interface ParsedQuery {
text: string;
forced: "youtube" | "soundcloud" | "local" | null;
}
function parsePrefix(raw: string): ParsedQuery {
const trimmed = raw.trim();
const match = /^(yt|youtube|sc|soundcloud|local|file):\s*(.+)$/is.exec(trimmed);
if (!match) return { text: trimmed, forced: null };
const [, prefix, rest] = match as unknown as [string, string, string];
const key = prefix.toLowerCase();
if (key === "sc" || key === "soundcloud") return { text: rest.trim(), forced: "soundcloud" };
if (key === "local" || key === "file") return { text: rest.trim(), forced: "local" };
return { text: rest.trim(), forced: "youtube" };
}
/** Turns whatever a user typed into a playable set of tracks. */
export async function resolveQuery(
rawQuery: string,
requestedBy: Requester,
maxTracks = config.MAX_QUEUE_SIZE,
): Promise<SearchResult> {
const { text, forced } = parsePrefix(rawQuery);
if (!text) throw new UserFacingError("Укажите название трека или ссылку");
if (forced === "local") {
const tracks = await local.search(text, config.SEARCH_RESULT_LIMIT, requestedBy);
if (tracks.length === 0) throw new UserFacingError("В медиатеке ничего не найдено");
return { tracks: tracks.slice(0, 1), playlist: null };
}
const url = asUrl(text);
if (url) {
const limit = Math.min(maxTracks, config.MAX_PLAYLIST_TRACKS);
const singleTrack = isTrackInsidePlaylist(url);
if (hostMatches(url, YOUTUBE_HOSTS) || hostMatches(url, SOUNDCLOUD_HOSTS)) {
return ytdlp.resolveUrl(text, requestedBy, limit, { singleTrack });
}
const probed = await direct.probe(text);
if (probed.isMedia) {
return { tracks: [direct.toTrack(text, requestedBy, probed)], playlist: null };
}
// Not a raw media URL — let yt-dlp try its extractors (Bandcamp, Vimeo, ...).
return ytdlp.resolveUrl(text, requestedBy, limit, { singleTrack });
}
if (local.isEnabled() && forced === null) {
const localHits = await local.search(text, 1, requestedBy);
if (localHits.length > 0 && localHits[0]) return { tracks: [localHits[0]], playlist: null };
}
const tracks = await searchWithFallback(text, forced, 1, requestedBy);
if (tracks.length === 0) throw new UserFacingError(NOTHING_FOUND);
return { tracks: tracks.slice(0, 1), playlist: null };
}
export type SearchSource = "all" | "youtube" | "soundcloud" | "local";
/** Round-robins two result lists so neither source buries the other. */
function interleave(a: Track[], b: Track[]): Track[] {
const merged: Track[] = [];
for (let i = 0; i < Math.max(a.length, b.length); i += 1) {
const first = a[i];
const second = b[i];
if (first) merged.push(first);
if (second) merged.push(second);
}
return merged;
}
/** Multi-result search used by the `search` command and the web panel. */
export async function searchTracks(
rawQuery: string,
requestedBy: Requester,
limit = config.SEARCH_RESULT_LIMIT,
source: SearchSource = "all",
): Promise<Track[]> {
const { text, forced } = parsePrefix(rawQuery);
if (!text) return [];
const url = asUrl(text);
if (url) {
const result = await resolveQuery(text, requestedBy);
return result.tracks;
}
// A prefix inside the query is an explicit instruction and outranks the picker.
const target: SearchSource = forced ?? source;
if (target === "local") return local.search(text, limit, requestedBy);
if (target === "youtube" || target === "soundcloud") {
return ytdlp.search(text, target, limit, requestedBy);
}
const [youtube, soundcloud, localHits] = await Promise.all([
ytdlp.search(text, "youtube", limit, requestedBy).catch(() => []),
ytdlp.search(text, "soundcloud", limit, requestedBy).catch(() => []),
local.isEnabled() ? local.search(text, 3, requestedBy).catch(() => []) : Promise.resolve([]),
]);
return [...localHits, ...interleave(youtube, soundcloud)].slice(0, limit);
}
/**
* YouTube silently returns nothing for queries its restricted mode dislikes —
* same query, same words, results in a browser but an empty list over the API.
* Falling back to SoundCloud rescues a good share of those.
*/
async function searchWithFallback(
text: string,
forced: "youtube" | "soundcloud" | "local" | null,
limit: number,
requestedBy: Requester,
): Promise<Track[]> {
const primary = forced === "soundcloud" ? "soundcloud" : "youtube";
const tracks = await ytdlp.search(text, primary, limit, requestedBy);
if (tracks.length > 0 || forced !== null) return tracks;
return ytdlp.search(text, "soundcloud", limit, requestedBy).catch(() => []);
}
export const NOTHING_FOUND =
"Ничего не найдено. YouTube иногда прячет результаты от ботов (например, из-за ограниченного режима) — попробуйте другие слова или вставьте ссылку на трек.";
export interface PlaybackInput {
/** Either a file path / URL for ffmpeg, or a piped stream. */
input: string | Readable;
inputOptions: string[];
cleanup(): void;
/** Resolves with a reason if the downloader died on its own, for reporting. */
failure?: Promise<string | null>;
/** Raw I420 frames to publish as a screen share, when video is on. */
video?: { stream: Readable; width: number; height: number };
}
export interface PlaybackOptions {
/** Publish the picture too; the server-wide switch still has to allow it. */
video?: boolean;
}
/** Only YouTube reliably carries a picture worth showing next to the audio. */
function canShowVideo(track: Track, wanted: boolean): boolean {
return wanted && config.VIDEO_ENABLED && track.source === "youtube" && !track.isLive;
}
const HTTP_RESILIENCE = [
"-reconnect", "1",
"-reconnect_streamed", "1",
"-reconnect_delay_max", "5",
];
/**
* ffmpeg speaks HTTP proxies only, and just for http(s) inputs — it has no SOCKS
* support. So an http(s) proxy is handed to ffmpeg for direct links, while
* anything else keeps ffmpeg off the network entirely (see openPlayback).
*/
function ffmpegProxyOptions(): string[] {
const proxy = config.YTDLP_PROXY;
if (!proxy || !/^https?:\/\//i.test(proxy)) return [];
return ["-http_proxy", proxy];
}
/** Opens an ffmpeg-compatible input for a track, optionally starting at an offset. */
export async function openPlayback(
track: Track,
seekSeconds = 0,
options: PlaybackOptions = {},
): Promise<PlaybackInput> {
const seekOptions = seekSeconds > 0 ? ["-ss", seekSeconds.toFixed(2)] : [];
if (track.source === "local") {
const filePath = await local.assertInsideLibrary(track.url);
return { input: filePath, inputOptions: seekOptions, cleanup: () => {} };
}
if (track.source === "direct") {
return {
input: track.url,
inputOptions: [...HTTP_RESILIENCE, ...ffmpegProxyOptions(), ...seekOptions],
cleanup: () => {},
};
}
// The format is checked before committing to the video pipeline: audio comes
// out of that same pipeline, so falling back afterwards would kill the sound.
if (
canShowVideo(track, options.video ?? false) &&
(await ytdlp.hasProgressiveVideo(track.url, config.VIDEO_HEIGHT))
) {
log.info({ title: track.title }, "playing with video");
const pipeline = openVideoPipeline(track.url, seekSeconds);
return {
input: pipeline.audio,
inputOptions: pipeline.audioInputOptions,
cleanup: () => pipeline.kill(),
video: { stream: pipeline.video, width: pipeline.width, height: pipeline.height },
};
}
// Everything goes through yt-dlp: YouTube stalls direct CDN URLs fetched by
// anything else, and ffmpeg would bypass a SOCKS proxy anyway. Seeking
// therefore costs a decode up to the offset instead of an HTTP range request.
const proc = ytdlp.openAudioStream(track.url);
return {
input: proc.stream,
inputOptions: seekOptions,
cleanup: () => proc.kill(),
failure: proc.failure,
};
}
import type { Readable } from "node:stream";
import { config } from "../config.js";
import { logger } from "../logger.js";
import { UserFacingError, type Requester, type SearchResult, type Track } from "../types.js";
import * as direct from "./direct.js";
import * as local from "./local.js";
import { openVideoPipeline } from "./video-pipeline.js";
import * as ytdlp from "./ytdlp.js";
export { checkAvailable as checkYtDlp, checkCookies, checkProxy, describeProxy } from "./ytdlp.js";
export { isEnabled as isLocalLibraryEnabled, listFiles as listLocalFiles } from "./local.js";
const log = logger.child({ mod: "sources" });
const YOUTUBE_HOSTS = ["youtube.com", "youtu.be", "music.youtube.com", "m.youtube.com"];
const SOUNDCLOUD_HOSTS = ["soundcloud.com", "on.soundcloud.com", "m.soundcloud.com"];
function asUrl(value: string): URL | null {
if (!/^https?:\/\//i.test(value)) return null;
try {
return new URL(value);
} catch {
return null;
}
}
function hostMatches(url: URL, hosts: string[]): boolean {
const host = url.hostname.replace(/^www\./, "");
return hosts.some((candidate) => host === candidate || host.endsWith(`.${candidate}`));
}
/**
* A YouTube link copied while a mix or playlist is open carries both `v` and
* `list`. People mean the track they were listening to — expanding the list
* would dump a whole radio station into the queue. The rule stays deliberately
* blunt (any `v=` means one track) so pasting a link is predictable; a
* `/playlist?list=…` URL is how you ask for the whole list.
*/
function isTrackInsidePlaylist(url: URL): boolean {
return Boolean(url.searchParams.get("v") && url.searchParams.get("list"));
}
interface ParsedQuery {
text: string;
forced: "youtube" | "soundcloud" | "local" | null;
}
function parsePrefix(raw: string): ParsedQuery {
const trimmed = raw.trim();
const match = /^(yt|youtube|sc|soundcloud|local|file):\s*(.+)$/is.exec(trimmed);
if (!match) return { text: trimmed, forced: null };
const [, prefix, rest] = match as unknown as [string, string, string];
const key = prefix.toLowerCase();
if (key === "sc" || key === "soundcloud") return { text: rest.trim(), forced: "soundcloud" };
if (key === "local" || key === "file") return { text: rest.trim(), forced: "local" };
return { text: rest.trim(), forced: "youtube" };
}
/** Turns whatever a user typed into a playable set of tracks. */
export async function resolveQuery(
rawQuery: string,
requestedBy: Requester,
maxTracks = config.MAX_QUEUE_SIZE,
): Promise<SearchResult> {
const { text, forced } = parsePrefix(rawQuery);
if (!text) throw new UserFacingError("Укажите название трека или ссылку");
if (forced === "local") {
const tracks = await local.search(text, config.SEARCH_RESULT_LIMIT, requestedBy);
if (tracks.length === 0) throw new UserFacingError("В медиатеке ничего не найдено");
return { tracks: tracks.slice(0, 1), playlist: null };
}
const url = asUrl(text);
if (url) {
const limit = Math.min(maxTracks, config.MAX_PLAYLIST_TRACKS);
const singleTrack = isTrackInsidePlaylist(url);
if (hostMatches(url, YOUTUBE_HOSTS) || hostMatches(url, SOUNDCLOUD_HOSTS)) {
return ytdlp.resolveUrl(text, requestedBy, limit, { singleTrack });
}
const probed = await direct.probe(text);
if (probed.isMedia) {
return { tracks: [direct.toTrack(text, requestedBy, probed)], playlist: null };
}
// Not a raw media URL — let yt-dlp try its extractors (Bandcamp, Vimeo, ...).
return ytdlp.resolveUrl(text, requestedBy, limit, { singleTrack });
}
if (local.isEnabled() && forced === null) {
const localHits = await local.search(text, 1, requestedBy);
if (localHits.length > 0 && localHits[0]) return { tracks: [localHits[0]], playlist: null };
}
const tracks = await searchWithFallback(text, forced, 1, requestedBy);
if (tracks.length === 0) throw new UserFacingError(NOTHING_FOUND);
return { tracks: tracks.slice(0, 1), playlist: null };
}
export type SearchSource = "all" | "youtube" | "soundcloud" | "local";
/** Round-robins two result lists so neither source buries the other. */
function interleave(a: Track[], b: Track[]): Track[] {
const merged: Track[] = [];
for (let i = 0; i < Math.max(a.length, b.length); i += 1) {
const first = a[i];
const second = b[i];
if (first) merged.push(first);
if (second) merged.push(second);
}
return merged;
}
/** Multi-result search used by the `search` command and the web panel. */
export async function searchTracks(
rawQuery: string,
requestedBy: Requester,
limit = config.SEARCH_RESULT_LIMIT,
source: SearchSource = "all",
): Promise<Track[]> {
const { text, forced } = parsePrefix(rawQuery);
if (!text) return [];
const url = asUrl(text);
if (url) {
const result = await resolveQuery(text, requestedBy);
return result.tracks;
}
// A prefix inside the query is an explicit instruction and outranks the picker.
const target: SearchSource = forced ?? source;
if (target === "local") return local.search(text, limit, requestedBy);
if (target === "youtube" || target === "soundcloud") {
return ytdlp.search(text, target, limit, requestedBy);
}
const [youtube, soundcloud, localHits] = await Promise.all([
ytdlp.search(text, "youtube", limit, requestedBy).catch(() => []),
ytdlp.search(text, "soundcloud", limit, requestedBy).catch(() => []),
local.isEnabled() ? local.search(text, 3, requestedBy).catch(() => []) : Promise.resolve([]),
]);
return [...localHits, ...interleave(youtube, soundcloud)].slice(0, limit);
}
/**
* YouTube silently returns nothing for queries its restricted mode dislikes —
* same query, same words, results in a browser but an empty list over the API.
* Falling back to SoundCloud rescues a good share of those.
*/
async function searchWithFallback(
text: string,
forced: "youtube" | "soundcloud" | "local" | null,
limit: number,
requestedBy: Requester,
): Promise<Track[]> {
const primary = forced === "soundcloud" ? "soundcloud" : "youtube";
const tracks = await ytdlp.search(text, primary, limit, requestedBy);
if (tracks.length > 0 || forced !== null) return tracks;
return ytdlp.search(text, "soundcloud", limit, requestedBy).catch(() => []);
}
export const NOTHING_FOUND =
"Ничего не найдено. YouTube иногда прячет результаты от ботов (например, из-за ограниченного режима) — попробуйте другие слова или вставьте ссылку на трек.";
export interface PlaybackInput {
/** Either a file path / URL for ffmpeg, or a piped stream. */
input: string | Readable;
inputOptions: string[];
cleanup(): void;
/** Resolves with a reason if the downloader died on its own, for reporting. */
failure?: Promise<string | null>;
/** Raw I420 frames to publish as a screen share, when video is on. */
video?: { stream: Readable; width: number; height: number };
}
export interface PlaybackOptions {
/** Publish the picture too; the server-wide switch still has to allow it. */
video?: boolean;
}
/** Only YouTube reliably carries a picture worth showing next to the audio. */
function canShowVideo(track: Track, wanted: boolean): boolean {
return wanted && config.VIDEO_ENABLED && track.source === "youtube" && !track.isLive;
}
const HTTP_RESILIENCE = [
"-reconnect", "1",
"-reconnect_streamed", "1",
"-reconnect_delay_max", "5",
];
/**
* ffmpeg speaks HTTP proxies only, and just for http(s) inputs — it has no SOCKS
* support. So an http(s) proxy is handed to ffmpeg for direct links, while
* anything else keeps ffmpeg off the network entirely (see openPlayback).
*/
function ffmpegProxyOptions(): string[] {
const proxy = config.YTDLP_PROXY;
if (!proxy || !/^https?:\/\//i.test(proxy)) return [];
return ["-http_proxy", proxy];
}
/** Opens an ffmpeg-compatible input for a track, optionally starting at an offset. */
export async function openPlayback(
track: Track,
seekSeconds = 0,
options: PlaybackOptions = {},
): Promise<PlaybackInput> {
const seekOptions = seekSeconds > 0 ? ["-ss", seekSeconds.toFixed(2)] : [];
if (track.source === "local") {
const filePath = await local.assertInsideLibrary(track.url);
return { input: filePath, inputOptions: seekOptions, cleanup: () => {} };
}
if (track.source === "direct") {
return {
input: track.url,
inputOptions: [...HTTP_RESILIENCE, ...ffmpegProxyOptions(), ...seekOptions],
cleanup: () => {},
};
}
// The format is checked before committing to the video pipeline: audio comes
// out of that same pipeline, so falling back afterwards would kill the sound.
const videoMode = canShowVideo(track, options.video ?? false)
? await ytdlp.selectVideoMode(track.url, config.VIDEO_HEIGHT)
: null;
if (videoMode) {
log.info({ title: track.title, mode: videoMode }, "playing with video");
const pipeline = openVideoPipeline(track.url, seekSeconds, videoMode);
return {
input: pipeline.audio,
inputOptions: pipeline.audioInputOptions,
cleanup: () => pipeline.kill(),
video: { stream: pipeline.video, width: pipeline.width, height: pipeline.height },
};
}
// Everything goes through yt-dlp: YouTube stalls direct CDN URLs fetched by
// anything else, and ffmpeg would bypass a SOCKS proxy anyway. Seeking
// therefore costs a decode up to the offset instead of an HTTP range request.
const proc = ytdlp.openAudioStream(track.url);
return {
input: proc.stream,
inputOptions: seekOptions,
cleanup: () => proc.kill(),
failure: proc.failure,
};
}
src/sources/video-pipeline.ts
+78 -40
View File
@@ -1,9 +1,9 @@
import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process";
import { spawn, type ChildProcess, type ChildProcessWithoutNullStreams } from "node:child_process";
import { createRequire } from "node:module";
import type { Readable } from "node:stream";
import type { Duplex, Readable, Writable } from "node:stream";
import { config } from "../config.js";
import { logger } from "../logger.js";
import { downloadArgs, progressiveFormat } from "./ytdlp.js";
import { AUDIO_FORMAT, downloadArgs, progressiveFormat, videoFormat, type VideoMode } from "./ytdlp.js";
const require = createRequire(import.meta.url);
/** ffmpeg-static ships the binary revoice.js already relies on. */
@@ -23,33 +23,35 @@ export interface VideoPipeline {
kill(): void;
}
/**
* One download, one decode, two outputs: PCM for the voice track and I420 frames
* for the screen share.
*
* yt-dlp does the fetching (so cookies and the proxy apply as everywhere else)
* and hands ffmpeg a progressive stream. `-re` paces the decode at playback
* speed: it keeps picture and sound together and stops the video from racing
* ahead — a decoded 720p frame is 1.4 MB, so an unpaced stream would eat memory
* in seconds.
*/
export function openVideoPipeline(pageUrl: string, seekSeconds = 0): VideoPipeline {
const { VIDEO_WIDTH: width, VIDEO_HEIGHT: height, VIDEO_FPS: fps } = config;
const ytdlp = spawn(
function spawnDownload(pageUrl: string, format: string) {
return spawn(
config.YTDLP_PATH,
[
...downloadArgs(),
"-f",
progressiveFormat(height),
"--no-playlist",
"--quiet",
"-o",
"-",
pageUrl,
],
[...downloadArgs(), "-f", format, "--no-playlist", "--quiet", "-o", "-", pageUrl],
{ stdio: ["ignore", "pipe", "pipe"], windowsHide: true },
);
}
/**
* Decodes a clip into the two things a call needs: PCM for the voice track and
* raw I420 frames for the screen share, both out of a single ffmpeg.
*
* A progressive format (one file with both streams) is the cheap path — one
* download, one decode. When YouTube offers no such format, which is now the
* common case, picture and sound are downloaded in parallel and fed to ffmpeg
* over separate pipes, and it pairs them by timestamp.
*
* Everything is fetched by yt-dlp, so cookies and the proxy apply as everywhere
* else — handing CDN URLs to ffmpeg instead does not work, YouTube stalls them.
* `-re` paces the decode at playback speed, which keeps the two in step and
* stops the video from racing ahead: a decoded 720p frame is 1.4 MB, so an
* unpaced stream would eat memory in seconds.
*/
export function openVideoPipeline(pageUrl: string, seekSeconds = 0, mode: VideoMode = "split"): VideoPipeline {
const { VIDEO_WIDTH: width, VIDEO_HEIGHT: height, VIDEO_FPS: fps } = config;
const split = mode === "split";
const videoSource = spawnDownload(pageUrl, split ? videoFormat(height) : progressiveFormat(height));
const audioSource = split ? spawnDownload(pageUrl, AUDIO_FORMAT) : null;
const filters = [
`scale=${width}:${height}:force_original_aspect_ratio=decrease`,
@@ -58,19 +60,25 @@ export function openVideoPipeline(pageUrl: string, seekSeconds = 0): VideoPipeli
"format=yuv420p",
].join(",");
const seek = seekSeconds > 0 ? ["-ss", seekSeconds.toFixed(2)] : [];
// Split: fd 0 video in, fd 3 audio in, fd 4 frames out.
// Progressive: fd 0 everything in, fd 3 frames out.
const videoOutFd = split ? 4 : 3;
const ffmpeg = spawn(
FFMPEG_PATH,
[
"-hide_banner",
"-loglevel",
"error",
...(seekSeconds > 0 ? ["-ss", seekSeconds.toFixed(2)] : []),
...seek,
"-re",
"-i",
"pipe:0",
...(split ? [...seek, "-re", "-i", "pipe:3"] : []),
// Audio branch → stdout.
"-map",
"0:a:0?",
split ? "1:a:0" : "0:a:0",
"-f",
"s16le",
"-ar",
@@ -78,27 +86,56 @@ export function openVideoPipeline(pageUrl: string, seekSeconds = 0): VideoPipeli
"-ac",
"2",
"pipe:1",
// Video branch → the extra pipe on fd 3.
// Video branch → its own pipe.
"-map",
"0:v:0",
"-vf",
filters,
"-f",
"rawvideo",
"pipe:3",
`pipe:${videoOutFd}`,
],
{ stdio: ["pipe", "pipe", "pipe", "pipe"], windowsHide: true },
) as ChildProcessWithoutNullStreams & { stdio: Readable[] };
{
stdio: split
? ["pipe", "pipe", "pipe", "pipe", "pipe"]
: ["pipe", "pipe", "pipe", "pipe"],
windowsHide: true,
},
) as ChildProcessWithoutNullStreams & { stdio: Array<Duplex & Readable> };
ytdlp.stdout.pipe(ffmpeg.stdin);
const videoOut = ffmpeg.stdio[videoOutFd] as Readable;
videoSource.stdout.pipe(ffmpeg.stdin);
ytdlp.stderr.setEncoding("utf8");
ytdlp.stderr.on("data", (chunk: string) => log.warn({ ytdlp: chunk.trim() }, "video download"));
const pipes: Array<Readable | Writable | Duplex> = [
videoSource.stdout,
ffmpeg.stdin,
ffmpeg.stdout,
videoOut,
];
if (audioSource) {
const audioIn = ffmpeg.stdio[3] as Duplex;
audioSource.stdout.pipe(audioIn);
pipes.push(audioSource.stdout, audioIn);
}
const children: ChildProcess[] = [videoSource, ffmpeg];
if (audioSource) children.push(audioSource);
for (const [name, child] of [
["video", videoSource],
["audio", audioSource],
] as const) {
if (!child) continue;
child.stderr.setEncoding("utf8");
child.stderr.on("data", (chunk: string) =>
log.warn({ stream: name, ytdlp: chunk.trim() }, "video pipeline download"),
);
}
ffmpeg.stderr.setEncoding("utf8");
ffmpeg.stderr.on("data", (chunk: string) => log.warn({ ffmpeg: chunk.trim() }, "ffmpeg"));
// A closed pipe when the other side goes away is expected, not an error.
for (const stream of [ytdlp.stdout, ffmpeg.stdin, ffmpeg.stdout]) {
for (const stream of pipes) {
stream.on("error", (err: NodeJS.ErrnoException) => {
if (err.code !== "EPIPE") log.debug({ err }, "pipeline stream error");
});
@@ -108,14 +145,15 @@ export function openVideoPipeline(pageUrl: string, seekSeconds = 0): VideoPipeli
return {
audio: ffmpeg.stdout,
audioInputOptions: ["-f", "s16le", "-ar", "48000", "-ac", "2"],
video: ffmpeg.stdio[3] as Readable,
video: videoOut,
width,
height,
kill: () => {
if (killed) return;
killed = true;
if (ytdlp.exitCode === null) ytdlp.kill("SIGKILL");
if (ffmpeg.exitCode === null) ffmpeg.kill("SIGKILL");
for (const child of children) {
if (child.exitCode === null) child.kill("SIGKILL");
}
},
};
}
src/sources/ytdlp.ts
+30 -16
View File
@@ -346,43 +346,57 @@ export function downloadArgs(): string[] {
return baseArgs({ download: true });
}
export type VideoMode = "progressive" | "split";
/**
* Checks whether YouTube still offers a progressive format (video and audio in
* one file) at this size. Only those can be streamed: anything that needs
* merging makes yt-dlp download both streams in full before writing a byte, and
* direct CDN URLs are no longer fetchable by ffmpeg — YouTube stalls them.
* Decides how a clip can be played with picture, in one lookup:
*
* - "progressive" — one file carries both streams. Cheapest: a single download
* and a single decode.
* - "split" — video and audio come separately, so they are downloaded in
* parallel and paired by ffmpeg. YouTube increasingly offers only this.
* - null — no usable picture; the track plays as audio.
*/
export async function hasProgressiveVideo(pageUrl: string, maxHeight: number): Promise<boolean> {
export async function selectVideoMode(pageUrl: string, maxHeight: number): Promise<VideoMode | null> {
try {
const { stdout } = await runYtDlp([
...baseArgs({ download: true }),
"-f",
progressiveFormat(maxHeight),
`b[height<=${maxHeight}]/bv*[height<=${maxHeight}]/b`,
"--no-playlist",
"--print",
"%(format_id)s",
"%(format_id)s|%(acodec)s|%(vcodec)s",
pageUrl,
]);
const formatId = stdout.trim();
if (!formatId) {
log.warn({ pageUrl }, "no progressive format offered, playing audio only");
return false;
const [formatId, acodec, vcodec] = stdout.trim().split("|");
if (!formatId || !vcodec || vcodec === "none") {
log.warn({ pageUrl }, "no video format offered, playing audio only");
return null;
}
log.info({ pageUrl, formatId }, "progressive format for video playback");
return true;
const mode: VideoMode = acodec && acodec !== "none" ? "progressive" : "split";
log.info({ pageUrl, formatId, mode }, "video format selected");
return mode;
} catch (err) {
// Worth seeing: this is the difference between "clip plays" and "sound only".
const reason = err instanceof Error ? err.message : String(err);
log.warn({ pageUrl, reason }, "format lookup failed, playing audio only");
return false;
return null;
}
}
/** Best single-file format within the height budget. */
/** Single file with both streams, when YouTube still offers one. */
export function progressiveFormat(maxHeight: number): string {
return `b[height<=${maxHeight}]/b`;
return `b[height<=${maxHeight}]`;
}
/** Video-only stream, paired with AUDIO_FORMAT by ffmpeg. */
export function videoFormat(maxHeight: number): string {
return `bv*[height<=${maxHeight}]/b[height<=${maxHeight}]/b`;
}
/** Audio stream to pair with the picture. */
export const AUDIO_FORMAT = "ba/b";
/** Private, disposable copy of the cookie jar for concurrent reads. */
async function copyCookies(): Promise<string | null> {
if (!config.YTDLP_COOKIES) return null;