Frame size alone does not make a picture sharp: LiveKit picks a
conservative bitrate for whatever resolution it is given, so 720p could
well look worse than a well-fed 480p. The encoder ceiling and codec are
now configurable, and the frame queue's memory budget with them, since
raising the resolution multiplies both the bytes per frame and the wait.
Startup logs what the track was published with, and the README explains
which knob to move first and what in the "video sync" line says the
machine has run out of headroom.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The diagnostics came back clean — aheadBy steady at 3.3 s, nothing
dropped, nothing re-pegged — so the remaining lip-sync error was a
constant, and the cause is the clock: media.seconds counts audio handed
to LiveKit, whose audio source holds up to a second (its default queue)
before anyone hears it. Frames released against that ran a queue-length
early, every time.
The clock now subtracts what is still queued, which the source reports
directly, so the compensation follows the real buffer instead of a
guessed constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The drift kept growing because the video branch could not keep real time:
VIDEO_HEIGHT is a bounding box, but it was also used to pick the source
format, so a 640x320 picture was decoded from a 720p stream — four times
the work for the same result. The download now asks for the smallest
standard rung that covers the output height.
Dropping late frames only helps when lateness is momentary; when the
source runs slower than the sound, every frame is late and the gap grows
without bound. The timeline is now re-pegged to the current audio
position instead: the picture jumps once and holds, so desync stays
bounded whatever the machine can manage.
A "video sync" line every ten seconds reports where each side is, how
many frames are queued, dropped and re-pegged, which is what separates a
slow decoder from a wrong frame rate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gating frames on absolute stream position overshot in the other
direction: audio starts flowing before the first frame is decoded, so
pinning frame 0 to audio second 0 puts the whole clip permanently in
debt, and frames arrive already past due.
The first frame now records the audio position it arrived at, and the
rest keep their spacing from there — the drift correction stays, the
startup difference is no longer treated as lateness. That offset is
logged, so the residual can be trimmed with VIDEO_SYNC_OFFSET_MS, which
the publisher now applies itself instead of ffmpeg (one mechanism, and it
works in both directions).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With the codec fixed the picture stopped stuttering but ran ahead of the
sound, and for a structural reason: the audio takes a longer road to the
call — our PCM goes through revoice's own ffmpeg and its buffer — while
frames went out the moment they were decoded.
Each frame now carries its position in the stream and waits until the
audio that belongs with it has actually played, using the player's own
playback position as the shared clock. Frames that fall more than 250 ms
behind are dropped rather than shown late, so the picture recovers by
itself instead of accumulating drift.
The queue is bounded by bytes and never by pausing the stream: one ffmpeg
feeds both outputs, so blocking the video pipe would stop the audio whose
clock we are waiting on — a deadlock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The panel still claimed it could not see anyone in voice while the bot
and the user were sitting in the same channel: Stoat's gateway does not
send voice states to bots, so the SDK's participant list stays empty no
matter what.
While the bot is connected its LiveKit room knows exactly who is there,
so that is now consulted alongside the SDK — presence is reported as
known, the panel names the channel the user is in, and the listener rule
has real data to work with instead of falling back to trust.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AlreadyConnected kept coming back after every rebuild. Shutdown closed
the HTTP server first, which blocks while the panel's WebSocket is open,
so Docker's grace period ran out and the process was killed before it
ever left the voice channels — and Stoat only clears a bot's voice state
when it disconnects, leaving the stale entry that refuses the next join.
Voice channels are now left first, and each shutdown step has its own
deadline so neither can eat the grace period.
A failed join_call also escaped as an unhandled rejection with a full
stack, because revoice awaits it inside an async executor it never
guards; the error is already latched and reported, so that call no longer
settles at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The panel said "you are not in a voice channel" while the client showed
the user sitting in the call, which also disabled the only button that
could summon the bot. The SDK learns voice participants from gateway
events alone — no endpoint to ask — so a bot that restarted, or missed a
VoiceChannelJoin, sees every channel as empty forever, and that looks
exactly like nobody listening.
The rule now refuses only on proven absence: when the bot can see people
in voice somewhere in the server and the requester is not among them.
With no presence data at all it accepts an explicitly chosen channel and
logs that it is trusting the request, and the panel falls back to the
channel picker instead of a dead button.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three problems visible at once on a running instance: the picture
stuttered and drifted behind the sound, it was letterboxed oddly, and it
appeared as a separate tile instead of coming from the bot.
- YouTube handed us AV1 (format 398). Software-decoding AV1 at 720p does
not sustain real time on a small server, which explains both the
stutter and the drift; H.264 is now requested first, VP9 second.
- The frame was padded into a fixed box, so a clip whose proportions
differed got black bars baked in and then more from the client. Size is
now a bounding box and the frame keeps the clip's own proportions.
- Video was published as a screen share, which every client renders as
its own tile. VIDEO_SOURCE=camera (the new default) puts it inside the
bot's tile; "screen" keeps the old behaviour.
VIDEO_SYNC_OFFSET_MS is there for the residual drift, since audio and
video travel as two separately published tracks. Measured loudnorm first
to rule it out as the cause of the desync: it adds 0 ms.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Playing with video only worked for clips YouTube still serves as a single
progressive file, and it mostly does not: the log showed "Requested
format is not available" and every track fell back to sound only.
A progressive format is still preferred when offered — one download, one
decode — but when there is none, video and audio are fetched in parallel
and handed to one ffmpeg over separate pipes, which pairs them by
timestamp. This also lifts the 360p ceiling: the height is now a CPU
choice rather than a limit.
The extra-descriptor plumbing (two inputs, two outputs on one ffmpeg) was
verified against synthetic media: exactly 5.0s of PCM and 120 whole
frames at 24 fps, no partial tail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
VIDEO_ENABLED is now permission rather than behaviour: it decides whether
the feature exists at all, while turning it on for a server is a toggle in
the panel or !video in chat, off by default. Video costs real CPU for
every playing channel, so that should be a deliberate choice rather than
something a config flag switches on everywhere.
With the env flag off the panel renders no toggle at all and !video says
so, and the switch applies from the next track — swapping tracks mid-play
would cut the current one.
The loop button no longer reads "выкл" either: it sat next to the video
button showing the same word, so the two states were indistinguishable.
Both now name what they do and rely on highlighting for state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The screen share never started because openPlayback had no video branch
left: removing the old direct-URL seek path took it with it, since it sat
between the anchors used for that edit. canShowVideo and the pipeline
import stayed behind, and nothing failed to compile, so it looked wired
up while every track quietly took the audio-only path.
Verified end to end: openPlayback now returns a 640x360 track, format 18
is selected, and 12 seconds of playback yields 202 frames alongside the
audio.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Whether a clip gets a picture is decided by a format lookup whose failure
was only logged at debug, so an audio-only fallback was indistinguishable
from video being broken. The reason is now visible at the default log
level, and taking the video path says so too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The video path leaves no trace in the log when it is switched off, so a
missing VIDEO_ENABLED in an existing .env looks exactly like a broken
screen share. Startup now states which mode it is in, and at what size.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The bot stayed in a channel everyone had left. revoice.js stores voice
users by participant identity but removes them by participant name — and
Stoat sets those to different things (the user id and username#tag) — so
its list never shrinks and the channel never looked empty to us.
Participants are now counted from the LiveKit room directly, and the
check also runs on a 30s interval, so a participant event that never
arrives cannot strand the bot either.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
VIDEO_ENABLED makes the bot publish a second LiveKit track with the
picture. Stoat only grants screen_share in the call token when the bot
has the Video permission and the instance has video enabled, so a refusal
is reported in chat and playback continues with sound alone.
Two constraints shaped the pipeline, both found by testing rather than
assumption:
- Only progressive formats can be streamed. Separate video+audio streams
make yt-dlp download both in full before muxing a single byte, and
direct CDN URLs handed to ffmpeg simply hang — YouTube no longer serves
them to other clients. That caps video at the 360p single file YouTube
offers, and the format is checked before committing to the video path,
since audio would otherwise come from the same broken pipeline.
- One ffmpeg with two outputs, paced by -re: an unpaced decode races
ahead of the sound and eats memory at 1.4 MB per frame.
The same CDN-URL finding removes the audio seek shortcut, which resolved
such a URL and would have hung the same way; seeking now decodes up to
the offset like it already did behind a proxy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
YouTube's bot checks started once traffic went through a proxy exit IP,
while direct downloads had worked. YTDLP_PROXY_SCOPE=search keeps the
proxy on search and metadata — where it is needed to get past filtered
results — and lets the audio stream go out directly. Default stays "all",
so nothing changes unless it is set.
Searches also now run against a throwaway copy of the cookie file: they
run in parallel and yt-dlp rewrites that file on exit, so two of them
could clobber the jar the downloads depend on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Downloads failed with "Sign in to confirm you're not a bot" even with
cookies and a working proxy. yt-dlp was also warning that no JavaScript
runtime was available: YouTube's player challenges now require one, and
without it extraction degrades into exactly those bot checks.
The image already ships Node, so yt-dlp is pointed at it via
--js-runtimes, but only after confirming the installed build understands
the flag — an older binary would otherwise fail on an unknown option.
Startup logs which runtime was picked and warns when there is none.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A proxy published as 127.0.0.1:1080 is reachable from the host and
nowhere else — not via host.docker.internal either, since loopback
publishing never leaves the host. The fix is a shared docker network and
addressing the proxy by container name, so deploy/ now carries a ready
compose.override.yml example for exactly that, parameterised by
PROXY_NETWORK.
compose.override.yml is gitignored so a local deployment tweak cannot
collide with future pulls, the way the domain edit did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A proxy pointed at 127.0.0.1 from inside a container reaches the
container itself, and the only sign was a connection error buried in the
first search. Startup now probes the proxy over TCP and says what is
wrong, naming host.docker.internal when loopback was configured.
The README also covers the follow-up trap: even that address fails when
the proxy listens on loopback only, so it shows how to check the bind
address and what to change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
compose.yml was tracked but had to be edited per deployment, so every
update collided with the local change ("your local changes would be
overwritten by merge"). The domain now comes from STOAT_DOMAIN via
compose interpolation, and host.docker.internal is mapped unconditionally
so a host-side proxy needs no edit either. Nothing in compose.yml has to
be touched any more.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
YTDLP_PROXY routes every yt-dlp call — search, metadata and the audio
stream, for YouTube and SoundCloud alike — through an http(s) or SOCKS
proxy such as a local Psiphon. Startup logs which proxy is in use with
any credentials stripped.
ffmpeg has no SOCKS support, so with a proxy configured playback always
goes through the yt-dlp pipe instead of a resolved CDN URL: nothing
escapes past the proxy, at the cost of slower seeking. Direct links keep
using ffmpeg, which gets the proxy only when it speaks http(s).
Verified against a dead proxy: requests fail through it rather than
quietly going direct.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A link copied from an open mix (watch?v=…&list=RD…) expanded into the
whole radio station — 500 entries from one paste. Any URL with a `v=`
parameter now resolves to that single track, and only a /playlist?list=…
URL expands. The rule is deliberately blunt rather than keyed on
start_radio, so pasting a link behaves the same way every time.
Playlist expansion is also capped separately from the queue limit
(MAX_PLAYLIST_TRACKS, 100 by default) and the chat says when a playlist
hit that ceiling.
Verified against the real yt-dlp: both mix links resolve to one track, a
plain video link to one, and a playlist URL to its 13 entries.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Errors reached people as raw tool output: "Unsupported URL:
https://...%D0%B0%D1%83..." says nothing to whoever pasted a link. The
common cases now map to plain descriptions with the next step where there
is one (age gate and bot checks point at YTDLP_COOKIES, a stale extractor
at updating yt-dlp), URLs are decoded, and anything unrecognised still
falls through verbatim.
The panel's prefix hint is gone now that the source picker does the same
job visibly; prefixes keep working and stay documented for chat.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Search now queries YouTube and SoundCloud together by default and
interleaves the two result lists so neither buries the other; a picker
left of the input narrows it to one source (plus the local library when
configured), and a prefix typed into the query still outranks it.
The panel was capped at 1180px, which left most of a wide screen empty —
it now scales to 1680px and gives the search column the extra room.
A dead link also reported "could not parse yt-dlp's response", which
described our parser rather than the problem; it now shows yt-dlp's own
error line, or says the link did not open.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
YouTube applies restricted-mode filtering to programmatic search: the
same query that returns a clip in a signed-in browser comes back as an
empty list over the API, with exit code 0 and no message at all, and
cookies do not change it. Verified locally — "gudium идол" returns three
results, "gudium аудиопорно" returns none, and so does the plain
youtube.com/results page for the same words.
An empty YouTube search now retries on SoundCloud, and when both come up
empty the message says the query may have been filtered and suggests
pasting a link, instead of implying the track does not exist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four things people hit while using the panel:
- The bot could be sent into a channel the requester was not in, and
playback could be started from nowhere. Playback now follows the
listener (REQUIRE_LISTENER, on by default), and the voice row shows
where you and the bot are instead of offering a free channel picker.
- Voice presence only refreshed on reload, because the SDK updates
channel participants without emitting an event. The socket now watches
that view and pushes changes.
- The bot left the channel whenever the queue ran dry. It now leaves only
after the last person does, EMPTY_TIMEOUT_SECONDS later (120 by
default), and stays put while anyone is still listening.
- A search that yielded nothing said nothing: yt-dlp can exit 0 with an
empty result, so that case now reports the reason (or "nothing found"),
and searches are logged with their result count.
The queue moved under the player so search owns the left column, and
elapsed time no longer renders as "LIVE" — formatDuration treated 0 as a
live stream, which also affected the chat's progress bar.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Switching tracks killed the process: revoice's #cleanUp() dereferences
this.fProc unconditionally, and its own ffmpeg error handler calls stop()
a second time after stop() has already nulled that field. Killing ffmpeg
is what triggers that error, so its handlers are detached first, the
instance's stop() is wrapped defensively (revoice calls it internally),
and an uncaughtException handler keeps the bot in the channel if the
dependency throws from another async callback.
Failed command-message deletions were logged at debug, i.e. invisible in
the default configuration; they now warn with Stoat's error type, and the
README says which permission the bot's role needs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The image shipped a year-old yt-dlp, which YouTube now rejects with
"The page needs to be reloaded". Bumped to 2026.08.19 and documented
rebuilding as the standard fix, including how to pass a newer tag
without waiting for a repository update.
A downloader dying mid-stream also looked exactly like a very short
track: ffmpeg saw EOF, the player advanced, and the channel only got
"queue finished". The failure reason now reaches the chat.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Joining a channel failed silently: revoice's join() runs an async
executor inside `new Promise`, so a rejected join_call never reaches
reject() — the promise hangs forever and the real error escapes as an
unhandled rejection. The client wrapper now latches API failures and
settles the join itself, translating Stoat's error codes (AlreadyConnected,
LiveKitUnavailable, UnknownNode, ...) into messages the chat can show.
A failed join also used to leave the connection object alive, which kept
the bot registered in the channel and made the next attempt fail with
AlreadyConnected; it is now destroyed on any failure. The LiveKit node
name is configurable via VOICE_NODE for instances that renamed it, and
the README documents how to clear a stuck voice state from Redis.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
revoice.js reads `room.isConnected()`, but @livekit/rtc-node turned that
into a getter, so every join threw "this.room.isConnected is not a
function" before the bot ever reached the channel. The player no longer
touches that getter: readiness comes from the connection's own join /
roomfetched events and is cleared when it reports the offline state.
Also delete the invoking chat message once a command is recognised
(DELETE_COMMAND_MESSAGES, on by default) so channels stay readable;
failures are non-fatal since it needs ManageMessages.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Cookies were already wired up but only mentioned in passing, and the
non-obvious parts were undocumented: yt-dlp has no username/password
support for YouTube, it rewrites the cookie file to persist rotated
cookies (so a read-only file expires early), and the export has to
happen in a private window that is logged out before closing.
Startup now reports whether the cookie file is usable, missing, or
read-only, and YTDLP_EXTRACTOR_ARGS is passed through for the cases
where YouTube blocks a server IP outright.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Cloning straight into /opt fails with "Permission denied", and a clone
left owned by root would also make the ./data bind mount unwritable for
the container's node user.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The instance's internal Caddy is only for Stoat itself, so the panel no
longer goes through it: the bot ships its own compose project, publishes
3005 on loopback, and the host's external Caddy gives it a domain.
extra_hosts pins the instance domain to host-gateway, so the bot reaches
the API, gateway and LiveKit through the external proxy with a valid
certificate instead of depending on router NAT loopback. The old
in-project layout stays available as a fallback example, together with a
step-by-step guide for when voice fails to connect.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Plays audio into Stoat voice channels over LiveKit and exposes the same
player through both chat commands and a browser panel, so the two never
drift apart: everything routes through a single MusicManager.
- core: per-server GuildPlayer (queue, loop, shuffle, seek, volume,
idle auto-leave) driving revoice.js/@livekit/rtc-node and ffmpeg
- sources: yt-dlp for YouTube/SoundCloud, direct media URLs and internet
radio, optional local library with path-traversal guards
- bot: 18 chat commands with aliases, plus !panel one-time login links
- api: Fastify REST + WebSocket, sessions authenticated against the
instance's own /auth/session/login (TOTP supported), permissions
re-checked against Stoat membership and roles on every request
- web: React panel with search, queue editing, seek and volume
- deploy: Dockerfile, compose.override.yml and Caddyfile snippets for
dropping the service into an existing /opt/stoat stack
Verified with npm run typecheck, both builds, and scripts/smoke-api.mjs
(9 API checks). Voice playback itself needs a live instance to test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
