Commit Graph
2 Commits
Author SHA1 Message Date
Leonid PershinandClaude Opus 5 fe62bcf53f Add collector settings and per-source purge
Every limit the fetcher was using was a constant. They are settings now, and
CollectOptions became the single place policy lives: AppSettings.ToCollectOptions
clamps them, and the HTTP layer's FetchOptions is projected from that. One
clamping site rather than two sets of ceilings drifting apart.

Clamping rather than validating, for the reason the proxy options already do it:
a hand-edited file must not stop the app from starting. A MaxItemBytes edited to
zero would otherwise refuse everything, and a zeroed concurrency would deadlock
the run outright - so both are pulled into range instead. An empty format filter
is read as "everything", because switching every format off is far more likely
to be a slip than an instruction to collect nothing.

The media root has an ordering problem - it is a setting that decides the paths
the container is built from - so the file is read once before the container
exists rather than making every path lazy for one value.

Purge is scoped to a source and lives on the Collect page, where the source is
already chosen. Content another source also holds survives, which is what the
index's reference count was for.

The showcase hint says out loud what a hard link means: editing the browsable
copy edits the original, and deleting it frees nothing until the last name goes.
That is surprising enough to belong in the UI rather than only in the code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 21:52:43 +03:00
Leonid PershinandClaude Opus 5 70fb3a1df3 Add the Collect page
The collector becomes usable. The page is a close copy of the parse page's
shape - same proxy gate, same batched flush to the observable collection, same
truncation cap that reports rather than truncates silently - because that shape
was built for streaming outcomes and the collector produces exactly those.

Two differences that are not cosmetic. The batch drops from 512 to 64: items
arrive at network speed, roughly one a second, and a batch of five hundred would
mean the list never visibly moved. And progress is explicitly indeterminate
until a source finishes listing, because a paginated listing genuinely does not
know its total until the last page - a bar pretending otherwise would be lying.

The input swaps shape with the source: an endpoint source wants one address, a
pasted-list source wants many lines. The proxy gate is unchanged in substance -
only network sources are gated, and the banner keeps its x:Name because the
headless tests find it that way.

Rendering the page caught a defect the tests could not: IsVisible sat on the
caption inside the header border rather than on the border, so hiding the text
left its padding and divider behind as an empty bar above the input.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 21:39:21 +03:00