390 lines
28 KiB
Markdown
390 lines
28 KiB
Markdown
# CLAUDE.md
|
||
|
||
Guidance for Claude Code (and other agents) working in this repository.
|
||
|
||
## Memory: EchoVault (read this first)
|
||
|
||
This project uses the **EchoVault** MCP server for persistent, cross-session memory.
|
||
Prior sessions store architectural decisions, fixed bugs, and gotchas there. Follow
|
||
this protocol every session:
|
||
|
||
1. **At session start — load context.** Call `memory_context` (project is
|
||
auto-detected from cwd) before doing any work. Use `memory_search` for specific
|
||
topics (e.g. "detector model", "classic-cv", "false positives").
|
||
2. **During work — search before re-deciding.** When the task touches an area that
|
||
may have prior context, `memory_search` it first instead of re-deriving decisions.
|
||
3. **Before ending a session — save what matters.** Call `memory_save` when you made
|
||
a design decision, fixed a bug (include root cause + fix), found a non-obvious
|
||
gotcha, or the user corrected/clarified a requirement. Pick the right `category`
|
||
(`decision` / `bug` / `pattern` / `learning` / `context`). Do **not** save trivia,
|
||
things obvious from the code, or duplicates.
|
||
|
||
EchoVault is the source of truth for *why* things are the way they are; this file is
|
||
the stable, high-level map. When they disagree, trust on-disk code first, then
|
||
EchoVault, then this file — and update whichever is stale.
|
||
|
||
## What this project is
|
||
|
||
**HVideoTool** is a Windows-first desktop GUI utility that **detects already-applied
|
||
censorship** (mosaic, pixelation, blur, black bars) in **images**, and draws outlines
|
||
over the detected censored regions. You open a **project** (see below); it runs each
|
||
image through a detector, draws the regions, and shows a detailed per-image list of
|
||
what it found.
|
||
|
||
> **Projects (this session).** Work is organized into **projects** — a project is a
|
||
> folder holding `project.json` (metadata + per-project settings) · `frames/` (the
|
||
> images) · `detections.json` (the detection cache) · `collections/Избранное` (the
|
||
> single default "favorites" collection). See `core/project.py`. The per-project settings (detector, model,
|
||
> threshold, restore engine) live in `project.json`; the global `settings.json` only
|
||
> seeds the **defaults for new projects**. The old "open a bare folder" flow is now
|
||
> "Импортировать папку как проект…" (copies images into a new project's `frames/`).
|
||
|
||
> **Scope is still narrow.** It used to extract frames from video, detect, and play
|
||
> back with overlays (a thread-based "project model"). That video *playback* pipeline
|
||
> was **removed**; the new "projects" are just an on-disk layout, NOT worker threads or
|
||
> playback. The tool remains a synchronous, single-threaded **image inspector**.
|
||
|
||
Keep this scope sharp:
|
||
|
||
- Primary job is **detection + overlay/inspection**. A **restoration** ("расцензурить")
|
||
step was added later (user-requested): on-demand (single frame **or** whole-project
|
||
batch into `restored/`), behind a `Restorer` interface. **Detection is YOLO-only and
|
||
restoration is DeepMosaics-only** — the noisy classic-CV detector (+ the `combined`
|
||
composite) and the cv2 `inpaint` baseline (filled but didn't reconstruct) were
|
||
**removed** as "works poorly". Detection is **multi-model** (ADetailer-style): drop YOLO
|
||
weights under `models/yolo/<category>/`, tick which ones are active in the toolbar
|
||
"Модели" menu, and a detect runs **all** ticked models and merges results (each tagged
|
||
with its category → its own overlay colour). DeepMosaics has two engines: **image** (per-frame) and
|
||
**video** (BVDNet, temporal — uses neighbour frames). Its GPL-3.0 network code is
|
||
**vendored** under `core/restore/_deepmosaics/` and run in-process (user supplies only
|
||
the weights). Because of that vendoring the **whole project is GPL-3.0**. LADA
|
||
(BasicVSR++) is a possible future engine, not wired.
|
||
- Still **no diffusion / ControlNet / SDXL**. (`xinsir/controlnet-union-sdxl-1.0` was
|
||
rejected early — a generative *conditioning* model, not a censorship restorer. Don't
|
||
reintroduce it.) Restoration, if upgraded, uses a mosaic-removal model (DeepMosaics/
|
||
LADA), not a general text-to-image diffusion pipeline.
|
||
- It detects **already-censored** regions, not "content that should be censored"
|
||
(i.e. not an NSFW classifier).
|
||
- Video is only a one-shot frame-extraction convenience (see below): "Создать из
|
||
ролика…" makes a new project and decodes the clip into its `frames/`. Detection and
|
||
restoration operate on the project's images.
|
||
|
||
## Target environment
|
||
|
||
- **OS:** Windows 11 x64 (primary). Use PowerShell syntax in commands.
|
||
- **Python:** 3.11+.
|
||
- **GPU:** NVIDIA + CUDA via PyTorch, for the YOLO detector and the DeepMosaics
|
||
restorer. CPU fallback works but is slow (esp. DeepMosaics / the temporal BVDNet).
|
||
|
||
## Tech stack (decided)
|
||
|
||
| Concern | Choice |
|
||
|----------------|---------------------------------------------|
|
||
| GUI | PySide6 (Qt 6) — LGPL |
|
||
| Image IO | OpenCV (`opencv-python`) + NumPy, unicode-safe via `core/imageio.py` |
|
||
| Detector | Ultralytics YOLO, multi-model ensemble (models/yolo/<category>/*.pt) |
|
||
|
||
Torch/CUDA + Ultralytics enter with the YOLO detector. Keep that dependency optional
|
||
(the `yolo` extra in `pyproject.toml` pulls only Ultralytics; torch is installed
|
||
separately per the README). Both detection (YOLO) and restoration (DeepMosaics) now
|
||
require torch — there's no longer a torch-free detector.
|
||
|
||
## Architecture (as implemented)
|
||
|
||
> This reflects the actual code on disk. The GUI is mostly synchronous, but the
|
||
> **heavy compute (detection + restoration) runs on a background thread** so the UI
|
||
> stays responsive — see `ui/workers.py` and the "Background jobs" bullet (this reverses
|
||
> the earlier "no worker threads" rule; `processEvents` can't unfreeze a single multi-
|
||
> second `detector.detect()`/DeepMosaics call). Work is organized into **projects**
|
||
> (`core/project.py`): a project folder holds `project.json` (metadata + per-project
|
||
> settings), `frames/` (the images), `detections.json` (the detection cache, at the
|
||
> project root — no longer a sidecar next to the images), and `collections/Избранное`
|
||
> (the favorites collection). The only video touch is a one-shot "Создать из ролика…"
|
||
> that creates a new project and decodes a clip into its `frames/` via the **ffmpeg
|
||
> CLI** (cv2.VideoCapture fallback; NOT PyAV). When code and this file disagree, trust
|
||
> the code.
|
||
|
||
```
|
||
hvideotool/
|
||
├── __main__.py # entry point + CLI (optional project path, --detector, --model)
|
||
├── app.py # QApplication bootstrap; run(config, target=None) — opens/auto-reopens a project
|
||
├── config.py # AppConfig + DetectionConfig/OverlayConfig (thresholds live here)
|
||
├── settings_store.py # new-project DEFAULTS + last/recent projects to ~/HVideoTool/settings.json
|
||
├── ui/
|
||
│ ├── main_window.py # the whole UI: toolbar + [file list | image view | detail table]
|
||
│ ├── workers.py # Job (QRunnable): runs detect/restore off-thread, results via Qt signals
|
||
│ └── image_view.py # renders an image + draws polygon/bbox overlays (QPainter); can highlight one
|
||
└── core/
|
||
├── imageio.py # unicode-safe imread/imwrite (np.fromfile + imdecode)
|
||
├── torch_info.py # probe torch/CUDA (gather/reason/install_hint) for the device badge; no Qt
|
||
├── project.py # Project: layout (project.json/frames/detections.json/collections) + per-project settings
|
||
├── video/
|
||
│ ├── extract.py # extract_frames(): ffmpeg CLI (cv2 fallback) -> JPGs; keyframe/step modes + downscale
|
||
│ └── frame.py # Frame dataclass (image BGR, index, pts) — the detector input type
|
||
├── restore/ # "un-censor" (DeepMosaics only; per-frame OR temporal)
|
||
│ ├── base.py # Restorer ABC: restore(image, dets, should_cancel) + restore_sequence (batch/temporal) + .temporal flag; Cancelled exc
|
||
│ ├── factory.py # build_restorer(name, config) -> deepmosaics | deepmosaics_video (lada = TODO)
|
||
│ ├── deepmosaics.py # DeepMosaicsRestorer (image, per-frame) + DeepMosaicsVideoRestorer (BVDNet, temporal); in-process, load once; uses _deepmosaics/
|
||
│ └── _deepmosaics/ # VENDORED DeepMosaics models/+util/ (GPL-3.0) — added to sys.path at import
|
||
└── detection/ # YOLO only, multi-model
|
||
├── base.py # Detector ABC: detect(frame) -> list[Detection]
|
||
├── factory.py # build_detector(config) -> MultiYoloDetector over config.detector_models
|
||
├── registry.py # discover_models()/category_of() — scans models/yolo/<category>/*.pt
|
||
├── multi.py # MultiYoloDetector — runs several YoloDetectors, concatenates results
|
||
├── types.py # Detection (type + score + bbox + polygon + label/category; .display); CensorType enum
|
||
├── cache.py # save/load the detection cache (detections.json); key = set of model basenames + conf/imgsz
|
||
└── yolo.py # YoloDetector — Ultralytics YOLO-seg; lazy torch/ultralytics; tags dets with a category label
|
||
```
|
||
|
||
### How it works
|
||
|
||
- `MainWindow` holds the config, builds the detector lazily via `build_detector`
|
||
(cached by the **selected-model set** + conf/imgsz in `_make_detector`), and keeps
|
||
`_results: dict[path -> list[Detection]]` as the detection cache.
|
||
- **Multi-model selection.** `config.detector_models` is the list of ticked YOLO weights
|
||
(paths under `models/yolo/<category>/`). The toolbar "Модели" `QToolButton`/`QMenu`
|
||
(`_rebuild_models_menu`, items grouped by category via `addSection`) toggles them
|
||
(`_on_model_toggled` → persist + `_invalidate_results`); "Добавить модель…" copies a
|
||
`.pt` into `models/yolo/<category>/`. On project open `_ensure_models` prunes vanished
|
||
paths and, if nothing is selected, default-ticks every discovered model. `build_detector`
|
||
builds one `YoloDetector` per selected model (tagged `label=category`) wrapped in a
|
||
`MultiYoloDetector` that concatenates their detections (no cross-model dedup). Each
|
||
`Detection` carries `label` (category) **and `model`** (the producing `.pt` stem, tagged in
|
||
`YoloDetector` — shown as its own "Модель" column in the detail table since a category folder
|
||
may hold several models); overlay colour + table group by `Detection.display`
|
||
(label, else the CensorType) via `OverlayConfig.colors` + a stable `palette` fallback.
|
||
- **Background jobs (`ui/workers.py`).** Detection and restoration are CPU-heavy and
|
||
would freeze the GUI, so they run on a `QThreadPool` thread via `Job` (a `QRunnable`
|
||
wrapping `fn(job)`); results return to the GUI through queued Qt signals
|
||
(`tick`/`progress`/`done`/`failed`). `MainWindow._start_job(fn, total, on_tick, on_done)`
|
||
starts one (only one at a time — `_busy` guards entry points), `_finish_job`/
|
||
`_on_job_failed` end it. `_make_detector`/`_make_restorer`, image reads, and
|
||
`engine.detect/restore` all run **inside the worker** (`_compute` is the pure
|
||
read+detect helper); the `fn` must touch NO Qt widgets — it emits plain data that the
|
||
GUI-thread slots (`_apply_detection`, restore `tick`) apply. `_begin_busy` disables
|
||
`model_action` for the duration (it'd race the running detector). This is the
|
||
deliberate exception to the old single-threaded rule (CPU YOLO/DeepMosaics per-call
|
||
latency can't be hidden with `processEvents`).
|
||
- **Device badge + CUDA diagnostics.** A clickable status-bar chip (`device_badge`) shows
|
||
"⚡ CUDA" (green) or "🖥 CPU" (orange). `_probe_device` runs `core/torch_info.gather()` in
|
||
a background `Job` at startup (it imports torch AND shells out to `nvidia-smi`, so it's off
|
||
the GUI thread) → `_set_device_badge`. `gather()` collects torch facts (version, built_cuda,
|
||
cuda_available, device_name) **and** NVIDIA facts (gpus, driver_version, max cuda_driver).
|
||
Clicking (`_show_device_info`) opens a diagnostic **QDialog** (not QMessageBox — its text
|
||
wasn't copyable): a read-only monospace `QPlainTextEdit` with `torch_info.analyze(info)` →
|
||
`{summary, details, steps, command}` — a verdict on *why* it's on CPU (CPU-only `+cpu`
|
||
build / no GPU / driver-too-old-for-built-CUDA) and the exact pip fix. Buttons: "Скопировать
|
||
команду установки" (`_copy_install_command` → the recommended cu121/cu118 command, picked by
|
||
`recommend_channel` from the driver's CUDA) and "Проверить заново" (re-runs `_probe_device`).
|
||
`core/torch_info.py` is pure (no Qt); subprocess uses `CREATE_NO_WINDOW` on Windows.
|
||
- **Projects (`core/project.py`).** `MainWindow._project` is the open `Project`; its
|
||
frames come from `project.frames_dir` (navigation/cache/tags work off `_files`). Entry
|
||
points: "Создать проект…"
|
||
(`_create_project`), "Открыть проект…" (`_open_project_dialog`), "Импортировать папку
|
||
как проект…" (`_import_folder_as_project` — copies images into a new project's
|
||
`frames/`, carries over an old `.hvideotool_detections.json` sidecar if present), and
|
||
a "Недавние проекты" submenu. `_open_project(project)` is the core open: it
|
||
`apply_to_config`s the project's settings, syncs the toolbar widgets without signal
|
||
loops (`_sync_settings_ui`), titles the window, records last/recent, and lists
|
||
`frames/`. On startup `app.run` opens the CLI `target` or auto-reopens
|
||
`settings_store.last_project()` (`_auto_open_last`).
|
||
- **Per-project settings.** Detector/model/threshold/restore engine live in
|
||
`project.json` (`Project.settings`, `_SETTING_KEYS`). `_persist_settings()` writes both
|
||
the global defaults (for new projects) **and** the open project. Global `settings.json`
|
||
is now only defaults + last/recent projects.
|
||
- Listing image files (`_IMAGE_EXTS`) from `frames/` uses a progress bar (bulk insert
|
||
with `setUpdatesEnabled(False)` + periodic `processEvents`), so a big project doesn't
|
||
freeze silently.
|
||
- **Viewing and detecting are decoupled on purpose** (so browsing stays instant even
|
||
with a slow CPU detector): selecting a file only *shows* it with its cached result
|
||
(header reads "не рассчитано" if none). Detection runs on **double-click**, the
|
||
"Рассчитать кадр" action (Space → `_recompute_current`, force-recomputes current), or
|
||
"Детектировать все" (whole folder, progress bar). Do NOT re-add auto-detect-on-select.
|
||
Results cache in `_results`; the file-list row gets a count suffix when computed.
|
||
Switching the model clears the cache (`_choose_model` → `_invalidate_results`).
|
||
- **Detection cache (persisted).** `_results` is mirrored to the project's
|
||
`detections.json` (`core/detection/cache.py`, at `project.cache_path`; keyed by
|
||
**basename** so it survives moving the project). `cache.save_results`/`load_results`
|
||
take the cache file and the image `base_dir` (= `project.frames_dir`) separately, since
|
||
the cache lives at the project root, not next to the images. It's tagged with
|
||
the detector identity (`_results_key` = detector + model + conf/imgsz); on open,
|
||
`_load_cached_results` reloads it **only on a key match** (else ignored, not shown as
|
||
current). Saved (`_save_results`, skipped when `_results` is empty so it never clobbers
|
||
a good cache with nothing) after detect-all (incl. cancel → partial), single
|
||
detect/recompute, move-to-collection, and on `closeEvent`.
|
||
**"Детектировать все" is incremental** (skips already-cached frames → resumes/top-ups);
|
||
**"Все заново"** (`_detect_all(force=True)`) clears the cache first (full regen);
|
||
**"Рассчитать кадр"/Space** always recomputes the one current frame. Empty list in the
|
||
cache = "checked, clean" (tinted green, no mark); `in _results` distinguishes it from
|
||
"not computed".
|
||
- **Favorites (curation).** Curation was simplified (user request) to a **single
|
||
default collection** — no create/select/browse UI. The "★ В избранное" button (left
|
||
pane, under the file list) / "В избранное" menu item / Ctrl+M → `_move_to_favorites`
|
||
**moves** (`shutil.move`, not copy) the `ExtendedSelection`-selected frames into
|
||
`project.favorites_dir` (`collections/Избранное`, `FAVORITES_DIR` in `core/project.py`,
|
||
created lazily on first move), removing them from list/`_files`/cache. `_unique_dest`
|
||
avoids clobbering (`foo.jpg` → `foo (1).jpg`). Use case: flag good frames into a
|
||
training/example set while inspecting detections.
|
||
- **Restoration ("Расцензурить кадр" / "Расцензурить все").** DeepMosaics-only, and it
|
||
**locates the mosaic itself** — so restoration is **fully decoupled from detection**: no
|
||
detector runs in either path (detections are passed as `[]`). Single-frame: a toolbar
|
||
action reads the current frame + runs `self._restorer` (built via `build_restorer`) **on
|
||
a background job** (`_restore_current`; `done` stores `_restored[path]` + shows it). "Показать
|
||
оригинал/результат" toggles (`_showing_restored`); "Сохранить результат" writes
|
||
`<stem>_restored.jpg` beside the frame. **Batch ("Расцензурить все" / "Все заново",
|
||
`_restore_all(force)`)** mirrors `_detect_all`: a single background job restores every
|
||
frame and writes results to the project's **`restored/`** dir (`Project.restored_dir`,
|
||
basename-mirrored, kept OUT of `frames/` so outputs aren't re-listed/re-restored); the
|
||
per-frame engine **skips frames already in `restored/`** unless `force` (resume). The
|
||
engines are **DeepMosaics** (`restore/deepmosaics.py`), run **in-process** from the
|
||
vendored `_deepmosaics/` code, loading the BiSeNet locator + generator **once** (lazy,
|
||
cached on the instance):
|
||
- `deepmosaics` (image, per-frame): reproduces `cleanmosaic_img_server` (locate mosaic
|
||
→ run generator on the crop → feather back), ~0.3 s/frame cached on CPU. Image model
|
||
`clean_youknow_resnet_9blocks.pth`.
|
||
- `deepmosaics_video` (**temporal, BVDNet**): `DeepMosaicsVideoRestorer`, `.temporal=True`.
|
||
Reproduces `cleanmosaic_video_fusion` — per target frame it feeds the net a window of
|
||
`T=5` neighbour frames sampled at step `S=3` around it (`N=2` each side, clamped at the
|
||
sequence edges) **plus its own previous output (recurrent)**, for temporal coherence.
|
||
Because of that recurrence it must run a **contiguous, ordered range** — it implements
|
||
`restore_sequence(count, get_frame, get_dets, emit, should_cancel)` (the batch run uses
|
||
it; single-frame `restore` degrades to a window of the same frame). Needs the **video**
|
||
weights `clean_youknow_video.pth` (+ `mosaic_position.pth` beside). `INPUT_SIZE=256`.
|
||
**`feed_restored`** (config `dm_feed_restored`, default on, checkbox in `RestoreDialog`):
|
||
when set, the **past** neighbours in the window (`j<i`) are taken from the engine's own
|
||
already-restored outputs (a small rolling cache, `reach=N*S` deep) instead of the original
|
||
mosaic frames — stronger temporal coherence. The **centre** (the frame being cleaned) and
|
||
**future** neighbours (`j>i`, not yet restored) stay original. Slightly out-of-distribution
|
||
for the BVDNet (trained on mosaic windows), so it's a toggle; off = faithful DeepMosaics.
|
||
|
||
`restore_sequence` is on the `Restorer` ABC (default = independent per-frame loop);
|
||
`_restore_all` dispatches on `restorer.temporal` (temporal → `restore_sequence` over the
|
||
whole range; per-frame → resumable loop with skip-existing). `should_cancel`
|
||
(= `lambda: job.cancelled`) is polled so "■ Стоп" stops it; engines raise `Cancelled`,
|
||
which `Job.run` reports as a clean cancel. Engine + weights are set in `RestoreDialog`
|
||
(Файл → Движок восстановления…) — the model dropdown shows image vs video weights per
|
||
selected engine — persisted, and built lazily/cached in `_make_restorer`. NOTE:
|
||
DeepMosaics locates mosaics itself (its `mosaic_position.pth`) — our detections aren't
|
||
passed to it. To add another engine (e.g. LADA), implement `core/restore/base.Restorer`
|
||
(set `.temporal` + override `restore_sequence` if it needs neighbours) and register it in
|
||
`restore/factory.build_restorer`.
|
||
- **Navigation bar** under the image (`_build_nav_bar`): prev/next frame (◀ ▶, keys
|
||
`,`/`.`), a scrubber `frame_slider` across the whole sequence, a `pos_label`
|
||
("row / n"), and jump-to-detection (◀ детекция / детекция ▶, keys `[`/`]`,
|
||
`_step_hit` scans `_results` for the next non-empty frame). The slider and file list
|
||
are kept in sync via `_update_nav` guarded by `_nav_sync` (avoids signal loops); all
|
||
navigation ultimately drives `file_list.setCurrentRow`. The scrubber is a custom
|
||
`MarkerSlider` (`ui/marker_slider.py`) that paints **two mark layers**: cyan ticks
|
||
(upper half) at frames with detections (`_refresh_marks` projects `_results`) and
|
||
**green ticks (lower half) at restored frames** (`_refresh_restored_marks` scans
|
||
`restored/` + in-memory `_restored`; called on load and after each restore op, not
|
||
per-frame); per-pixel deduped so big folders stay cheap. Under the scrubber a
|
||
**progress summary** `stats_label` reads "Кадров: N · детектировано: D/N (с цензурой:
|
||
H) · расцензурено: R/N" (`_update_counts_label`, cheap counts; `_restored_count`
|
||
cached by `_refresh_restored_marks`). File-list rows are tinted too (`_tag_file`):
|
||
red = censorship found, green = checked & clean. Both reset on `_invalidate_results`.
|
||
- **Cancellation (cooperative).** A single "■ Стоп" toolbar action (Esc) cancels the
|
||
running op. `_begin_busy(total)` / `_end_busy()` toggle `self._busy` + the Stop button +
|
||
the progress bar (`total=None` → indeterminate). For **background jobs** (detection,
|
||
restore) `_request_cancel` calls `self._job.cancel()`; the worker loop checks
|
||
`job.cancelled` between frames and restore polls it via `should_cancel`. The still-
|
||
synchronous loops (`_load_folder` listing, import copy, video extraction — its
|
||
`progress` cb returns `not self._cancel`) check `self._cancel` between `processEvents`
|
||
ticks. Entry points guard with `if self._busy: return` (notably `_move_to_favorites`,
|
||
which mutates `_files` that a detect-all job reads — so a snapshot/pending list is used).
|
||
`closeEvent` cancels a running job and `waitForDone(3000)` before tearing down.
|
||
- `image_view.ImageView` draws the image scaled-to-fit plus overlays. Overlay
|
||
visibility/threshold are applied at paint time. Selecting a row in the detail table
|
||
calls `set_highlight(i)` — that detection is drawn boldly (even below threshold) and
|
||
the rest dim. The detail table lists ALL detections (sorted by score), so sub-threshold
|
||
hits are still visible for debugging; the threshold only affects what's drawn.
|
||
|
||
### Separation of concerns
|
||
|
||
- `ui/` must not import `torch` / `ultralytics` directly. It builds detectors only via
|
||
`core/detection/factory.build_detector` and talks to `core/` through the `Detector`
|
||
interface and the `Detection`/`CensorType` types.
|
||
- Detection is YOLO-only (multi-model) and restoration is DeepMosaics-only. New detection
|
||
kinds plug in by adding more `.pt` under `models/yolo/<category>/` — no code change. The
|
||
toolbar shows a "Модели" menu of checkable models (no detector dropdown); restoration
|
||
engines are chosen in `RestoreDialog`. If you re-add a different engine *kind*, implement
|
||
`core/detection/base.Detector` / `core/restore/base.Restorer` and register it in the
|
||
respective `factory`. (No legacy-settings migration is kept while in active development —
|
||
old `settings.json`/`project.json` keys are simply ignored, not coerced.)
|
||
|
||
## Commands
|
||
|
||
```powershell
|
||
python -m venv .venv; .\.venv\Scripts\Activate.ps1
|
||
pip install -e ".[yolo]" # YOLO needs ultralytics; install torch separately (README)
|
||
|
||
python -m hvideotool # reopen the last project (or create/open one in-app)
|
||
python -m hvideotool "C:\path\to\MyProject" --model models\lada_mosaic_detection_model_v4_accurate.pt
|
||
```
|
||
|
||
No formal test suite. Headless sanity check: set `QT_QPA_PLATFORM=offscreen`, build a
|
||
`MainWindow`, `Project.create(tmp)` + copy a few images into `frames/`,
|
||
`_open_project(project)`, drive `file_list.setCurrentRow(...)`, and read
|
||
`detail_table` / `detail_header`. Or run `build_detector(config).detect(...)` on a
|
||
frame directly.
|
||
|
||
## Conventions
|
||
|
||
- Match the style of surrounding code; keep `core/` free of Qt where reasonable.
|
||
- Type hints on public functions and the `Detector` interface.
|
||
- Model weights (`.pt`) and large media are **not** committed — keep them in `models/`
|
||
and `.gitignore`d.
|
||
- User-facing strings / README are in Russian; code identifiers and this file in English.
|
||
|
||
## Gotchas
|
||
|
||
- **Models live under `models/yolo/<category>/`.** Discovery (`detection/registry.py`)
|
||
scans that tree; the **category folder is the detection label + overlay colour** (e.g.
|
||
`models/yolo/mosaic/lada.pt` → "mosaic", `models/yolo/face/…` → "face"). On open
|
||
`_ensure_models` default-ticks all discovered models if the project has no selection.
|
||
Put a **censorship** model in `mosaic/` (LADA `lada_mosaic_detection_model_v4_accurate.pt`);
|
||
a generic COCO model (e.g. `yolo11n-seg.pt`) would detect people/objects → noise.
|
||
Selecting many models multiplies per-frame time (each runs in turn).
|
||
- **classic-CV / inpaint were removed (worked poorly).** The classic-CV detector was
|
||
noisy/approximate on real footage (false positives on skin/hair/fabric/JPEG; missed
|
||
real mosaic after downscale) and the `combined` mode + cv2 `inpaint` baseline went with
|
||
it. Detection is YOLO-only, restoration is DeepMosaics-only. No backward-compat shims
|
||
while in active development — stale keys in old `settings.json`/`project.json` are just
|
||
ignored (a project with no valid model selection default-ticks all discovered models).
|
||
- **Domain matters.** LADA is trained on REAL video (JAV). It detects some anime mosaic
|
||
but not all. The real anime fix is *retraining* a YOLO11-seg (see `scripts/training/`).
|
||
- **YOLO detector = LADA weights** ([HF `ladaapp/lada`](https://huggingface.co/ladaapp/lada)).
|
||
YOLO **segmentation** model, classes `{0: mosaic_nsfw, 1: mosaic_sfw_head}` → both map
|
||
to `CensorType.MOSAIC` (`_name_to_type` matches "mosaic" in the class name). Detects
|
||
mosaic only. Weights + Ultralytics are AGPL-3.0 (accepted). `yolo.py` lazy-imports
|
||
`torch`/`ultralytics`.
|
||
- **No model weights in the repo.** Code must fail with a clear, actionable message
|
||
when the model path is missing — not a raw stack trace (`factory._require_model`,
|
||
`YoloDetector.__init__`).
|
||
- **CUDA/torch install is environment-specific.** Don't add torch to core deps; it
|
||
stays out (the `yolo` extra pulls only Ultralytics) and is installed separately.
|
||
- **CPU-only torch must not request CUDA.** A `+cpu` torch build raises "Torch not
|
||
compiled with CUDA enabled" the moment something calls `.cuda()`. Both engines guard
|
||
for this: `YoloDetector` picks `cuda` only when `torch.cuda.is_available()` (even an
|
||
explicit `yolo_device="cuda"` is downgraded to cpu); both `DeepMosaicsRestorer._ensure_loaded`
|
||
and `DeepMosaicsVideoRestorer._ensure_loaded` force `gpu_id="-1"` when CUDA is absent (the
|
||
vendored `model_util.todevice` / `data.im2tensor`/`to_tensor` call `.cuda()` for any
|
||
`gpu_id != "-1"`, e.g. the `dm_gpu="0"` default). So a wrong/CPU-only torch falls back to
|
||
CPU instead of crashing (the temporal BVDNet engine is heavy on CPU, though).
|
||
- **QImage from a numpy buffer must be `.copy()`d** (see `ImageView.set_image`),
|
||
otherwise it aliases a buffer that gets freed → garbage/crash.
|
||
- **Always use `core/imageio.py`** (`imread_unicode`/`imwrite_unicode`) for images —
|
||
`cv2.imread`/`imwrite` silently fail on non-ASCII Windows paths.
|
||
- Don't reintroduce any generative / ControlNet dependency, nor the removed video
|
||
*playback pipeline* (PyAV, producer/consumer worker threads, player, project session).
|
||
(The new `core/project.py` is an on-disk layout, not that thread-based "project
|
||
model".) NOTE: a **single** background `Job` thread for detect/restore (`ui/workers.py`)
|
||
IS in scope now (keeps the GUI responsive) — that's different from the rejected
|
||
multi-thread video pipeline. The one allowed video touch is
|
||
`core/video/extract.py` (one-shot decode → a new project's `frames/`, behind "Создать
|
||
из ролика…"): ffmpeg CLI — `_find_ffmpeg()` prefers PATH, else the binary bundled by
|
||
the `imageio-ffmpeg` dep, else cv2 fallback. Keyframe-only `-skip_frame nokey` is
|
||
~10× faster than every-frame; `-hwaccel` does NOT help (GPU transfer overhead). Use
|
||
ffmpeg/cv2, not PyAV, and keep it synchronous. Decoding every frame is the inherent
|
||
cost — the speed lever is decoding *fewer* frames (keyframes).
|