Implement multi-model detection in HVideoTool: updated the detection system to support multiple YOLO models simultaneously, enhancing detection capabilities. Reflected changes in the UI with a new model selection menu and updated documentation in README and CLAUDE.md to guide users on model management and configuration.

This commit is contained in:
Leonid Pershin
2026-06-07 07:05:50 +03:00
parent ac02ca27a8
commit 0996ca7bb9
14 changed files with 353 additions and 156 deletions
+41 -25
View File
@@ -51,7 +51,10 @@ Keep this scope sharp:
batch into `restored/`), behind a `Restorer` interface. **Detection is YOLO-only and
restoration is DeepMosaics-only** — the noisy classic-CV detector (+ the `combined`
composite) and the cv2 `inpaint` baseline (filled but didn't reconstruct) were
**removed** as "works poorly". DeepMosaics has two engines: **image** (per-frame) and
**removed** as "works poorly". Detection is **multi-model** (ADetailer-style): drop YOLO
weights under `models/yolo/<category>/`, tick which ones are active in the toolbar
"Модели" menu, and a detect runs **all** ticked models and merges results (each tagged
with its category → its own overlay colour). DeepMosaics has two engines: **image** (per-frame) and
**video** (BVDNet, temporal — uses neighbour frames). Its GPL-3.0 network code is
**vendored** under `core/restore/_deepmosaics/` and run in-process (user supplies only
the weights). Because of that vendoring the **whole project is GPL-3.0**. LADA
@@ -79,7 +82,7 @@ Keep this scope sharp:
|----------------|---------------------------------------------|
| GUI | PySide6 (Qt 6) — LGPL |
| Image IO | OpenCV (`opencv-python`) + NumPy, unicode-safe via `core/imageio.py` |
| Detector | Ultralytics YOLO (LADA weights) behind a pluggable interface (YOLO-only) |
| Detector | Ultralytics YOLO, multi-model ensemble (models/yolo/<category>/*.pt) |
Torch/CUDA + Ultralytics enter with the YOLO detector. Keep that dependency optional
(the `yolo` extra in `pyproject.toml` pulls only Ultralytics; torch is installed
@@ -123,19 +126,31 @@ hvideotool/
│ ├── factory.py # build_restorer(name, config) -> deepmosaics | deepmosaics_video (lada = TODO)
│ ├── deepmosaics.py # DeepMosaicsRestorer (image, per-frame) + DeepMosaicsVideoRestorer (BVDNet, temporal); in-process, load once; uses _deepmosaics/
│ └── _deepmosaics/ # VENDORED DeepMosaics models/+util/ (GPL-3.0) — added to sys.path at import
└── detection/ # YOLO only
└── detection/ # YOLO only, multi-model
├── base.py # Detector ABC: detect(frame) -> list[Detection]
├── factory.py # build_detector(config) -> yolo (the only kind)
├── types.py # Detection (+ to_dict/from_dict), CensorType enum
├── cache.py # save/load the project detection cache (detections.json): cache_file + base_dir args
── yolo.py # YoloDetector — Ultralytics YOLO-seg; lazy-imports torch/ultralytics
├── factory.py # build_detector(config) -> MultiYoloDetector over config.detector_models
├── registry.py # discover_models()/category_of() — scans models/yolo/<category>/*.pt
├── multi.py # MultiYoloDetector — runs several YoloDetectors, concatenates results
── types.py # Detection (type + score + bbox + polygon + label/category; .display); CensorType enum
├── cache.py # save/load the detection cache (detections.json); key = set of model basenames + conf/imgsz
└── yolo.py # YoloDetector — Ultralytics YOLO-seg; lazy torch/ultralytics; tags dets with a category label
```
### How it works
- `MainWindow` holds the config, builds the detector lazily via `build_detector`
(cached by detector+model+conf in `_make_detector`), and keeps `_results: dict[path
-> list[Detection]]` as the detection cache.
(cached by the **selected-model set** + conf/imgsz in `_make_detector`), and keeps
`_results: dict[path -> list[Detection]]` as the detection cache.
- **Multi-model selection.** `config.detector_models` is the list of ticked YOLO weights
(paths under `models/yolo/<category>/`). The toolbar "Модели" `QToolButton`/`QMenu`
(`_rebuild_models_menu`, items grouped by category via `addSection`) toggles them
(`_on_model_toggled` → persist + `_invalidate_results`); "Добавить модель…" copies a
`.pt` into `models/yolo/<category>/`. On project open `_ensure_models` prunes vanished
paths and, if nothing is selected, default-ticks every discovered model. `build_detector`
builds one `YoloDetector` per selected model (tagged `label=category`) wrapped in a
`MultiYoloDetector` that concatenates their detections (no cross-model dedup). Each
`Detection` carries `label` (category); overlay colour + table group by `Detection.display`
(label, else the CensorType) via `OverlayConfig.colors` + a stable `palette` fallback.
- **Background jobs (`ui/workers.py`).** Detection and restoration are CPU-heavy and
would freeze the GUI, so they run on a `QThreadPool` thread via `Job` (a `QRunnable`
wrapping `fn(job)`); results return to the GUI through queued Qt signals
@@ -276,11 +291,13 @@ hvideotool/
- `ui/` must not import `torch` / `ultralytics` directly. It builds detectors only via
`core/detection/factory.build_detector` and talks to `core/` through the `Detector`
interface and the `Detection`/`CensorType` types.
- Detection is YOLO-only and restoration is DeepMosaics-only. If you re-add an engine
kind, implement `core/detection/base.Detector` / `core/restore/base.Restorer`, register
the string in the respective `factory`, and add it to `DETECTORS`/`RESTORERS` in
`config.py` (and `normalize_config`). There's no detector dropdown anymore — the toolbar
just shows "Детектор: YOLO"; restoration engines are chosen in `RestoreDialog`.
- Detection is YOLO-only (multi-model) and restoration is DeepMosaics-only. New detection
kinds plug in by adding more `.pt` under `models/yolo/<category>/` — no code change. The
toolbar shows a "Модели" menu of checkable models (no detector dropdown); restoration
engines are chosen in `RestoreDialog`. If you re-add a different engine *kind*, implement
`core/detection/base.Detector` / `core/restore/base.Restorer` and register it in the
respective `factory`. (No legacy-settings migration is kept while in active development —
old `settings.json`/`project.json` keys are simply ignored, not coerced.)
## Commands
@@ -308,20 +325,19 @@ frame directly.
## Gotchas
- **Wrong YOLO model = "noise".** The YOLO detector needs a **censorship** model
(LADA `models\lada_mosaic_detection_model_v4_accurate.pt`). If `model_path` points at
a generic COCO model (e.g. the `yolo11n-seg.pt` in the repo root, which Ultralytics
auto-downloads / is the training base), it detects people/objects and maps them to
`CensorType.UNKNOWN` → purple boxes that look like noise. This was a real user trap.
**A model is auto-picked on project open** via `MainWindow._ensure_model()`
`_auto_find_model()`: it scans `./models/**.pt` and matches only filenames containing
`lada`/`mosaic` (so it skips the COCO `yolo11n-seg.pt` trap), no prompt; otherwise the
user picks via "Модель…" (`_choose_model`).
- **Models live under `models/yolo/<category>/`.** Discovery (`detection/registry.py`)
scans that tree; the **category folder is the detection label + overlay colour** (e.g.
`models/yolo/mosaic/lada.pt` → "mosaic", `models/yolo/face/…` → "face"). On open
`_ensure_models` default-ticks all discovered models if the project has no selection.
Put a **censorship** model in `mosaic/` (LADA `lada_mosaic_detection_model_v4_accurate.pt`);
a generic COCO model (e.g. `yolo11n-seg.pt`) would detect people/objects → noise.
Selecting many models multiplies per-frame time (each runs in turn).
- **classic-CV / inpaint were removed (worked poorly).** The classic-CV detector was
noisy/approximate on real footage (false positives on skin/hair/fabric/JPEG; missed
real mosaic after downscale) and the `combined` mode + cv2 `inpaint` baseline went with
it. Detection is YOLO-only, restoration is DeepMosaics-only. `normalize_config` coerces
any leftover `classic`/`combined`/`inpaint` in old settings/projects to `yolo`/`deepmosaics`.
it. Detection is YOLO-only, restoration is DeepMosaics-only. No backward-compat shims
while in active development — stale keys in old `settings.json`/`project.json` are just
ignored (a project with no valid model selection default-ticks all discovered models).
- **Domain matters.** LADA is trained on REAL video (JAV). It detects some anime mosaic
but not all. The real anime fix is *retraining* a YOLO11-seg (see `scripts/training/`).
- **YOLO detector = LADA weights** ([HF `ladaapp/lada`](https://huggingface.co/ladaapp/lada)).