HVideoTool/CLAUDE.md

CLAUDE.md

Guidance for Claude Code (and other agents) working in this repository.

Memory: EchoVault (read this first)

This project uses the EchoVault MCP server for persistent, cross-session memory. Prior sessions store architectural decisions, fixed bugs, and gotchas there. Follow this protocol every session:

1. At session start — load context. Call memory_context (project is auto-detected from cwd) before doing any work. Use memory_search for specific topics (e.g. "detector model", "classic-cv", "false positives").
2. During work — search before re-deciding. When the task touches an area that may have prior context, memory_search it first instead of re-deriving decisions.
3. Before ending a session — save what matters. Call memory_save when you made a design decision, fixed a bug (include root cause + fix), found a non-obvious gotcha, or the user corrected/clarified a requirement. Pick the right category (decision / bug / pattern / learning / context). Do not save trivia, things obvious from the code, or duplicates.

EchoVault is the source of truth for why things are the way they are; this file is the stable, high-level map. When they disagree, trust on-disk code first, then EchoVault, then this file — and update whichever is stale.

What this project is

HVideoTool is a Windows-first desktop GUI utility that detects already-applied censorship (mosaic, pixelation, blur, black bars) in images, and draws outlines over the detected censored regions. You open a project (see below); it runs each image through a detector, draws the regions, and shows a detailed per-image list of what it found.

Projects (this session). Work is organized into projects — a project is a folder holding project.json (metadata + per-project settings) · frames/ (the images) · detections.json (the detection cache) · collections/Избранное (the single default "favorites" collection). See core/project.py. The per-project settings (detector, model, threshold, restore engine) live in project.json; the global settings.json only seeds the defaults for new projects. The old "open a bare folder" flow is now "Импортировать папку как проект…" (copies images into a new project's frames/).

Scope is still narrow. It used to extract frames from video, detect, and play back with overlays (a thread-based "project model"). That video playback pipeline was removed; the new "projects" are just an on-disk layout, NOT worker threads or playback. The tool remains a synchronous, single-threaded image inspector.

Keep this scope sharp:

• Primary job is detection + overlay/inspection. A restoration ("расцензурить") step was added later (user-requested): on-demand (single frame or whole-project batch into restored/), behind a Restorer interface. Detection is YOLO-only and restoration is DeepMosaics-only — the noisy classic-CV detector (+ the combined composite) and the cv2 inpaint baseline (filled but didn't reconstruct) were removed as "works poorly". Detection is multi-model (ADetailer-style): drop YOLO weights under models/yolo/<category>/, tick which ones are active in the toolbar "Модели" menu, and a detect runs all ticked models and merges results (each tagged with its category → its own overlay colour). DeepMosaics has two engines: image (per-frame) and video (BVDNet, temporal — uses neighbour frames). Its GPL-3.0 network code is vendored under core/restore/_deepmosaics/ and run in-process (user supplies only the weights). Because of that vendoring the whole project is GPL-3.0. LADA (BasicVSR++) is a possible future engine, not wired.
• Still no diffusion / ControlNet / SDXL. (xinsir/controlnet-union-sdxl-1.0 was rejected early — a generative conditioning model, not a censorship restorer. Don't reintroduce it.) Restoration, if upgraded, uses a mosaic-removal model (DeepMosaics/ LADA), not a general text-to-image diffusion pipeline.
• It detects already-censored regions, not "content that should be censored" (i.e. not an NSFW classifier).
• Video is only a one-shot frame-extraction convenience (see below): "Создать из ролика…" makes a new project and decodes the clip into its frames/. Detection and restoration operate on the project's images.

Target environment

• OS: Windows 11 x64 (primary). Use PowerShell syntax in commands.
• Python: 3.11+.
• GPU: NVIDIA + CUDA via PyTorch, for the YOLO detector and the DeepMosaics restorer. CPU fallback works but is slow (esp. DeepMosaics / the temporal BVDNet).

Tech stack (decided)

Concern	Choice
GUI	PySide6 (Qt 6) — LGPL
Image IO	OpenCV (opencv-python) + NumPy, unicode-safe via core/imageio.py
Detector	Ultralytics YOLO, multi-model ensemble (models/yolo//*.pt)

Torch/CUDA + Ultralytics enter with the YOLO detector. Keep that dependency optional (the yolo extra in pyproject.toml pulls only Ultralytics; torch is installed separately per the README). Both detection (YOLO) and restoration (DeepMosaics) now require torch — there's no longer a torch-free detector.

Architecture (as implemented)

This reflects the actual code on disk. The GUI is mostly synchronous, but the heavy compute (detection + restoration) runs on a background thread so the UI stays responsive — see ui/workers.py and the "Background jobs" bullet (this reverses the earlier "no worker threads" rule; processEvents can't unfreeze a single multi- second detector.detect()/DeepMosaics call). Work is organized into projects (core/project.py): a project folder holds project.json (metadata + per-project settings), frames/ (the images), detections.json (the detection cache, at the project root — no longer a sidecar next to the images), and collections/Избранное (the favorites collection). The only video touch is a one-shot "Создать из ролика…" that creates a new project and decodes a clip into its frames/ via the ffmpeg CLI (cv2.VideoCapture fallback; NOT PyAV). When code and this file disagree, trust the code.

hvideotool/
├── __main__.py          # entry point + CLI (optional project path, --detector, --model)
├── app.py               # QApplication bootstrap; run(config, target=None) — opens/auto-reopens a project
├── config.py            # AppConfig + DetectionConfig/OverlayConfig (thresholds live here)
├── settings_store.py    # new-project DEFAULTS + last/recent projects to ~/HVideoTool/settings.json
├── ui/
│   ├── main_window.py   # the whole UI: toolbar + [file list | image view | detail table]
│   ├── workers.py       # Job (QRunnable): runs detect/restore off-thread, results via Qt signals
│   └── image_view.py    # renders an image + draws polygon/bbox overlays (QPainter); can highlight one
└── core/
    ├── imageio.py       # unicode-safe imread/imwrite (np.fromfile + imdecode)
    ├── torch_info.py    # probe torch/CUDA (gather/reason/install_hint) for the device badge; no Qt
    ├── project.py       # Project: layout (project.json/frames/detections.json/collections) + per-project settings
    ├── video/
    │   ├── extract.py   # extract_frames(): ffmpeg CLI (cv2 fallback) -> JPGs; keyframe/step modes + downscale
    │   └── frame.py     # Frame dataclass (image BGR, index, pts) — the detector input type
    ├── restore/         # "un-censor" (DeepMosaics only; per-frame OR temporal)
    │   ├── base.py      #   Restorer ABC: restore(image, dets, should_cancel) + restore_sequence (batch/temporal) + .temporal flag; Cancelled exc
    │   ├── factory.py   #   build_restorer(name, config) -> deepmosaics | deepmosaics_video (lada = TODO)
    │   ├── deepmosaics.py #  DeepMosaicsRestorer (image, per-frame) + DeepMosaicsVideoRestorer (BVDNet, temporal); in-process, load once; uses _deepmosaics/
    │   └── _deepmosaics/ #  VENDORED DeepMosaics models/+util/ (GPL-3.0) — added to sys.path at import
    └── detection/        # YOLO only, multi-model
        ├── base.py      # Detector ABC: detect(frame) -> list[Detection]
        ├── factory.py   # build_detector(config) -> MultiYoloDetector over config.detector_models
        ├── registry.py  # discover_models()/category_of() — scans models/yolo/<category>/*.pt
        ├── multi.py     # MultiYoloDetector — runs several YoloDetectors, concatenates results
        ├── types.py     # Detection (type + score + bbox + polygon + label/category; .display); CensorType enum
        ├── cache.py     # save/load the detection cache (detections.json); key = set of model basenames + conf/imgsz
        └── yolo.py      # YoloDetector — Ultralytics YOLO-seg; lazy torch/ultralytics; tags dets with a category label

How it works

• MainWindow holds the config, builds the detector lazily via build_detector (cached by the selected-model set + conf/imgsz in _make_detector), and keeps _results: dict[path -> list[Detection]] as the detection cache.

• Multi-model selection. config.detector_models is the list of ticked YOLO weights (paths under models/yolo/<category>/). The toolbar "Модели" QToolButton/QMenu (_rebuild_models_menu, items grouped by category via addSection) toggles them (_on_model_toggled → persist + _invalidate_results); "Добавить модель…" copies a .pt into models/yolo/<category>/. On project open _ensure_models prunes vanished paths and, if nothing is selected, default-ticks every discovered model. build_detector builds one YoloDetector per selected model (tagged label=category) wrapped in a MultiYoloDetector that concatenates their detections (no cross-model dedup). Each Detection carries label (category); overlay colour + table group by Detection.display (label, else the CensorType) via OverlayConfig.colors + a stable palette fallback.

• Background jobs (ui/workers.py). Detection and restoration are CPU-heavy and would freeze the GUI, so they run on a QThreadPool thread via Job (a QRunnable wrapping fn(job)); results return to the GUI through queued Qt signals (tick/progress/done/failed). MainWindow._start_job(fn, total, on_tick, on_done) starts one (only one at a time — _busy guards entry points), _finish_job/ _on_job_failed end it. _make_detector/_make_restorer, image reads, and engine.detect/restore all run inside the worker (_compute is the pure read+detect helper); the fn must touch NO Qt widgets — it emits plain data that the GUI-thread slots (_apply_detection, restore tick) apply. _begin_busy disables model_action for the duration (it'd race the running detector). This is the deliberate exception to the old single-threaded rule (CPU YOLO/DeepMosaics per-call latency can't be hidden with processEvents).

• Device badge + CUDA diagnostics. A clickable status-bar chip (device_badge) shows "⚡ CUDA" (green) or "🖥 CPU" (orange). _probe_device runs core/torch_info.gather() in a background Job at startup (it imports torch AND shells out to nvidia-smi, so it's off the GUI thread) → _set_device_badge. gather() collects torch facts (version, built_cuda, cuda_available, device_name) and NVIDIA facts (gpus, driver_version, max cuda_driver). Clicking (_show_device_info) opens a diagnostic QDialog (not QMessageBox — its text wasn't copyable): a read-only monospace QPlainTextEdit with torch_info.analyze(info) → {summary, details, steps, command} — a verdict on why it's on CPU (CPU-only +cpu build / no GPU / driver-too-old-for-built-CUDA) and the exact pip fix. Buttons: "Скопировать команду установки" (_copy_install_command → the recommended cu121/cu118 command, picked by recommend_channel from the driver's CUDA) and "Проверить заново" (re-runs _probe_device). core/torch_info.py is pure (no Qt); subprocess uses CREATE_NO_WINDOW on Windows.

• Projects (core/project.py). MainWindow._project is the open Project; its frames come from project.frames_dir (navigation/cache/tags work off _files). Entry points: "Создать проект…" (_create_project), "Открыть проект…" (_open_project_dialog), "Импортировать папку как проект…" (_import_folder_as_project — copies images into a new project's frames/, carries over an old .hvideotool_detections.json sidecar if present), and a "Недавние проекты" submenu. _open_project(project) is the core open: it apply_to_configs the project's settings, syncs the toolbar widgets without signal loops (_sync_settings_ui), titles the window, records last/recent, and lists frames/. On startup app.run opens the CLI target or auto-reopens settings_store.last_project() (_auto_open_last).

• Per-project settings. Detector/model/threshold/restore engine live in project.json (Project.settings, _SETTING_KEYS). _persist_settings() writes both the global defaults (for new projects) and the open project. Global settings.json is now only defaults + last/recent projects.

• Listing image files (_IMAGE_EXTS) from frames/ uses a progress bar (bulk insert with setUpdatesEnabled(False) + periodic processEvents), so a big project doesn't freeze silently.

• Viewing and detecting are decoupled on purpose (so browsing stays instant even with a slow CPU detector): selecting a file only shows it with its cached result (header reads "не рассчитано" if none). Detection runs on double-click, the "Рассчитать кадр" action (Space → _recompute_current, force-recomputes current), or "Детектировать все" (whole folder, progress bar). Do NOT re-add auto-detect-on-select. Results cache in _results; the file-list row gets a count suffix when computed. Switching the model clears the cache (_choose_model → _invalidate_results).

• Detection cache (persisted). _results is mirrored to the project's detections.json (core/detection/cache.py, at project.cache_path; keyed by basename so it survives moving the project). cache.save_results/load_results take the cache file and the image base_dir (= project.frames_dir) separately, since the cache lives at the project root, not next to the images. It's tagged with the detector identity (_results_key = detector + model + conf/imgsz); on open, _load_cached_results reloads it only on a key match (else ignored, not shown as current). Saved (_save_results, skipped when _results is empty so it never clobbers a good cache with nothing) after detect-all (incl. cancel → partial), single detect/recompute, move-to-collection, and on closeEvent. "Детектировать все" is incremental (skips already-cached frames → resumes/top-ups); "Все заново" (_detect_all(force=True)) clears the cache first (full regen); "Рассчитать кадр"/Space always recomputes the one current frame. Empty list in the cache = "checked, clean" (tinted green, no mark); in _results distinguishes it from "not computed".

• Favorites (curation). Curation was simplified (user request) to a single default collection — no create/select/browse UI. The "★ В избранное" button (left pane, under the file list) / "В избранное" menu item / Ctrl+M → _move_to_favorites moves (shutil.move, not copy) the ExtendedSelection-selected frames into project.favorites_dir (collections/Избранное, FAVORITES_DIR in core/project.py, created lazily on first move), removing them from list/_files/cache. _unique_dest avoids clobbering (foo.jpg → foo (1).jpg). Use case: flag good frames into a training/example set while inspecting detections.

• Restoration ("Расцензурить кадр" / "Расцензурить все"). DeepMosaics-only, and it locates the mosaic itself — so restoration is fully decoupled from detection: no detector runs in either path (detections are passed as []). Single-frame: a toolbar action reads the current frame + runs self._restorer (built via build_restorer) on a background job (_restore_current; done stores _restored[path] + shows it). "Показать оригинал/результат" toggles (_showing_restored); "Сохранить результат" writes <stem>_restored.jpg beside the frame. Batch ("Расцензурить все" / "Все заново", _restore_all(force)) mirrors _detect_all: a single background job restores every frame and writes results to the project's restored/ dir (Project.restored_dir, basename-mirrored, kept OUT of frames/ so outputs aren't re-listed/re-restored); the per-frame engine skips frames already in restored/ unless force (resume). The engines are DeepMosaics (restore/deepmosaics.py), run in-process from the vendored _deepmosaics/ code, loading the BiSeNet locator + generator once (lazy, cached on the instance):

  • deepmosaics (image, per-frame): reproduces cleanmosaic_img_server (locate mosaic → run generator on the crop → feather back), ~0.3 s/frame cached on CPU. Image model clean_youknow_resnet_9blocks.pth.
  • deepmosaics_video (temporal, BVDNet): DeepMosaicsVideoRestorer, .temporal=True. Reproduces cleanmosaic_video_fusion — per target frame it feeds the net a window of T=5 neighbour frames sampled at step S=3 around it (N=2 each side, clamped at the sequence edges) plus its own previous output (recurrent), for temporal coherence. Because of that recurrence it must run a contiguous, ordered range — it implements restore_sequence(count, get_frame, get_dets, emit, should_cancel) (the batch run uses it; single-frame restore degrades to a window of the same frame). Needs the video weights clean_youknow_video.pth (+ mosaic_position.pth beside). INPUT_SIZE=256.

  restore_sequence is on the Restorer ABC (default = independent per-frame loop); _restore_all dispatches on restorer.temporal (temporal → restore_sequence over the whole range; per-frame → resumable loop with skip-existing). should_cancel (= lambda: job.cancelled) is polled so "■ Стоп" stops it; engines raise Cancelled, which Job.run reports as a clean cancel. Engine + weights are set in RestoreDialog (Файл → Движок восстановления…) — the model dropdown shows image vs video weights per selected engine — persisted, and built lazily/cached in _make_restorer. NOTE: DeepMosaics locates mosaics itself (its mosaic_position.pth) — our detections aren't passed to it. To add another engine (e.g. LADA), implement core/restore/base.Restorer (set .temporal + override restore_sequence if it needs neighbours) and register it in restore/factory.build_restorer.

• Navigation bar under the image (_build_nav_bar): prev/next frame (◀ ▶, keys ,/.), a scrubber frame_slider across the whole sequence, a pos_label ("row / n"), and jump-to-detection (◀ детекция / детекция ▶, keys [/], _step_hit scans _results for the next non-empty frame). The slider and file list are kept in sync via _update_nav guarded by _nav_sync (avoids signal loops); all navigation ultimately drives file_list.setCurrentRow. The scrubber is a custom MarkerSlider (ui/marker_slider.py) that paints cyan ticks at frames with detections (_refresh_marks projects _results onto row indices; per-pixel deduped so big folders stay cheap). File-list rows are tinted too (_tag_file): red = censorship found, green = checked & clean. Both reset on _invalidate_results.

• Cancellation (cooperative). A single "■ Стоп" toolbar action (Esc) cancels the running op. _begin_busy(total) / _end_busy() toggle self._busy + the Stop button + the progress bar (total=None → indeterminate). For background jobs (detection, restore) _request_cancel calls self._job.cancel(); the worker loop checks job.cancelled between frames and restore polls it via should_cancel. The still- synchronous loops (_load_folder listing, import copy, video extraction — its progress cb returns not self._cancel) check self._cancel between processEvents ticks. Entry points guard with if self._busy: return (notably _move_to_favorites, which mutates _files that a detect-all job reads — so a snapshot/pending list is used). closeEvent cancels a running job and waitForDone(3000) before tearing down.

• image_view.ImageView draws the image scaled-to-fit plus overlays. Overlay visibility/threshold are applied at paint time. Selecting a row in the detail table calls set_highlight(i) — that detection is drawn boldly (even below threshold) and the rest dim. The detail table lists ALL detections (sorted by score), so sub-threshold hits are still visible for debugging; the threshold only affects what's drawn.

Separation of concerns

• ui/ must not import torch / ultralytics directly. It builds detectors only via core/detection/factory.build_detector and talks to core/ through the Detector interface and the Detection/CensorType types.
• Detection is YOLO-only (multi-model) and restoration is DeepMosaics-only. New detection kinds plug in by adding more .pt under models/yolo/<category>/ — no code change. The toolbar shows a "Модели" menu of checkable models (no detector dropdown); restoration engines are chosen in RestoreDialog. If you re-add a different engine kind, implement core/detection/base.Detector / core/restore/base.Restorer and register it in the respective factory. (No legacy-settings migration is kept while in active development — old settings.json/project.json keys are simply ignored, not coerced.)

Commands

python -m venv .venv; .\.venv\Scripts\Activate.ps1
pip install -e ".[yolo]"               # YOLO needs ultralytics; install torch separately (README)

python -m hvideotool                   # reopen the last project (or create/open one in-app)
python -m hvideotool "C:\path\to\MyProject" --model models\lada_mosaic_detection_model_v4_accurate.pt

No formal test suite. Headless sanity check: set QT_QPA_PLATFORM=offscreen, build a MainWindow, Project.create(tmp) + copy a few images into frames/, _open_project(project), drive file_list.setCurrentRow(...), and read detail_table / detail_header. Or run build_detector(config).detect(...) on a frame directly.

Conventions

• Match the style of surrounding code; keep core/ free of Qt where reasonable.
• Type hints on public functions and the Detector interface.
• Model weights (.pt) and large media are not committed — keep them in models/ and .gitignored.
• User-facing strings / README are in Russian; code identifiers and this file in English.

Gotchas

• Models live under models/yolo/<category>/. Discovery (detection/registry.py) scans that tree; the category folder is the detection label + overlay colour (e.g. models/yolo/mosaic/lada.pt → "mosaic", models/yolo/face/… → "face"). On open _ensure_models default-ticks all discovered models if the project has no selection. Put a censorship model in mosaic/ (LADA lada_mosaic_detection_model_v4_accurate.pt); a generic COCO model (e.g. yolo11n-seg.pt) would detect people/objects → noise. Selecting many models multiplies per-frame time (each runs in turn).
• classic-CV / inpaint were removed (worked poorly). The classic-CV detector was noisy/approximate on real footage (false positives on skin/hair/fabric/JPEG; missed real mosaic after downscale) and the combined mode + cv2 inpaint baseline went with it. Detection is YOLO-only, restoration is DeepMosaics-only. No backward-compat shims while in active development — stale keys in old settings.json/project.json are just ignored (a project with no valid model selection default-ticks all discovered models).
• Domain matters. LADA is trained on REAL video (JAV). It detects some anime mosaic but not all. The real anime fix is retraining a YOLO11-seg (see scripts/training/).
• YOLO detector = LADA weights (HF ladaapp/lada). YOLO segmentation model, classes {0: mosaic_nsfw, 1: mosaic_sfw_head} → both map to CensorType.MOSAIC (_name_to_type matches "mosaic" in the class name). Detects mosaic only. Weights + Ultralytics are AGPL-3.0 (accepted). yolo.py lazy-imports torch/ultralytics.
• No model weights in the repo. Code must fail with a clear, actionable message when the model path is missing — not a raw stack trace (factory._require_model, YoloDetector.__init__).
• CUDA/torch install is environment-specific. Don't add torch to core deps; it stays out (the yolo extra pulls only Ultralytics) and is installed separately.
• CPU-only torch must not request CUDA. A +cpu torch build raises "Torch not compiled with CUDA enabled" the moment something calls .cuda(). Both engines guard for this: YoloDetector picks cuda only when torch.cuda.is_available() (even an explicit yolo_device="cuda" is downgraded to cpu); both DeepMosaicsRestorer._ensure_loaded and DeepMosaicsVideoRestorer._ensure_loaded force gpu_id="-1" when CUDA is absent (the vendored model_util.todevice / data.im2tensor/to_tensor call .cuda() for any gpu_id != "-1", e.g. the dm_gpu="0" default). So a wrong/CPU-only torch falls back to CPU instead of crashing (the temporal BVDNet engine is heavy on CPU, though).
• QImage from a numpy buffer must be .copy()d (see ImageView.set_image), otherwise it aliases a buffer that gets freed → garbage/crash.
• Always use core/imageio.py (imread_unicode/imwrite_unicode) for images — cv2.imread/imwrite silently fail on non-ASCII Windows paths.
• Don't reintroduce any generative / ControlNet dependency, nor the removed video playback pipeline (PyAV, producer/consumer worker threads, player, project session). (The new core/project.py is an on-disk layout, not that thread-based "project model".) NOTE: a single background Job thread for detect/restore (ui/workers.py) IS in scope now (keeps the GUI responsive) — that's different from the rejected multi-thread video pipeline. The one allowed video touch is core/video/extract.py (one-shot decode → a new project's frames/, behind "Создать из ролика…"): ffmpeg CLI — _find_ffmpeg() prefers PATH, else the binary bundled by the imageio-ffmpeg dep, else cv2 fallback. Keyframe-only -skip_frame nokey is ~10× faster than every-frame; -hwaccel does NOT help (GPU transfer overhead). Use ffmpeg/cv2, not PyAV, and keep it synchronous. Decoding every frame is the inherent cost — the speed lever is decoding fewer frames (keyframes).