diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..9a15d6b --- /dev/null +++ b/.gitattributes @@ -0,0 +1,14 @@ +# Keep LF in the repository regardless of the OS that edits the files. +* text=auto eol=lf + +# cmd.exe needs CRLF in batch files +*.bat text eol=crlf +*.cmd text eol=crlf + +*.png binary +*.jpg binary +*.webp binary +*.ogg binary +*.wav binary +*.zip binary +*.rpa binary diff --git a/README.md b/README.md index 60c35e0..52917e0 100644 --- a/README.md +++ b/README.md @@ -1,102 +1,102 @@ -# PGames — локальная библиотека визуальных новелл - -Загружаете архив с игрой на Ren'Py — сервер распаковывает его, переводит -скрипты и ассеты в собственный формат **PGVN** (JSON-скрипты, WebP, Opus, WebM) -и удаляет исходники. Играете в браузере во встроенном плеере. У каждой игры -может быть несколько версий (форков) от разных пользователей; версии делят -общие ассеты, поэтому форк почти не занимает места. - -Подробности устройства — в [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). - -## Установка на VM - -```bash -git clone pgames && cd pgames -cp .env.example .env -# обязательно поменяйте SECRET_KEY (openssl rand -hex 32) и пароли -docker compose up -d --build -``` - -Приложение слушает `http://:8080` (порт — `HTTP_PORT` в `.env`). -Caddy на хосте Proxmox: - -``` -games.example.lan { - reverse_proxy 192.168.1.50:8080 -} -``` - -Если в браузере сайт открывается по HTTPS через Caddy, оставьте -`COOKIE_SECURE=true`; при доступе напрямую по HTTP поставьте `false`, иначе -браузер не сохранит сессию. - -## Первый вход - -Логин `admin` / пароль из `ADMIN_PASSWORD` (по умолчанию `admin`). Сразу после -входа нужно задать свой логин и пароль. - -Регистрации нет: пользователей создаёт админ в разделе «Пользователи». -Сервер генерирует пароль и показывает его один раз — передайте его человеку, -при первом входе он задаст свой. Права настраиваются через роли -(«Роли»): `admin`, `moderator`, `player` плюс свои. - -## Импорт уже распакованных игр - -Если игры уже лежат на VM распакованными, архивировать их не нужно. -Подключите папку к контейнеру (только чтение) и запустите импорт: - -```bash -docker compose run --rm -v /srv/games:/games:ro backend \ - python -m app.import_dir "/games/Some-Game" --title "Some Game" -``` - -Папка не изменяется, сконвертированная копия попадает в библиотеку как -обычная загрузка. Проверить, насколько хорошо конвертер понимает игру, -не трогая медиа (минуты вместо часов), можно сухим прогоном: - -```bash -docker compose run --rm -v /srv/games:/games:ro backend \ - python -W ignore -m app.converter.analyze "/games/Some-Game" -``` - -## Данные и бэкап - -* `./data/blobs` — сконвертированные игры (неизменяемые файлы по sha256); -* том `db` — PostgreSQL (пользователи, игры, версии, сохранения). - -Для бэкапа достаточно `pg_dump` и копии `data/blobs`. - -## Разработка - -На Windows тестовый стенд поднимается одной командой: `start-test.bat`. -Он запускает отдельный compose-проект `pgames-test` на http://localhost:8090 -(свой `.env.test`, данные в `data-test\`), генерирует тестовую игру -`data-test\sample.zip` и открывает браузер. `start-test.bat stop` — остановить, -`reset` — удалить стенд вместе с данными, `logs` — логи бэкенда и воркера. - -```bash -cd backend && pip install -r requirements.txt pytest && python -m pytest -cd frontend && npm install && npm run dev # проксирует /api на :8000 -python -m tests.make_sample sample.zip # тестовая игра (из backend/) -``` - -## Что поддерживает конвертер Ren'Py - -`label`, реплики (в т.ч. атрибуты, `extend`, `centered`, NVL), `scene/show/hide` -с `at`/`as`/`behind`/`zorder`/`with`, переходы (`dissolve`, `fade`, -`Fade(...)`, `vpunch`, wipe/slide…), `menu` с условиями и `set`, `if/elif/else`, -`while`, `jump/call/return` (включая `expression`), `$`/`python:`/`init python:` -(подмножество Python: переменные, выражения, списки/словари, циклы), -`define/default`, `Character(...)`, `image` (файлы, цвета, `Transform`, `im.*`, -`Composite`, `ConditionSwitch`, `Movie`, покадровые ATL-анимации с `pause`/`repeat`), -`layeredimage` (always/attribute/group/if), `transform` со статическими -свойствами, `Shake`, `camera`, вложенные метки и именованные `menu`, -`call label(args)` и метки с параметрами, `play/queue/stop/voice` и свои -каналы (`register_channel`), `pause`, `.rpa`, `.rpyc` (через unrpyc), -автоопределение картинок из `images/`, GUI-текстбокс и шрифты игры. - -Пока **не** поддерживается: `screen`-язык (кастомные экраны, карты, -мини-игры), движение и твины в ATL, функции/классы в `python` и картинки, -которые игра вычисляет своими функциями. Такие места перечислены в отчёте -конвертации у версии. Игры-«песочницы», где навигация построена на экранах, -пока не играбельны. +# PGames — локальная библиотека визуальных новелл + +Загружаете архив с игрой на Ren'Py — сервер распаковывает его, переводит +скрипты и ассеты в собственный формат **PGVN** (JSON-скрипты, WebP, Opus, WebM) +и удаляет исходники. Играете в браузере во встроенном плеере. У каждой игры +может быть несколько версий (форков) от разных пользователей; версии делят +общие ассеты, поэтому форк почти не занимает места. + +Подробности устройства — в [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). + +## Установка на VM + +```bash +git clone pgames && cd pgames +cp .env.example .env +# обязательно поменяйте SECRET_KEY (openssl rand -hex 32) и пароли +docker compose up -d --build +``` + +Приложение слушает `http://:8080` (порт — `HTTP_PORT` в `.env`). +Caddy на хосте Proxmox: + +``` +games.example.lan { + reverse_proxy 192.168.1.50:8080 +} +``` + +Если в браузере сайт открывается по HTTPS через Caddy, оставьте +`COOKIE_SECURE=true`; при доступе напрямую по HTTP поставьте `false`, иначе +браузер не сохранит сессию. + +## Первый вход + +Логин `admin` / пароль из `ADMIN_PASSWORD` (по умолчанию `admin`). Сразу после +входа нужно задать свой логин и пароль. + +Регистрации нет: пользователей создаёт админ в разделе «Пользователи». +Сервер генерирует пароль и показывает его один раз — передайте его человеку, +при первом входе он задаст свой. Права настраиваются через роли +(«Роли»): `admin`, `moderator`, `player` плюс свои. + +## Импорт уже распакованных игр + +Если игры уже лежат на VM распакованными, архивировать их не нужно. +Подключите папку к контейнеру (только чтение) и запустите импорт: + +```bash +docker compose run --rm -v /srv/games:/games:ro backend \ + python -m app.import_dir "/games/Some-Game" --title "Some Game" +``` + +Папка не изменяется, сконвертированная копия попадает в библиотеку как +обычная загрузка. Проверить, насколько хорошо конвертер понимает игру, +не трогая медиа (минуты вместо часов), можно сухим прогоном: + +```bash +docker compose run --rm -v /srv/games:/games:ro backend \ + python -W ignore -m app.converter.analyze "/games/Some-Game" +``` + +## Данные и бэкап + +* `./data/blobs` — сконвертированные игры (неизменяемые файлы по sha256); +* том `db` — PostgreSQL (пользователи, игры, версии, сохранения). + +Для бэкапа достаточно `pg_dump` и копии `data/blobs`. + +## Разработка + +На Windows тестовый стенд поднимается одной командой: `start-test.bat`. +Он запускает отдельный compose-проект `pgames-test` на http://localhost:8090 +(свой `.env.test`, данные в `data-test\`), генерирует тестовую игру +`data-test\sample.zip` и открывает браузер. `start-test.bat stop` — остановить, +`reset` — удалить стенд вместе с данными, `logs` — логи бэкенда и воркера. + +```bash +cd backend && pip install -r requirements.txt pytest && python -m pytest +cd frontend && npm install && npm run dev # проксирует /api на :8000 +python -m tests.make_sample sample.zip # тестовая игра (из backend/) +``` + +## Что поддерживает конвертер Ren'Py + +`label`, реплики (в т.ч. атрибуты, `extend`, `centered`, NVL), `scene/show/hide` +с `at`/`as`/`behind`/`zorder`/`with`, переходы (`dissolve`, `fade`, +`Fade(...)`, `vpunch`, wipe/slide…), `menu` с условиями и `set`, `if/elif/else`, +`while`, `jump/call/return` (включая `expression`), `$`/`python:`/`init python:` +(подмножество Python: переменные, выражения, списки/словари, циклы), +`define/default`, `Character(...)`, `image` (файлы, цвета, `Transform`, `im.*`, +`Composite`, `ConditionSwitch`, `Movie`, покадровые ATL-анимации с `pause`/`repeat`), +`layeredimage` (always/attribute/group/if), `transform` со статическими +свойствами, `Shake`, `camera`, вложенные метки и именованные `menu`, +`call label(args)` и метки с параметрами, `play/queue/stop/voice` и свои +каналы (`register_channel`), `pause`, `.rpa`, `.rpyc` (через unrpyc), +автоопределение картинок из `images/`, GUI-текстбокс и шрифты игры. + +Пока **не** поддерживается: `screen`-язык (кастомные экраны, карты, +мини-игры), движение и твины в ATL, функции/классы в `python` и картинки, +которые игра вычисляет своими функциями. Такие места перечислены в отчёте +конвертации у версии. Игры-«песочницы», где навигация построена на экранах, +пока не играбельны. diff --git a/backend/alembic.ini b/backend/alembic.ini index 180bff8..b430bfc 100644 --- a/backend/alembic.ini +++ b/backend/alembic.ini @@ -1,24 +1,24 @@ -[alembic] -script_location = migrations -prepend_sys_path = . - -[loggers] -keys = root - -[handlers] -keys = console - -[formatters] -keys = generic - -[logger_root] -level = WARN -handlers = console - -[handler_console] -class = logging.StreamHandler -args = (sys.stderr,) -formatter = generic - -[formatter_generic] -format = %(levelname)-5.5s [%(name)s] %(message)s +[alembic] +script_location = migrations +prepend_sys_path = . + +[loggers] +keys = root + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = WARN +handlers = console + +[handler_console] +class = logging.StreamHandler +args = (sys.stderr,) +formatter = generic + +[formatter_generic] +format = %(levelname)-5.5s [%(name)s] %(message)s diff --git a/backend/app/blobs.py b/backend/app/blobs.py index b333d89..16dc407 100644 --- a/backend/app/blobs.py +++ b/backend/app/blobs.py @@ -1,69 +1,69 @@ -"""Content-addressed blob store. - -A blob id is ```` (e.g. ``3fa1….webp``); the file lives at -``blobs/3f/a1/``. The extension is part of the id so nginx can serve -blobs with the right Content-Type straight from disk. -""" - -import hashlib -import os -import re -import shutil -import tempfile -from pathlib import Path - -from app.config import get_settings - -BLOB_ID_RE = re.compile(r"^[0-9a-f]{64}(\.[a-z0-9]{1,8})?$") -_CHUNK = 1 << 20 - - -def blob_path(blob_id: str) -> Path: - if not BLOB_ID_RE.match(blob_id): - raise ValueError(f"bad blob id {blob_id!r}") - return get_settings().blobs_dir / blob_id[:2] / blob_id[2:4] / blob_id - - -def _finalize(tmp: Path, digest: str, ext: str) -> tuple[str, int]: - blob_id = digest + ext.lower() - dest = blob_path(blob_id) - size = tmp.stat().st_size - if dest.exists(): - tmp.unlink() - else: - dest.parent.mkdir(parents=True, exist_ok=True) - tmp.chmod(0o644) # mkstemp creates 0600; nginx serves blobs as another user - os.replace(tmp, dest) - return blob_id, size - - -def _tmp_file() -> Path: - root = get_settings().blobs_dir / "tmp" - root.mkdir(parents=True, exist_ok=True) - fd, name = tempfile.mkstemp(dir=root) - os.close(fd) - return Path(name) - - -def put_file(src: Path, ext: str, move: bool = False) -> tuple[str, int]: - """Store a file, returning (blob id, size). ``move`` consumes ``src``.""" - h = hashlib.sha256() - with src.open("rb") as f: - while chunk := f.read(_CHUNK): - h.update(chunk) - tmp = _tmp_file() - if move: - shutil.move(src, tmp) - else: - shutil.copyfile(src, tmp) - return _finalize(tmp, h.hexdigest(), ext) - - -def put_bytes(data: bytes, ext: str) -> tuple[str, int]: - tmp = _tmp_file() - tmp.write_bytes(data) - return _finalize(tmp, hashlib.sha256(data).hexdigest(), ext) - - -def read_bytes(blob_id: str) -> bytes: - return blob_path(blob_id).read_bytes() +"""Content-addressed blob store. + +A blob id is ```` (e.g. ``3fa1….webp``); the file lives at +``blobs/3f/a1/``. The extension is part of the id so nginx can serve +blobs with the right Content-Type straight from disk. +""" + +import hashlib +import os +import re +import shutil +import tempfile +from pathlib import Path + +from app.config import get_settings + +BLOB_ID_RE = re.compile(r"^[0-9a-f]{64}(\.[a-z0-9]{1,8})?$") +_CHUNK = 1 << 20 + + +def blob_path(blob_id: str) -> Path: + if not BLOB_ID_RE.match(blob_id): + raise ValueError(f"bad blob id {blob_id!r}") + return get_settings().blobs_dir / blob_id[:2] / blob_id[2:4] / blob_id + + +def _finalize(tmp: Path, digest: str, ext: str) -> tuple[str, int]: + blob_id = digest + ext.lower() + dest = blob_path(blob_id) + size = tmp.stat().st_size + if dest.exists(): + tmp.unlink() + else: + dest.parent.mkdir(parents=True, exist_ok=True) + tmp.chmod(0o644) # mkstemp creates 0600; nginx serves blobs as another user + os.replace(tmp, dest) + return blob_id, size + + +def _tmp_file() -> Path: + root = get_settings().blobs_dir / "tmp" + root.mkdir(parents=True, exist_ok=True) + fd, name = tempfile.mkstemp(dir=root) + os.close(fd) + return Path(name) + + +def put_file(src: Path, ext: str, move: bool = False) -> tuple[str, int]: + """Store a file, returning (blob id, size). ``move`` consumes ``src``.""" + h = hashlib.sha256() + with src.open("rb") as f: + while chunk := f.read(_CHUNK): + h.update(chunk) + tmp = _tmp_file() + if move: + shutil.move(src, tmp) + else: + shutil.copyfile(src, tmp) + return _finalize(tmp, h.hexdigest(), ext) + + +def put_bytes(data: bytes, ext: str) -> tuple[str, int]: + tmp = _tmp_file() + tmp.write_bytes(data) + return _finalize(tmp, hashlib.sha256(data).hexdigest(), ext) + + +def read_bytes(blob_id: str) -> bytes: + return blob_path(blob_id).read_bytes() diff --git a/backend/app/converter/assets.py b/backend/app/converter/assets.py index 83cd62c..eed2d48 100644 --- a/backend/app/converter/assets.py +++ b/backend/app/converter/assets.py @@ -1,86 +1,86 @@ -"""Media conversion into web-friendly formats.""" - -import shutil -import subprocess -import uuid -from pathlib import Path - -from PIL import Image, ImageSequence - -IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp", ".bmp", ".gif", ".tga", ".avif"} -AUDIO_EXTS = {".ogg", ".mp3", ".wav", ".opus", ".flac", ".m4a", ".oga"} -VIDEO_EXTS = {".webm", ".ogv", ".mp4", ".mkv", ".avi", ".mpg", ".mpeg", ".m4v"} -FONT_EXTS = {".ttf", ".otf", ".woff", ".woff2", ".ttc"} - -WEBP_MAX = 16383 - - -def kind_of(path: Path) -> str | None: - ext = path.suffix.lower() - if ext in IMAGE_EXTS: - return "image" - if ext in AUDIO_EXTS: - return "audio" - if ext in VIDEO_EXTS: - return "video" - if ext in FONT_EXTS: - return "font" - return None - - -def target_ext(path: Path) -> str: - kind = kind_of(path) - return {"image": ".webp", "audio": ".ogg", "video": ".webm"}.get(kind, path.suffix.lower()) - - -def convert(src: Path, out_dir: Path) -> tuple[Path, str]: - """Convert one file into out_dir. Returns (output file, extension used for the blob).""" - kind = kind_of(src) - tmp = str(out_dir / uuid.uuid4().hex) - if kind == "image": - return _image(src, Path(tmp)) - if kind == "audio": - return _ffmpeg(src, Path(tmp + ".ogg"), - ["-vn", "-c:a", "libopus", "-b:a", "112k", "-vbr", "on"]), ".ogg" - if kind == "video": - if src.suffix.lower() == ".webm": - out = Path(tmp + ".webm") - shutil.copyfile(src, out) - return out, ".webm" - return _ffmpeg(src, Path(tmp + ".webm"), - ["-c:v", "libvpx-vp9", "-crf", "34", "-b:v", "0", "-deadline", "realtime", - "-cpu-used", "8", "-row-mt", "1", "-c:a", "libopus", "-b:a", "112k"] - ), ".webm" - out = Path(tmp + src.suffix.lower()) - shutil.copyfile(src, out) - return out, src.suffix.lower() - - -def _image(src: Path, tmp: Path) -> tuple[Path, str]: - with Image.open(src) as img: - animated = getattr(img, "is_animated", False) - if max(img.size) > WEBP_MAX: - out = tmp.with_suffix(".png") - img.save(out, "PNG", optimize=False) - return out, ".png" - out = tmp.with_suffix(".webp") - if animated: - frames = [f.convert("RGBA") for f in ImageSequence.Iterator(img)] - frames[0].save(out, "WEBP", save_all=True, append_images=frames[1:], quality=85, - duration=img.info.get("duration", 100), loop=0) - return out, ".webp" - has_alpha = img.mode in ("RGBA", "LA", "PA") or (img.mode == "P" and "transparency" in img.info) - img = img.convert("RGBA" if has_alpha else "RGB") - img.save(out, "WEBP", quality=88, method=4, alpha_quality=95) - return out, ".webp" - - -def _ffmpeg(src: Path, out: Path, args: list[str]) -> Path: - proc = subprocess.run( - ["ffmpeg", "-nostdin", "-hide_banner", "-loglevel", "error", "-y", "-i", str(src), - *args, str(out)], - capture_output=True, text=True, timeout=3600, - ) - if proc.returncode != 0: - raise RuntimeError(f"ffmpeg: {proc.stderr.strip()[-300:]}") - return out +"""Media conversion into web-friendly formats.""" + +import shutil +import subprocess +import uuid +from pathlib import Path + +from PIL import Image, ImageSequence + +IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp", ".bmp", ".gif", ".tga", ".avif"} +AUDIO_EXTS = {".ogg", ".mp3", ".wav", ".opus", ".flac", ".m4a", ".oga"} +VIDEO_EXTS = {".webm", ".ogv", ".mp4", ".mkv", ".avi", ".mpg", ".mpeg", ".m4v"} +FONT_EXTS = {".ttf", ".otf", ".woff", ".woff2", ".ttc"} + +WEBP_MAX = 16383 + + +def kind_of(path: Path) -> str | None: + ext = path.suffix.lower() + if ext in IMAGE_EXTS: + return "image" + if ext in AUDIO_EXTS: + return "audio" + if ext in VIDEO_EXTS: + return "video" + if ext in FONT_EXTS: + return "font" + return None + + +def target_ext(path: Path) -> str: + kind = kind_of(path) + return {"image": ".webp", "audio": ".ogg", "video": ".webm"}.get(kind, path.suffix.lower()) + + +def convert(src: Path, out_dir: Path) -> tuple[Path, str]: + """Convert one file into out_dir. Returns (output file, extension used for the blob).""" + kind = kind_of(src) + tmp = str(out_dir / uuid.uuid4().hex) + if kind == "image": + return _image(src, Path(tmp)) + if kind == "audio": + return _ffmpeg(src, Path(tmp + ".ogg"), + ["-vn", "-c:a", "libopus", "-b:a", "112k", "-vbr", "on"]), ".ogg" + if kind == "video": + if src.suffix.lower() == ".webm": + out = Path(tmp + ".webm") + shutil.copyfile(src, out) + return out, ".webm" + return _ffmpeg(src, Path(tmp + ".webm"), + ["-c:v", "libvpx-vp9", "-crf", "34", "-b:v", "0", "-deadline", "realtime", + "-cpu-used", "8", "-row-mt", "1", "-c:a", "libopus", "-b:a", "112k"] + ), ".webm" + out = Path(tmp + src.suffix.lower()) + shutil.copyfile(src, out) + return out, src.suffix.lower() + + +def _image(src: Path, tmp: Path) -> tuple[Path, str]: + with Image.open(src) as img: + animated = getattr(img, "is_animated", False) + if max(img.size) > WEBP_MAX: + out = tmp.with_suffix(".png") + img.save(out, "PNG", optimize=False) + return out, ".png" + out = tmp.with_suffix(".webp") + if animated: + frames = [f.convert("RGBA") for f in ImageSequence.Iterator(img)] + frames[0].save(out, "WEBP", save_all=True, append_images=frames[1:], quality=85, + duration=img.info.get("duration", 100), loop=0) + return out, ".webp" + has_alpha = img.mode in ("RGBA", "LA", "PA") or (img.mode == "P" and "transparency" in img.info) + img = img.convert("RGBA" if has_alpha else "RGB") + img.save(out, "WEBP", quality=88, method=4, alpha_quality=95) + return out, ".webp" + + +def _ffmpeg(src: Path, out: Path, args: list[str]) -> Path: + proc = subprocess.run( + ["ffmpeg", "-nostdin", "-hide_banner", "-loglevel", "error", "-y", "-i", str(src), + *args, str(out)], + capture_output=True, text=True, timeout=3600, + ) + if proc.returncode != 0: + raise RuntimeError(f"ffmpeg: {proc.stderr.strip()[-300:]}") + return out diff --git a/backend/app/converter/pyexpr.py b/backend/app/converter/pyexpr.py index 1f1313a..8223a67 100644 --- a/backend/app/converter/pyexpr.py +++ b/backend/app/converter/pyexpr.py @@ -1,189 +1,189 @@ -"""Translate the subset of Python used by visual novels into PGVN IR. - -Expressions become JSON trees ``{"k": kind, ...}``; statements become -``{"op": ..., ...}`` nodes executed by the player's JS interpreter. -Anything outside the subset raises :class:`Unsupported` so the caller can -report it and degrade gracefully. -""" - -import ast -from typing import Any - -Expr = dict[str, Any] -Stmt = dict[str, Any] - -BIN_OPS = { - ast.Add: "+", ast.Sub: "-", ast.Mult: "*", ast.Div: "/", ast.FloorDiv: "//", - ast.Mod: "%", ast.Pow: "**", ast.BitAnd: "&", ast.BitOr: "|", ast.BitXor: "^", - ast.LShift: "<<", ast.RShift: ">>", -} -CMP_OPS = { - ast.Eq: "==", ast.NotEq: "!=", ast.Lt: "<", ast.LtE: "<=", ast.Gt: ">", ast.GtE: ">=", - ast.In: "in", ast.NotIn: "not in", ast.Is: "is", ast.IsNot: "is not", -} -UNARY_OPS = {ast.Not: "not", ast.USub: "-", ast.UAdd: "+", ast.Invert: "~"} - -# Modules whose import is harmless: the player provides the functions it supports. -KNOWN_MODULES = {"random", "math", "renpy", "store", "time", "datetime"} - - -class Unsupported(Exception): - pass - - -def const(v: Any) -> Expr: - return {"k": "const", "v": v} - - -def parse_expr(source: str) -> Expr: - try: - tree = ast.parse(source.strip(), mode="eval") - except SyntaxError as e: - raise Unsupported(f"синтаксическая ошибка в выражении: {source!r}") from e - return expr(tree.body) - - -def parse_stmts(source: str) -> list[Stmt]: - try: - tree = ast.parse(_dedent(source)) - except SyntaxError as e: - raise Unsupported(f"синтаксическая ошибка в python: {e.msg} ({source[:60]!r})") from e - return stmts(tree.body) - - -def _dedent(source: str) -> str: - import textwrap - return textwrap.dedent(source) - - -def literal(source: str) -> Any: - """Evaluate a constant expression at conversion time, or raise Unsupported.""" - try: - return ast.literal_eval(source.strip()) - except (ValueError, SyntaxError, TypeError, MemoryError, RecursionError) as e: - raise Unsupported(f"не константа: {source!r}") from e - - -def expr(node: ast.AST) -> Expr: - match node: - case ast.Constant(value=v): - if isinstance(v, bytes): - v = v.decode("utf-8", "replace") - if isinstance(v, complex) or v is Ellipsis: - raise Unsupported("неподдерживаемая константа") - return const(v) - case ast.Name(id=name): - return {"k": "name", "id": name} - case ast.Attribute(value=ast.Name(id="store"), attr=attr): - return {"k": "name", "id": attr} - case ast.Attribute(value=value, attr=attr): - return {"k": "attr", "o": expr(value), "a": attr} - case ast.Subscript(value=value, slice=ast.Slice(lower=lo, upper=hi, step=st)): - return {"k": "slice", "o": expr(value), "lo": _opt(lo), "hi": _opt(hi), "st": _opt(st)} - case ast.Subscript(value=value, slice=index): - return {"k": "sub", "o": expr(value), "i": expr(index)} - case ast.BinOp(left=l, op=op, right=r): - if type(op) not in BIN_OPS: - raise Unsupported(f"оператор {type(op).__name__}") - return {"k": "bin", "op": BIN_OPS[type(op)], "l": expr(l), "r": expr(r)} - case ast.BoolOp(op=op, values=values): - return {"k": "bool", "op": "and" if isinstance(op, ast.And) else "or", - "v": [expr(v) for v in values]} - case ast.UnaryOp(op=op, operand=operand): - return {"k": "unary", "op": UNARY_OPS[type(op)], "v": expr(operand)} - case ast.Compare(left=l, ops=ops, comparators=rs): - return {"k": "cmp", "l": expr(l), "ops": [CMP_OPS[type(o)] for o in ops], - "r": [expr(r) for r in rs]} - case ast.IfExp(test=t, body=b, orelse=e): - return {"k": "if", "c": expr(t), "t": expr(b), "e": expr(e)} - case ast.List(elts=elts) | ast.Set(elts=elts): - return {"k": "list", "v": [expr(e) for e in elts]} - case ast.Tuple(elts=elts): - return {"k": "tuple", "v": [expr(e) for e in elts]} - case ast.Dict(keys=keys, values=values): - if any(k is None for k in keys): - raise Unsupported("распаковка ** в словаре") - return {"k": "dict", "v": [[expr(k), expr(v)] for k, v in zip(keys, values)]} - case ast.Call(func=func, args=args, keywords=kws): - if any(isinstance(a, ast.Starred) for a in args) or any(k.arg is None for k in kws): - raise Unsupported("распаковка аргументов *args/**kwargs") - return {"k": "call", "f": expr(func), "a": [expr(a) for a in args], - "kw": {k.arg: expr(k.value) for k in kws}} - case ast.JoinedStr(values=values): - parts: list[Any] = [] - for v in values: - if isinstance(v, ast.Constant): - parts.append(v.value) - elif isinstance(v, ast.FormattedValue): - parts.append(expr(v.value)) - return {"k": "fmt", "v": parts} - case (ast.ListComp(elt=elt, generators=[ast.comprehension(target=t, iter=it, ifs=ifs, - is_async=0)]) - | ast.GeneratorExp(elt=elt, generators=[ast.comprehension(target=t, iter=it, - ifs=ifs, is_async=0)]) - | ast.SetComp(elt=elt, generators=[ast.comprehension(target=t, iter=it, ifs=ifs, - is_async=0)])): - return {"k": "comp", "e": expr(elt), "t": target(t), "it": expr(it), - "if": [expr(i) for i in ifs]} - case ast.Lambda(): - raise Unsupported("lambda") - raise Unsupported(f"выражение {type(node).__name__}") - - -def _opt(node: ast.AST | None) -> Expr | None: - return None if node is None else expr(node) - - -def target(node: ast.AST) -> Expr: - match node: - case ast.Name() | ast.Attribute() | ast.Subscript(): - return expr(node) - case ast.Tuple(elts=elts) | ast.List(elts=elts): - return {"k": "tuple", "v": [target(e) for e in elts]} - raise Unsupported(f"присваивание в {type(node).__name__}") - - -def stmts(nodes: list[ast.stmt]) -> list[Stmt]: - out: list[Stmt] = [] - for n in nodes: - out.extend(stmt(n)) - return out - - -def stmt(node: ast.stmt) -> list[Stmt]: - match node: - case ast.Assign(targets=targets, value=value): - v = expr(value) - return [{"op": "set", "t": target(t), "v": v} for t in targets] - case ast.AnnAssign(target=t, value=value) if value is not None: - return [{"op": "set", "t": target(t), "v": expr(value)}] - case ast.AugAssign(target=t, op=op, value=value): - if type(op) not in BIN_OPS: - raise Unsupported(f"оператор {type(op).__name__}") - return [{"op": "aug", "t": target(t), "o": BIN_OPS[type(op)], "v": expr(value)}] - case ast.Expr(value=ast.Constant()): - return [] # docstring / bare literal - case ast.Expr(value=value): - return [{"op": "eval", "e": expr(value)}] - case ast.If(test=test, body=body, orelse=orelse): - return [{"op": "if", "c": expr(test), "then": stmts(body), "else": stmts(orelse)}] - case ast.While(test=test, body=body, orelse=[]): - return [{"op": "while", "c": expr(test), "body": stmts(body)}] - case ast.For(target=t, iter=it, body=body, orelse=[]): - return [{"op": "for", "t": target(t), "it": expr(it), "body": stmts(body)}] - case ast.Delete(targets=targets): - return [{"op": "del", "t": target(t)} for t in targets] - case ast.Pass() | ast.Global() | ast.Nonlocal(): - return [] - case ast.Break(): - return [{"op": "break"}] - case ast.Continue(): - return [{"op": "continue"}] - case ast.Import(names=names) | ast.ImportFrom(names=names): - mod = getattr(node, "module", None) or names[0].name - if mod.split(".")[0] in KNOWN_MODULES: - return [] - raise Unsupported(f"импорт модуля {mod}") - case ast.FunctionDef(name=name) | ast.ClassDef(name=name): - raise Unsupported(f"определение {name}()") - raise Unsupported(f"оператор {type(node).__name__}") +"""Translate the subset of Python used by visual novels into PGVN IR. + +Expressions become JSON trees ``{"k": kind, ...}``; statements become +``{"op": ..., ...}`` nodes executed by the player's JS interpreter. +Anything outside the subset raises :class:`Unsupported` so the caller can +report it and degrade gracefully. +""" + +import ast +from typing import Any + +Expr = dict[str, Any] +Stmt = dict[str, Any] + +BIN_OPS = { + ast.Add: "+", ast.Sub: "-", ast.Mult: "*", ast.Div: "/", ast.FloorDiv: "//", + ast.Mod: "%", ast.Pow: "**", ast.BitAnd: "&", ast.BitOr: "|", ast.BitXor: "^", + ast.LShift: "<<", ast.RShift: ">>", +} +CMP_OPS = { + ast.Eq: "==", ast.NotEq: "!=", ast.Lt: "<", ast.LtE: "<=", ast.Gt: ">", ast.GtE: ">=", + ast.In: "in", ast.NotIn: "not in", ast.Is: "is", ast.IsNot: "is not", +} +UNARY_OPS = {ast.Not: "not", ast.USub: "-", ast.UAdd: "+", ast.Invert: "~"} + +# Modules whose import is harmless: the player provides the functions it supports. +KNOWN_MODULES = {"random", "math", "renpy", "store", "time", "datetime"} + + +class Unsupported(Exception): + pass + + +def const(v: Any) -> Expr: + return {"k": "const", "v": v} + + +def parse_expr(source: str) -> Expr: + try: + tree = ast.parse(source.strip(), mode="eval") + except SyntaxError as e: + raise Unsupported(f"синтаксическая ошибка в выражении: {source!r}") from e + return expr(tree.body) + + +def parse_stmts(source: str) -> list[Stmt]: + try: + tree = ast.parse(_dedent(source)) + except SyntaxError as e: + raise Unsupported(f"синтаксическая ошибка в python: {e.msg} ({source[:60]!r})") from e + return stmts(tree.body) + + +def _dedent(source: str) -> str: + import textwrap + return textwrap.dedent(source) + + +def literal(source: str) -> Any: + """Evaluate a constant expression at conversion time, or raise Unsupported.""" + try: + return ast.literal_eval(source.strip()) + except (ValueError, SyntaxError, TypeError, MemoryError, RecursionError) as e: + raise Unsupported(f"не константа: {source!r}") from e + + +def expr(node: ast.AST) -> Expr: + match node: + case ast.Constant(value=v): + if isinstance(v, bytes): + v = v.decode("utf-8", "replace") + if isinstance(v, complex) or v is Ellipsis: + raise Unsupported("неподдерживаемая константа") + return const(v) + case ast.Name(id=name): + return {"k": "name", "id": name} + case ast.Attribute(value=ast.Name(id="store"), attr=attr): + return {"k": "name", "id": attr} + case ast.Attribute(value=value, attr=attr): + return {"k": "attr", "o": expr(value), "a": attr} + case ast.Subscript(value=value, slice=ast.Slice(lower=lo, upper=hi, step=st)): + return {"k": "slice", "o": expr(value), "lo": _opt(lo), "hi": _opt(hi), "st": _opt(st)} + case ast.Subscript(value=value, slice=index): + return {"k": "sub", "o": expr(value), "i": expr(index)} + case ast.BinOp(left=l, op=op, right=r): + if type(op) not in BIN_OPS: + raise Unsupported(f"оператор {type(op).__name__}") + return {"k": "bin", "op": BIN_OPS[type(op)], "l": expr(l), "r": expr(r)} + case ast.BoolOp(op=op, values=values): + return {"k": "bool", "op": "and" if isinstance(op, ast.And) else "or", + "v": [expr(v) for v in values]} + case ast.UnaryOp(op=op, operand=operand): + return {"k": "unary", "op": UNARY_OPS[type(op)], "v": expr(operand)} + case ast.Compare(left=l, ops=ops, comparators=rs): + return {"k": "cmp", "l": expr(l), "ops": [CMP_OPS[type(o)] for o in ops], + "r": [expr(r) for r in rs]} + case ast.IfExp(test=t, body=b, orelse=e): + return {"k": "if", "c": expr(t), "t": expr(b), "e": expr(e)} + case ast.List(elts=elts) | ast.Set(elts=elts): + return {"k": "list", "v": [expr(e) for e in elts]} + case ast.Tuple(elts=elts): + return {"k": "tuple", "v": [expr(e) for e in elts]} + case ast.Dict(keys=keys, values=values): + if any(k is None for k in keys): + raise Unsupported("распаковка ** в словаре") + return {"k": "dict", "v": [[expr(k), expr(v)] for k, v in zip(keys, values)]} + case ast.Call(func=func, args=args, keywords=kws): + if any(isinstance(a, ast.Starred) for a in args) or any(k.arg is None for k in kws): + raise Unsupported("распаковка аргументов *args/**kwargs") + return {"k": "call", "f": expr(func), "a": [expr(a) for a in args], + "kw": {k.arg: expr(k.value) for k in kws}} + case ast.JoinedStr(values=values): + parts: list[Any] = [] + for v in values: + if isinstance(v, ast.Constant): + parts.append(v.value) + elif isinstance(v, ast.FormattedValue): + parts.append(expr(v.value)) + return {"k": "fmt", "v": parts} + case (ast.ListComp(elt=elt, generators=[ast.comprehension(target=t, iter=it, ifs=ifs, + is_async=0)]) + | ast.GeneratorExp(elt=elt, generators=[ast.comprehension(target=t, iter=it, + ifs=ifs, is_async=0)]) + | ast.SetComp(elt=elt, generators=[ast.comprehension(target=t, iter=it, ifs=ifs, + is_async=0)])): + return {"k": "comp", "e": expr(elt), "t": target(t), "it": expr(it), + "if": [expr(i) for i in ifs]} + case ast.Lambda(): + raise Unsupported("lambda") + raise Unsupported(f"выражение {type(node).__name__}") + + +def _opt(node: ast.AST | None) -> Expr | None: + return None if node is None else expr(node) + + +def target(node: ast.AST) -> Expr: + match node: + case ast.Name() | ast.Attribute() | ast.Subscript(): + return expr(node) + case ast.Tuple(elts=elts) | ast.List(elts=elts): + return {"k": "tuple", "v": [target(e) for e in elts]} + raise Unsupported(f"присваивание в {type(node).__name__}") + + +def stmts(nodes: list[ast.stmt]) -> list[Stmt]: + out: list[Stmt] = [] + for n in nodes: + out.extend(stmt(n)) + return out + + +def stmt(node: ast.stmt) -> list[Stmt]: + match node: + case ast.Assign(targets=targets, value=value): + v = expr(value) + return [{"op": "set", "t": target(t), "v": v} for t in targets] + case ast.AnnAssign(target=t, value=value) if value is not None: + return [{"op": "set", "t": target(t), "v": expr(value)}] + case ast.AugAssign(target=t, op=op, value=value): + if type(op) not in BIN_OPS: + raise Unsupported(f"оператор {type(op).__name__}") + return [{"op": "aug", "t": target(t), "o": BIN_OPS[type(op)], "v": expr(value)}] + case ast.Expr(value=ast.Constant()): + return [] # docstring / bare literal + case ast.Expr(value=value): + return [{"op": "eval", "e": expr(value)}] + case ast.If(test=test, body=body, orelse=orelse): + return [{"op": "if", "c": expr(test), "then": stmts(body), "else": stmts(orelse)}] + case ast.While(test=test, body=body, orelse=[]): + return [{"op": "while", "c": expr(test), "body": stmts(body)}] + case ast.For(target=t, iter=it, body=body, orelse=[]): + return [{"op": "for", "t": target(t), "it": expr(it), "body": stmts(body)}] + case ast.Delete(targets=targets): + return [{"op": "del", "t": target(t)} for t in targets] + case ast.Pass() | ast.Global() | ast.Nonlocal(): + return [] + case ast.Break(): + return [{"op": "break"}] + case ast.Continue(): + return [{"op": "continue"}] + case ast.Import(names=names) | ast.ImportFrom(names=names): + mod = getattr(node, "module", None) or names[0].name + if mod.split(".")[0] in KNOWN_MODULES: + return [] + raise Unsupported(f"импорт модуля {mod}") + case ast.FunctionDef(name=name) | ast.ClassDef(name=name): + raise Unsupported(f"определение {name}()") + raise Unsupported(f"оператор {type(node).__name__}") diff --git a/backend/app/converter/rpa.py b/backend/app/converter/rpa.py index 80aa135..6fb0e82 100644 --- a/backend/app/converter/rpa.py +++ b/backend/app/converter/rpa.py @@ -1,109 +1,109 @@ -"""Reader for Ren'Py .rpa archives (RPA-2.0, RPA-3.x and common variants). - -The index is a zlib-compressed pickle, so it is loaded with a restricted -unpickler that only allows plain containers — archives are untrusted input. -""" - -import io -import pickle -import zlib -from collections.abc import Callable -from pathlib import Path - - -class RpaError(Exception): - pass - - -class _SafeUnpickler(pickle.Unpickler): - _ALLOWED = {("_codecs", "encode"), ("builtins", "set"), ("builtins", "frozenset"), - ("__builtin__", "set"), ("builtins", "bytearray"), ("__builtin__", "bytearray"), - ("builtins", "bytes"), ("__builtin__", "bytes")} - - def find_class(self, module, name): - if (module, name) in self._ALLOWED: - import builtins - import codecs - return codecs.encode if module == "_codecs" else getattr(builtins, name) - raise pickle.UnpicklingError(f"запрещённый тип в индексе RPA: {module}.{name}") - - -def _text(v) -> str: - return v.decode("utf-8", "replace") if isinstance(v, bytes) else str(v) - - -def read_index(path: Path) -> dict[str, list[tuple[int, int, bytes]]]: - with path.open("rb") as f: - header = f.readline(200).decode("latin-1").strip() - parts = header.split() - if not parts or not parts[0].startswith("RPA-"): - raise RpaError(f"{path.name}: неизвестный формат архива ({header[:20]!r})") - version = parts[0] - try: - offset = int(parts[1], 16) - key = 0 - if version != "RPA-2.0": - for token in parts[2:]: - key ^= int(token, 16) - except (IndexError, ValueError) as e: - raise RpaError(f"{path.name}: повреждён заголовок") from e - f.seek(offset) - try: - raw = _SafeUnpickler(io.BytesIO(zlib.decompress(f.read())), encoding="bytes").load() - except (zlib.error, pickle.UnpicklingError, EOFError) as e: - raise RpaError(f"{path.name}: не удалось прочитать индекс: {e}") from e - - index: dict[str, list[tuple[int, int, bytes]]] = {} - for name, entries in raw.items(): - fixed = [] - for entry in entries: - off, length = entry[0] ^ key, entry[1] ^ key - prefix = entry[2] if len(entry) > 2 else b"" - if isinstance(prefix, str): - prefix = prefix.encode("latin-1") - fixed.append((off, length, prefix)) - index[_text(name)] = fixed - return index - - -def extract_rpa(path: Path, dest: Path, only: Callable[[str], bool] | None = None) -> int: - """Extract files of an archive into dest (all, or those `only` accepts).""" - index = read_index(path) - root = dest.resolve() - count = 0 - with path.open("rb") as f: - for name, entries in index.items(): - if only is not None and not only(name): - continue - target = (dest / name).resolve() - if not target.is_relative_to(root): - continue - target.parent.mkdir(parents=True, exist_ok=True) - with target.open("wb") as out: - for off, length, prefix in entries: - out.write(prefix) - f.seek(off) - remaining = length - len(prefix) - while remaining > 0: - chunk = f.read(min(remaining, 1 << 20)) - if not chunk: - break - out.write(chunk) - remaining -= len(chunk) - count += 1 - return count - - -def write_rpa(path: Path, files: dict[str, bytes], key: int = 0x42424242) -> None: - """Write an RPA-3.0 archive (used by tests).""" - with path.open("wb") as f: - f.write(b"RPA-3.0 " + b"0" * 16 + b" " + b"0" * 8 + b"\n") - index = {} - for name, data in files.items(): - off = f.tell() - f.write(data) - index[name] = [(off ^ key, len(data) ^ key, b"")] - idx_off = f.tell() - f.write(zlib.compress(pickle.dumps(index, protocol=2))) - f.seek(0) - f.write(f"RPA-3.0 {idx_off:016x} {key:08x}\n".encode()) +"""Reader for Ren'Py .rpa archives (RPA-2.0, RPA-3.x and common variants). + +The index is a zlib-compressed pickle, so it is loaded with a restricted +unpickler that only allows plain containers — archives are untrusted input. +""" + +import io +import pickle +import zlib +from collections.abc import Callable +from pathlib import Path + + +class RpaError(Exception): + pass + + +class _SafeUnpickler(pickle.Unpickler): + _ALLOWED = {("_codecs", "encode"), ("builtins", "set"), ("builtins", "frozenset"), + ("__builtin__", "set"), ("builtins", "bytearray"), ("__builtin__", "bytearray"), + ("builtins", "bytes"), ("__builtin__", "bytes")} + + def find_class(self, module, name): + if (module, name) in self._ALLOWED: + import builtins + import codecs + return codecs.encode if module == "_codecs" else getattr(builtins, name) + raise pickle.UnpicklingError(f"запрещённый тип в индексе RPA: {module}.{name}") + + +def _text(v) -> str: + return v.decode("utf-8", "replace") if isinstance(v, bytes) else str(v) + + +def read_index(path: Path) -> dict[str, list[tuple[int, int, bytes]]]: + with path.open("rb") as f: + header = f.readline(200).decode("latin-1").strip() + parts = header.split() + if not parts or not parts[0].startswith("RPA-"): + raise RpaError(f"{path.name}: неизвестный формат архива ({header[:20]!r})") + version = parts[0] + try: + offset = int(parts[1], 16) + key = 0 + if version != "RPA-2.0": + for token in parts[2:]: + key ^= int(token, 16) + except (IndexError, ValueError) as e: + raise RpaError(f"{path.name}: повреждён заголовок") from e + f.seek(offset) + try: + raw = _SafeUnpickler(io.BytesIO(zlib.decompress(f.read())), encoding="bytes").load() + except (zlib.error, pickle.UnpicklingError, EOFError) as e: + raise RpaError(f"{path.name}: не удалось прочитать индекс: {e}") from e + + index: dict[str, list[tuple[int, int, bytes]]] = {} + for name, entries in raw.items(): + fixed = [] + for entry in entries: + off, length = entry[0] ^ key, entry[1] ^ key + prefix = entry[2] if len(entry) > 2 else b"" + if isinstance(prefix, str): + prefix = prefix.encode("latin-1") + fixed.append((off, length, prefix)) + index[_text(name)] = fixed + return index + + +def extract_rpa(path: Path, dest: Path, only: Callable[[str], bool] | None = None) -> int: + """Extract files of an archive into dest (all, or those `only` accepts).""" + index = read_index(path) + root = dest.resolve() + count = 0 + with path.open("rb") as f: + for name, entries in index.items(): + if only is not None and not only(name): + continue + target = (dest / name).resolve() + if not target.is_relative_to(root): + continue + target.parent.mkdir(parents=True, exist_ok=True) + with target.open("wb") as out: + for off, length, prefix in entries: + out.write(prefix) + f.seek(off) + remaining = length - len(prefix) + while remaining > 0: + chunk = f.read(min(remaining, 1 << 20)) + if not chunk: + break + out.write(chunk) + remaining -= len(chunk) + count += 1 + return count + + +def write_rpa(path: Path, files: dict[str, bytes], key: int = 0x42424242) -> None: + """Write an RPA-3.0 archive (used by tests).""" + with path.open("wb") as f: + f.write(b"RPA-3.0 " + b"0" * 16 + b" " + b"0" * 8 + b"\n") + index = {} + for name, data in files.items(): + off = f.tell() + f.write(data) + index[name] = [(off ^ key, len(data) ^ key, b"")] + idx_off = f.tell() + f.write(zlib.compress(pickle.dumps(index, protocol=2))) + f.seek(0) + f.write(f"RPA-3.0 {idx_off:016x} {key:08x}\n".encode()) diff --git a/backend/app/import_dir.py b/backend/app/import_dir.py index 32a4292..2f72e97 100644 --- a/backend/app/import_dir.py +++ b/backend/app/import_dir.py @@ -1,48 +1,48 @@ -"""Import an already unpacked game from a folder on the server. - - python -m app.import_dir /games/Some-Game [--title "Name"] [--owner admin] - -The folder is only read; the converted copy goes to the blob store like an upload. -""" - -import argparse -import asyncio -from pathlib import Path - -from sqlalchemy import select - -from app.db import SessionLocal -from app.models import Game, GameVersion, Role, User -from app.worker import convert - - -async def main() -> None: - ap = argparse.ArgumentParser() - ap.add_argument("path", type=Path) - ap.add_argument("--title") - ap.add_argument("--owner", help="username; defaults to the first admin") - args = ap.parse_args() - if not args.path.is_dir(): - raise SystemExit(f"{args.path} is not a directory") - - async with SessionLocal() as db: - stmt = select(User).join(Role).order_by(User.id) - stmt = stmt.where(User.username == args.owner) if args.owner else stmt.where(Role.name == "admin") - owner = await db.scalar(stmt.limit(1)) - if owner is None: - raise SystemExit("owner not found") - game = Game(title=args.title or args.path.name, created_by_id=owner.id) - db.add(game) - await db.flush() - version = GameVersion(game_id=game.id, owner_id=owner.id, name="Оригинал", - upload_name=args.path.name, status="importing") - db.add(version) - await db.commit() - game_id, version_id = game.id, version.id - - ok = await convert(version_id, args.path, args.path.name) - print(f"game {game_id}, version {version_id}: {'ready' if ok else 'failed'}") - - -if __name__ == "__main__": - asyncio.run(main()) +"""Import an already unpacked game from a folder on the server. + + python -m app.import_dir /games/Some-Game [--title "Name"] [--owner admin] + +The folder is only read; the converted copy goes to the blob store like an upload. +""" + +import argparse +import asyncio +from pathlib import Path + +from sqlalchemy import select + +from app.db import SessionLocal +from app.models import Game, GameVersion, Role, User +from app.worker import convert + + +async def main() -> None: + ap = argparse.ArgumentParser() + ap.add_argument("path", type=Path) + ap.add_argument("--title") + ap.add_argument("--owner", help="username; defaults to the first admin") + args = ap.parse_args() + if not args.path.is_dir(): + raise SystemExit(f"{args.path} is not a directory") + + async with SessionLocal() as db: + stmt = select(User).join(Role).order_by(User.id) + stmt = stmt.where(User.username == args.owner) if args.owner else stmt.where(Role.name == "admin") + owner = await db.scalar(stmt.limit(1)) + if owner is None: + raise SystemExit("owner not found") + game = Game(title=args.title or args.path.name, created_by_id=owner.id) + db.add(game) + await db.flush() + version = GameVersion(game_id=game.id, owner_id=owner.id, name="Оригинал", + upload_name=args.path.name, status="importing") + db.add(version) + await db.commit() + game_id, version_id = game.id, version.id + + ok = await convert(version_id, args.path, args.path.name) + print(f"game {game_id}, version {version_id}: {'ready' if ok else 'failed'}") + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/backend/app/main.py b/backend/app/main.py index 9e47f3e..858aa0a 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -1,41 +1,41 @@ -import logging -from contextlib import asynccontextmanager - -from fastapi import FastAPI, HTTPException -from fastapi.responses import FileResponse - -from app.blobs import BLOB_ID_RE, blob_path -from app.bootstrap import bootstrap -from app.config import get_settings -from app.db import SessionLocal -from app.routers import auth, games, roles, users, versions - -logging.basicConfig(level=logging.INFO) - - -@asynccontextmanager -async def lifespan(_: FastAPI): - if get_settings().secret_key == "change-me": - logging.getLogger(__name__).warning("SECRET_KEY is the default value — set it in .env") - async with SessionLocal() as db: - await bootstrap(db) - yield - - -app = FastAPI(title="pgames-manager", lifespan=lifespan, docs_url="/api/docs", - openapi_url="/api/openapi.json") -for r in (auth, users, roles, games, versions): - app.include_router(r.router) - - -@app.get("/api/health") -async def health() -> dict[str, str]: - return {"status": "ok"} - - -@app.get("/blobs/{blob_id}") -async def blob(blob_id: str) -> FileResponse: - """Fallback for running without nginx; in compose nginx serves /blobs itself.""" - if not BLOB_ID_RE.match(blob_id) or not (path := blob_path(blob_id)).is_file(): - raise HTTPException(404) - return FileResponse(path, headers={"Cache-Control": "public, max-age=31536000, immutable"}) +import logging +from contextlib import asynccontextmanager + +from fastapi import FastAPI, HTTPException +from fastapi.responses import FileResponse + +from app.blobs import BLOB_ID_RE, blob_path +from app.bootstrap import bootstrap +from app.config import get_settings +from app.db import SessionLocal +from app.routers import auth, games, roles, users, versions + +logging.basicConfig(level=logging.INFO) + + +@asynccontextmanager +async def lifespan(_: FastAPI): + if get_settings().secret_key == "change-me": + logging.getLogger(__name__).warning("SECRET_KEY is the default value — set it in .env") + async with SessionLocal() as db: + await bootstrap(db) + yield + + +app = FastAPI(title="pgames-manager", lifespan=lifespan, docs_url="/api/docs", + openapi_url="/api/openapi.json") +for r in (auth, users, roles, games, versions): + app.include_router(r.router) + + +@app.get("/api/health") +async def health() -> dict[str, str]: + return {"status": "ok"} + + +@app.get("/blobs/{blob_id}") +async def blob(blob_id: str) -> FileResponse: + """Fallback for running without nginx; in compose nginx serves /blobs itself.""" + if not BLOB_ID_RE.match(blob_id) or not (path := blob_path(blob_id)).is_file(): + raise HTTPException(404) + return FileResponse(path, headers={"Cache-Control": "public, max-age=31536000, immutable"}) diff --git a/backend/app/models.py b/backend/app/models.py index fef2904..2f9642c 100644 --- a/backend/app/models.py +++ b/backend/app/models.py @@ -1,99 +1,99 @@ -from datetime import datetime -from typing import Any - -from sqlalchemy import ( - ARRAY, - BigInteger, - Boolean, - DateTime, - ForeignKey, - Integer, - String, - Text, - UniqueConstraint, - func, -) -from sqlalchemy.dialects.postgresql import JSONB -from sqlalchemy.orm import Mapped, mapped_column, relationship - -from app.db import Base - - -class Role(Base): - __tablename__ = "roles" - - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64), unique=True) - permissions: Mapped[list[str]] = mapped_column(ARRAY(String), default=list) - is_system: Mapped[bool] = mapped_column(Boolean, default=False) - - -class User(Base): - __tablename__ = "users" - - id: Mapped[int] = mapped_column(primary_key=True) - username: Mapped[str] = mapped_column(String(64), unique=True) - password_hash: Mapped[str] = mapped_column(String(255)) - role_id: Mapped[int] = mapped_column(ForeignKey("roles.id")) - is_active: Mapped[bool] = mapped_column(Boolean, default=True) - must_change_password: Mapped[bool] = mapped_column(Boolean, default=True) - must_change_username: Mapped[bool] = mapped_column(Boolean, default=False) - token_version: Mapped[int] = mapped_column(Integer, default=0) - created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) - - role: Mapped[Role] = relationship(lazy="joined") - - @property - def permissions(self) -> list[str]: - return sorted(self.role.permissions) if self.is_active else [] - - -class Game(Base): - __tablename__ = "games" - - id: Mapped[int] = mapped_column(primary_key=True) - title: Mapped[str] = mapped_column(String(255)) - description: Mapped[str] = mapped_column(Text, default="") - tags: Mapped[list[str]] = mapped_column(ARRAY(String), default=list) - engine: Mapped[str] = mapped_column(String(32), default="renpy") - cover_hash: Mapped[str | None] = mapped_column(String(80)) - created_by_id: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL")) - created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) - - created_by: Mapped[User | None] = relationship(lazy="joined") - - -class GameVersion(Base): - __tablename__ = "game_versions" - - id: Mapped[int] = mapped_column(primary_key=True) - game_id: Mapped[int] = mapped_column(ForeignKey("games.id", ondelete="CASCADE"), index=True) - parent_id: Mapped[int | None] = mapped_column(ForeignKey("game_versions.id", ondelete="SET NULL")) - owner_id: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL")) - name: Mapped[str] = mapped_column(String(255)) - description: Mapped[str] = mapped_column(Text, default="") - # uploading -> queued -> converting -> ready | failed; importing = app.import_dir run - status: Mapped[str] = mapped_column(String(16), default="uploading", index=True) - progress: Mapped[str] = mapped_column(String(255), default="") - upload_name: Mapped[str | None] = mapped_column(String(255)) - manifest: Mapped[dict[str, Any] | None] = mapped_column(JSONB) - report: Mapped[dict[str, Any] | None] = mapped_column(JSONB) - size_bytes: Mapped[int] = mapped_column(BigInteger, default=0) - created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) - - owner: Mapped[User | None] = relationship(lazy="joined") - - -class Save(Base): - __tablename__ = "saves" - __table_args__ = (UniqueConstraint("user_id", "version_id", "slot"),) - - id: Mapped[int] = mapped_column(primary_key=True) - user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE")) - version_id: Mapped[int] = mapped_column(ForeignKey("game_versions.id", ondelete="CASCADE")) - slot: Mapped[str] = mapped_column(String(32)) - title: Mapped[str] = mapped_column(String(255), default="") - data: Mapped[dict[str, Any]] = mapped_column(JSONB) - updated_at: Mapped[datetime] = mapped_column( - DateTime(timezone=True), server_default=func.now(), onupdate=func.now() - ) +from datetime import datetime +from typing import Any + +from sqlalchemy import ( + ARRAY, + BigInteger, + Boolean, + DateTime, + ForeignKey, + Integer, + String, + Text, + UniqueConstraint, + func, +) +from sqlalchemy.dialects.postgresql import JSONB +from sqlalchemy.orm import Mapped, mapped_column, relationship + +from app.db import Base + + +class Role(Base): + __tablename__ = "roles" + + id: Mapped[int] = mapped_column(primary_key=True) + name: Mapped[str] = mapped_column(String(64), unique=True) + permissions: Mapped[list[str]] = mapped_column(ARRAY(String), default=list) + is_system: Mapped[bool] = mapped_column(Boolean, default=False) + + +class User(Base): + __tablename__ = "users" + + id: Mapped[int] = mapped_column(primary_key=True) + username: Mapped[str] = mapped_column(String(64), unique=True) + password_hash: Mapped[str] = mapped_column(String(255)) + role_id: Mapped[int] = mapped_column(ForeignKey("roles.id")) + is_active: Mapped[bool] = mapped_column(Boolean, default=True) + must_change_password: Mapped[bool] = mapped_column(Boolean, default=True) + must_change_username: Mapped[bool] = mapped_column(Boolean, default=False) + token_version: Mapped[int] = mapped_column(Integer, default=0) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + + role: Mapped[Role] = relationship(lazy="joined") + + @property + def permissions(self) -> list[str]: + return sorted(self.role.permissions) if self.is_active else [] + + +class Game(Base): + __tablename__ = "games" + + id: Mapped[int] = mapped_column(primary_key=True) + title: Mapped[str] = mapped_column(String(255)) + description: Mapped[str] = mapped_column(Text, default="") + tags: Mapped[list[str]] = mapped_column(ARRAY(String), default=list) + engine: Mapped[str] = mapped_column(String(32), default="renpy") + cover_hash: Mapped[str | None] = mapped_column(String(80)) + created_by_id: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL")) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + + created_by: Mapped[User | None] = relationship(lazy="joined") + + +class GameVersion(Base): + __tablename__ = "game_versions" + + id: Mapped[int] = mapped_column(primary_key=True) + game_id: Mapped[int] = mapped_column(ForeignKey("games.id", ondelete="CASCADE"), index=True) + parent_id: Mapped[int | None] = mapped_column(ForeignKey("game_versions.id", ondelete="SET NULL")) + owner_id: Mapped[int | None] = mapped_column(ForeignKey("users.id", ondelete="SET NULL")) + name: Mapped[str] = mapped_column(String(255)) + description: Mapped[str] = mapped_column(Text, default="") + # uploading -> queued -> converting -> ready | failed; importing = app.import_dir run + status: Mapped[str] = mapped_column(String(16), default="uploading", index=True) + progress: Mapped[str] = mapped_column(String(255), default="") + upload_name: Mapped[str | None] = mapped_column(String(255)) + manifest: Mapped[dict[str, Any] | None] = mapped_column(JSONB) + report: Mapped[dict[str, Any] | None] = mapped_column(JSONB) + size_bytes: Mapped[int] = mapped_column(BigInteger, default=0) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + + owner: Mapped[User | None] = relationship(lazy="joined") + + +class Save(Base): + __tablename__ = "saves" + __table_args__ = (UniqueConstraint("user_id", "version_id", "slot"),) + + id: Mapped[int] = mapped_column(primary_key=True) + user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE")) + version_id: Mapped[int] = mapped_column(ForeignKey("game_versions.id", ondelete="CASCADE")) + slot: Mapped[str] = mapped_column(String(32)) + title: Mapped[str] = mapped_column(String(255), default="") + data: Mapped[dict[str, Any]] = mapped_column(JSONB) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), server_default=func.now(), onupdate=func.now() + ) diff --git a/backend/app/routers/versions.py b/backend/app/routers/versions.py index 00f3557..5549d7a 100644 --- a/backend/app/routers/versions.py +++ b/backend/app/routers/versions.py @@ -1,205 +1,205 @@ -from typing import Annotated, Any - -from fastapi import APIRouter, Depends, HTTPException, Request, status -from sqlalchemy import select -from starlette.requests import ClientDisconnect - -from app.deps import DB, CurrentUser, has_perm, require -from app.models import GameVersion, Save, User -from app.routers.games import get_game -from app.schemas import SaveDetailOut, SaveIn, SaveOut, VersionDetailOut, VersionIn, VersionOut -from app.uploads import remove_upload, upload_path - -router = APIRouter(prefix="/api", tags=["versions"]) -Player = Annotated[User, Depends(require("games.play"))] -Uploader = Annotated[User, Depends(require("games.upload"))] - -ARCHIVE_EXTS = (".zip", ".7z", ".rar", ".tar", ".tar.gz", ".tgz", ".tar.bz2", ".tar.xz") - - -def can_see(user: User, v: GameVersion) -> bool: - return v.status == "ready" or v.owner_id == user.id or can_manage(user, v) - - -def can_manage(user: User, v: GameVersion) -> bool: - return ( - v.owner_id == user.id - or has_perm(user, "versions.manage_any") - or has_perm(user, "games.delete") - ) - - -async def get_version(db: DB, user: User, version_id: int) -> GameVersion: - v = await db.get(GameVersion, version_id) - if not v or not can_see(user, v): - raise HTTPException(status.HTTP_404_NOT_FOUND, "Версия не найдена") - return v - - -@router.post("/games/{game_id}/upload", response_model=VersionOut, status_code=201) -async def upload_version( - game_id: int, request: Request, user: Uploader, db: DB, filename: str, name: str = "Оригинал" -) -> GameVersion: - """Stream a raw archive body to disk and queue it for conversion.""" - game = await get_game(db, game_id) - if not filename.lower().endswith(ARCHIVE_EXTS): - raise HTTPException(status.HTTP_400_BAD_REQUEST, - f"Поддерживаемые архивы: {', '.join(ARCHIVE_EXTS)}") - version = GameVersion(game_id=game.id, owner_id=user.id, name=name.strip() or "Оригинал", - upload_name=filename, status="uploading") - db.add(version) - await db.commit() - - path = upload_path(version.id) - path.parent.mkdir(parents=True, exist_ok=True) - size = 0 - try: - with path.open("wb") as f: - async for chunk in request.stream(): - f.write(chunk) - size += len(chunk) - except Exception as e: - remove_upload(version.id) - await db.delete(version) - await db.commit() - if isinstance(e, ClientDisconnect): - # usually a proxy in front cut the body off (e.g. its own 413) - raise HTTPException(status.HTTP_400_BAD_REQUEST, "Загрузка прервана") from None - raise - if size == 0: - remove_upload(version.id) - await db.delete(version) - await db.commit() - raise HTTPException(status.HTTP_400_BAD_REQUEST, "Пустой файл") - - version.status = "queued" - version.progress = "В очереди на конвертацию" - version.size_bytes = size - await db.commit() - return version - - -@router.get("/versions/{version_id}", response_model=VersionDetailOut) -async def version_detail(version_id: int, user: Player, db: DB) -> GameVersion: - return await get_version(db, user, version_id) - - -@router.get("/versions/{version_id}/manifest") -async def version_manifest(version_id: int, user: Player, db: DB) -> dict[str, Any]: - v = await get_version(db, user, version_id) - if v.status != "ready" or not v.manifest: - raise HTTPException(status.HTTP_409_CONFLICT, "Версия ещё не готова") - return v.manifest - - -@router.post("/versions/{version_id}/fork", response_model=VersionOut, status_code=201) -async def fork_version(version_id: int, body: VersionIn, - user: Annotated[User, Depends(require("games.play", "versions.fork"))], - db: DB) -> GameVersion: - parent = await get_version(db, user, version_id) - if parent.status != "ready": - raise HTTPException(status.HTTP_409_CONFLICT, "Форкнуть можно только готовую версию") - fork = GameVersion( - game_id=parent.game_id, - parent_id=parent.id, - owner_id=user.id, - name=body.name.strip(), - description=body.description, - status="ready", - manifest=parent.manifest, - report=parent.report, - size_bytes=parent.size_bytes, - ) - db.add(fork) - await db.commit() - return fork - - -@router.patch("/versions/{version_id}", response_model=VersionOut) -async def update_version(version_id: int, body: VersionIn, user: CurrentUser, db: DB) -> GameVersion: - v = await get_version(db, user, version_id) - if not can_manage(user, v): - raise HTTPException(status.HTTP_403_FORBIDDEN, "Нет прав на эту версию") - v.name = body.name.strip() - v.description = body.description - await db.commit() - return v - - -@router.post("/versions/{version_id}/retry", response_model=VersionOut) -async def retry_version(version_id: int, user: CurrentUser, db: DB) -> GameVersion: - v = await get_version(db, user, version_id) - if not can_manage(user, v): - raise HTTPException(status.HTTP_403_FORBIDDEN, "Нет прав на эту версию") - if v.status != "failed" or not upload_path(v.id).exists(): - raise HTTPException(status.HTTP_409_CONFLICT, "Повторная конвертация невозможна") - v.status = "queued" - v.progress = "В очереди на конвертацию" - await db.commit() - return v - - -@router.delete("/versions/{version_id}", status_code=204) -async def delete_version(version_id: int, user: CurrentUser, db: DB) -> None: - v = await get_version(db, user, version_id) - if not can_manage(user, v): - raise HTTPException(status.HTTP_403_FORBIDDEN, "Нет прав на эту версию") - if v.status in ("converting", "importing"): - raise HTTPException(status.HTTP_409_CONFLICT, "Версия сейчас конвертируется") - await db.delete(v) - await db.commit() - remove_upload(version_id) - - -# --- saves ----------------------------------------------------------------- - -def _check_slot(slot: str) -> str: - if not slot or len(slot) > 32 or not all(c.isalnum() or c in "_-" for c in slot): - raise HTTPException(status.HTTP_400_BAD_REQUEST, "Неверный слот") - return slot - - -@router.get("/versions/{version_id}/saves", response_model=list[SaveOut]) -async def list_saves(version_id: int, user: Player, db: DB) -> list[Save]: - await get_version(db, user, version_id) - return list((await db.scalars( - select(Save).where(Save.user_id == user.id, Save.version_id == version_id) - .order_by(Save.updated_at.desc()) - )).all()) - - -async def _find_save(db: DB, user: User, version_id: int, slot: str) -> Save | None: - return await db.scalar(select(Save).where( - Save.user_id == user.id, Save.version_id == version_id, Save.slot == _check_slot(slot))) - - -@router.get("/versions/{version_id}/saves/{slot}", response_model=SaveDetailOut) -async def get_save(version_id: int, slot: str, user: Player, db: DB) -> Save: - save = await _find_save(db, user, version_id, slot) - if not save: - raise HTTPException(status.HTTP_404_NOT_FOUND, "Сохранение не найдено") - return save - - -@router.put("/versions/{version_id}/saves/{slot}", response_model=SaveOut) -async def put_save(version_id: int, slot: str, body: SaveIn, user: Player, db: DB) -> Save: - await get_version(db, user, version_id) - save = await _find_save(db, user, version_id, slot) - if save is None: - save = Save(user_id=user.id, version_id=version_id, slot=slot, title=body.title[:255], - data=body.data) - db.add(save) - else: - save.title = body.title[:255] - save.data = body.data - await db.commit() - await db.refresh(save) - return save - - -@router.delete("/versions/{version_id}/saves/{slot}", status_code=204) -async def delete_save(version_id: int, slot: str, user: Player, db: DB) -> None: - save = await _find_save(db, user, version_id, slot) - if save: - await db.delete(save) - await db.commit() +from typing import Annotated, Any + +from fastapi import APIRouter, Depends, HTTPException, Request, status +from sqlalchemy import select +from starlette.requests import ClientDisconnect + +from app.deps import DB, CurrentUser, has_perm, require +from app.models import GameVersion, Save, User +from app.routers.games import get_game +from app.schemas import SaveDetailOut, SaveIn, SaveOut, VersionDetailOut, VersionIn, VersionOut +from app.uploads import remove_upload, upload_path + +router = APIRouter(prefix="/api", tags=["versions"]) +Player = Annotated[User, Depends(require("games.play"))] +Uploader = Annotated[User, Depends(require("games.upload"))] + +ARCHIVE_EXTS = (".zip", ".7z", ".rar", ".tar", ".tar.gz", ".tgz", ".tar.bz2", ".tar.xz") + + +def can_see(user: User, v: GameVersion) -> bool: + return v.status == "ready" or v.owner_id == user.id or can_manage(user, v) + + +def can_manage(user: User, v: GameVersion) -> bool: + return ( + v.owner_id == user.id + or has_perm(user, "versions.manage_any") + or has_perm(user, "games.delete") + ) + + +async def get_version(db: DB, user: User, version_id: int) -> GameVersion: + v = await db.get(GameVersion, version_id) + if not v or not can_see(user, v): + raise HTTPException(status.HTTP_404_NOT_FOUND, "Версия не найдена") + return v + + +@router.post("/games/{game_id}/upload", response_model=VersionOut, status_code=201) +async def upload_version( + game_id: int, request: Request, user: Uploader, db: DB, filename: str, name: str = "Оригинал" +) -> GameVersion: + """Stream a raw archive body to disk and queue it for conversion.""" + game = await get_game(db, game_id) + if not filename.lower().endswith(ARCHIVE_EXTS): + raise HTTPException(status.HTTP_400_BAD_REQUEST, + f"Поддерживаемые архивы: {', '.join(ARCHIVE_EXTS)}") + version = GameVersion(game_id=game.id, owner_id=user.id, name=name.strip() or "Оригинал", + upload_name=filename, status="uploading") + db.add(version) + await db.commit() + + path = upload_path(version.id) + path.parent.mkdir(parents=True, exist_ok=True) + size = 0 + try: + with path.open("wb") as f: + async for chunk in request.stream(): + f.write(chunk) + size += len(chunk) + except Exception as e: + remove_upload(version.id) + await db.delete(version) + await db.commit() + if isinstance(e, ClientDisconnect): + # usually a proxy in front cut the body off (e.g. its own 413) + raise HTTPException(status.HTTP_400_BAD_REQUEST, "Загрузка прервана") from None + raise + if size == 0: + remove_upload(version.id) + await db.delete(version) + await db.commit() + raise HTTPException(status.HTTP_400_BAD_REQUEST, "Пустой файл") + + version.status = "queued" + version.progress = "В очереди на конвертацию" + version.size_bytes = size + await db.commit() + return version + + +@router.get("/versions/{version_id}", response_model=VersionDetailOut) +async def version_detail(version_id: int, user: Player, db: DB) -> GameVersion: + return await get_version(db, user, version_id) + + +@router.get("/versions/{version_id}/manifest") +async def version_manifest(version_id: int, user: Player, db: DB) -> dict[str, Any]: + v = await get_version(db, user, version_id) + if v.status != "ready" or not v.manifest: + raise HTTPException(status.HTTP_409_CONFLICT, "Версия ещё не готова") + return v.manifest + + +@router.post("/versions/{version_id}/fork", response_model=VersionOut, status_code=201) +async def fork_version(version_id: int, body: VersionIn, + user: Annotated[User, Depends(require("games.play", "versions.fork"))], + db: DB) -> GameVersion: + parent = await get_version(db, user, version_id) + if parent.status != "ready": + raise HTTPException(status.HTTP_409_CONFLICT, "Форкнуть можно только готовую версию") + fork = GameVersion( + game_id=parent.game_id, + parent_id=parent.id, + owner_id=user.id, + name=body.name.strip(), + description=body.description, + status="ready", + manifest=parent.manifest, + report=parent.report, + size_bytes=parent.size_bytes, + ) + db.add(fork) + await db.commit() + return fork + + +@router.patch("/versions/{version_id}", response_model=VersionOut) +async def update_version(version_id: int, body: VersionIn, user: CurrentUser, db: DB) -> GameVersion: + v = await get_version(db, user, version_id) + if not can_manage(user, v): + raise HTTPException(status.HTTP_403_FORBIDDEN, "Нет прав на эту версию") + v.name = body.name.strip() + v.description = body.description + await db.commit() + return v + + +@router.post("/versions/{version_id}/retry", response_model=VersionOut) +async def retry_version(version_id: int, user: CurrentUser, db: DB) -> GameVersion: + v = await get_version(db, user, version_id) + if not can_manage(user, v): + raise HTTPException(status.HTTP_403_FORBIDDEN, "Нет прав на эту версию") + if v.status != "failed" or not upload_path(v.id).exists(): + raise HTTPException(status.HTTP_409_CONFLICT, "Повторная конвертация невозможна") + v.status = "queued" + v.progress = "В очереди на конвертацию" + await db.commit() + return v + + +@router.delete("/versions/{version_id}", status_code=204) +async def delete_version(version_id: int, user: CurrentUser, db: DB) -> None: + v = await get_version(db, user, version_id) + if not can_manage(user, v): + raise HTTPException(status.HTTP_403_FORBIDDEN, "Нет прав на эту версию") + if v.status in ("converting", "importing"): + raise HTTPException(status.HTTP_409_CONFLICT, "Версия сейчас конвертируется") + await db.delete(v) + await db.commit() + remove_upload(version_id) + + +# --- saves ----------------------------------------------------------------- + +def _check_slot(slot: str) -> str: + if not slot or len(slot) > 32 or not all(c.isalnum() or c in "_-" for c in slot): + raise HTTPException(status.HTTP_400_BAD_REQUEST, "Неверный слот") + return slot + + +@router.get("/versions/{version_id}/saves", response_model=list[SaveOut]) +async def list_saves(version_id: int, user: Player, db: DB) -> list[Save]: + await get_version(db, user, version_id) + return list((await db.scalars( + select(Save).where(Save.user_id == user.id, Save.version_id == version_id) + .order_by(Save.updated_at.desc()) + )).all()) + + +async def _find_save(db: DB, user: User, version_id: int, slot: str) -> Save | None: + return await db.scalar(select(Save).where( + Save.user_id == user.id, Save.version_id == version_id, Save.slot == _check_slot(slot))) + + +@router.get("/versions/{version_id}/saves/{slot}", response_model=SaveDetailOut) +async def get_save(version_id: int, slot: str, user: Player, db: DB) -> Save: + save = await _find_save(db, user, version_id, slot) + if not save: + raise HTTPException(status.HTTP_404_NOT_FOUND, "Сохранение не найдено") + return save + + +@router.put("/versions/{version_id}/saves/{slot}", response_model=SaveOut) +async def put_save(version_id: int, slot: str, body: SaveIn, user: Player, db: DB) -> Save: + await get_version(db, user, version_id) + save = await _find_save(db, user, version_id, slot) + if save is None: + save = Save(user_id=user.id, version_id=version_id, slot=slot, title=body.title[:255], + data=body.data) + db.add(save) + else: + save.title = body.title[:255] + save.data = body.data + await db.commit() + await db.refresh(save) + return save + + +@router.delete("/versions/{version_id}/saves/{slot}", status_code=204) +async def delete_save(version_id: int, slot: str, user: Player, db: DB) -> None: + save = await _find_save(db, user, version_id, slot) + if save: + await db.delete(save) + await db.commit() diff --git a/backend/app/worker.py b/backend/app/worker.py index 215ad7b..9dec60f 100644 --- a/backend/app/worker.py +++ b/backend/app/worker.py @@ -1,116 +1,116 @@ -"""Conversion worker: polls the DB for queued versions and converts them.""" - -import asyncio -import logging -import shutil -from pathlib import Path - -from sqlalchemy import select, update - -from app.blobs import read_bytes -from app.config import get_settings -from app.converter.pipeline import convert_archive -from app.db import SessionLocal -from app.models import Game, GameVersion -from app.routers.games import make_cover -from app.uploads import remove_upload, upload_path - -logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") -log = logging.getLogger("worker") - - -async def set_progress(version_id: int, text: str) -> None: - async with SessionLocal() as db: - await db.execute(update(GameVersion).where(GameVersion.id == version_id) - .values(progress=text[:255])) - await db.commit() - - -async def claim() -> GameVersion | None: - async with SessionLocal() as db: - version_id = await db.scalar( - select(GameVersion.id).where(GameVersion.status == "queued").order_by(GameVersion.id) - .limit(1).with_for_update(skip_locked=True) - ) - if version_id is None: - return None - v = await db.get(GameVersion, version_id) - v.status = "converting" - v.progress = "Начало конвертации" - await db.commit() - return v - - -async def process(v: GameVersion) -> None: - ok = await convert(v.id, upload_path(v.id), v.upload_name or "") - if ok: - remove_upload(v.id) # sources are not kept once converted - - -async def convert(version_id: int, source: Path, filename: str) -> bool: - """Run the converter for a version and store the outcome. Returns success.""" - loop = asyncio.get_running_loop() - work = get_settings().work_dir / str(version_id) - - def progress(text: str) -> None: - asyncio.run_coroutine_threadsafe(set_progress(version_id, text), loop) - - log.info("converting version %s (%s)", version_id, filename) - try: - result = await asyncio.to_thread(convert_archive, source, filename, work, progress) - except Exception as e: # noqa: BLE001 - log.exception("version %s failed", version_id) - async with SessionLocal() as db: - version = await db.get(GameVersion, version_id) - if version: - version.status = "failed" - version.progress = f"Ошибка: {e}"[:255] - version.report = {"error": str(e)} - await db.commit() - return False - finally: - shutil.rmtree(work, ignore_errors=True) - - async with SessionLocal() as db: - version = await db.get(GameVersion, version_id) - if version is None: # deleted while converting - return True - version.status = "ready" - version.progress = "" - version.manifest = result.manifest - version.report = result.report - version.size_bytes = result.size - game = await db.get(Game, version.game_id) - if game and not game.cover_hash and result.cover_source: - try: - game.cover_hash = await asyncio.to_thread(make_cover, - read_bytes(result.cover_source)) - except OSError: - pass - if game and result.title and game.title == source.name: - game.title = result.title - await db.commit() - log.info("version %s ready: %s", version_id, result.report.get("media")) - return True - - -async def main() -> None: - settings = get_settings() - for d in (settings.blobs_dir, settings.uploads_dir, settings.work_dir): - d.mkdir(parents=True, exist_ok=True) - async with SessionLocal() as db: - # jobs interrupted by a restart go back to the queue - await db.execute(update(GameVersion).where(GameVersion.status == "converting") - .values(status="queued")) - await db.commit() - log.info("worker started") - while True: - v = await claim() - if v is None: - await asyncio.sleep(settings.worker_poll_seconds) - continue - await process(v) - - -if __name__ == "__main__": - asyncio.run(main()) +"""Conversion worker: polls the DB for queued versions and converts them.""" + +import asyncio +import logging +import shutil +from pathlib import Path + +from sqlalchemy import select, update + +from app.blobs import read_bytes +from app.config import get_settings +from app.converter.pipeline import convert_archive +from app.db import SessionLocal +from app.models import Game, GameVersion +from app.routers.games import make_cover +from app.uploads import remove_upload, upload_path + +logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") +log = logging.getLogger("worker") + + +async def set_progress(version_id: int, text: str) -> None: + async with SessionLocal() as db: + await db.execute(update(GameVersion).where(GameVersion.id == version_id) + .values(progress=text[:255])) + await db.commit() + + +async def claim() -> GameVersion | None: + async with SessionLocal() as db: + version_id = await db.scalar( + select(GameVersion.id).where(GameVersion.status == "queued").order_by(GameVersion.id) + .limit(1).with_for_update(skip_locked=True) + ) + if version_id is None: + return None + v = await db.get(GameVersion, version_id) + v.status = "converting" + v.progress = "Начало конвертации" + await db.commit() + return v + + +async def process(v: GameVersion) -> None: + ok = await convert(v.id, upload_path(v.id), v.upload_name or "") + if ok: + remove_upload(v.id) # sources are not kept once converted + + +async def convert(version_id: int, source: Path, filename: str) -> bool: + """Run the converter for a version and store the outcome. Returns success.""" + loop = asyncio.get_running_loop() + work = get_settings().work_dir / str(version_id) + + def progress(text: str) -> None: + asyncio.run_coroutine_threadsafe(set_progress(version_id, text), loop) + + log.info("converting version %s (%s)", version_id, filename) + try: + result = await asyncio.to_thread(convert_archive, source, filename, work, progress) + except Exception as e: # noqa: BLE001 + log.exception("version %s failed", version_id) + async with SessionLocal() as db: + version = await db.get(GameVersion, version_id) + if version: + version.status = "failed" + version.progress = f"Ошибка: {e}"[:255] + version.report = {"error": str(e)} + await db.commit() + return False + finally: + shutil.rmtree(work, ignore_errors=True) + + async with SessionLocal() as db: + version = await db.get(GameVersion, version_id) + if version is None: # deleted while converting + return True + version.status = "ready" + version.progress = "" + version.manifest = result.manifest + version.report = result.report + version.size_bytes = result.size + game = await db.get(Game, version.game_id) + if game and not game.cover_hash and result.cover_source: + try: + game.cover_hash = await asyncio.to_thread(make_cover, + read_bytes(result.cover_source)) + except OSError: + pass + if game and result.title and game.title == source.name: + game.title = result.title + await db.commit() + log.info("version %s ready: %s", version_id, result.report.get("media")) + return True + + +async def main() -> None: + settings = get_settings() + for d in (settings.blobs_dir, settings.uploads_dir, settings.work_dir): + d.mkdir(parents=True, exist_ok=True) + async with SessionLocal() as db: + # jobs interrupted by a restart go back to the queue + await db.execute(update(GameVersion).where(GameVersion.status == "converting") + .values(status="queued")) + await db.commit() + log.info("worker started") + while True: + v = await claim() + if v is None: + await asyncio.sleep(settings.worker_poll_seconds) + continue + await process(v) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/frontend/src/pages/GamePage.tsx b/frontend/src/pages/GamePage.tsx index 7b6a789..c042481 100644 --- a/frontend/src/pages/GamePage.tsx +++ b/frontend/src/pages/GamePage.tsx @@ -1,233 +1,233 @@ -import { useCallback, useEffect, useState } from "react"; -import { Link, useNavigate, useParams } from "react-router-dom"; -import { api, formatSize, uploadRaw, type GameDetail, type Version } from "../api"; -import { useAuth } from "../auth"; - -const STATUS: Record = { - uploading: "загружается", - queued: "в очереди", - converting: "конвертируется", - importing: "импортируется", - ready: "готова", - failed: "ошибка", -}; - -export function GamePage() { - const { gameId } = useParams(); - const navigate = useNavigate(); - const { me, can } = useAuth(); - const [game, setGame] = useState(null); - const [error, setError] = useState(""); - const [editing, setEditing] = useState(false); - - const load = useCallback(async () => { - try { - setGame(await api.get(`/api/games/${gameId}`)); - } catch (e) { - setError((e as Error).message); - } - }, [gameId]); - - useEffect(() => { - void load(); - }, [load]); - - // poll while something is being converted - const busy = game?.versions.some((v) => ["uploading", "queued", "converting", "importing"].includes(v.status)); - useEffect(() => { - if (!busy) return; - const t = setInterval(() => void load(), 2000); - return () => clearInterval(t); - }, [busy, load]); - - if (error) return
{error}
; - if (!game) return
Загрузка…
; - - // the backend also lets uploaders edit only games they created - const canEdit = can("games.edit") || can("games.upload"); - const run = async (fn: () => Promise) => { - setError(""); - try { - await fn(); - await load(); - } catch (e) { - setError((e as Error).message); - } - }; - - const fork = (v: Version) => { - const name = prompt("Название новой версии", `${v.name} (${me?.username})`); - if (name) void run(() => api.post(`/api/versions/${v.id}/fork`, { name })); - }; - - const uploadCover = (file: File) => - run(() => uploadRaw("PUT", `/api/games/${game.id}/cover`, file, () => {})); - - const uploadVersion = (file: File) => { - const name = prompt("Название версии", file.name); - if (name) - void run(() => uploadRaw("POST", `/api/games/${game.id}/upload?filename=${encodeURIComponent(file.name)}&name=${encodeURIComponent(name)}`, - file, () => {})); - }; - - const byParent = new Map(); - for (const v of game.versions) { - const parent = game.versions.some((p) => p.id === v.parent_id) ? v.parent_id : null; - byParent.set(parent, [...(byParent.get(parent) ?? []), v]); - } - const renderTree = (parent: number | null, depth: number): JSX.Element[] => - (byParent.get(parent) ?? []).flatMap((v) => [ - navigate(`/play/${v.id}`)} onFork={() => fork(v)} - onRetry={() => void run(() => api.post(`/api/versions/${v.id}/retry`))} - onDelete={() => confirm(`Удалить версию «${v.name}»?`) && void run(() => api.del(`/api/versions/${v.id}`))} />, - ...renderTree(v.id, depth + 1), - ]); - - return ( -
- ← Библиотека -
-
- {game.cover_hash ? : {game.title[0]}} -
-
- {editing ? ( - { setEditing(false); void load(); }} /> - ) : ( - <> -

{game.title}

-
{game.tags.map((t) => {t})}
-

{game.description || Нет описания}

-
- {canEdit && } - {canEdit && ( - - )} - {can("games.upload") && ( - - )} - {can("games.delete") && ( - - )} -
- - )} -
-
- {error &&
{error}
} -

Версии

-
{renderTree(null, 0)}
-
- ); -} - -function VersionRow(props: { - v: Version; depth: number; canManage: boolean; canFork: boolean; - onPlay: () => void; onFork: () => void; onRetry: () => void; onDelete: () => void; -}) { - const { v } = props; - const [report, setReport] = useState(undefined); - const toggleReport = async () => { - if (report !== undefined) setReport(undefined); - else setReport((await api.get(`/api/versions/${v.id}`)).report ?? null); - }; - return ( -
-
-
- {props.depth > 0 && "↳ "}{v.name} - {STATUS[v.status]} -
- {v.owner?.username ?? "—"} · {new Date(v.created_at).toLocaleDateString()} · {formatSize(v.size_bytes)} - {v.progress && ` · ${v.progress}`} -
-
-
- {v.status === "ready" && } - {v.status === "ready" && props.canFork && } - {(v.status === "ready" || v.status === "failed") && } - {v.status === "failed" && props.canManage && } - {props.canManage && !["converting", "importing"].includes(v.status) && } -
-
- {report !== undefined && } -
- ); -} - -function Report({ report }: { report: Record | null }) { - if (!report) return
Отчёта нет
; - if (report.error) return
{report.error}
; - return ( -
-
- Скриптов: {report.scripts} (декомпилировано {report.decompiled}) - Меток: {report.labels} - Реплик: {report.stats?.say ?? 0} - Меню: {report.stats?.menu ?? 0} - Картинок: {report.media?.image}, аудио: {report.media?.audio}, видео: {report.media?.video} - Размер: {formatSize(report.sourceSize)} → {formatSize(report.outputSize)} - Время: {report.durationSec} с -
- {report.missingFiles?.length > 0 && ( -
- Не найдены файлы ({report.missingFiles.length}) -
    {report.missingFiles.map((f: string) =>
  • {f}
    • )}
-
- )} - {report.warnings?.length > 0 && ( -
- Предупреждения ({report.warnings.reduce((s: number, w: any) => s + w.count, 0)}) - - - {report.warnings.map((w: any) => ( - - - - - - ))} - -
{w.count}×{w.message}{w.example}
-
- )} -
- ); -} - -function GameEditor({ game, onDone }: { game: GameDetail; onDone: () => void }) { - const [title, setTitle] = useState(game.title); - const [description, setDescription] = useState(game.description); - const [tags, setTags] = useState(game.tags.join(", ")); - const [error, setError] = useState(""); - const save = async () => { - try { - await api.patch(`/api/games/${game.id}`, { - title, description, tags: tags.split(",").map((t) => t.trim()).filter(Boolean), - }); - onDone(); - } catch (e) { - setError((e as Error).message); - } - }; - return ( -
- setTitle(e.target.value)} /> -