chore: add CI, lint config, login rate limit and split large modules

- Rate-limit failed logins (8 per 15 min per IP or username, 429 + Retry-After)
- Add ruff/mypy config and GitHub Actions workflow (ruff, pytest, tsc, tests, build)
- Apply ruff autofixes (import order, pyupgrade)
- Extract rpy_helpers.py, engine-util.ts, TranslationParts.tsx and
  translation-types.ts from oversized modules
- Add dependency-free React UI tests
- Refresh README and ARCHITECTURE (Naninovel/Tomie, status and plans)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
mrleo1nidandClaude Sonnet 5.5 committed 2026-10-25 11:23:22 +03:00
1 parent 57a4edcbfd
commit bbbcc6a5d8
42 files changed
+678 -391

No files matched your search

+45
View File
@@ -0,0 +1,45 @@
name: CI
on:
push:
branches: [master]
pull_request:
jobs:
backend:
runs-on: ubuntu-latest
defaults:
run:
working-directory: backend
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
cache-dependency-path: backend/requirements*.txt
- name: Install system tools
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends ffmpeg
- run: pip install -r requirements-dev.txt ruff mypy
- run: ruff check .
- name: Type check (informational, existing backlog)
run: mypy app
continue-on-error: true
- run: python -m pytest -q
frontend:
runs-on: ubuntu-latest
defaults:
run:
working-directory: frontend
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: npm ci
- run: npm run typecheck
- run: npm test
- run: npm run build
+18 -3
View File
@@ -1,8 +1,9 @@
# PGames — локальная библиотека визуальных новелл # PGames — локальная библиотека визуальных новелл
Загружаете архив с игрой на Ren'Py — сервер распаковывает его, переводит Загружаете архив с игрой на Ren'Py или Unity-игру на Naninovel (включая
отдельный адаптер Tomie) — сервер распаковывает его, переводит
скрипты и ассеты в собственный формат **PGVN** (JSON-скрипты, WebP, Opus, WebM) скрипты и ассеты в собственный формат **PGVN** (JSON-скрипты, WebP, Opus, WebM)
и удаляет исходники. Играете в браузере во встроенном плеере. У каждой игры и удаляет исходники. Игры других движков отклоняются на входе. Играете в браузере во встроенном плеере. У каждой игры
может быть несколько версий (форков) от разных пользователей; версии делят может быть несколько версий (форков) от разных пользователей; версии делят
общие ассеты, поэтому форк почти не занимает места. общие ассеты, поэтому форк почти не занимает места.
@@ -81,11 +82,23 @@ docker compose run --rm -v /srv/games:/games:ro backend \
`reset` — удалить стенд вместе с данными, `logs` — логи бэкенда и воркера. `reset` — удалить стенд вместе с данными, `logs` — логи бэкенда и воркера.
```bash ```bash
cd backend && pip install -r requirements.txt pytest && python -m pytest cd backend && pip install -r requirements-dev.txt ruff && ruff check . && python -m pytest
cd frontend && npm install && npm run dev # проксирует /api на :8000 cd frontend && npm install && npm run dev # проксирует /api на :8000
cd frontend && npm run typecheck && npm test
python -m tests.make_sample sample.zip # тестовая игра (из backend/) python -m tests.make_sample sample.zip # тестовая игра (из backend/)
``` ```
Тесты конвертера требуют `ffmpeg` в `PATH`. Те же проверки (ruff, pytest,
`tsc`, тесты и сборка фронтенда) выполняет GitHub Actions
(`.github/workflows/ci.yml`); `mypy` запускается там как справочная проверка.
Каталоги `data/` и `data-test/` содержат библиотеку и тестовый стенд (гигабайты),
они в `.gitignore` — не добавляйте их в коммиты и в образы Docker.
Вход ограничен по числу неудачных попыток: 8 ошибок за 15 минут по IP или по
логину блокируют вход на оставшееся время (ответ 429 с `Retry-After`). Счётчик
хранится в памяти процесса и сбрасывается при перезапуске backend.
## Что поддерживает конвертер Ren'Py ## Что поддерживает конвертер Ren'Py
`label`, реплики (в т.ч. атрибуты, `extend`, `centered`, NVL), `scene/show/hide` `label`, реплики (в т.ч. атрибуты, `extend`, `centered`, NVL), `scene/show/hide`
@@ -101,6 +114,8 @@ python -m tests.make_sample sample.zip # тестовая игра (из ba
каналы (`register_channel`), `pause`, `.rpa`, `.rpyc` (через unrpyc), каналы (`register_channel`), `pause`, `.rpa`, `.rpyc` (через unrpyc),
автоопределение картинок из `images/`, GUI-текстбокс и шрифты игры. автоопределение картинок из `images/`, GUI-текстбокс и шрифты игры.
Для Naninovel/Tomie см. [docs/IMPORT.md](docs/IMPORT.md).
Пока **не** поддерживается: `screen`-язык (кастомные экраны, карты, Пока **не** поддерживается: `screen`-язык (кастомные экраны, карты,
мини-игры), движение и твины в ATL, функции/классы в `python` и картинки, мини-игры), движение и твины в ATL, функции/классы в `python` и картинки,
которые игра вычисляет своими функциями. Такие места перечислены в отчёте которые игра вычисляет своими функциями. Такие места перечислены в отчёте
+2 -2
View File
@@ -12,12 +12,12 @@ import sys
import tempfile import tempfile
import time import time
from collections import Counter from collections import Counter
from dataclasses import asdict
from pathlib import Path from pathlib import Path
from app.converter import assets, locales from app.converter import assets, locales
from app.converter.compatibility import Issue, audit_ir
from dataclasses import asdict
from app.converter.archive import ArchiveError, find_game_dir from app.converter.archive import ArchiveError, find_game_dir
from app.converter.compatibility import Issue, audit_ir
from app.converter.pipeline import _auto_images, _is_engine_file, _resolve_image, anchors from app.converter.pipeline import _auto_images, _is_engine_file, _resolve_image, anchors
from app.converter.rpa import RpaError, extract_rpa, read_index from app.converter.rpa import RpaError, extract_rpa, read_index
from app.converter.rpy_lexer import ParseError from app.converter.rpy_lexer import ParseError
+1 -1
View File
@@ -1,7 +1,7 @@
"""Engine detection and fail-closed import diagnostics. Never runs game executables.""" """Engine detection and fail-closed import diagnostics. Never runs game executables."""
import os
from dataclasses import asdict, dataclass from dataclasses import asdict, dataclass
from pathlib import Path, PurePosixPath from pathlib import Path, PurePosixPath
import os
@dataclass(frozen=True) @dataclass(frozen=True)
+1 -1
View File
@@ -8,11 +8,11 @@
import hashlib import hashlib
import re import re
from uuid import uuid4
from collections import Counter, defaultdict from collections import Counter, defaultdict
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from uuid import uuid4
from app.converter.rpy_lexer import Cursor, Line, ParseError, parse_tree from app.converter.rpy_lexer import Cursor, Line, ParseError, parse_tree
from app.converter.source import read_source from app.converter.source import read_source
+1 -1
View File
@@ -1,7 +1,7 @@
"""Compile the supported Naninovel command subset to the shared PGVN player.""" """Compile the supported Naninovel command subset to the shared PGVN player."""
from collections import Counter
import hashlib import hashlib
import re import re
from collections import Counter
from app.converter import nani_expr from app.converter import nani_expr
from app.converter.compatibility import CompatibilityError, Issue from app.converter.compatibility import CompatibilityError, Issue
+1 -1
View File
@@ -1,6 +1,6 @@
"""Parse Naninovel expressions into data, never Python/JavaScript source code.""" """Parse Naninovel expressions into data, never Python/JavaScript source code."""
import re
import math import math
import re
TOKEN = re.compile(r'''\s*(?:(\d+(?:\.\d*)?(?:[eE][+-]?\d+)?|\.\d+)|([A-Za-z_]\w*)|("(?:\\.|[^"\\])*"|'(?:\\.|[^'\\])*')|(&&|\|\||==|!=|<>|<=|>=|<<|>>|[()+\-*/%<>=!~?:,&|^]))''') TOKEN = re.compile(r'''\s*(?:(\d+(?:\.\d*)?(?:[eE][+-]?\d+)?|\.\d+)|([A-Za-z_]\w*)|("(?:\\.|[^"\\])*"|'(?:\\.|[^'\\])*')|(&&|\|\||==|!=|<>|<=|>=|<<|>>|[()+\-*/%<>=!~?:,&|^]))''')
PRECEDENCE = {"or": 2, "||": 2, "and": 3, "&&": 3, "|": 4, "^": 5, PRECEDENCE = {"or": 2, "||": 2, "and": 3, "&&": 3, "|": 4, "^": 5,
+4 -4
View File
@@ -1,9 +1,8 @@
"""Recover Naninovel data from Unity builds without executing their managed code.""" """Recover Naninovel data from Unity builds without executing their managed code."""
from dataclasses import dataclass, field
from pathlib import Path
import hashlib import hashlib
import json import json
import re from dataclasses import dataclass, field
from pathlib import Path
from app.converter.compatibility import CompatibilityError, Issue from app.converter.compatibility import CompatibilityError, Issue
from app.converter.unity_bundle import unpack from app.converter.unity_bundle import unpack
@@ -133,8 +132,9 @@ def read_package(root: Path, work: Path, *, media: bool = False) -> Package:
def _read_package(root: Path, work: Path, package: Package, *, media: bool) -> Package: def _read_package(root: Path, work: Path, package: Package, *, media: bool) -> Package:
import UnityPy import UnityPy
from app.converter.tomie_profile import extract_tables, identify
from app.converter.unity_scripts import ScriptCodec, script_objects from app.converter.unity_scripts import ScriptCodec, script_objects
from app.converter.tomie_profile import identify, extract_tables
package.profile = identify(root) package.profile = identify(root)
if package.profile: if package.profile:
+2 -3
View File
@@ -1,8 +1,7 @@
"""Naninovel importer: compiled Unity data -> validated PGVN package.""" """Naninovel importer: compiled Unity data -> validated PGVN package."""
from pathlib import Path, PurePosixPath
import json import json
import re
import tempfile import tempfile
from pathlib import Path, PurePosixPath
from app.blobs import put_bytes, put_file from app.blobs import put_bytes, put_file
from app.converter import assets, locales from app.converter import assets, locales
@@ -158,8 +157,8 @@ def _convert_package(root: Path, work: Path, progress, package):
def _convert_tomie(root, work, progress, package): def _convert_tomie(root, work, progress, package):
from app.converter.pipeline import Result, anchors from app.converter.pipeline import Result, anchors
from app.converter.tomie_format import RUNTIME, modules, prepare
from app.converter.tomie_assets import export_media as export_tomie_media from app.converter.tomie_assets import export_media as export_tomie_media
from app.converter.tomie_format import RUNTIME, modules, prepare
native = prepare(package) # all script checks precede media decoding/blob writes native = prepare(package) # all script checks precede media decoding/blob writes
scripts = modules(native, package.texts) scripts = modules(native, package.texts)
+7 -1
View File
@@ -15,12 +15,18 @@ from app.blobs import put_bytes, put_file
from app.config import get_settings from app.config import get_settings
from app.converter import assets, locales from app.converter import assets, locales
from app.converter.archive import extract, find_game_dir from app.converter.archive import extract, find_game_dir
from app.converter.compatibility import (
CompatibilityError,
Issue,
audit_ir,
detect,
require_supported_scripts,
)
from app.converter.rpa import RpaError, extract_rpa from app.converter.rpa import RpaError, extract_rpa
from app.converter.rpy_lexer import ParseError from app.converter.rpy_lexer import ParseError
from app.converter.rpy_parser import RpyParser, ScriptData from app.converter.rpy_parser import RpyParser, ScriptData
from app.converter.rpyc import decompile_missing from app.converter.rpyc import decompile_missing
from app.converter.source import read_source from app.converter.source import read_source
from app.converter.compatibility import CompatibilityError, Issue, audit_ir, detect, require_supported_scripts
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
FORMAT = "pgvn/1" FORMAT = "pgvn/1"
+3 -3
View File
@@ -1,9 +1,9 @@
"""Read-only compatibility check, shared by uploads, CLI and the converter.""" """Read-only compatibility check, shared by uploads, CLI and the converter."""
from pathlib import Path
import tempfile
import zipfile
import shutil import shutil
import subprocess import subprocess
import tempfile
import zipfile
from pathlib import Path
from app.converter.archive import extract from app.converter.archive import extract
from app.converter.compatibility import CompatibilityError, Issue, detect from app.converter.compatibility import CompatibilityError, Issue, detect
+130
View File
@@ -0,0 +1,130 @@
"""Module-level helpers of the Ren'Py parser (Python snippets, ATL props, constants)."""
import ast
import re
import warnings
from typing import Any
from app.converter.pyexpr import Expr
Node = dict[str, Any]
POSITION_PROPS = {"xpos", "ypos", "xanchor", "yanchor", "xalign", "yalign", "xcenter",
"ycenter", "zoom", "xzoom", "yzoom", "alpha", "rotate", "xoffset", "yoffset",
"align", "pos", "anchor", "xysize", "size"}
_SHOW_KEYWORDS = {"at", "as", "behind", "onlayer", "with", "zorder"}
_NUMBER_LINE = re.compile(r"^\d+(\.\d*)?$|^\.\d+$")
_CALL_LINE = re.compile(r"^([A-Za-z_][\w.]*)\s*\(")
_DISPLAYABLE_CALLS = {"Movie", "Solid", "Transform", "Composite", "LiveComposite", "Fixed",
"ConditionSwitch", "Image", "im.Scale", "im.FactorScale", "im.Flip",
"im.Composite", "im.Crop", "im.Grayscale", "im.Sepia", "Frame", "Flatten"}
_MEDIA_EXTS = {"png", "jpg", "jpeg", "webp", "gif", "bmp", "avif", "ogg", "mp3", "wav", "opus",
"flac", "webm", "ogv", "mp4", "mkv", "avi", "m4a"}
def _dedent(source: str) -> str:
import textwrap
return textwrap.dedent(source)
def _parse_py(source: str, mode: str = "exec") -> ast.AST:
# decompiled scripts are full of "\{" in plain strings: don't spam SyntaxWarning
with warnings.catch_warnings():
warnings.simplefilter("ignore")
return ast.parse(_dedent(source), mode=mode)
def _dotted_name(e: Expr) -> str:
if e["k"] == "name":
return e["id"]
if e["k"] == "attr":
return f"{_dotted_name(e['o'])}.{e['a']}"
return ""
def _assign_name(t: ast.AST) -> str | None:
if isinstance(t, ast.Name):
return t.id
if isinstance(t, ast.Attribute):
inner = _assign_name(t.value)
return f"{inner}.{t.attr}" if inner else None
return None
def _const(e: Expr | None) -> Any:
if e is None:
return None
if e.get("k") == "const":
return e["v"]
if e.get("k") in ("tuple", "list"):
return [_const(x) for x in e["v"]]
if e.get("k") == "unary" and e["op"] == "-":
v = _const(e["v"])
return -v if isinstance(v, (int, float)) else None
return None
def _num(s: str | None) -> float | None:
try:
return float(s) if s is not None else None
except ValueError:
return None
def _static_props(kw: dict[str, Expr]) -> dict[str, Any]:
props = {}
for k, v in kw.items():
if k in POSITION_PROPS:
val = _const(v)
if val is not None:
props[k] = val
return props
def _atl_props(text: str) -> dict[str, Any]:
"""Extract static `prop value` pairs from one ATL line, ignoring timing."""
tokens = text.replace(",", " ").split()
props: dict[str, Any] = {}
i = 0
while i < len(tokens) - 1:
key = tokens[i]
if key in POSITION_PROPS:
raw = tokens[i + 1]
try:
props[key] = ast.literal_eval(raw)
except (ValueError, SyntaxError):
if raw.startswith("(") and i + 2 < len(tokens):
try:
props[key] = list(ast.literal_eval(raw + "," + tokens[i + 2]))
i += 1
except (ValueError, SyntaxError):
pass
i += 2
else:
i += 1
return props
def _split_top(source: str) -> list[str]:
parts, depth, cur = [], 0, []
for ch in source:
if ch in "([{":
depth += 1
elif ch in ")]}":
depth -= 1
if ch == "," and depth == 0:
parts.append("".join(cur))
cur = []
else:
cur.append(ch)
if "".join(cur).strip():
parts.append("".join(cur))
return [p.strip() for p in parts if p.strip()]
def _jump_or_call(op: str, target: Expr) -> Node:
if target.get("k") == "const":
return {"op": op, "target": target["v"]}
return {"op": op, "expr": target}
+19 -122
View File
@@ -1,14 +1,30 @@
"""Ren'Py script (.rpy) → PGVN IR.""" """Ren'Py script (.rpy) → PGVN IR."""
import ast import ast
import re
import warnings
from collections import Counter from collections import Counter
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any from typing import Any
from app.converter import pyexpr from app.converter import pyexpr
from app.converter.pyexpr import Expr, Stmt, Unsupported, const from app.converter.pyexpr import Expr, Stmt, Unsupported, const
from app.converter.rpy_helpers import (
_CALL_LINE,
_DISPLAYABLE_CALLS,
_MEDIA_EXTS,
_NUMBER_LINE,
_SHOW_KEYWORDS,
POSITION_PROPS,
_assign_name,
_atl_props,
_const,
_dedent,
_dotted_name,
_jump_or_call,
_num,
_parse_py,
_split_top,
_static_props,
)
from app.converter.rpy_lexer import Cursor, Line, ParseError, parse_tree from app.converter.rpy_lexer import Cursor, Line, ParseError, parse_tree
Node = dict[str, Any] Node = dict[str, Any]
@@ -41,9 +57,6 @@ TRANSITION_CLASSES = {"Dissolve", "Fade", "ImageDissolve", "AlphaDissolve", "Pix
POSITIONS = {"left", "right", "center", "truecenter", "topleft", "topright", "top", POSITIONS = {"left", "right", "center", "truecenter", "topleft", "topright", "top",
"offscreenleft", "offscreenright", "default", "reset"} "offscreenleft", "offscreenright", "default", "reset"}
POSITION_PROPS = {"xpos", "ypos", "xanchor", "yanchor", "xalign", "yalign", "xcenter",
"ycenter", "zoom", "xzoom", "yzoom", "alpha", "rotate", "xoffset", "yoffset",
"align", "pos", "anchor", "xysize", "size"}
CHANNELS = {"music", "sound", "voice", "audio", "movie"} CHANNELS = {"music", "sound", "voice", "audio", "movie"}
SKIP_BLOCKS = {"screen", "style", "translate", "testcase"} SKIP_BLOCKS = {"screen", "style", "translate", "testcase"}
BUILTIN_CHARACTERS = {"narrator", "centered", "vcentered", "extend", "nvl", "adv", "name_only"} BUILTIN_CHARACTERS = {"narrator", "centered", "vcentered", "extend", "nvl", "adv", "name_only"}
@@ -1100,7 +1113,7 @@ class RpyParser:
if c.string_raw() is not None: if c.string_raw() is not None:
c.pos = save c.pos = save
break break
p = c.image_part() or (("@" if c.char("@") else None)) p = c.image_part() or ("@" if c.char("@") else None)
if p is None: if p is None:
c.pos = save c.pos = save
break break
@@ -1130,119 +1143,3 @@ class RpyParser:
c.rest() c.rest()
self.d.stats["say"] += 1 self.d.stats["say"] += 1
return [node] return [node]
_SHOW_KEYWORDS = {"at", "as", "behind", "onlayer", "with", "zorder"}
_NUMBER_LINE = re.compile(r"^\d+(\.\d*)?$|^\.\d+$")
_CALL_LINE = re.compile(r"^([A-Za-z_][\w.]*)\s*\(")
_DISPLAYABLE_CALLS = {"Movie", "Solid", "Transform", "Composite", "LiveComposite", "Fixed",
"ConditionSwitch", "Image", "im.Scale", "im.FactorScale", "im.Flip",
"im.Composite", "im.Crop", "im.Grayscale", "im.Sepia", "Frame", "Flatten"}
_MEDIA_EXTS = {"png", "jpg", "jpeg", "webp", "gif", "bmp", "avif", "ogg", "mp3", "wav", "opus",
"flac", "webm", "ogv", "mp4", "mkv", "avi", "m4a"}
def _dedent(source: str) -> str:
import textwrap
return textwrap.dedent(source)
def _parse_py(source: str, mode: str = "exec") -> ast.AST:
# decompiled scripts are full of "\{" in plain strings: don't spam SyntaxWarning
with warnings.catch_warnings():
warnings.simplefilter("ignore")
return ast.parse(_dedent(source), mode=mode)
def _dotted_name(e: Expr) -> str:
if e["k"] == "name":
return e["id"]
if e["k"] == "attr":
return f"{_dotted_name(e['o'])}.{e['a']}"
return ""
def _assign_name(t: ast.AST) -> str | None:
if isinstance(t, ast.Name):
return t.id
if isinstance(t, ast.Attribute):
inner = _assign_name(t.value)
return f"{inner}.{t.attr}" if inner else None
return None
def _const(e: Expr | None) -> Any:
if e is None:
return None
if e.get("k") == "const":
return e["v"]
if e.get("k") in ("tuple", "list"):
return [_const(x) for x in e["v"]]
if e.get("k") == "unary" and e["op"] == "-":
v = _const(e["v"])
return -v if isinstance(v, (int, float)) else None
return None
def _num(s: str | None) -> float | None:
try:
return float(s) if s is not None else None
except ValueError:
return None
def _static_props(kw: dict[str, Expr]) -> dict[str, Any]:
props = {}
for k, v in kw.items():
if k in POSITION_PROPS:
val = _const(v)
if val is not None:
props[k] = val
return props
def _atl_props(text: str) -> dict[str, Any]:
"""Extract static `prop value` pairs from one ATL line, ignoring timing."""
tokens = text.replace(",", " ").split()
props: dict[str, Any] = {}
i = 0
while i < len(tokens) - 1:
key = tokens[i]
if key in POSITION_PROPS:
raw = tokens[i + 1]
try:
props[key] = ast.literal_eval(raw)
except (ValueError, SyntaxError):
if raw.startswith("(") and i + 2 < len(tokens):
try:
props[key] = list(ast.literal_eval(raw + "," + tokens[i + 2]))
i += 1
except (ValueError, SyntaxError):
pass
i += 2
else:
i += 1
return props
def _split_top(source: str) -> list[str]:
parts, depth, cur = [], 0, []
for ch in source:
if ch in "([{":
depth += 1
elif ch in ")]}":
depth -= 1
if ch == "," and depth == 0:
parts.append("".join(cur))
cur = []
else:
cur.append(ch)
if "".join(cur).strip():
parts.append("".join(cur))
return [p.strip() for p in parts if p.strip()]
def _jump_or_call(op: str, target: Expr) -> Node:
if target.get("k") == "const":
return {"op": op, "target": target["v"]}
return {"op": op, "expr": target}
+1 -1
View File
@@ -1,7 +1,7 @@
"""Decode source text without silently replacing characters in localisations.""" """Decode source text without silently replacing characters in localisations."""
import io import io
from pathlib import Path
import tokenize import tokenize
from pathlib import Path
from app.converter.rpy_lexer import ParseError from app.converter.rpy_lexer import ParseError
+1 -1
View File
@@ -1,6 +1,6 @@
"""Recover the pinned Tomie build's media and ResourceManager actor bindings.""" """Recover the pinned Tomie build's media and ResourceManager actor bindings."""
from pathlib import Path
import hashlib import hashlib
from pathlib import Path
from app.blobs import put_file from app.blobs import put_file
from app.converter import assets from app.converter import assets
+3 -2
View File
@@ -1,7 +1,7 @@
"""Version-pinned Tomie metadata. Reads IL constants; never executes managed code.""" """Version-pinned Tomie metadata. Reads IL constants; never executes managed code."""
import hashlib import hashlib
from pathlib import Path
import re import re
from pathlib import Path
PROFILE = "tomie/expansion-1.3801" PROFILE = "tomie/expansion-1.3801"
ASSEMBLY_HASH = "1c6aec12894362f1b9c62858a5a4c286eb93aee175ed1ef339218dc5a7b2729b" ASSEMBLY_HASH = "1c6aec12894362f1b9c62858a5a4c286eb93aee175ed1ef339218dc5a7b2729b"
@@ -110,8 +110,9 @@ def runtime_data(package) -> dict:
def validate_expressions(scripts: dict) -> dict: def validate_expressions(scripts: dict) -> dict:
"""Check the full recovered expression corpus, including dynamic templates.""" """Check the full recovered expression corpus, including dynamic templates."""
from app.converter.nani_expr import assignment, parse
from collections import Counter from collections import Counter
from app.converter.nani_expr import assignment, parse
expressions, errors = {}, Counter() expressions, errors = {}, Counter()
examples = {} examples = {}
@@ -9,7 +9,6 @@ from app.converter.compatibility import Issue
from app.converter.nani_expr import assignment, parse from app.converter.nani_expr import assignment, parse
from app.converter.tomie_profile import FUNCTIONS, validate_expressions from app.converter.tomie_profile import FUNCTIONS, validate_expressions
ARGUMENTS = { ARGUMENTS = {
"ModifyBackground": "AppearanceAndTransition Duration Scale TintColor", "ModifyBackground": "AppearanceAndTransition Duration Scale TintColor",
"ModifyTextPrinter": "Duration HideOther IdAndAppearance MakeDefault ScenePosition", "ModifyTextPrinter": "Duration HideOther IdAndAppearance MakeDefault ScenePosition",
+1 -1
View File
@@ -3,8 +3,8 @@ from pathlib import Path
def unpack(source: Path, target: Path) -> list[Path]: def unpack(source: Path, target: Path) -> list[Path]:
from UnityPy.streams import EndianBinaryReader
from UnityPy.helpers.CompressionHelper import DECOMPRESSION_MAP from UnityPy.helpers.CompressionHelper import DECOMPRESSION_MAP
from UnityPy.streams import EndianBinaryReader
target.mkdir(parents=True, exist_ok=True) target.mkdir(parents=True, exist_ok=True)
with source.open("rb") as stream: with source.open("rb") as stream:
+1 -1
View File
@@ -98,8 +98,8 @@ class ScriptCodec:
helper.read_typetree_boost, helper.get_ref_type_node = old_boost, old_resolve helper.read_typetree_boost, helper.get_ref_type_node = old_boost, old_resolve
def read(self, obj): def read(self, obj):
from UnityPy.streams import EndianBinaryWriter
from UnityPy.helpers import TypeTreeHelper as helper from UnityPy.helpers import TypeTreeHelper as helper
from UnityPy.streams import EndianBinaryWriter
# Bundles can retain exact schemas which differ from generated metadata # Bundles can retain exact schemas which differ from generated metadata
# (notably editor-compiled Init). Prefer those schemas when available. # (notably editor-compiled Init). Prefer those schemas when available.
self.templates = {(ref.m_AssemblyName.removesuffix(".dll"), ref.m_NameSpace, ref.m_ClassName): ref.node self.templates = {(ref.m_AssemblyName.removesuffix(".dll"), ref.m_NameSpace, ref.m_ClassName): ref.node
+1 -1
View File
@@ -25,8 +25,8 @@ async def main() -> None:
if not args.path.is_dir(): if not args.path.is_dir():
raise SystemExit(f"{args.path} is not a directory") raise SystemExit(f"{args.path} is not a directory")
from app.converter.preflight import inspect_directory
from app.converter.compatibility import CompatibilityError from app.converter.compatibility import CompatibilityError
from app.converter.preflight import inspect_directory
try: try:
report = await asyncio.to_thread(inspect_directory, args.path) report = await asyncio.to_thread(inspect_directory, args.path)
except CompatibilityError as exc: except CompatibilityError as exc:
+2 -3
View File
@@ -6,11 +6,9 @@ python -m app.import_library /games --import --owner admin
import argparse import argparse
import asyncio import asyncio
import json import json
from pathlib import Path
import tempfile
import sys import sys
from pathlib import Path
from app.converter.archive import ArchiveError, extract
from app.converter.compatibility import CompatibilityError, Issue from app.converter.compatibility import CompatibilityError, Issue
from app.converter.preflight import inspect_archive, inspect_directory from app.converter.preflight import inspect_archive, inspect_directory
@@ -52,6 +50,7 @@ def check(source: Path) -> dict:
async def import_one(source: Path, report: dict, username: str | None) -> dict: async def import_one(source: Path, report: dict, username: str | None) -> dict:
from sqlalchemy import select from sqlalchemy import select
from app.db import SessionLocal from app.db import SessionLocal
from app.models import Game, GameVersion, Role, User from app.models import Game, GameVersion, Role, User
from app.worker import convert from app.worker import convert
+2 -2
View File
@@ -2,12 +2,12 @@
import copy import copy
import difflib import difflib
import json import json
from uuid import uuid4, uuid5, NAMESPACE_URL from uuid import NAMESPACE_URL, uuid4, uuid5
from app.blobs import BLOB_ID_RE from app.blobs import BLOB_ID_RE
from app.pgvn import expression, statements
from app.converter.locales import walk from app.converter.locales import walk
from app.converter.pipeline import anchors from app.converter.pipeline import anchors
from app.pgvn import expression, statements
from app.translation import file_json, put_json from app.translation import file_json, put_json
OPS = set("label camera say scene show hide with play stop queue voice pause menu if while jump call return exec window noop nvl".split()) OPS = set("label camera say scene show hide with play stop queue voice pause menu if while jump call return exec window noop nvl".split())
+55
View File
@@ -0,0 +1,55 @@
import time
from collections import defaultdict, deque
from fastapi import Request
class FailureLimiter:
"""Sliding-window counter of failed attempts, kept in process memory.
Several uvicorn workers would each count separately, which only loosens the
limit; the service runs a single backend process.
"""
def __init__(self, max_failures: int, window_seconds: float) -> None:
self.max_failures = max_failures
self.window = window_seconds
self._hits: dict[str, deque[float]] = defaultdict(deque)
def _prune(self, key: str, now: float) -> deque[float]:
hits = self._hits[key]
while hits and now - hits[0] > self.window:
hits.popleft()
if not hits:
self._hits.pop(key, None)
return hits
def retry_after(self, *keys: str) -> int:
"""Seconds until a blocked key is allowed again, 0 if none is blocked."""
now = time.monotonic()
wait = 0.0
for key in keys:
hits = self._prune(key, now)
if len(hits) >= self.max_failures:
wait = max(wait, hits[0] + self.window - now)
return int(wait) + 1 if wait > 0 else 0
def fail(self, *keys: str) -> None:
now = time.monotonic()
for key in keys:
self._prune(key, now)
self._hits[key].append(now)
def reset(self, *keys: str) -> None:
for key in keys:
self._hits.pop(key, None)
def client_ip(request: Request) -> str:
forwarded = request.headers.get("x-forwarded-for")
if forwarded:
return forwarded.split(",")[0].strip()
return request.client.host if request.client else "unknown"
login_limiter = FailureLimiter(max_failures=8, window_seconds=15 * 60)
+12 -2
View File
@@ -1,9 +1,10 @@
from fastapi import APIRouter, HTTPException, Response, status from fastapi import APIRouter, HTTPException, Request, Response, status
from sqlalchemy import select from sqlalchemy import select
from app.config import get_settings from app.config import get_settings
from app.deps import DB, SessionUser from app.deps import DB, SessionUser
from app.models import User from app.models import User
from app.ratelimit import client_ip, login_limiter
from app.schemas import ChangeCredentialsIn, LoginIn, MeOut from app.schemas import ChangeCredentialsIn, LoginIn, MeOut
from app.security import ( from app.security import (
COOKIE_NAME, COOKIE_NAME,
@@ -30,10 +31,19 @@ def _set_cookie(response: Response, user: User) -> None:
@router.post("/login", response_model=MeOut) @router.post("/login", response_model=MeOut)
async def login(body: LoginIn, response: Response, db: DB) -> User: async def login(body: LoginIn, request: Request, response: Response, db: DB) -> User:
keys = (f"ip:{client_ip(request)}", f"user:{body.username.lower()}")
if wait := login_limiter.retry_after(*keys):
raise HTTPException(
status.HTTP_429_TOO_MANY_REQUESTS,
f"Слишком много неудачных попыток входа. Повторите через {wait} с",
headers={"Retry-After": str(wait)},
)
user = await db.scalar(select(User).where(User.username == body.username)) user = await db.scalar(select(User).where(User.username == body.username))
if not user or not user.is_active or not verify_password(user.password_hash, body.password): if not user or not user.is_active or not verify_password(user.password_hash, body.password):
login_limiter.fail(*keys)
raise HTTPException(status.HTTP_401_UNAUTHORIZED, "Неверный логин или пароль") raise HTTPException(status.HTTP_401_UNAUTHORIZED, "Неверный логин или пароль")
login_limiter.reset(*keys)
_set_cookie(response, user) _set_cookie(response, user)
return user return user
+1 -1
View File
@@ -7,11 +7,11 @@ from sqlalchemy import and_, exists, func, or_, select, true
from starlette.concurrency import run_in_threadpool from starlette.concurrency import run_in_threadpool
from app.blobs import put_bytes from app.blobs import put_bytes
from app.uploads import remove_upload
from app.deps import DB, CurrentUser, has_perm, require from app.deps import DB, CurrentUser, has_perm, require
from app.models import Game, GameVersion, User from app.models import Game, GameVersion, User
from app.schemas import GameDetailOut, GameIn, GameOut, GameUpdateIn, VersionOut from app.schemas import GameDetailOut, GameIn, GameOut, GameUpdateIn, VersionOut
from app.translation import version_languages from app.translation import version_languages
from app.uploads import remove_upload
router = APIRouter(prefix="/api/games", tags=["games"]) router = APIRouter(prefix="/api/games", tags=["games"])
Player = Annotated[User, Depends(require("games.play"))] Player = Annotated[User, Depends(require("games.play"))]
+11 -4
View File
@@ -1,19 +1,26 @@
from typing import Annotated, Any
import asyncio import asyncio
from typing import Annotated, Any
from fastapi import APIRouter, Depends, HTTPException, Request, status from fastapi import APIRouter, Depends, HTTPException, Request, status
from sqlalchemy import select from sqlalchemy import select
from starlette.concurrency import run_in_threadpool from starlette.concurrency import run_in_threadpool
from starlette.requests import ClientDisconnect from starlette.requests import ClientDisconnect
from app import translation
from app.deps import DB, CurrentUser, has_perm, require from app.deps import DB, CurrentUser, has_perm, require
from app.models import Game, GameVersion, Revision, Save, User, VersionHistory from app.models import Game, GameVersion, Revision, Save, User, VersionHistory
from app.routers.games import get_game from app.routers.games import get_game
from app.schemas import ( from app.schemas import (
ForkIn, HistoryOut, PublishIn, SaveDetailOut, SaveIn, SaveOut, VersionDetailOut, VersionIn, ForkIn,
HistoryOut,
PublishIn,
SaveDetailOut,
SaveIn,
SaveOut,
VersionDetailOut,
VersionIn,
VersionOut, VersionOut,
) )
from app import translation
from app.uploads import remove_upload, upload_path from app.uploads import remove_upload, upload_path
router = APIRouter(prefix="/api", tags=["versions"]) router = APIRouter(prefix="/api", tags=["versions"])
@@ -79,9 +86,9 @@ async def upload_version(
await db.commit() await db.commit()
raise HTTPException(status.HTTP_400_BAD_REQUEST, "Пустой файл") raise HTTPException(status.HTTP_400_BAD_REQUEST, "Пустой файл")
from app.converter.preflight import probe_archive
from app.converter.archive import ArchiveError from app.converter.archive import ArchiveError
from app.converter.compatibility import CompatibilityError from app.converter.compatibility import CompatibilityError
from app.converter.preflight import probe_archive
try: try:
probe = await asyncio.to_thread(probe_archive, path, filename) probe = await asyncio.to_thread(probe_archive, path, filename)
except (CompatibilityError, ArchiveError, OSError, ValueError) as exc: except (CompatibilityError, ArchiveError, OSError, ValueError) as exc:
+1 -1
View File
@@ -384,7 +384,7 @@ def export_po(manifest: dict[str, Any], lang: str, game_title: str) -> str:
f"msgstr {_po_quote(e.get('t', ''))}", ""] f"msgstr {_po_quote(e.get('t', ''))}", ""]
for src, usage in idx.strings.items(): for src, usage in idx.strings.items():
e = strings.get(src) or {} e = strings.get(src) or {}
out += [f"#. {usage}", f'msgctxt "string"', f"msgid {_po_quote(src)}", out += [f"#. {usage}", 'msgctxt "string"', f"msgid {_po_quote(src)}",
f"msgstr {_po_quote(e.get('t', ''))}", ""] f"msgstr {_po_quote(e.get('t', ''))}", ""]
return "\n".join(out) return "\n".join(out)
+1 -1
View File
@@ -9,8 +9,8 @@ from sqlalchemy import select, update
from app.blobs import read_bytes from app.blobs import read_bytes
from app.config import get_settings from app.config import get_settings
from app.converter.pipeline import convert_archive
from app.converter.compatibility import CompatibilityError from app.converter.compatibility import CompatibilityError
from app.converter.pipeline import convert_archive
from app.db import SessionLocal from app.db import SessionLocal
from app.models import Game, GameVersion from app.models import Game, GameVersion
from app.routers.games import make_cover from app.routers.games import make_cover
+5
View File
@@ -0,0 +1,5 @@
[mypy]
python_version = 3.12
ignore_missing_imports = True
check_untyped_defs = False
exclude = (migrations|tests)/
+10
View File
@@ -0,0 +1,10 @@
target-version = "py312"
line-length = 100
extend-exclude = ["migrations/versions"]
[lint]
select = ["E", "F", "W", "I", "B", "UP"]
ignore = ["E501", "E741", "E702", "B904", "B905", "B007", "F402"]
[lint.per-file-ignores]
"tests/*" = ["F811"]
+12 -6
View File
@@ -1,13 +1,17 @@
import asyncio import asyncio
import json import json
from pathlib import Path
from types import SimpleNamespace
import zipfile import zipfile
from types import SimpleNamespace
import pytest import pytest
from app.config import get_settings from app.config import get_settings
from app.converter.compatibility import CompatibilityError, audit_ir, detect, require_supported_scripts from app.converter.compatibility import (
CompatibilityError,
audit_ir,
detect,
require_supported_scripts,
)
from app.converter.nani_compiler import compile_package from app.converter.nani_compiler import compile_package
from app.converter.nani_expr import assignment, parse from app.converter.nani_expr import assignment, parse
from app.converter.nani_package import Package, decode_script, text_records from app.converter.nani_package import Package, decode_script, text_records
@@ -92,8 +96,8 @@ def test_renpy_atl_animation_is_rejected_instead_of_becoming_static():
def test_source_encoding_is_preserved_or_explicitly_rejected(tmp_path): def test_source_encoding_is_preserved_or_explicitly_rejected(tmp_path):
from app.converter.source import read_source
from app.converter.rpy_lexer import ParseError from app.converter.rpy_lexer import ParseError
from app.converter.source import read_source
source = tmp_path / "script.rpy" source = tmp_path / "script.rpy"
source.write_bytes('# coding: cp1251\nlabel start:\n "Привет"'.encode("cp1251")) source.write_bytes('# coding: cp1251\nlabel start:\n "Привет"'.encode("cp1251"))
assert "Привет" in read_source(source) assert "Привет" in read_source(source)
@@ -222,8 +226,8 @@ def test_naninovel_resources_keep_keys_delimiters_and_context():
def test_naninovel_import_localization_and_mod_editor_roundtrip(tmp_path, monkeypatch): def test_naninovel_import_localization_and_mod_editor_roundtrip(tmp_path, monkeypatch):
from app.blobs import read_bytes from app.blobs import read_bytes
from app.converter.naninovel import convert_naninovel from app.converter.naninovel import convert_naninovel
from app.translation import apply_changes, locale_tables, text_index
from app.mods import document, validate from app.mods import document, validate
from app.translation import apply_changes, locale_tables, text_index
monkeypatch.setenv("DATA_DIR", str(tmp_path / "library")) monkeypatch.setenv("DATA_DIR", str(tmp_path / "library"))
get_settings.cache_clear() get_settings.cache_clear()
@@ -250,6 +254,7 @@ def test_naninovel_import_localization_and_mod_editor_roundtrip(tmp_path, monkey
def test_unity_codec_restores_global_hooks_after_error(): def test_unity_codec_restores_global_hooks_after_error():
from UnityPy.helpers import TypeTreeHelper as helper from UnityPy.helpers import TypeTreeHelper as helper
from app.converter.unity_scripts import ScriptCodec from app.converter.unity_scripts import ScriptCodec
codec = ScriptCodec.__new__(ScriptCodec) codec = ScriptCodec.__new__(ScriptCodec)
before = (helper.read_value, helper.write_value, helper.read_typetree_boost, helper.get_ref_type_node) before = (helper.read_value, helper.write_value, helper.read_typetree_boost, helper.get_ref_type_node)
@@ -288,9 +293,10 @@ def test_unused_renpy_translation_entries_are_not_script_blockers(tmp_path):
def test_upload_rejects_unsupported_engine_without_queueing(tmp_path, monkeypatch): def test_upload_rejects_unsupported_engine_without_queueing(tmp_path, monkeypatch):
from fastapi import HTTPException
from app.routers import versions from app.routers import versions
from app.uploads import upload_path from app.uploads import upload_path
from fastapi import HTTPException
source = tmp_path / "unsupported.zip" source = tmp_path / "unsupported.zip"
with zipfile.ZipFile(source, "w") as archive: with zipfile.ZipFile(source, "w") as archive:
+2 -2
View File
@@ -1,12 +1,11 @@
"""Integration against PostgreSQL in a temporary schema, never touches library rows.""" """Integration against PostgreSQL in a temporary schema, never touches library rows."""
import asyncio import asyncio
import copy
import os import os
from uuid import uuid4 from uuid import uuid4
import httpx import httpx
import pytest import pytest
from sqlalchemy import select, text from sqlalchemy import text
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app import db as database from app import db as database
@@ -128,6 +127,7 @@ def test_complete_mod_workflow(pg_manifest, monkeypatch):
assert repeated_lines.count("Upstream addition") == 1 assert repeated_lines.count("Upstream addition") == 1
# A real replacement is converted, and a stale upload is refused. # A real replacement is converted, and a stale upload is refused.
import io import io
from PIL import Image from PIL import Image
image = io.BytesIO() image = io.BytesIO()
Image.new("RGB", (32, 32), "red").save(image, "PNG") Image.new("RGB", (32, 32), "red").save(image, "PNG")
+25
View File
@@ -0,0 +1,25 @@
from app.ratelimit import FailureLimiter
def test_blocks_after_max_failures_and_resets():
limiter = FailureLimiter(max_failures=3, window_seconds=60)
for _ in range(2):
limiter.fail("ip:1", "user:a")
assert limiter.retry_after("ip:1", "user:a") == 0
limiter.fail("ip:1", "user:a")
assert limiter.retry_after("ip:1", "user:a") > 0
assert limiter.retry_after("ip:2", "user:b") == 0
limiter.reset("ip:1", "user:a")
assert limiter.retry_after("ip:1", "user:a") == 0
def test_window_expires(monkeypatch):
import app.ratelimit as rl
now = [1000.0]
monkeypatch.setattr(rl.time, "monotonic", lambda: now[0])
limiter = FailureLimiter(max_failures=1, window_seconds=10)
limiter.fail("k")
assert limiter.retry_after("k") > 0
now[0] += 11
assert limiter.retry_after("k") == 0
+2 -2
View File
@@ -6,11 +6,11 @@ import pytest
def test_native_translation_and_mod_edits_rebuild_the_same_vm_input(tmp_path, monkeypatch): def test_native_translation_and_mod_edits_rebuild_the_same_vm_input(tmp_path, monkeypatch):
from app import mods from app import mods
from app.config import get_settings from app.config import get_settings
from app.converter.naninovel import _convert_package
from app.converter.nani_package import Package, text_records from app.converter.nani_package import Package, text_records
from app.converter.naninovel import _convert_package
from app.converter.tomie_format import unpack_modules from app.converter.tomie_format import unpack_modules
from app.converter.tomie_profile import PROFILE from app.converter.tomie_profile import PROFILE
from app.translation import apply_changes, text_index, locale_tables from app.translation import apply_changes, locale_tables, text_index
monkeypatch.setenv("DATA_DIR", str(tmp_path / "data")) monkeypatch.setenv("DATA_DIR", str(tmp_path / "data"))
get_settings.cache_clear() get_settings.cache_clear()
package = Package(profile=PROFILE, defaults={"points": 3}, function_data={"names": ["Tomie"]}) package = Package(profile=PROFILE, defaults={"points": 3}, function_data={"names": ["Tomie"]})
+9 -3
View File
@@ -1,12 +1,18 @@
import json
import os
from pathlib import Path from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
import os
import json
import pytest import pytest
from app.converter.nani_expr import assignment, parse from app.converter.nani_expr import assignment, parse
from app.converter.tomie_profile import ASSEMBLY_HASH, FUNCTIONS, constant_value, extract_tables, identify from app.converter.tomie_profile import (
ASSEMBLY_HASH,
FUNCTIONS,
constant_value,
extract_tables,
identify,
)
def instruction(op, operand=None): def instruction(op, operand=None):
+17 -6
View File
@@ -131,10 +131,21 @@ LOCKED`), Redis не нужен.
* меню выборов, история реплик, skip/auto, быстрые и слотовые сохранения * меню выборов, история реплик, skip/auto, быстрые и слотовые сохранения
на сервере, громкость каналов, предзагрузка ближайших картинок. на сервере, громкость каналов, предзагрузка ближайших картинок.
## Дорожная карта ## Состояние и планы
* Реализованы редактор PGVN-форка, замена ассетов, ревизии, трёхстороннее слияние Реализовано:
и перенос на новый оригинал — см. [MODDING.md](MODDING.md).
* Поддержка screens/ATL-трансформаций в IR. * редактор PGVN-форка, замена ассетов, ревизии, трёхстороннее слияние и перенос
* Другие движки (TyranoBuilder, RPG Maker MV) через тот же IR. на новый оригинал — см. [MODDING.md](MODDING.md);
* Сборщик мусора доступен как `python -m app.gc`, включая периодический режим. * импорт Naninovel и адаптер Tomie, пакетный импорт и строгая проверка
совместимости — см. [IMPORT.md](IMPORT.md);
* редактор переводов — см. [LOCALIZATION.md](LOCALIZATION.md);
* сборщик мусора (`python -m app.gc`, есть периодический режим);
* ограничение неудачных попыток входа (`app/ratelimit.py`), CI в GitHub Actions.
Планы:
* поддержка `screen`-языка и ATL-анимаций с движением в IR;
* другие движки (TyranoBuilder, RPG Maker MV) через тот же IR;
* дальнейшее дробление `engine.ts` на модули (рендер сцены, UI, сохранения);
* типизация backend под `mypy` (сейчас проверка справочная).
+6 -161
View File
@@ -1,84 +1,16 @@
import { useCallback, useEffect, useMemo, useRef, useState, type ReactNode } from "react"; import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { Link, useNavigate, useParams, useSearchParams } from "react-router-dom"; import { Link, useNavigate, useParams, useSearchParams } from "react-router-dom";
import { api, uploadRaw, type Version } from "../api"; import { api, uploadRaw, type Version } from "../api";
import { EmptyState, ErrorNote, PageHeader, Spinner } from "../ui/Bits"; import { EmptyState, ErrorNote, PageHeader, Spinner } from "../ui/Bits";
import { Icon } from "../ui/Icon"; import { Icon } from "../ui/Icon";
import { useUi } from "../ui/Ui"; import { useUi } from "../ui/Ui";
import { GlossaryEditor, ProgressBar, highlight } from "./TranslationParts";
import {
FILTERS, PAGE, STATE_BADGE, STATE_LABEL, USAGE,
type GlossaryTerm, type Item, type LangStats, type LineState, type Overview, type Tab,
} from "./translation-types";
import "./translate.css"; import "./translate.css";
type LineState = "untranslated" | "outdated" | "translated" | "fuzzy" | "machine" | "approved";
interface LangStats {
code: string;
name: string;
font: string | null;
lines: number;
translated: number;
progress: number;
states: Record<LineState, number>;
strings: number;
stringsTranslated: number;
}
interface Overview {
sourceLang: string | null;
editable: boolean;
canApprove: boolean;
lines: number;
strings: number;
modules: { path: string; lines: number }[];
languages: LangStats[];
knownLanguages: Record<string, string>;
}
interface Item {
key: string;
source: string;
translation: string;
state: LineState;
// lines
nodeId?: string;
module?: string;
label?: string;
who?: string;
prev?: string | null;
next?: string | null;
scene?: string | null;
// strings
usage?: string;
}
interface GlossaryTerm {
src: string;
t: string;
note: string;
}
type Tab = "lines" | "strings" | "glossary";
const STATE_LABEL: Record<LineState, string> = {
untranslated: "Не переведено",
outdated: "Устарело",
fuzzy: "Неточно",
machine: "Машинный",
translated: "Переведено",
approved: "Одобрено",
};
const STATE_BADGE: Record<LineState, string> = {
untranslated: "badge-plain",
outdated: "badge-error",
fuzzy: "badge-warn",
machine: "badge-warn",
translated: "badge-ok",
approved: "badge-ok",
};
const FILTERS: [string, string][] = [
["todo", "К работе"], ["all", "Все"], ["untranslated", "Не переведено"], ["outdated", "Устарело"],
["fuzzy", "Неточно"], ["machine", "Машинный"], ["translated", "Переведено"], ["approved", "Одобрено"],
];
const PAGE = 40;
const USAGE: Record<string, string> = { name: "имя персонажа", menu: "пункт меню", text: "текст на экране" };
export function TranslationEditor() { export function TranslationEditor() {
const { versionId } = useParams(); const { versionId } = useParams();
const [params, setParams] = useSearchParams(); const [params, setParams] = useSearchParams();
@@ -275,29 +207,6 @@ export function TranslationEditor() {
); );
} }
function ProgressBar({ stats }: { stats: LangStats }) {
const total = stats.lines || 1;
const seg = (state: LineState, cls: string) =>
stats.states[state] ? <div className={cls} style={{ width: `${(stats.states[state] / total) * 100}%` }}
title={`${STATE_LABEL[state]}: ${stats.states[state]}`} /> : null;
return (
<div className="tr-progress">
<div className="tr-bar">
{seg("approved", "seg-approved")}
{seg("translated", "seg-ok")}
{seg("fuzzy", "seg-warn")}
{seg("machine", "seg-warn")}
{seg("outdated", "seg-error")}
</div>
<span className="muted small">
{stats.translated} из {stats.lines} реплик ({Math.round(stats.progress * 100)}%)
{stats.states.outdated ? ` · устарело ${stats.states.outdated}` : ""}
{" · "}строк {stats.stringsTranslated}/{stats.strings}
</span>
</div>
);
}
function EntriesView({ versionId, lang, tab, ov, onStats }: { function EntriesView({ versionId, lang, tab, ov, onStats }: {
versionId: string; lang: string; tab: Tab; ov: Overview; onStats: (s: Partial<LangStats>) => void; versionId: string; lang: string; tab: Tab; ov: Overview; onStats: (s: Partial<LangStats>) => void;
}) { }) {
@@ -524,67 +433,3 @@ function Row({ item, versionId, lang, editable, canApprove, terms, onChange, onB
</div> </div>
); );
} }
function highlight(text: string, terms: GlossaryTerm[]): ReactNode {
if (!terms.length) return text;
const escaped = terms.map((t) => t.src.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")).sort((a, b) => b.length - a.length);
const re = new RegExp(`(${escaped.join("|")})`, "gi");
return text.split(re).map((part, i) => {
const term = i % 2 ? terms.find((t) => t.src.toLowerCase() === part.toLowerCase()) : undefined;
return term ? <mark key={i} title={`${term.t}${term.note ? ` — ${term.note}` : ""}`}>{part}</mark> : part;
});
}
function GlossaryEditor({ versionId, lang, editable, terms, onChange }: {
versionId: string; lang: string; editable: boolean; terms: GlossaryTerm[]; onChange: (t: GlossaryTerm[]) => void;
}) {
const ui = useUi();
const [rows, setRows] = useState<GlossaryTerm[]>(terms.length ? terms : [{ src: "", t: "", note: "" }]);
const save = async () => {
try {
await api.put(`/api/versions/${versionId}/i18n/${lang}`, { glossary: rows });
onChange(rows.filter((r) => r.src.trim()));
ui.toast("Глоссарий сохранён", "ok");
} catch (e) {
ui.toast((e as Error).message, "error");
}
};
const set = (i: number, key: keyof GlossaryTerm, value: string) =>
setRows((rs) => rs.map((r, n) => n === i ? { ...r, [key]: value } : r));
return (
<div className="card">
<p className="muted small" style={{ marginTop: 0 }}>
Имена и термины, которые нужно переводить единообразно. В репликах они подсвечиваются,
при наведении виден принятый перевод.
</p>
<div className="table-wrap">
<table className="table tr-glossary">
<thead><tr><th>Оригинал</th><th>Перевод</th><th>Заметка</th><th /></tr></thead>
<tbody>
{rows.map((r, i) => (
<tr key={i}>
<td><input value={r.src} readOnly={!editable} onChange={(e) => set(i, "src", e.target.value)} /></td>
<td><input value={r.t} readOnly={!editable} onChange={(e) => set(i, "t", e.target.value)} lang={lang} /></td>
<td><input value={r.note} readOnly={!editable} onChange={(e) => set(i, "note", e.target.value)} /></td>
<td>
{editable && (
<button className="btn btn-sm btn-ghost btn-icon btn-danger" aria-label="Удалить термин"
onClick={() => setRows((rs) => rs.filter((_, n) => n !== i))}><Icon name="x" size={14} /></button>
)}
</td>
</tr>
))}
</tbody>
</table>
</div>
{editable && (
<div className="actions" style={{ marginTop: "var(--space-3)" }}>
<button className="btn btn-sm" onClick={() => setRows((rs) => [...rs, { src: "", t: "", note: "" }])}>
<Icon name="plus" size={16} />Термин
</button>
<button className="btn btn-sm btn-primary" onClick={() => void save()}><Icon name="check" size={16} />Сохранить</button>
</div>
)}
</div>
);
}
+93
View File
@@ -0,0 +1,93 @@
import { useState, type ReactNode } from "react";
import { api } from "../api";
import { Icon } from "../ui/Icon";
import { useUi } from "../ui/Ui";
import { STATE_LABEL, type GlossaryTerm, type LangStats, type LineState } from "./translation-types";
export function ProgressBar({ stats }: { stats: LangStats }) {
const total = stats.lines || 1;
const seg = (state: LineState, cls: string) =>
stats.states[state] ? <div className={cls} style={{ width: `${(stats.states[state] / total) * 100}%` }}
title={`${STATE_LABEL[state]}: ${stats.states[state]}`} /> : null;
return (
<div className="tr-progress">
<div className="tr-bar">
{seg("approved", "seg-approved")}
{seg("translated", "seg-ok")}
{seg("fuzzy", "seg-warn")}
{seg("machine", "seg-warn")}
{seg("outdated", "seg-error")}
</div>
<span className="muted small">
{stats.translated} из {stats.lines} реплик ({Math.round(stats.progress * 100)}%)
{stats.states.outdated ? ` · устарело ${stats.states.outdated}` : ""}
{" · "}строк {stats.stringsTranslated}/{stats.strings}
</span>
</div>
);
}
export function highlight(text: string, terms: GlossaryTerm[]): ReactNode {
if (!terms.length) return text;
const escaped = terms.map((t) => t.src.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")).sort((a, b) => b.length - a.length);
const re = new RegExp(`(${escaped.join("|")})`, "gi");
return text.split(re).map((part, i) => {
const term = i % 2 ? terms.find((t) => t.src.toLowerCase() === part.toLowerCase()) : undefined;
return term ? <mark key={i} title={`${term.t}${term.note ? ` — ${term.note}` : ""}`}>{part}</mark> : part;
});
}
export function GlossaryEditor({ versionId, lang, editable, terms, onChange }: {
versionId: string; lang: string; editable: boolean; terms: GlossaryTerm[]; onChange: (t: GlossaryTerm[]) => void;
}) {
const ui = useUi();
const [rows, setRows] = useState<GlossaryTerm[]>(terms.length ? terms : [{ src: "", t: "", note: "" }]);
const save = async () => {
try {
await api.put(`/api/versions/${versionId}/i18n/${lang}`, { glossary: rows });
onChange(rows.filter((r) => r.src.trim()));
ui.toast("Глоссарий сохранён", "ok");
} catch (e) {
ui.toast((e as Error).message, "error");
}
};
const set = (i: number, key: keyof GlossaryTerm, value: string) =>
setRows((rs) => rs.map((r, n) => n === i ? { ...r, [key]: value } : r));
return (
<div className="card">
<p className="muted small" style={{ marginTop: 0 }}>
Имена и термины, которые нужно переводить единообразно. В репликах они подсвечиваются,
при наведении виден принятый перевод.
</p>
<div className="table-wrap">
<table className="table tr-glossary">
<thead><tr><th>Оригинал</th><th>Перевод</th><th>Заметка</th><th /></tr></thead>
<tbody>
{rows.map((r, i) => (
<tr key={i}>
<td><input value={r.src} readOnly={!editable} onChange={(e) => set(i, "src", e.target.value)} /></td>
<td><input value={r.t} readOnly={!editable} onChange={(e) => set(i, "t", e.target.value)} lang={lang} /></td>
<td><input value={r.note} readOnly={!editable} onChange={(e) => set(i, "note", e.target.value)} /></td>
<td>
{editable && (
<button className="btn btn-sm btn-ghost btn-icon btn-danger" aria-label="Удалить термин"
onClick={() => setRows((rs) => rs.filter((_, n) => n !== i))}><Icon name="x" size={14} /></button>
)}
</td>
</tr>
))}
</tbody>
</table>
</div>
{editable && (
<div className="actions" style={{ marginTop: "var(--space-3)" }}>
<button className="btn btn-sm" onClick={() => setRows((rs) => [...rs, { src: "", t: "", note: "" }])}>
<Icon name="plus" size={16} />Термин
</button>
<button className="btn btn-sm btn-primary" onClick={() => void save()}><Icon name="check" size={16} />Сохранить</button>
</div>
)}
</div>
);
}
+74
View File
@@ -0,0 +1,74 @@
// Shared types and labels of the translation editor.
export type LineState = "untranslated" | "outdated" | "translated" | "fuzzy" | "machine" | "approved";
export interface LangStats {
code: string;
name: string;
font: string | null;
lines: number;
translated: number;
progress: number;
states: Record<LineState, number>;
strings: number;
stringsTranslated: number;
}
export interface Overview {
sourceLang: string | null;
editable: boolean;
canApprove: boolean;
lines: number;
strings: number;
modules: { path: string; lines: number }[];
languages: LangStats[];
knownLanguages: Record<string, string>;
}
export interface Item {
key: string;
source: string;
translation: string;
state: LineState;
// lines
nodeId?: string;
module?: string;
label?: string;
who?: string;
prev?: string | null;
next?: string | null;
scene?: string | null;
// strings
usage?: string;
}
export interface GlossaryTerm {
src: string;
t: string;
note: string;
}
export type Tab = "lines" | "strings" | "glossary";
export const STATE_LABEL: Record<LineState, string> = {
untranslated: "Не переведено",
outdated: "Устарело",
fuzzy: "Неточно",
machine: "Машинный",
translated: "Переведено",
approved: "Одобрено",
};
export const STATE_BADGE: Record<LineState, string> = {
untranslated: "badge-plain",
outdated: "badge-error",
fuzzy: "badge-warn",
machine: "badge-warn",
translated: "badge-ok",
approved: "badge-ok",
};
export const FILTERS: [string, string][] = [
["todo", "К работе"], ["all", "Все"], ["untranslated", "Не переведено"], ["outdated", "Устарело"],
["fuzzy", "Неточно"], ["machine", "Машинный"], ["translated", "Переведено"], ["approved", "Одобрено"],
];
export const PAGE = 40;
export const USAGE: Record<string, string> = { name: "имя персонажа", menu: "пункт меню", text: "текст на экране" };
+51
View File
@@ -0,0 +1,51 @@
// Small shared pieces of the engine: save API types, settings and DOM/storage helpers.
import type { Volumes } from "./audio";
import type { SaveData } from "./types";
export interface SaveMeta {
slot: string;
title: string;
updated_at: string;
}
export interface SaveApi<T = SaveData> {
list(): Promise<SaveMeta[]>;
get(slot: string): Promise<T | null>;
put(slot: string, title: string, data: T): Promise<void>;
remove(slot: string): Promise<void>;
}
export interface Settings extends Volumes {
cps: number; // characters per second, 0 = instant
autoDelay: number; // seconds per 20 characters in auto mode
}
export const DEFAULT_SETTINGS: Settings = { cps: 45, autoDelay: 1.2, music: 0.7, sound: 0.8, voice: 1 };
export const SLOTS = 12;
export class Restart extends Error {}
export function el<K extends keyof HTMLElementTagNameMap>(tag: K, cls = "", text = ""): HTMLElementTagNameMap[K] {
const e = document.createElement(tag);
if (cls) e.className = cls;
if (text) e.textContent = text;
return e;
}
export function loadJSON<T>(key: string, fallback: T): T {
try {
const raw = localStorage.getItem(key);
return raw ? { ...fallback, ...JSON.parse(raw) } : fallback;
} catch {
return fallback;
}
}
export function saveJSON(key: string, value: unknown) {
try {
localStorage.setItem(key, JSON.stringify(value));
} catch {
/* storage unavailable */
}
}
+3 -47
View File
@@ -1,35 +1,17 @@
// PGVN interpreter + in-game UI. // PGVN interpreter + in-game UI.
import { AudioSystem, type Volumes } from "./audio"; import { AudioSystem } from "./audio";
import { JumpSignal, PyDict, Python, decodeValue, encodeValue, pyStr, truthy, type Scope } from "./python"; import { JumpSignal, PyDict, Python, decodeValue, encodeValue, pyStr, truthy, type Scope } from "./python";
import { Renderer, type ResolvedImage } from "./renderer"; import { Renderer, type ResolvedImage } from "./renderer";
import { fastPoint, interpolate, parseText, plainText, renderText, waitPoints } from "./text"; import { fastPoint, interpolate, parseText, plainText, renderText, waitPoints } from "./text";
import { languageName, rememberLanguage, type Translator } from "./i18n"; import { languageName, rememberLanguage, type Translator } from "./i18n";
import { naniText } from "./naninovel"; import { naniText } from "./naninovel";
import { DEFAULT_SETTINGS, SLOTS, Restart, el, loadJSON, saveJSON, type SaveApi, type SaveMeta, type Settings } from "./engine-util";
import type { import type {
CharacterDef, Expr, Frame, GameData, ImageDef, Label, Layer, Manifest, Node, SaveData, ScriptModule, CharacterDef, Expr, Frame, GameData, ImageDef, Label, Layer, Manifest, Node, SaveData, ScriptModule,
} from "./types"; } from "./types";
export interface SaveMeta { export type { SaveApi, SaveMeta } from "./engine-util";
slot: string;
title: string;
updated_at: string;
}
export interface SaveApi<T = SaveData> {
list(): Promise<SaveMeta[]>;
get(slot: string): Promise<T | null>;
put(slot: string, title: string, data: T): Promise<void>;
remove(slot: string): Promise<void>;
}
interface Settings extends Volumes {
cps: number; // characters per second, 0 = instant
autoDelay: number; // seconds per 20 characters in auto mode
}
const DEFAULT_SETTINGS: Settings = { cps: 45, autoDelay: 1.2, music: 0.7, sound: 0.8, voice: 1 };
const SLOTS = 12;
interface LabelRef { interface LabelRef {
label: Label; label: Label;
@@ -37,32 +19,6 @@ interface LabelRef {
index: number; index: number;
} }
class Restart extends Error {}
function el<K extends keyof HTMLElementTagNameMap>(tag: K, cls = "", text = ""): HTMLElementTagNameMap[K] {
const e = document.createElement(tag);
if (cls) e.className = cls;
if (text) e.textContent = text;
return e;
}
function loadJSON<T>(key: string, fallback: T): T {
try {
const raw = localStorage.getItem(key);
return raw ? { ...fallback, ...JSON.parse(raw) } : fallback;
} catch {
return fallback;
}
}
function saveJSON(key: string, value: unknown) {
try {
localStorage.setItem(key, JSON.stringify(value));
} catch {
/* storage unavailable */
}
}
export class Engine { export class Engine {
private labels = new Map<string, LabelRef>(); private labels = new Map<string, LabelRef>();
private anchors = new Map<string, { label: string; path: number[]; params?: [string, Expr | null][] | null }>(); private anchors = new Map<string, { label: string; path: number[]; params?: [string, Expr | null][] | null }>();
+42
View File
@@ -0,0 +1,42 @@
import assert from "node:assert/strict";
import test from "node:test";
import { fileURLToPath } from "node:url";
import { build } from "esbuild";
// Bundle React together with the code under test so that both share one React instance.
async function load(entry) {
const result = await build({
stdin: { contents: entry, resolveDir: fileURLToPath(new URL("../src/", import.meta.url)), loader: "tsx" },
bundle: true, write: false, format: "esm", platform: "node", jsx: "automatic", logLevel: "silent",
banner: { js: "import { createRequire } from \"node:module\"; const require = createRequire(process.cwd() + \"/package.json\");" },
});
const code = Buffer.from(result.outputFiles[0].text).toString("base64");
return import(`data:text/javascript;base64,${code}`);
}
const { renderToStaticMarkup, h, ProgressBar, highlight } = await load(`
export { renderToStaticMarkup } from "react-dom/server";
export { createElement as h } from "react";
export { ProgressBar, highlight } from "./pages/TranslationParts";
`);
const stats = {
lines: 10, translated: 5, progress: 0.5, strings: 4, stringsTranslated: 2,
states: { untranslated: 5, outdated: 1, translated: 3, fuzzy: 0, machine: 0, approved: 1 },
};
test("ProgressBar renders counts and only non-empty segments", () => {
const html = renderToStaticMarkup(h(ProgressBar, { stats }));
assert.match(html, /5 из 10 реплик \(50%\)/);
assert.match(html, /устарело 1/);
assert.match(html, /seg-approved/);
assert.doesNotMatch(html, /seg-warn/);
});
test("highlight wraps glossary terms case-insensitively and escapes regex characters", () => {
const terms = [{ src: "Eileen", t: "Эйлин", note: "" }, { src: "a+b", t: "x", note: "" }];
const html = renderToStaticMarkup(h("p", null, highlight("eileen met a+b", terms)));
assert.match(html, /<mark title="Эйлин">eileen<\/mark>/);
assert.match(html, /<mark title="x">a\+b<\/mark>/);
assert.equal(highlight("plain", []), "plain");
});