Seed Assistent personas as overlay folders and tighten Ollama/Assistent glue.

gpu-rent now writes personas/<id>/ on the VM (not legacy personas.json), adds seed-personas/doctor checks, and shortens mid/high keep-alive now that Assistent parks the LLM before Generate.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Leonid Pershin
2026-08-22 01:00:14 +03:00
co-authored by Cursor
parent 789fa26918
commit 03ba4cb6ed
22 changed files with 430 additions and 53 deletions
+4 -2
View File
@@ -1,6 +1,8 @@
# Copy to assistent-personas.yaml (never commit the working copy). # Copy to assistent-personas.yaml (never commit the working copy).
# Synced to the VM on each up → /mnt/swarm_data/Assistent/personas.yaml + personas.json # Synced to the VM on each up / seed-personas →
# Assistent reads personas.json (overlay). Yaml is the laptop source for gpu-rent seed. # /mnt/swarm_data/Assistent/personas/<id>/{persona.json,extra.md}
# /mnt/swarm_data/Assistent/_base/assistant.json (default_persona, num_ctx)
# Yaml is the laptop source for gpu-rent seed; Assistent reads overlay folders.
default: neutral default: neutral
+2
View File
@@ -18,6 +18,8 @@
Файл маленький (danbooru.csv ≈ 34 MB). Killer из‑за него не откладываем, но на первом bootstrap качаем **до** старта UI, вместе с extensions. Файл маленький (danbooru.csv ≈ 34 MB). Killer из‑за него не откладываем, но на первом bootstrap качаем **до** старта UI, вместе с extensions.
Swarm Assistent индексирует тот же csv в FTS-таблицу `tags` внутри `Assistent/memory/assistent.sqlite` (без эмбедов). Первый чат после смены sha занимает несколько секунд; дальше `lookup_tags` ищет канон/алиасы. В `memory_hits` csv не попадает.
## Откуда файл ## Откуда файл
Дефолт — то, на что ссылается сама документация SwarmUI: Дефолт — то, на что ссылается сама документация SwarmUI:
+1
View File
@@ -75,6 +75,7 @@ gpu-rent up --yes --ollama
| `gpu-rent ssh` | Оболочка на VM | | `gpu-rent ssh` | Оболочка на VM |
| `gpu-rent seed-models` | Докачать новые строки Civitai-манифеста на живой диск | | `gpu-rent seed-models` | Докачать новые строки Civitai-манифеста на живой диск |
| `gpu-rent seed-extensions` | Доклонировать/обновить git-репы; restart swarmui | | `gpu-rent seed-extensions` | Доклонировать/обновить git-репы; restart swarmui |
| `gpu-rent seed-personas` | Пуш `assistent-personas.yaml` → overlay на VM (без полного up) |
| `gpu-rent push` / `push-models` | Локальные деревья → VM | | `gpu-rent push` / `push-models` | Локальные деревья → VM |
| `gpu-rent pull-output` | VM `Output/``./Output` | | `gpu-rent pull-output` | VM `Output/``./Output` |
| `gpu-rent capture` / `capture all` | Инвентарь VM → merge **ссылок** в `models.yaml` + `extensions.yaml` (веса не качать) | | `gpu-rent capture` / `capture all` | Инвентарь VM → merge **ссылок** в `models.yaml` + `extensions.yaml` (веса не качать) |
+1 -1
View File
@@ -10,7 +10,7 @@
| Civitai хост | Дефолт API `civitai.red` (полный каталог). `.com` — SFW-витрина, NSFW с неё часто 404. Ссылки `.com`/`.red`/`.green` в манифесте принимаем. 404 → один retry на второй хост. Один токен на оба домена | | Civitai хост | Дефолт API `civitai.red` (полный каталог). `.com` — SFW-витрина, NSFW с неё часто 404. Ссылки `.com`/`.red`/`.green` в манифесте принимаем. 404 → один retry на второй хост. Один токен на оба домена |
| Пул GPU | Перед `up`/`flavors` сканируем `SCAN_POOLS` (дефолт `ru-6,ru-7`) по compute URL. Сегменты `ru-6a/b/c` берём из Nova `availability_zones`, не отдельными пулами. Пресет размера `FLAVOR_SIZE_PRESET` (дефолт `cheap` = 1×GPU, min vCPU/RAM). Диск уже есть — AZ диска, не дешёвый чужой сегмент. Первый запуск — дешёвый SKU в сегменте, где есть карта из `FLAVOR_PREFERENCE`. `.env` автоматом не пишем | | Пул GPU | Перед `up`/`flavors` сканируем `SCAN_POOLS` (дефолт `ru-6,ru-7`) по compute URL. Сегменты `ru-6a/b/c` берём из Nova `availability_zones`, не отдельными пулами. Пресет размера `FLAVOR_SIZE_PRESET` (дефолт `cheap` = 1×GPU, min vCPU/RAM). Диск уже есть — AZ диска, не дешёвый чужой сегмент. Первый запуск — дешёвый SKU в сегменте, где есть карта из `FLAVOR_PREFERENCE`. `.env` автоматом не пишем |
| Манифест моделей | `<repo>/models.yaml`, типы: checkpoint / lora / vae / embedding / controlnet / upscaler. В git только `models.example.yaml` | | Манифест моделей | `<repo>/models.yaml`, типы: checkpoint / lora / vae / embedding / controlnet / upscaler. В git только `models.example.yaml` |
| Расширения | `<repo>/extensions.yaml`: git-репы `swarmui``src/Extensions`, `comfy` → DLNodes. Поле `requires: none\|ollama\|any-llm` фильтрует по `LLM_RUNTIME`. Пример: swarm-assistent с `requires: ollama` | | Расширения | `<repo>/extensions.yaml`: git-репы `swarmui``src/Extensions`, `comfy` → DLNodes. Поле `requires: none\|ollama\|any-llm` фильтрует по `LLM_RUNTIME`. Пример: swarm-assistent с `requires: ollama`. Personas: `assistent-personas.yaml` → overlay `Assistent/personas/<id>/` на `up` / `seed-personas` |
| Autocomplete | До первого старта: скачать word-list в `Data/Autocompletions`, прописать `DefaultUser.AutoComplete.Source`. На каждом `up` сверить GitHub blob sha и обновить файл, если изменился. Дефолт: `tags/danbooru.csv` из a1111-sd-webui-tagcomplete (как в доке SwarmUI) | | Autocomplete | До первого старта: скачать word-list в `Data/Autocompletions`, прописать `DefaultUser.AutoComplete.Source`. На каждом `up` сверить GitHub blob sha и обновить файл, если изменился. Дефолт: `tags/danbooru.csv` из a1111-sd-webui-tagcomplete (как в доке SwarmUI) |
| Доступ | Браузер на туннеле; MCP переключается на облако, пока оно живо; HTTP API SwarmUI через тот же туннель | | Доступ | Браузер на туннеле; MCP переключается на облако, пока оно живо; HTTP API SwarmUI через тот же туннель |
| Preemptible | По умолчанию всегда. Обычный сервер — только `--no-spot` | | Preemptible | По умолчанию всегда. Обычный сервер — только `--no-spot` |
+1 -1
View File
@@ -53,7 +53,7 @@ comfy:
| `ollama` | только при `LLM_RUNTIME=ollama` | | `ollama` | только при `LLM_RUNTIME=ollama` |
| `any-llm` | при `LLM_RUNTIME=ollama` | | `any-llm` | при `LLM_RUNTIME=ollama` |
Строки с несовпавшим `requires` пропускаются (лог), остальные ставятся как обычно. В `extensions.example.yaml` по умолчанию — **swarm-assistent** с `requires: ollama` (чат + доска Generate/Ref + persona/Cards, ветка `main`). На `up` сидится `assistent-personas.yaml``/mnt/swarm_data/Assistent/personas.yaml` **и** `personas.json` (Assistent читает только **json**). Строки с несовпавшим `requires` пропускаются (лог), остальные ставятся как обычно. В `extensions.example.yaml` по умолчанию — **swarm-assistent** с `requires: ollama` (чат + доска Generate/Ref + persona/Cards, ветка `main`). На `up``gpu-rent seed-personas`) сидится `assistent-personas.yaml` overlay `/mnt/swarm_data/Assistent/personas/<id>/persona.json` + `extra.md`, плюс `_base/assistant.json` (`default_persona`, опционально `num_ctx` с GPU tier). Legacy `personas.json` больше не пишется.
В git репозитория gpu-rent не коммитить рабочий список с лишними приватными URL сверх примера. В git репозитория gpu-rent не коммитить рабочий список с лишними приватными URL сверх примера.
+14 -7
View File
@@ -81,18 +81,20 @@ $env:OLLAMA_HOST = "http://127.0.0.1:17811"
После успешного тега — **warmup**: 1-token `POST /api/chat`, чтобы веса (~6 GB VL) легли в VRAM до первого сообщения в Assistent. То же при `gpu-rent tunnel`, если `/api/ps` пуст. Промах warmup — warning, GPU не гасим. После успешного тега — **warmup**: 1-token `POST /api/chat`, чтобы веса (~6 GB VL) легли в VRAM до первого сообщения в Assistent. То же при `gpu-rent tunnel`, если `/api/ps` пуст. Промах warmup — warning, GPU не гасим.
Qwen3-VL (`huihui_ai/qwen3-vl-abliterated:…`) требует **Ollama ≥ 0.12.7**. Тег `:8b-instruct` — чат Instruct; `:latest` у этой библиотеки — Thinking, не ставить дефолтом.
### Пресеты (меню) ### Пресеты (меню)
| # | ключ | tag / смысл | | # | ключ | tag / смысл |
| --- | --- | --- | | --- | --- | --- |
| 1 | **recommended** | `huihui_ai/qwen2.5-vl-abliterated:7b` — RU + vision, ~6GB | | 1 | **recommended** | `huihui_ai/qwen3-vl-abliterated:8b-instruct` — RU + vision, ~6.1GB; запасной `huihui_ai/qwen2.5-vl-abliterated:7b` |
| 2 | light | `…:3b` — то же, мало VRAM (~3GB) | | 2 | light | `…:3b` — то же, мало VRAM (~3GB) |
| 3 | text | `huihui_ai/qwen2.5-abliterate:7b` — RU, без vision (~5GB) | | 3 | text | `huihui_ai/qwen2.5-abliterate:7b` — RU, без vision (~5GB) |
| 4 | big | `huihui_ai/qwen2.5-vl-abliterated:32b` — RU + vision (~21GB) | | 4 | big | `huihui_ai/qwen2.5-vl-abliterated:32b` — RU + vision (~21GB) |
| 5 | empty | только runtime | | 5 | empty | только runtime |
| — | keep | не трогать yaml | | — | keep | не трогать yaml |
Все пресеты (кроме `empty`) — **abliterated** Qwen2.5 с нормальным русским. Официальные censored-теги (`qwen2.5vl:…`) в меню нет. Рекомендуемый пресет — **abliterated Qwen3-VL Instruct**; light / text / big пока Qwen2.5. Официальные censored-теги (`qwen2.5vl:…`) в меню нет.
`default: true` в yaml — preferred в логе; pull идёт по всему списку. `default: true` в yaml — preferred в логе; pull идёт по всему списку.
@@ -105,18 +107,22 @@ Unit `gpu-rent-ollama` читает `/mnt/swarm_data/.gpu-rent-gpu.json`:
| Tier (VRAM) | Flash Attn | KEEP_ALIVE | KV cache | GPU_OVERHEAD | CONTEXT | | Tier (VRAM) | Flash Attn | KEEP_ALIVE | KV cache | GPU_OVERHEAD | CONTEXT |
| --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- |
| low (&lt;16GiB) | off | 2m | q4_0 | 6GiB | 8k | | low (&lt;16GiB) | off | 2m | q4_0 | 6GiB | 8k |
| mid (1623) | on* | 15m | q8_0 | 10GiB | 16k | | mid (1623) | on* | 5m | q8_0 | 10GiB | 16k |
| high (2447) | on* | 15m | q8_0 | 14GiB | 16k | | high (2447) | on* | 5m | q8_0 | 14GiB | 16k |
| ultra (≥48) | on* | 30m | q8_0 | 20GiB | 32k | | ultra (≥48) | on* | 30m | q8_0 | 20GiB | 32k |
\*Flash на Ampere+ (compute ≥ 8.0). `NUM_PARALLEL=2`, `MAX_LOADED_MODELS=2` (chat VL + memory embed). Memory models use `use: memory` and a CPU Modelfile (`num_gpu 0`) so embed does not steal VRAM from the chat model. Ollama default `num_ctx` is 4096; we set `OLLAMA_CONTEXT_LENGTH` so Assistent + vision fits. Env: `/mnt/swarm_data/.gpu-rent-ollama.env`. \*Flash на Ampere+ (compute ≥ 8.0). `NUM_PARALLEL=2`, `MAX_LOADED_MODELS=2` (chat VL + memory embed). Memory models use `use: memory` and a CPU Modelfile (`num_gpu 0`) so embed does not steal VRAM from the chat model. Ollama default `num_ctx` is 4096; we set `OLLAMA_CONTEXT_LENGTH` so Assistent + vision fits. Mid/high keep-alive is **5m** because Assistent parks the chat model before Generate (`AssistentParkLlm`) and warms after. Env: `/mnt/swarm_data/.gpu-rent-ollama.env`.
Personas: laptop `assistent-personas.yaml` → on `up` / `seed-personas` overlay folders under `/mnt/swarm_data/Assistent/personas/<id>/` (+ `_base/assistant.json` with `default_persona` / `num_ctx`). See [extensions.md](extensions.md).
`ollama-models.yaml` entries: `ollama-models.yaml` entries:
```yaml ```yaml
- name: huihui_ai/qwen2.5-vl-abliterated:7b - name: huihui_ai/qwen3-vl-abliterated:8b-instruct
use: chat use: chat
default: true default: true
- name: huihui_ai/qwen2.5-vl-abliterated:7b
use: chat
- name: nomic-embed-text - name: nomic-embed-text
use: memory use: memory
``` ```
@@ -129,7 +135,8 @@ Roles are written to `/mnt/swarm_data/Assistent/ollama-roles.json` for the Assis
Busy (не гасить GPU): Busy (не гасить GPU):
- `ollama pull` (маркер младше ~45 мин; старше сбрасывается); - очередь / loading SwarmUI (Generate) — даже если Ollama `/api/ps` пуст после ParkLlm;
- `ollama pull` (маркер младше ~3 ч; refresh во время pull);
- загруженная модель в Ollama. - загруженная модель в Ollama.
Ошибка установки LLM на `up`**fail** (не тихий лог). Ошибка установки LLM на `up`**fail** (не тихий лог).
+7 -2
View File
@@ -2,13 +2,18 @@
# name = exact tag for `ollama pull`. # name = exact tag for `ollama pull`.
# use: chat — Assistent header select; use: memory — settings memory model. # use: chat — Assistent header select; use: memory — settings memory model.
# Requirement: uncensored (abliterated) + Russian for chat. Empty models: [] → no pull. # Requirement: uncensored (abliterated) + Russian for chat. Empty models: [] → no pull.
# Qwen3-VL tags need Ollama ≥ 0.12.7. Pin :8b-instruct — :latest is Thinking.
models: models:
# Recommended (~6GB): vision + RU/EN, abliterated # Recommended (~6.1GB): Qwen3-VL Instruct, vision + RU/EN, abliterated
- name: huihui_ai/qwen2.5-vl-abliterated:7b - name: huihui_ai/qwen3-vl-abliterated:8b-instruct
use: chat use: chat
default: true default: true
# Previous default — still in Assistent chat select
- name: huihui_ai/qwen2.5-vl-abliterated:7b
use: chat
# Vector memory (CPU via num_gpu 0 Modelfile after pull) — not shown in chat select # Vector memory (CPU via num_gpu 0 Modelfile after pull) — not shown in chat select
- name: nomic-embed-text - name: nomic-embed-text
use: memory use: memory
+5
View File
@@ -40,6 +40,11 @@ def collect_access_links(cfg: Config, *, tunneled: bool) -> list[AccessLink]:
links.extend( links.extend(
[ [
AccessLink("SwarmUI UI", base, "браузер"), AccessLink("SwarmUI UI", base, "браузер"),
AccessLink(
"Assistent",
base,
"вкладка Assistent в SwarmUI",
),
AccessLink("SwarmUI API", f"{base}/API/", "HTTP JSON"), AccessLink("SwarmUI API", f"{base}/API/", "HTTP JSON"),
AccessLink("SwarmUI MCP", f"{base}/mcp", "Cursor mcp.json"), AccessLink("SwarmUI MCP", f"{base}/mcp", "Cursor mcp.json"),
] ]
+22
View File
@@ -324,6 +324,16 @@ def status() -> None:
table.add_row("idle-killer", f"{killer_line} · arm ok (сессия)") table.add_row("idle-killer", f"{killer_line} · arm ok (сессия)")
else: else:
table.add_row("idle-killer", killer_line) table.add_row("idle-killer", killer_line)
try:
from gpu_rent.provision import count_wanted_models_on_vm
wanted_n = count_wanted_models_on_vm(cfg, state.floating_ip)
table.add_row(
"Assistent wanted",
f"{wanted_n} в очереди" if wanted_n else "пусто",
)
except Exception:
table.add_row("Assistent wanted", "")
except GpuRentError as exc: except GpuRentError as exc:
table.add_row("диск used/free", f"SSH: {exc}") table.add_row("диск used/free", f"SSH: {exc}")
table.add_row("idle-killer", "нет SSH") table.add_row("idle-killer", "нет SSH")
@@ -932,6 +942,18 @@ def seed_extensions_cmd() -> None:
_die(exc) _die(exc)
@app.command("seed-personas")
def seed_personas_cmd() -> None:
"""Push assistent-personas.yaml → VM Assistent overlay (без полного up)."""
try:
from gpu_rent.provision import seed_assistent_personas
cfg, host = _live()
seed_assistent_personas(cfg, host, log)
except GpuRentError as exc:
_die(exc)
capture_app = typer.Typer( capture_app = typer.Typer(
help=( help=(
"Снять с VM инвентарь → локальные манифесты (только ссылки, без весов). " "Снять с VM инвентарь → локальные манифесты (только ссылки, без весов). "
+63 -1
View File
@@ -15,7 +15,7 @@ from gpu_rent.inventory import (
pick_volume_type, pick_volume_type,
rank_flavors, rank_flavors,
) )
from gpu_rent.manifests import parse_extensions, parse_models from gpu_rent.manifests import parse_extensions, parse_models, repo_dirname, repo_matches_runtime
from gpu_rent.os_client import ( from gpu_rent.os_client import (
compute_quotas, compute_quotas,
connect, connect,
@@ -243,10 +243,72 @@ def run_doctor() -> list[Check]:
_civitai(cfg, checks) _civitai(cfg, checks)
_huggingface(cfg, checks) _huggingface(cfg, checks)
_local_manifests(cfg, checks) _local_manifests(cfg, checks)
_llm_assistent(cfg, checks)
_local_folders(cfg, checks) _local_folders(cfg, checks)
return checks return checks
def _llm_assistent(cfg: Config, checks: list[Check]) -> None:
"""When Ollama is on, warn if Assistent stack pieces are missing locally."""
from gpu_rent.llm_runtime import normalize_runtime, parse_ollama_models
from gpu_rent.paths import ollama_models_manifest_path
runtime = normalize_runtime(getattr(cfg, "llm_runtime", "none"))
if runtime != "ollama":
checks.append(Check("LLM / Assistent", True, False, f"LLM_RUNTIME={runtime}"))
return
ollama_path = Path(
getattr(cfg, "ollama_models_manifest", None) or ollama_models_manifest_path()
)
if not ollama_path.is_file():
checks.append(
Check(
"ollama-models.yaml",
False,
True,
f"нет {ollama_path} — Assistent будет пустой. gpu-rent setup / скопируй example",
)
)
else:
try:
entries = parse_ollama_models(ollama_path)
chat = [e for e in entries if getattr(e, "use", "chat") != "memory"]
checks.append(
Check(
"ollama-models.yaml",
True,
True,
f"{len(entries)} тег(ов), chat={len(chat)}",
)
)
except Exception as exc:
checks.append(Check("ollama-models.yaml", False, True, str(exc)))
try:
repos = parse_extensions(cfg.extensions_manifest)
except ConfigError as exc:
checks.append(Check("Assistent ext", False, True, str(exc)))
return
has_assistent = any(
"assistent" in repo_dirname(r).lower()
or "assistent" in (r.url or "").lower()
for r in repos
if repo_matches_runtime(r, runtime)
)
if has_assistent:
checks.append(Check("Assistent ext", True, False, "swarm-assistent в extensions.yaml"))
else:
checks.append(
Check(
"Assistent ext",
True,
False,
"нет swarm-assistent (requires:ollama) в extensions.yaml — чат не поставится",
)
)
def _civitai(cfg: Config, checks: list[Check]) -> None: def _civitai(cfg: Config, checks: list[Check]) -> None:
if not cfg.civitai_api_token: if not cfg.civitai_api_token:
checks.append( checks.append(
+10 -9
View File
@@ -21,10 +21,13 @@ VALID_RUNTIMES = frozenset({"none", "ollama"})
MEMORY_EMBED_MODEL = "nomic-embed-text" MEMORY_EMBED_MODEL = "nomic-embed-text"
OLLAMA_PRESETS: dict[str, list[str]] = { OLLAMA_PRESETS: dict[str, list[str]] = {
# Requirement: uncensored (abliterated) + solid Russian. Qwen2.5 family. # Requirement: uncensored (abliterated) + solid Russian. Vision for prompt help.
# Vision tags preferred for SwarmUI prompt help with images. # Pin :8b-instruct — :latest on this library is Thinking, not chat.
# Memory embed (nomic) is appended separately with use: memory. # Qwen3-VL needs Ollama ≥ 0.12.7. Memory embed (nomic) appended with use: memory.
"recommended": ["huihui_ai/qwen2.5-vl-abliterated:7b"], # ~6GB "recommended": [
"huihui_ai/qwen3-vl-abliterated:8b-instruct", # ~6.1GB, default
"huihui_ai/qwen2.5-vl-abliterated:7b", # ~6GB, second chat tag
],
"light": ["huihui_ai/qwen2.5-vl-abliterated:3b"], # ~3GB "light": ["huihui_ai/qwen2.5-vl-abliterated:3b"], # ~3GB
"text": ["huihui_ai/qwen2.5-abliterate:7b"], # ~5GB, no vision "text": ["huihui_ai/qwen2.5-abliterate:7b"], # ~5GB, no vision
"big": ["huihui_ai/qwen2.5-vl-abliterated:32b"], # ~21GB "big": ["huihui_ai/qwen2.5-vl-abliterated:32b"], # ~21GB
@@ -32,7 +35,7 @@ OLLAMA_PRESETS: dict[str, list[str]] = {
} }
OLLAMA_PRESET_LABELS: dict[str, str] = { OLLAMA_PRESET_LABELS: dict[str, str] = {
"recommended": "VL 7B abliterate — RU + картинки (~6GB)", "recommended": "VL 8B Instruct abliterate — RU + картинки (~6GB)",
"light": "VL 3B abliterate — RU + картинки, мало VRAM (~3GB)", "light": "VL 3B abliterate — RU + картинки, мало VRAM (~3GB)",
"text": "7B abliterate text — RU, без vision (~5GB)", "text": "7B abliterate text — RU, без vision (~5GB)",
"big": "VL 32B abliterate — RU + картинки (~21GB)", "big": "VL 32B abliterate — RU + картинки (~21GB)",
@@ -40,10 +43,8 @@ OLLAMA_PRESET_LABELS: dict[str, str] = {
"keep": "не менять ollama-models.yaml", "keep": "не менять ollama-models.yaml",
} }
# Deprecated text blob — prefer menu helpers below. # Deprecated: use ollama_preset_menu / OLLAMA_PRESET_LABELS
PRESET_HELP = "\n".join( PRESET_HELP = "" # kept empty; menus use OLLAMA_PRESET_LABELS
f"{k}{v}" for k, v in OLLAMA_PRESET_LABELS.items() if k != "keep"
)
LLM_RUNTIME_LABELS: dict[str, str] = { LLM_RUNTIME_LABELS: dict[str, str] = {
"none": "только SwarmUI", "none": "только SwarmUI",
+4 -4
View File
@@ -77,16 +77,16 @@ def ollama_tune_for(info: GpuInfo) -> OllamaTune:
note = "ultra: flash+q8 KV, 20GiB reserved for Swarm, ctx 32k, keep 30m" note = "ultra: flash+q8 KV, 20GiB reserved for Swarm, ctx 32k, keep 30m"
elif info.tier == TIER_HIGH: elif info.tier == TIER_HIGH:
overhead = 14 * 1024**3 overhead = 14 * 1024**3
keep = "15m" keep = "5m" # Assistent parks LLM before Generate; short keep is enough
kv = "q8_0" kv = "q8_0"
ctx = 16384 ctx = 16384
note = "high: flash+q8 KV, 14GiB reserved for Swarm, ctx 16k, keep 15m" note = "high: flash+q8 KV, 14GiB reserved for Swarm, ctx 16k, keep 5m"
elif info.tier == TIER_MID: elif info.tier == TIER_MID:
overhead = 10 * 1024**3 overhead = 10 * 1024**3
keep = "15m" keep = "5m"
kv = "q8_0" kv = "q8_0"
ctx = 16384 ctx = 16384
note = "mid: flash+q8 KV, 10GiB reserved for Swarm, ctx 16k, keep 15m" note = "mid: flash+q8 KV, 10GiB reserved for Swarm, ctx 16k, keep 5m"
else: else:
overhead = 6 * 1024**3 overhead = 6 * 1024**3
keep = "2m" keep = "2m"
+133 -12
View File
@@ -710,8 +710,55 @@ def seed_swarmui_api_keys(cfg: Config, host: str, log: Log) -> None:
run_ssh(cfg, host, "rm -f /tmp/gpu-rent-swarm-api-keys.json", check=False) run_ssh(cfg, host, "rm -f /tmp/gpu-rent-swarm-api-keys.json", check=False)
def _safe_persona_id(raw: object) -> str | None:
s = str(raw or "").strip().replace("\\", "/")
if not s or ".." in s or "/" in s:
return None
import re
if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_\-]{0,63}", s):
return None
return s
def _overlay_num_ctx(cfg: Config, host: str) -> int | None:
"""Prefer GPU-tier context from .gpu-rent-gpu.json when present."""
raw = run_ssh(
cfg,
host,
f"test -f {DATA}/.gpu-rent-gpu.json && cat {DATA}/.gpu-rent-gpu.json || true",
check=False,
timeout=15,
).strip()
if not raw:
return None
try:
probe = json.loads(raw)
vram = int(probe.get("vram_mib") or probe.get("memory_total_mib") or 0)
except (json.JSONDecodeError, TypeError, ValueError):
return None
if vram <= 0:
return None
from gpu_rent.perf_tiers import GpuInfo, ollama_tune_for, tier_for_vram_mib
info = GpuInfo(
name=str(probe.get("name") or "gpu"),
vram_mib=vram,
compute_cap=str(probe.get("compute_cap") or "0.0"),
uuid=str(probe.get("uuid") or ""),
tier=tier_for_vram_mib(vram),
)
return ollama_tune_for(info).context_length
def seed_assistent_personas(cfg: Config, host: str, log: Log) -> None: def seed_assistent_personas(cfg: Config, host: str, log: Log) -> None:
"""Push local assistent-personas.yaml → VM Assistent/personas.yaml + personas.json.""" """Push local assistent-personas.yaml → VM Assistent/personas/<id>/ overlay.
Laptop yaml stays the editor; on VM we write persona.json + extra.md and
overlay assistant.json (default_persona, optional num_ctx). Does not clobber
voice/likes/dislikes/rules already on disk. No longer writes legacy
personas.json / personas.yaml dumps.
"""
from gpu_rent.paths import assistent_personas_example_path, assistent_personas_manifest_path from gpu_rent.paths import assistent_personas_example_path, assistent_personas_manifest_path
local = Path( local = Path(
@@ -728,24 +775,96 @@ def seed_assistent_personas(cfg: Config, host: str, log: Log) -> None:
if not text.strip(): if not text.strip():
log("assistent-personas: пустой файл — skip") log("assistent-personas: пустой файл — skip")
return return
remote_dir = f"{DATA}/Assistent"
remote_yaml = f"{remote_dir}/personas.yaml"
remote_json = f"{remote_dir}/personas.json"
run_ssh(cfg, host, f"mkdir -p {shlex.quote(remote_dir)}", check=False)
put_text(cfg, host, remote_yaml, text if text.endswith("\n") else text + "\n")
try: try:
import yaml import yaml
data = yaml.safe_load(text) or {} data = yaml.safe_load(text) or {}
except Exception as exc:
log(f"assistent-personas: yaml parse — {exc}")
return
if not isinstance(data, dict):
log("assistent-personas: корень должен быть mapping — skip")
return
remote_dir = f"{DATA}/Assistent"
run_ssh(cfg, host, f"mkdir -p {shlex.quote(remote_dir)}/personas {shlex.quote(remote_dir)}/_base", check=False)
default_id = _safe_persona_id(data.get("default")) or "neutral"
personas = data.get("personas") or []
written = 0
if isinstance(personas, list):
for row in personas:
if not isinstance(row, dict):
continue
pid = _safe_persona_id(row.get("id"))
if not pid:
continue
title = str(row.get("title") or pid).strip() or pid
prompt = str(row.get("prompt") or "").strip()
pdir = f"{remote_dir}/personas/{pid}"
run_ssh(cfg, host, f"mkdir -p {shlex.quote(pdir)}", check=False)
meta = {"title": title, "tagline": title, "accent": "#8b949e"}
put_text( put_text(
cfg, cfg,
host, host,
remote_json, f"{pdir}/persona.json",
json.dumps(data, ensure_ascii=False, indent=2) + "\n", json.dumps(meta, ensure_ascii=False, indent=2) + "\n",
) )
except Exception as exc: if prompt:
log(f"assistent-personas: json convert — {exc}") put_text(
log(f"assistent-personas → {remote_yaml}") cfg,
host,
f"{pdir}/extra.md",
prompt if prompt.endswith("\n") else prompt + "\n",
)
written += 1
assistant_overlay: dict = {"default_persona": default_id}
num_ctx = _overlay_num_ctx(cfg, host)
if num_ctx:
assistant_overlay["num_ctx"] = num_ctx
put_text(
cfg,
host,
f"{remote_dir}/_base/assistant.json",
json.dumps(assistant_overlay, ensure_ascii=False, indent=2) + "\n",
)
# Drop legacy dumps so Assistent does not double-inject prompts.
run_ssh(
cfg,
host,
f"rm -f {shlex.quote(remote_dir + '/personas.json')} "
f"{shlex.quote(remote_dir + '/personas.yaml')}",
check=False,
)
ctx_note = f", num_ctx={num_ctx}" if num_ctx else ""
log(f"assistent-personas → overlay personas/{written} (default={default_id}{ctx_note})")
def count_wanted_models_on_vm(cfg: Config, host: str) -> int:
"""Count entries in Assistent wanted queue on the VM (best-effort)."""
remote = f"{DATA}/.gpu-rent-wanted-models.yaml"
raw = run_ssh(
cfg,
host,
f"test -f {shlex.quote(remote)} && cat {shlex.quote(remote)} || true",
check=False,
timeout=20,
).strip()
if not raw:
return 0
try:
import yaml
data = yaml.safe_load(raw) or {}
except Exception:
return sum(1 for line in raw.splitlines() if line.strip().startswith("- url:"))
n = 0
if isinstance(data, dict):
for rows in data.values():
if isinstance(rows, list):
n += sum(1 for r in rows if isinstance(r, dict) and r.get("url"))
return n
def merge_wanted_models_from_vm(cfg: Config, host: str, log: Log) -> int: def merge_wanted_models_from_vm(cfg: Config, host: str, log: Log) -> int:
@@ -1072,6 +1191,8 @@ def provision_llm(cfg: Config, host: str, log: Log) -> None:
) )
# Drop LLM units that should not hold VRAM for this runtime. # Drop LLM units that should not hold VRAM for this runtime.
# Tombstone: still stop gpu-rent-llamacpp if an old disk left that unit behind
# (llamacpp runtime was removed; do not reinstall it).
if runtime == "none": if runtime == "none":
log("LLM: none — останавливаю gpu-rent-ollama / gpu-rent-llamacpp если были") log("LLM: none — останавливаю gpu-rent-ollama / gpu-rent-llamacpp если были")
_stop_units("gpu-rent-ollama", "gpu-rent-llamacpp") _stop_units("gpu-rent-ollama", "gpu-rent-llamacpp")
@@ -1083,7 +1204,7 @@ def provision_llm(cfg: Config, host: str, log: Log) -> None:
return return
still: list[str] = [] still: list[str] = []
if runtime == "ollama": if runtime == "ollama":
_stop_units("gpu-rent-llamacpp") _stop_units("gpu-rent-llamacpp") # tombstone: disable leftover llamacpp unit
log("LLM: ставим/запускаем Ollama") log("LLM: ставим/запускаем Ollama")
run_script_sudo( run_script_sudo(
cfg, cfg,
+5 -1
View File
@@ -129,7 +129,11 @@ def swarm_busy(swarm_url: str, timeout: float = 8.0) -> tuple[bool, str]:
def llm_busy(timeout: float = 3.0) -> tuple[bool, str]: def llm_busy(timeout: float = 3.0) -> tuple[bool, str]:
"""Ollama pull / loaded models count as busy.""" """Ollama pull / loaded models count as busy.
Empty /api/ps after AssistentParkLlm is NOT busy by itself — Swarm queue
is checked separately in main() and keeps the GPU alive during Generate.
"""
pull_marker = DATA / ".gpu-rent-ollama-pulling" pull_marker = DATA / ".gpu-rent-ollama-pulling"
if pull_marker.is_file(): if pull_marker.is_file():
try: try:
+2 -2
View File
@@ -47,9 +47,9 @@ except ValueError:
if gib >= 48: if gib >= 48:
tier, overhead, keep, kv, flash, ctx = "ultra", 20 * 1024**3, "30m", "q8_0", True, 32768 tier, overhead, keep, kv, flash, ctx = "ultra", 20 * 1024**3, "30m", "q8_0", True, 32768
elif gib >= 24: elif gib >= 24:
tier, overhead, keep, kv, flash, ctx = "high", 14 * 1024**3, "15m", "q8_0", True, 16384 tier, overhead, keep, kv, flash, ctx = "high", 14 * 1024**3, "5m", "q8_0", True, 16384
elif gib >= 16: elif gib >= 16:
tier, overhead, keep, kv, flash, ctx = "mid", 10 * 1024**3, "15m", "q8_0", True, 16384 tier, overhead, keep, kv, flash, ctx = "mid", 10 * 1024**3, "5m", "q8_0", True, 16384
else: else:
tier, overhead, keep, kv, flash, ctx = "low", 6 * 1024**3, "2m", "q4_0", False, 8192 tier, overhead, keep, kv, flash, ctx = "low", 6 * 1024**3, "2m", "q4_0", False, 8192
flash = bool(flash and (ampere or "A100" in name.upper() or "H100" in name.upper() or gib >= 16)) flash = bool(flash and (ampere or "A100" in name.upper() or "H100" in name.upper() or gib >= 16))
+3 -2
View File
@@ -16,9 +16,10 @@ def test_collect_links_swarm_and_ollama(monkeypatch):
links = collect_access_links(_Cfg(), tunneled=True) links = collect_access_links(_Cfg(), tunneled=True)
labels = [x.label for x in links] labels = [x.label for x in links]
assert "SwarmUI UI" in labels assert "SwarmUI UI" in labels
assert "Assistent" in labels
assert "SwarmUI MCP" in labels assert "SwarmUI MCP" in labels
assert "Ollama API" in labels assert "Ollama API" in labels
assert any("17811" in x.url for x in links) assert any(x.label == "Assistent" and "вкладка" in x.note for x in links)
def test_collect_links_no_tunnel(): def test_collect_links_no_tunnel():
@@ -57,7 +58,7 @@ def test_access_panel_hides_stale_llm_error_when_ollama_ok(monkeypatch):
{ {
"name": "ollama", "name": "ollama",
"ok": True, "ok": True,
"detail": "1 models (huihui_ai/qwen2.5-vl-abliterated:7b)", "detail": "1 models (huihui_ai/qwen3-vl-abliterated:8b-instruct)",
} }
], ],
} }
+81
View File
@@ -0,0 +1,81 @@
"""Unit tests for assistent personas overlay seed helpers."""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from gpu_rent.provision import _safe_persona_id, count_wanted_models_on_vm, seed_assistent_personas
def test_safe_persona_id():
assert _safe_persona_id("neutral") == "neutral"
assert _safe_persona_id("lewd_v2") == "lewd_v2"
assert _safe_persona_id("../x") is None
assert _safe_persona_id("a/b") is None
assert _safe_persona_id("") is None
def test_seed_assistent_personas_writes_overlay(tmp_path: Path, monkeypatch):
yaml_path = tmp_path / "assistent-personas.yaml"
yaml_path.write_text(
"default: cinema\n"
"personas:\n"
" - id: cinema\n"
" title: Кино\n"
" prompt: |\n"
" You are a DP.\n"
" - id: neutral\n"
" title: Нейтральный\n"
" prompt: Calm.\n",
encoding="utf-8",
)
puts: dict[str, str] = {}
ssh_cmds: list[str] = []
def fake_put(cfg, host, remote, text, mode=0o644):
puts[remote] = text
def fake_ssh(cfg, host, cmd, check=False, timeout=60):
ssh_cmds.append(cmd)
if ".gpu-rent-gpu.json" in cmd:
return json.dumps({"vram_mib": 24576, "name": "4090", "compute_cap": "8.9"})
return ""
monkeypatch.setattr("gpu_rent.provision.put_text", fake_put)
monkeypatch.setattr("gpu_rent.provision.run_ssh", fake_ssh)
monkeypatch.setattr(
"gpu_rent.paths.assistent_personas_manifest_path", lambda: yaml_path
)
monkeypatch.setattr(
"gpu_rent.paths.assistent_personas_example_path", lambda: tmp_path / "missing"
)
cfg = MagicMock()
cfg.assistent_personas_manifest = str(yaml_path)
logs: list[str] = []
seed_assistent_personas(cfg, "1.2.3.4", logs.append)
assert any("personas/cinema/persona.json" in p for p in puts)
assert any("personas/cinema/extra.md" in p for p in puts)
assert "/mnt/swarm_data/Assistent/_base/assistant.json" in puts
asst = json.loads(puts["/mnt/swarm_data/Assistent/_base/assistant.json"])
assert asst["default_persona"] == "cinema"
assert asst["num_ctx"] == 16384
assert any("rm -f" in c and "personas.json" in c for c in ssh_cmds)
assert not any(p.endswith("personas.yaml") for p in puts)
assert not any(p.endswith("personas.json") for p in puts)
assert any("overlay personas/2" in m for m in logs)
def test_count_wanted_models(monkeypatch):
raw = "lora:\n - url: https://civitai.red/x?modelVersionId=1\n title: A\n"
monkeypatch.setattr(
"gpu_rent.provision.run_ssh",
lambda *a, **k: raw,
)
assert count_wanted_models_on_vm(MagicMock(), "h") == 1
+61
View File
@@ -0,0 +1,61 @@
"""Doctor LLM / Assistent local checks."""
from __future__ import annotations
from pathlib import Path
from gpu_rent.doctor import Check, _llm_assistent
class _Cfg:
llm_runtime = "ollama"
ollama_models_manifest = None
extensions_manifest = None
def test_llm_assistent_warns_without_ollama_yaml(tmp_path: Path, monkeypatch):
cfg = _Cfg()
cfg.ollama_models_manifest = tmp_path / "missing-ollama.yaml"
cfg.extensions_manifest = tmp_path / "extensions.yaml"
cfg.extensions_manifest.write_text(
"swarmui:\n"
" - url: https://gitea.example/swarm-assistent.git\n"
" dir: swarm-assistent\n"
" requires: ollama\n",
encoding="utf-8",
)
monkeypatch.setattr(
"gpu_rent.paths.ollama_models_manifest_path",
lambda: cfg.ollama_models_manifest,
)
checks: list[Check] = []
_llm_assistent(cfg, checks)
names = {c.name: c for c in checks}
assert names["ollama-models.yaml"].ok is False
assert names["ollama-models.yaml"].blocking is True
assert names["Assistent ext"].ok is True
def test_llm_assistent_ok_with_manifests(tmp_path: Path, monkeypatch):
ollama = tmp_path / "ollama-models.yaml"
ollama.write_text(
"models:\n - name: foo:7b\n use: chat\n default: true\n",
encoding="utf-8",
)
ext = tmp_path / "extensions.yaml"
ext.write_text(
"swarmui:\n"
" - url: https://gitea.example/swarm-assistent.git\n"
" dir: swarm-assistent\n"
" requires: ollama\n",
encoding="utf-8",
)
cfg = _Cfg()
cfg.ollama_models_manifest = ollama
cfg.extensions_manifest = ext
cfg.llm_runtime = "ollama"
checks: list[Check] = []
_llm_assistent(cfg, checks)
by = {c.name: c for c in checks}
assert by["ollama-models.yaml"].ok
assert "swarm-assistent" in by["Assistent ext"].detail
+3 -1
View File
@@ -56,7 +56,9 @@ def test_write_preset(tmp_path: Path):
path = tmp_path / "out.yaml" path = tmp_path / "out.yaml"
write_ollama_models_preset(path, "recommended") write_ollama_models_preset(path, "recommended")
entries = parse_ollama_models(path) entries = parse_ollama_models(path)
assert entries[0].name == "huihui_ai/qwen2.5-vl-abliterated:7b" assert entries[0].name == "huihui_ai/qwen3-vl-abliterated:8b-instruct"
assert entries[0].default is True
assert entries[1].name == "huihui_ai/qwen2.5-vl-abliterated:7b"
write_ollama_models_preset(path, "big") write_ollama_models_preset(path, "big")
assert parse_ollama_models(path)[0].name == "huihui_ai/qwen2.5-vl-abliterated:32b" assert parse_ollama_models(path)[0].name == "huihui_ai/qwen2.5-vl-abliterated:32b"
write_ollama_models_preset(path, "text") write_ollama_models_preset(path, "text")
+2 -2
View File
@@ -43,10 +43,10 @@ def test_warmup_skips_when_ps_has_model(monkeypatch):
def fake_urlopen(req, timeout=None): def fake_urlopen(req, timeout=None):
url = getattr(req, "full_url", str(req)) url = getattr(req, "full_url", str(req))
assert "/api/ps" in url assert "/api/ps" in url
return FakeResp('{"models":[{"name":"huihui_ai/qwen2.5-vl-abliterated:7b"}]}') return FakeResp('{"models":[{"name":"huihui_ai/qwen3-vl-abliterated:8b-instruct"}]}')
monkeypatch.setattr(_mod.urllib.request, "urlopen", fake_urlopen) monkeypatch.setattr(_mod.urllib.request, "urlopen", fake_urlopen)
msg = _mod.warmup("huihui_ai/qwen2.5-vl-abliterated:7b", keep_alive="15m", num_ctx=16384, timeout=5) msg = _mod.warmup("huihui_ai/qwen3-vl-abliterated:8b-instruct", keep_alive="15m", num_ctx=16384, timeout=5)
assert "skip" in msg assert "skip" in msg
assert "VRAM" in msg assert "VRAM" in msg
+2 -2
View File
@@ -82,6 +82,6 @@ def test_ollama_mid_4090_context_16k():
) )
tune = ollama_tune_for(info) tune = ollama_tune_for(info)
assert tune.context_length == 16384 assert tune.context_length == 16384
assert tune.keep_alive == "15m" assert tune.keep_alive == "5m"
assert "OLLAMA_CONTEXT_LENGTH=16384" in "\n".join(ollama_env_lines(tune)) assert "OLLAMA_CONTEXT_LENGTH=16384" in "\n".join(ollama_env_lines(tune))
assert "OLLAMA_KEEP_ALIVE=15m" in "\n".join(ollama_env_lines(tune)) assert "OLLAMA_KEEP_ALIVE=5m" in "\n".join(ollama_env_lines(tune))
+1 -1
View File
@@ -86,7 +86,7 @@ def test_install_ollama_skips_restart_when_unit_unchanged():
text = files("gpu_rent.remote").joinpath("install_ollama.sh").read_text(encoding="utf-8") text = files("gpu_rent.remote").joinpath("install_ollama.sh").read_text(encoding="utf-8")
assert "cmp -s" in text assert "cmp -s" in text
assert "skip restart" in text assert "skip restart" in text
assert '"mid", 10 * 1024**3, "15m"' in text assert '"mid", 10 * 1024**3, "5m"' in text
def test_provision_llm_skips_on_api_tags_not_cli_list(): def test_provision_llm_skips_on_api_tags_not_cli_list():