Seed Assistent personas as overlay folders and tighten Ollama/Assistent glue.

gpu-rent now writes personas/<id>/ on the VM (not legacy personas.json), adds seed-personas/doctor checks, and shortens mid/high keep-alive now that Assistent parks the LLM before Generate.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Leonid Pershin
2026-08-22 01:00:14 +03:00
co-authored by Cursor
parent 789fa26918
commit 03ba4cb6ed
22 changed files with 430 additions and 53 deletions
+4 -2
View File
@@ -1,6 +1,8 @@
# Copy to assistent-personas.yaml (never commit the working copy).
# Synced to the VM on each up → /mnt/swarm_data/Assistent/personas.yaml + personas.json
# Assistent reads personas.json (overlay). Yaml is the laptop source for gpu-rent seed.
# Synced to the VM on each up / seed-personas →
# /mnt/swarm_data/Assistent/personas/<id>/{persona.json,extra.md}
# /mnt/swarm_data/Assistent/_base/assistant.json (default_persona, num_ctx)
# Yaml is the laptop source for gpu-rent seed; Assistent reads overlay folders.
default: neutral
+2
View File
@@ -18,6 +18,8 @@
Файл маленький (danbooru.csv ≈ 34 MB). Killer из‑за него не откладываем, но на первом bootstrap качаем **до** старта UI, вместе с extensions.
Swarm Assistent индексирует тот же csv в FTS-таблицу `tags` внутри `Assistent/memory/assistent.sqlite` (без эмбедов). Первый чат после смены sha занимает несколько секунд; дальше `lookup_tags` ищет канон/алиасы. В `memory_hits` csv не попадает.
## Откуда файл
Дефолт — то, на что ссылается сама документация SwarmUI:
+1
View File
@@ -75,6 +75,7 @@ gpu-rent up --yes --ollama
| `gpu-rent ssh` | Оболочка на VM |
| `gpu-rent seed-models` | Докачать новые строки Civitai-манифеста на живой диск |
| `gpu-rent seed-extensions` | Доклонировать/обновить git-репы; restart swarmui |
| `gpu-rent seed-personas` | Пуш `assistent-personas.yaml` → overlay на VM (без полного up) |
| `gpu-rent push` / `push-models` | Локальные деревья → VM |
| `gpu-rent pull-output` | VM `Output/``./Output` |
| `gpu-rent capture` / `capture all` | Инвентарь VM → merge **ссылок** в `models.yaml` + `extensions.yaml` (веса не качать) |
+1 -1
View File
@@ -10,7 +10,7 @@
| Civitai хост | Дефолт API `civitai.red` (полный каталог). `.com` — SFW-витрина, NSFW с неё часто 404. Ссылки `.com`/`.red`/`.green` в манифесте принимаем. 404 → один retry на второй хост. Один токен на оба домена |
| Пул GPU | Перед `up`/`flavors` сканируем `SCAN_POOLS` (дефолт `ru-6,ru-7`) по compute URL. Сегменты `ru-6a/b/c` берём из Nova `availability_zones`, не отдельными пулами. Пресет размера `FLAVOR_SIZE_PRESET` (дефолт `cheap` = 1×GPU, min vCPU/RAM). Диск уже есть — AZ диска, не дешёвый чужой сегмент. Первый запуск — дешёвый SKU в сегменте, где есть карта из `FLAVOR_PREFERENCE`. `.env` автоматом не пишем |
| Манифест моделей | `<repo>/models.yaml`, типы: checkpoint / lora / vae / embedding / controlnet / upscaler. В git только `models.example.yaml` |
| Расширения | `<repo>/extensions.yaml`: git-репы `swarmui``src/Extensions`, `comfy` → DLNodes. Поле `requires: none\|ollama\|any-llm` фильтрует по `LLM_RUNTIME`. Пример: swarm-assistent с `requires: ollama` |
| Расширения | `<repo>/extensions.yaml`: git-репы `swarmui``src/Extensions`, `comfy` → DLNodes. Поле `requires: none\|ollama\|any-llm` фильтрует по `LLM_RUNTIME`. Пример: swarm-assistent с `requires: ollama`. Personas: `assistent-personas.yaml` → overlay `Assistent/personas/<id>/` на `up` / `seed-personas` |
| Autocomplete | До первого старта: скачать word-list в `Data/Autocompletions`, прописать `DefaultUser.AutoComplete.Source`. На каждом `up` сверить GitHub blob sha и обновить файл, если изменился. Дефолт: `tags/danbooru.csv` из a1111-sd-webui-tagcomplete (как в доке SwarmUI) |
| Доступ | Браузер на туннеле; MCP переключается на облако, пока оно живо; HTTP API SwarmUI через тот же туннель |
| Preemptible | По умолчанию всегда. Обычный сервер — только `--no-spot` |
+1 -1
View File
@@ -53,7 +53,7 @@ comfy:
| `ollama` | только при `LLM_RUNTIME=ollama` |
| `any-llm` | при `LLM_RUNTIME=ollama` |
Строки с несовпавшим `requires` пропускаются (лог), остальные ставятся как обычно. В `extensions.example.yaml` по умолчанию — **swarm-assistent** с `requires: ollama` (чат + доска Generate/Ref + persona/Cards, ветка `main`). На `up` сидится `assistent-personas.yaml``/mnt/swarm_data/Assistent/personas.yaml` **и** `personas.json` (Assistent читает только **json**).
Строки с несовпавшим `requires` пропускаются (лог), остальные ставятся как обычно. В `extensions.example.yaml` по умолчанию — **swarm-assistent** с `requires: ollama` (чат + доска Generate/Ref + persona/Cards, ветка `main`). На `up``gpu-rent seed-personas`) сидится `assistent-personas.yaml` overlay `/mnt/swarm_data/Assistent/personas/<id>/persona.json` + `extra.md`, плюс `_base/assistant.json` (`default_persona`, опционально `num_ctx` с GPU tier). Legacy `personas.json` больше не пишется.
В git репозитория gpu-rent не коммитить рабочий список с лишними приватными URL сверх примера.
+14 -7
View File
@@ -81,18 +81,20 @@ $env:OLLAMA_HOST = "http://127.0.0.1:17811"
После успешного тега — **warmup**: 1-token `POST /api/chat`, чтобы веса (~6 GB VL) легли в VRAM до первого сообщения в Assistent. То же при `gpu-rent tunnel`, если `/api/ps` пуст. Промах warmup — warning, GPU не гасим.
Qwen3-VL (`huihui_ai/qwen3-vl-abliterated:…`) требует **Ollama ≥ 0.12.7**. Тег `:8b-instruct` — чат Instruct; `:latest` у этой библиотеки — Thinking, не ставить дефолтом.
### Пресеты (меню)
| # | ключ | tag / смысл |
| --- | --- | --- |
| 1 | **recommended** | `huihui_ai/qwen2.5-vl-abliterated:7b` — RU + vision, ~6GB |
| 1 | **recommended** | `huihui_ai/qwen3-vl-abliterated:8b-instruct` — RU + vision, ~6.1GB; запасной `huihui_ai/qwen2.5-vl-abliterated:7b` |
| 2 | light | `…:3b` — то же, мало VRAM (~3GB) |
| 3 | text | `huihui_ai/qwen2.5-abliterate:7b` — RU, без vision (~5GB) |
| 4 | big | `huihui_ai/qwen2.5-vl-abliterated:32b` — RU + vision (~21GB) |
| 5 | empty | только runtime |
| — | keep | не трогать yaml |
Все пресеты (кроме `empty`) — **abliterated** Qwen2.5 с нормальным русским. Официальные censored-теги (`qwen2.5vl:…`) в меню нет.
Рекомендуемый пресет — **abliterated Qwen3-VL Instruct**; light / text / big пока Qwen2.5. Официальные censored-теги (`qwen2.5vl:…`) в меню нет.
`default: true` в yaml — preferred в логе; pull идёт по всему списку.
@@ -105,18 +107,22 @@ Unit `gpu-rent-ollama` читает `/mnt/swarm_data/.gpu-rent-gpu.json`:
| Tier (VRAM) | Flash Attn | KEEP_ALIVE | KV cache | GPU_OVERHEAD | CONTEXT |
| --- | --- | --- | --- | --- | --- |
| low (&lt;16GiB) | off | 2m | q4_0 | 6GiB | 8k |
| mid (1623) | on* | 15m | q8_0 | 10GiB | 16k |
| high (2447) | on* | 15m | q8_0 | 14GiB | 16k |
| mid (1623) | on* | 5m | q8_0 | 10GiB | 16k |
| high (2447) | on* | 5m | q8_0 | 14GiB | 16k |
| ultra (≥48) | on* | 30m | q8_0 | 20GiB | 32k |
\*Flash на Ampere+ (compute ≥ 8.0). `NUM_PARALLEL=2`, `MAX_LOADED_MODELS=2` (chat VL + memory embed). Memory models use `use: memory` and a CPU Modelfile (`num_gpu 0`) so embed does not steal VRAM from the chat model. Ollama default `num_ctx` is 4096; we set `OLLAMA_CONTEXT_LENGTH` so Assistent + vision fits. Env: `/mnt/swarm_data/.gpu-rent-ollama.env`.
\*Flash на Ampere+ (compute ≥ 8.0). `NUM_PARALLEL=2`, `MAX_LOADED_MODELS=2` (chat VL + memory embed). Memory models use `use: memory` and a CPU Modelfile (`num_gpu 0`) so embed does not steal VRAM from the chat model. Ollama default `num_ctx` is 4096; we set `OLLAMA_CONTEXT_LENGTH` so Assistent + vision fits. Mid/high keep-alive is **5m** because Assistent parks the chat model before Generate (`AssistentParkLlm`) and warms after. Env: `/mnt/swarm_data/.gpu-rent-ollama.env`.
Personas: laptop `assistent-personas.yaml` → on `up` / `seed-personas` overlay folders under `/mnt/swarm_data/Assistent/personas/<id>/` (+ `_base/assistant.json` with `default_persona` / `num_ctx`). See [extensions.md](extensions.md).
`ollama-models.yaml` entries:
```yaml
- name: huihui_ai/qwen2.5-vl-abliterated:7b
- name: huihui_ai/qwen3-vl-abliterated:8b-instruct
use: chat
default: true
- name: huihui_ai/qwen2.5-vl-abliterated:7b
use: chat
- name: nomic-embed-text
use: memory
```
@@ -129,7 +135,8 @@ Roles are written to `/mnt/swarm_data/Assistent/ollama-roles.json` for the Assis
Busy (не гасить GPU):
- `ollama pull` (маркер младше ~45 мин; старше сбрасывается);
- очередь / loading SwarmUI (Generate) — даже если Ollama `/api/ps` пуст после ParkLlm;
- `ollama pull` (маркер младше ~3 ч; refresh во время pull);
- загруженная модель в Ollama.
Ошибка установки LLM на `up`**fail** (не тихий лог).
+7 -2
View File
@@ -2,13 +2,18 @@
# name = exact tag for `ollama pull`.
# use: chat — Assistent header select; use: memory — settings memory model.
# Requirement: uncensored (abliterated) + Russian for chat. Empty models: [] → no pull.
# Qwen3-VL tags need Ollama ≥ 0.12.7. Pin :8b-instruct — :latest is Thinking.
models:
# Recommended (~6GB): vision + RU/EN, abliterated
- name: huihui_ai/qwen2.5-vl-abliterated:7b
# Recommended (~6.1GB): Qwen3-VL Instruct, vision + RU/EN, abliterated
- name: huihui_ai/qwen3-vl-abliterated:8b-instruct
use: chat
default: true
# Previous default — still in Assistent chat select
- name: huihui_ai/qwen2.5-vl-abliterated:7b
use: chat
# Vector memory (CPU via num_gpu 0 Modelfile after pull) — not shown in chat select
- name: nomic-embed-text
use: memory
+5
View File
@@ -40,6 +40,11 @@ def collect_access_links(cfg: Config, *, tunneled: bool) -> list[AccessLink]:
links.extend(
[
AccessLink("SwarmUI UI", base, "браузер"),
AccessLink(
"Assistent",
base,
"вкладка Assistent в SwarmUI",
),
AccessLink("SwarmUI API", f"{base}/API/", "HTTP JSON"),
AccessLink("SwarmUI MCP", f"{base}/mcp", "Cursor mcp.json"),
]
+22
View File
@@ -324,6 +324,16 @@ def status() -> None:
table.add_row("idle-killer", f"{killer_line} · arm ok (сессия)")
else:
table.add_row("idle-killer", killer_line)
try:
from gpu_rent.provision import count_wanted_models_on_vm
wanted_n = count_wanted_models_on_vm(cfg, state.floating_ip)
table.add_row(
"Assistent wanted",
f"{wanted_n} в очереди" if wanted_n else "пусто",
)
except Exception:
table.add_row("Assistent wanted", "")
except GpuRentError as exc:
table.add_row("диск used/free", f"SSH: {exc}")
table.add_row("idle-killer", "нет SSH")
@@ -932,6 +942,18 @@ def seed_extensions_cmd() -> None:
_die(exc)
@app.command("seed-personas")
def seed_personas_cmd() -> None:
"""Push assistent-personas.yaml → VM Assistent overlay (без полного up)."""
try:
from gpu_rent.provision import seed_assistent_personas
cfg, host = _live()
seed_assistent_personas(cfg, host, log)
except GpuRentError as exc:
_die(exc)
capture_app = typer.Typer(
help=(
"Снять с VM инвентарь → локальные манифесты (только ссылки, без весов). "
+63 -1
View File
@@ -15,7 +15,7 @@ from gpu_rent.inventory import (
pick_volume_type,
rank_flavors,
)
from gpu_rent.manifests import parse_extensions, parse_models
from gpu_rent.manifests import parse_extensions, parse_models, repo_dirname, repo_matches_runtime
from gpu_rent.os_client import (
compute_quotas,
connect,
@@ -243,10 +243,72 @@ def run_doctor() -> list[Check]:
_civitai(cfg, checks)
_huggingface(cfg, checks)
_local_manifests(cfg, checks)
_llm_assistent(cfg, checks)
_local_folders(cfg, checks)
return checks
def _llm_assistent(cfg: Config, checks: list[Check]) -> None:
"""When Ollama is on, warn if Assistent stack pieces are missing locally."""
from gpu_rent.llm_runtime import normalize_runtime, parse_ollama_models
from gpu_rent.paths import ollama_models_manifest_path
runtime = normalize_runtime(getattr(cfg, "llm_runtime", "none"))
if runtime != "ollama":
checks.append(Check("LLM / Assistent", True, False, f"LLM_RUNTIME={runtime}"))
return
ollama_path = Path(
getattr(cfg, "ollama_models_manifest", None) or ollama_models_manifest_path()
)
if not ollama_path.is_file():
checks.append(
Check(
"ollama-models.yaml",
False,
True,
f"нет {ollama_path} — Assistent будет пустой. gpu-rent setup / скопируй example",
)
)
else:
try:
entries = parse_ollama_models(ollama_path)
chat = [e for e in entries if getattr(e, "use", "chat") != "memory"]
checks.append(
Check(
"ollama-models.yaml",
True,
True,
f"{len(entries)} тег(ов), chat={len(chat)}",
)
)
except Exception as exc:
checks.append(Check("ollama-models.yaml", False, True, str(exc)))
try:
repos = parse_extensions(cfg.extensions_manifest)
except ConfigError as exc:
checks.append(Check("Assistent ext", False, True, str(exc)))
return
has_assistent = any(
"assistent" in repo_dirname(r).lower()
or "assistent" in (r.url or "").lower()
for r in repos
if repo_matches_runtime(r, runtime)
)
if has_assistent:
checks.append(Check("Assistent ext", True, False, "swarm-assistent в extensions.yaml"))
else:
checks.append(
Check(
"Assistent ext",
True,
False,
"нет swarm-assistent (requires:ollama) в extensions.yaml — чат не поставится",
)
)
def _civitai(cfg: Config, checks: list[Check]) -> None:
if not cfg.civitai_api_token:
checks.append(
+10 -9
View File
@@ -21,10 +21,13 @@ VALID_RUNTIMES = frozenset({"none", "ollama"})
MEMORY_EMBED_MODEL = "nomic-embed-text"
OLLAMA_PRESETS: dict[str, list[str]] = {
# Requirement: uncensored (abliterated) + solid Russian. Qwen2.5 family.
# Vision tags preferred for SwarmUI prompt help with images.
# Memory embed (nomic) is appended separately with use: memory.
"recommended": ["huihui_ai/qwen2.5-vl-abliterated:7b"], # ~6GB
# Requirement: uncensored (abliterated) + solid Russian. Vision for prompt help.
# Pin :8b-instruct — :latest on this library is Thinking, not chat.
# Qwen3-VL needs Ollama ≥ 0.12.7. Memory embed (nomic) appended with use: memory.
"recommended": [
"huihui_ai/qwen3-vl-abliterated:8b-instruct", # ~6.1GB, default
"huihui_ai/qwen2.5-vl-abliterated:7b", # ~6GB, second chat tag
],
"light": ["huihui_ai/qwen2.5-vl-abliterated:3b"], # ~3GB
"text": ["huihui_ai/qwen2.5-abliterate:7b"], # ~5GB, no vision
"big": ["huihui_ai/qwen2.5-vl-abliterated:32b"], # ~21GB
@@ -32,7 +35,7 @@ OLLAMA_PRESETS: dict[str, list[str]] = {
}
OLLAMA_PRESET_LABELS: dict[str, str] = {
"recommended": "VL 7B abliterate — RU + картинки (~6GB)",
"recommended": "VL 8B Instruct abliterate — RU + картинки (~6GB)",
"light": "VL 3B abliterate — RU + картинки, мало VRAM (~3GB)",
"text": "7B abliterate text — RU, без vision (~5GB)",
"big": "VL 32B abliterate — RU + картинки (~21GB)",
@@ -40,10 +43,8 @@ OLLAMA_PRESET_LABELS: dict[str, str] = {
"keep": "не менять ollama-models.yaml",
}
# Deprecated text blob — prefer menu helpers below.
PRESET_HELP = "\n".join(
f"{k}{v}" for k, v in OLLAMA_PRESET_LABELS.items() if k != "keep"
)
# Deprecated: use ollama_preset_menu / OLLAMA_PRESET_LABELS
PRESET_HELP = "" # kept empty; menus use OLLAMA_PRESET_LABELS
LLM_RUNTIME_LABELS: dict[str, str] = {
"none": "только SwarmUI",
+4 -4
View File
@@ -77,16 +77,16 @@ def ollama_tune_for(info: GpuInfo) -> OllamaTune:
note = "ultra: flash+q8 KV, 20GiB reserved for Swarm, ctx 32k, keep 30m"
elif info.tier == TIER_HIGH:
overhead = 14 * 1024**3
keep = "15m"
keep = "5m" # Assistent parks LLM before Generate; short keep is enough
kv = "q8_0"
ctx = 16384
note = "high: flash+q8 KV, 14GiB reserved for Swarm, ctx 16k, keep 15m"
note = "high: flash+q8 KV, 14GiB reserved for Swarm, ctx 16k, keep 5m"
elif info.tier == TIER_MID:
overhead = 10 * 1024**3
keep = "15m"
keep = "5m"
kv = "q8_0"
ctx = 16384
note = "mid: flash+q8 KV, 10GiB reserved for Swarm, ctx 16k, keep 15m"
note = "mid: flash+q8 KV, 10GiB reserved for Swarm, ctx 16k, keep 5m"
else:
overhead = 6 * 1024**3
keep = "2m"
+133 -12
View File
@@ -710,8 +710,55 @@ def seed_swarmui_api_keys(cfg: Config, host: str, log: Log) -> None:
run_ssh(cfg, host, "rm -f /tmp/gpu-rent-swarm-api-keys.json", check=False)
def _safe_persona_id(raw: object) -> str | None:
s = str(raw or "").strip().replace("\\", "/")
if not s or ".." in s or "/" in s:
return None
import re
if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_\-]{0,63}", s):
return None
return s
def _overlay_num_ctx(cfg: Config, host: str) -> int | None:
"""Prefer GPU-tier context from .gpu-rent-gpu.json when present."""
raw = run_ssh(
cfg,
host,
f"test -f {DATA}/.gpu-rent-gpu.json && cat {DATA}/.gpu-rent-gpu.json || true",
check=False,
timeout=15,
).strip()
if not raw:
return None
try:
probe = json.loads(raw)
vram = int(probe.get("vram_mib") or probe.get("memory_total_mib") or 0)
except (json.JSONDecodeError, TypeError, ValueError):
return None
if vram <= 0:
return None
from gpu_rent.perf_tiers import GpuInfo, ollama_tune_for, tier_for_vram_mib
info = GpuInfo(
name=str(probe.get("name") or "gpu"),
vram_mib=vram,
compute_cap=str(probe.get("compute_cap") or "0.0"),
uuid=str(probe.get("uuid") or ""),
tier=tier_for_vram_mib(vram),
)
return ollama_tune_for(info).context_length
def seed_assistent_personas(cfg: Config, host: str, log: Log) -> None:
"""Push local assistent-personas.yaml → VM Assistent/personas.yaml + personas.json."""
"""Push local assistent-personas.yaml → VM Assistent/personas/<id>/ overlay.
Laptop yaml stays the editor; on VM we write persona.json + extra.md and
overlay assistant.json (default_persona, optional num_ctx). Does not clobber
voice/likes/dislikes/rules already on disk. No longer writes legacy
personas.json / personas.yaml dumps.
"""
from gpu_rent.paths import assistent_personas_example_path, assistent_personas_manifest_path
local = Path(
@@ -728,24 +775,96 @@ def seed_assistent_personas(cfg: Config, host: str, log: Log) -> None:
if not text.strip():
log("assistent-personas: пустой файл — skip")
return
remote_dir = f"{DATA}/Assistent"
remote_yaml = f"{remote_dir}/personas.yaml"
remote_json = f"{remote_dir}/personas.json"
run_ssh(cfg, host, f"mkdir -p {shlex.quote(remote_dir)}", check=False)
put_text(cfg, host, remote_yaml, text if text.endswith("\n") else text + "\n")
try:
import yaml
data = yaml.safe_load(text) or {}
except Exception as exc:
log(f"assistent-personas: yaml parse — {exc}")
return
if not isinstance(data, dict):
log("assistent-personas: корень должен быть mapping — skip")
return
remote_dir = f"{DATA}/Assistent"
run_ssh(cfg, host, f"mkdir -p {shlex.quote(remote_dir)}/personas {shlex.quote(remote_dir)}/_base", check=False)
default_id = _safe_persona_id(data.get("default")) or "neutral"
personas = data.get("personas") or []
written = 0
if isinstance(personas, list):
for row in personas:
if not isinstance(row, dict):
continue
pid = _safe_persona_id(row.get("id"))
if not pid:
continue
title = str(row.get("title") or pid).strip() or pid
prompt = str(row.get("prompt") or "").strip()
pdir = f"{remote_dir}/personas/{pid}"
run_ssh(cfg, host, f"mkdir -p {shlex.quote(pdir)}", check=False)
meta = {"title": title, "tagline": title, "accent": "#8b949e"}
put_text(
cfg,
host,
remote_json,
json.dumps(data, ensure_ascii=False, indent=2) + "\n",
f"{pdir}/persona.json",
json.dumps(meta, ensure_ascii=False, indent=2) + "\n",
)
except Exception as exc:
log(f"assistent-personas: json convert — {exc}")
log(f"assistent-personas → {remote_yaml}")
if prompt:
put_text(
cfg,
host,
f"{pdir}/extra.md",
prompt if prompt.endswith("\n") else prompt + "\n",
)
written += 1
assistant_overlay: dict = {"default_persona": default_id}
num_ctx = _overlay_num_ctx(cfg, host)
if num_ctx:
assistant_overlay["num_ctx"] = num_ctx
put_text(
cfg,
host,
f"{remote_dir}/_base/assistant.json",
json.dumps(assistant_overlay, ensure_ascii=False, indent=2) + "\n",
)
# Drop legacy dumps so Assistent does not double-inject prompts.
run_ssh(
cfg,
host,
f"rm -f {shlex.quote(remote_dir + '/personas.json')} "
f"{shlex.quote(remote_dir + '/personas.yaml')}",
check=False,
)
ctx_note = f", num_ctx={num_ctx}" if num_ctx else ""
log(f"assistent-personas → overlay personas/{written} (default={default_id}{ctx_note})")
def count_wanted_models_on_vm(cfg: Config, host: str) -> int:
"""Count entries in Assistent wanted queue on the VM (best-effort)."""
remote = f"{DATA}/.gpu-rent-wanted-models.yaml"
raw = run_ssh(
cfg,
host,
f"test -f {shlex.quote(remote)} && cat {shlex.quote(remote)} || true",
check=False,
timeout=20,
).strip()
if not raw:
return 0
try:
import yaml
data = yaml.safe_load(raw) or {}
except Exception:
return sum(1 for line in raw.splitlines() if line.strip().startswith("- url:"))
n = 0
if isinstance(data, dict):
for rows in data.values():
if isinstance(rows, list):
n += sum(1 for r in rows if isinstance(r, dict) and r.get("url"))
return n
def merge_wanted_models_from_vm(cfg: Config, host: str, log: Log) -> int:
@@ -1072,6 +1191,8 @@ def provision_llm(cfg: Config, host: str, log: Log) -> None:
)
# Drop LLM units that should not hold VRAM for this runtime.
# Tombstone: still stop gpu-rent-llamacpp if an old disk left that unit behind
# (llamacpp runtime was removed; do not reinstall it).
if runtime == "none":
log("LLM: none — останавливаю gpu-rent-ollama / gpu-rent-llamacpp если были")
_stop_units("gpu-rent-ollama", "gpu-rent-llamacpp")
@@ -1083,7 +1204,7 @@ def provision_llm(cfg: Config, host: str, log: Log) -> None:
return
still: list[str] = []
if runtime == "ollama":
_stop_units("gpu-rent-llamacpp")
_stop_units("gpu-rent-llamacpp") # tombstone: disable leftover llamacpp unit
log("LLM: ставим/запускаем Ollama")
run_script_sudo(
cfg,
+5 -1
View File
@@ -129,7 +129,11 @@ def swarm_busy(swarm_url: str, timeout: float = 8.0) -> tuple[bool, str]:
def llm_busy(timeout: float = 3.0) -> tuple[bool, str]:
"""Ollama pull / loaded models count as busy."""
"""Ollama pull / loaded models count as busy.
Empty /api/ps after AssistentParkLlm is NOT busy by itself — Swarm queue
is checked separately in main() and keeps the GPU alive during Generate.
"""
pull_marker = DATA / ".gpu-rent-ollama-pulling"
if pull_marker.is_file():
try:
+2 -2
View File
@@ -47,9 +47,9 @@ except ValueError:
if gib >= 48:
tier, overhead, keep, kv, flash, ctx = "ultra", 20 * 1024**3, "30m", "q8_0", True, 32768
elif gib >= 24:
tier, overhead, keep, kv, flash, ctx = "high", 14 * 1024**3, "15m", "q8_0", True, 16384
tier, overhead, keep, kv, flash, ctx = "high", 14 * 1024**3, "5m", "q8_0", True, 16384
elif gib >= 16:
tier, overhead, keep, kv, flash, ctx = "mid", 10 * 1024**3, "15m", "q8_0", True, 16384
tier, overhead, keep, kv, flash, ctx = "mid", 10 * 1024**3, "5m", "q8_0", True, 16384
else:
tier, overhead, keep, kv, flash, ctx = "low", 6 * 1024**3, "2m", "q4_0", False, 8192
flash = bool(flash and (ampere or "A100" in name.upper() or "H100" in name.upper() or gib >= 16))
+3 -2
View File
@@ -16,9 +16,10 @@ def test_collect_links_swarm_and_ollama(monkeypatch):
links = collect_access_links(_Cfg(), tunneled=True)
labels = [x.label for x in links]
assert "SwarmUI UI" in labels
assert "Assistent" in labels
assert "SwarmUI MCP" in labels
assert "Ollama API" in labels
assert any("17811" in x.url for x in links)
assert any(x.label == "Assistent" and "вкладка" in x.note for x in links)
def test_collect_links_no_tunnel():
@@ -57,7 +58,7 @@ def test_access_panel_hides_stale_llm_error_when_ollama_ok(monkeypatch):
{
"name": "ollama",
"ok": True,
"detail": "1 models (huihui_ai/qwen2.5-vl-abliterated:7b)",
"detail": "1 models (huihui_ai/qwen3-vl-abliterated:8b-instruct)",
}
],
}
+81
View File
@@ -0,0 +1,81 @@
"""Unit tests for assistent personas overlay seed helpers."""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from gpu_rent.provision import _safe_persona_id, count_wanted_models_on_vm, seed_assistent_personas
def test_safe_persona_id():
assert _safe_persona_id("neutral") == "neutral"
assert _safe_persona_id("lewd_v2") == "lewd_v2"
assert _safe_persona_id("../x") is None
assert _safe_persona_id("a/b") is None
assert _safe_persona_id("") is None
def test_seed_assistent_personas_writes_overlay(tmp_path: Path, monkeypatch):
yaml_path = tmp_path / "assistent-personas.yaml"
yaml_path.write_text(
"default: cinema\n"
"personas:\n"
" - id: cinema\n"
" title: Кино\n"
" prompt: |\n"
" You are a DP.\n"
" - id: neutral\n"
" title: Нейтральный\n"
" prompt: Calm.\n",
encoding="utf-8",
)
puts: dict[str, str] = {}
ssh_cmds: list[str] = []
def fake_put(cfg, host, remote, text, mode=0o644):
puts[remote] = text
def fake_ssh(cfg, host, cmd, check=False, timeout=60):
ssh_cmds.append(cmd)
if ".gpu-rent-gpu.json" in cmd:
return json.dumps({"vram_mib": 24576, "name": "4090", "compute_cap": "8.9"})
return ""
monkeypatch.setattr("gpu_rent.provision.put_text", fake_put)
monkeypatch.setattr("gpu_rent.provision.run_ssh", fake_ssh)
monkeypatch.setattr(
"gpu_rent.paths.assistent_personas_manifest_path", lambda: yaml_path
)
monkeypatch.setattr(
"gpu_rent.paths.assistent_personas_example_path", lambda: tmp_path / "missing"
)
cfg = MagicMock()
cfg.assistent_personas_manifest = str(yaml_path)
logs: list[str] = []
seed_assistent_personas(cfg, "1.2.3.4", logs.append)
assert any("personas/cinema/persona.json" in p for p in puts)
assert any("personas/cinema/extra.md" in p for p in puts)
assert "/mnt/swarm_data/Assistent/_base/assistant.json" in puts
asst = json.loads(puts["/mnt/swarm_data/Assistent/_base/assistant.json"])
assert asst["default_persona"] == "cinema"
assert asst["num_ctx"] == 16384
assert any("rm -f" in c and "personas.json" in c for c in ssh_cmds)
assert not any(p.endswith("personas.yaml") for p in puts)
assert not any(p.endswith("personas.json") for p in puts)
assert any("overlay personas/2" in m for m in logs)
def test_count_wanted_models(monkeypatch):
raw = "lora:\n - url: https://civitai.red/x?modelVersionId=1\n title: A\n"
monkeypatch.setattr(
"gpu_rent.provision.run_ssh",
lambda *a, **k: raw,
)
assert count_wanted_models_on_vm(MagicMock(), "h") == 1
+61
View File
@@ -0,0 +1,61 @@
"""Doctor LLM / Assistent local checks."""
from __future__ import annotations
from pathlib import Path
from gpu_rent.doctor import Check, _llm_assistent
class _Cfg:
llm_runtime = "ollama"
ollama_models_manifest = None
extensions_manifest = None
def test_llm_assistent_warns_without_ollama_yaml(tmp_path: Path, monkeypatch):
cfg = _Cfg()
cfg.ollama_models_manifest = tmp_path / "missing-ollama.yaml"
cfg.extensions_manifest = tmp_path / "extensions.yaml"
cfg.extensions_manifest.write_text(
"swarmui:\n"
" - url: https://gitea.example/swarm-assistent.git\n"
" dir: swarm-assistent\n"
" requires: ollama\n",
encoding="utf-8",
)
monkeypatch.setattr(
"gpu_rent.paths.ollama_models_manifest_path",
lambda: cfg.ollama_models_manifest,
)
checks: list[Check] = []
_llm_assistent(cfg, checks)
names = {c.name: c for c in checks}
assert names["ollama-models.yaml"].ok is False
assert names["ollama-models.yaml"].blocking is True
assert names["Assistent ext"].ok is True
def test_llm_assistent_ok_with_manifests(tmp_path: Path, monkeypatch):
ollama = tmp_path / "ollama-models.yaml"
ollama.write_text(
"models:\n - name: foo:7b\n use: chat\n default: true\n",
encoding="utf-8",
)
ext = tmp_path / "extensions.yaml"
ext.write_text(
"swarmui:\n"
" - url: https://gitea.example/swarm-assistent.git\n"
" dir: swarm-assistent\n"
" requires: ollama\n",
encoding="utf-8",
)
cfg = _Cfg()
cfg.ollama_models_manifest = ollama
cfg.extensions_manifest = ext
cfg.llm_runtime = "ollama"
checks: list[Check] = []
_llm_assistent(cfg, checks)
by = {c.name: c for c in checks}
assert by["ollama-models.yaml"].ok
assert "swarm-assistent" in by["Assistent ext"].detail
+3 -1
View File
@@ -56,7 +56,9 @@ def test_write_preset(tmp_path: Path):
path = tmp_path / "out.yaml"
write_ollama_models_preset(path, "recommended")
entries = parse_ollama_models(path)
assert entries[0].name == "huihui_ai/qwen2.5-vl-abliterated:7b"
assert entries[0].name == "huihui_ai/qwen3-vl-abliterated:8b-instruct"
assert entries[0].default is True
assert entries[1].name == "huihui_ai/qwen2.5-vl-abliterated:7b"
write_ollama_models_preset(path, "big")
assert parse_ollama_models(path)[0].name == "huihui_ai/qwen2.5-vl-abliterated:32b"
write_ollama_models_preset(path, "text")
+2 -2
View File
@@ -43,10 +43,10 @@ def test_warmup_skips_when_ps_has_model(monkeypatch):
def fake_urlopen(req, timeout=None):
url = getattr(req, "full_url", str(req))
assert "/api/ps" in url
return FakeResp('{"models":[{"name":"huihui_ai/qwen2.5-vl-abliterated:7b"}]}')
return FakeResp('{"models":[{"name":"huihui_ai/qwen3-vl-abliterated:8b-instruct"}]}')
monkeypatch.setattr(_mod.urllib.request, "urlopen", fake_urlopen)
msg = _mod.warmup("huihui_ai/qwen2.5-vl-abliterated:7b", keep_alive="15m", num_ctx=16384, timeout=5)
msg = _mod.warmup("huihui_ai/qwen3-vl-abliterated:8b-instruct", keep_alive="15m", num_ctx=16384, timeout=5)
assert "skip" in msg
assert "VRAM" in msg
+2 -2
View File
@@ -82,6 +82,6 @@ def test_ollama_mid_4090_context_16k():
)
tune = ollama_tune_for(info)
assert tune.context_length == 16384
assert tune.keep_alive == "15m"
assert tune.keep_alive == "5m"
assert "OLLAMA_CONTEXT_LENGTH=16384" in "\n".join(ollama_env_lines(tune))
assert "OLLAMA_KEEP_ALIVE=15m" in "\n".join(ollama_env_lines(tune))
assert "OLLAMA_KEEP_ALIVE=5m" in "\n".join(ollama_env_lines(tune))
+1 -1
View File
@@ -86,7 +86,7 @@ def test_install_ollama_skips_restart_when_unit_unchanged():
text = files("gpu_rent.remote").joinpath("install_ollama.sh").read_text(encoding="utf-8")
assert "cmp -s" in text
assert "skip restart" in text
assert '"mid", 10 * 1024**3, "15m"' in text
assert '"mid", 10 * 1024**3, "5m"' in text
def test_provision_llm_skips_on_api_tags_not_cli_list():