Add multi-turn Assistent session diagnostics to the Debug API.

POST /assistent/session + /chat with rich per-turn traces (patch, Exact merge, compact_context); chat-eval wraps one session turn. Docs playbook and unit/HTTP tests included.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Leonid Pershin
2026-08-23 07:32:58 +03:00
co-authored by Cursor
parent 719d77efd9
commit b83d1d1e9c
6 changed files with 1965 additions and 292 deletions
+38 -10
View File
@@ -133,31 +133,59 @@ Exit 0 → можно `up`. Exit 1 → причина в таблице / кра
| Путь | Зачем |
| --- | --- |
| `/assistent` | Сводка: extension + overlay + roles + memory + live API + hints + playbook |
| `/assistent/diagnose` | То же + journal по умолчанию + session store meta |
| `/assistent/extension` | DLL, Sqlite deps, Tab/Assets, git HEAD |
| `/assistent/overlay` | personas на VM vs локальный `assistent-personas/` |
| `/assistent/roles` | `ollama-roles.json``/api/tags` (`default_chat`) |
| `/assistent/memory` | `assistent.sqlite` + counts |
| `/assistent/api` | `AssistentListPersonas/Models/Memory/Chats` через туннель (или SSH) |
| `/assistent/api?chat_smoke=1` | + 1-token Ollama `/api/chat` (кратко грузит модель) |
| `/assistent/chat-eval` | **Opt-in** `GetNewSession``AssistentChat` (полный ответ ассистента) |
| `/assistent/logs` | journalctl swarmui: build/load/Sqlite |
| `/assistent?logs=1` | Сводка + journal |
| `/assistent/session` | **POST** — multi-turn debug chat (in-memory, TTL ~45m) |
| `/assistent/session/{id}/chat` | **POST** — ход + объект `trace` |
| `/assistent/session/{id}` | **GET** / **DELETE** — состояние / сброс |
| `/assistent/chat-eval` | One-shot: session + один chat + delete |
Симптомы → куда смотреть: поле `playbook` в `/assistent`.
**`/assistent/chat-eval`** — не входит в `/snapshot`. Может загрузить chat-модель в VRAM и записать чат в Sqlite (если DLL на месте). Предпочтительно `POST` с JSON; для агентов удобен и `GET` с query.
**Session / chat-eval** не входят в `/snapshot`. Могут грузить VRAM, писать sqlite chat, тратить биллинг GPU. `compactContext` / Exact merge в Assistent HTTP API **нет** — sidecar реконструирует best-effort (`trace.gaps`).
#### Agent playbook (curl)
```bash
# POST (предпочтительно)
curl -sS -X POST http://127.0.0.1:17821/assistent/chat-eval \
-H "Content-Type: application/json" \
-d "{\"message\":\"какой checkpoint и какие steps/cfg должны быть\",\"persona\":\"leonid\",\"timeout\":120}"
BASE=http://127.0.0.1:17821
# GET (агенты / быстрый smoke)
curl -sS "http://127.0.0.1:17821/assistent/chat-eval?message=ping&timeout=90"
# Static+live health (no chat)
curl -sS "$BASE/assistent/diagnose" | jq '{ok,hints,playbook,session_store}'
# Multi-turn conversational diagnostics
SID=$(curl -sS -X POST "$BASE/assistent/session" \
-H 'Content-Type: application/json' \
-d '{"persona":"neutral","pack":"ordinary","context":{"krea_profile":"turbo","checkpoint":"krea-turbo"}}' \
| jq -r .debug_session_id)
curl -sS -X POST "$BASE/assistent/session/$SID/chat" \
-H 'Content-Type: application/json' \
-d '{"message":"какой checkpoint и какие steps/cfg должны быть"}' \
| jq '{ok,reply,trace:{timings_ms,patch,exact_merge,compact_context,system_layers,gaps}}'
curl -sS -X POST "$BASE/assistent/session/$SID/chat" \
-H 'Content-Type: application/json' \
-d '{"message":"сделай generate с этими params"}' \
| jq '{ok, trace:{patch,exact_merge}}'
curl -sS "$BASE/assistent/session/$SID" | jq .session
curl -sS -X DELETE "$BASE/assistent/session/$SID"
# One-shot convenience (session + one chat + delete)
curl -sS -X POST "$BASE/assistent/chat-eval" \
-H 'Content-Type: application/json' \
-d '{"message":"какие steps/cfg для turbo?","persona":"leonid","timeout":120}' \
| jq '{ok,reply,patch,trace}'
```
Ответ: `ok`, `ms` / `chat_ms`, `model` / `preferred`, `reply` / `reply_prose`, `patch` (prompt/steps/cfg/aspect/actions…), `errors`, `hints`. Туннель `:swarmui_local_port` (обычно 17801), иначе SSH на VM `:7801`.
Ответ chat: `ok`, `reply`, `trace` (`timings_ms`, `patch`, `exact_merge`, `compact_context`, `system_chars` / `system_layers`, `skills`, `hops`, `gaps`, `errors`/`warnings`/`hints`). Sqlite SaveChat fail — soft error, если текст ответа есть.
После `up --no-tunnel` процесс завершается и sidecar гаснет — держи отдельно:
@@ -165,7 +193,7 @@ curl -sS "http://127.0.0.1:17821/assistent/chat-eval?message=ping&timeout=90"
gpu-rent debug
```
Мутаций конфига/GPU нет (restart/hold/stop — как раньше через CLI). `chat_smoke` только пингует Ollama; `chat-eval` — полноценный AssistentChat (opt-in).
Мутаций конфига/GPU нет (restart/hold/stop — как раньше через CLI). `chat_smoke` только пингует Ollama; session/chat-eval — полноценный AssistentChat (opt-in).
---
+300 -30
View File
@@ -1,8 +1,8 @@
"""Localhost debug HTTP sidecar for gpu-rent.
Bind only 127.0.0.1. Started from `up` / `tunnel` / `debug`.
Mostly read-only; `/assistent/chat-eval` is an opt-in AssistentChat probe
(may load VRAM / persist chat) and is not part of `/snapshot`.
Mostly read-only; Assistent session / chat-eval endpoints are opt-in
(may load VRAM / persist chat) and are not part of `/snapshot`.
"""
from __future__ import annotations
@@ -63,16 +63,25 @@ _CHAT_EVAL_PARAMS = [
},
]
_WARN_VRAM = (
"WARNING: may load chat model into VRAM, may write assistent.sqlite chat, "
"and incurs GPU billing if the VM is up. Not part of /snapshot. "
"compactContext / Exact merge are reconstructed client-side "
"(Assistent HTTP API does not return them)."
)
_OPENAPI: dict[str, Any] = {
"openapi": "3.0.3",
"info": {
"title": "gpu-rent Debug API",
"version": "1.1.0",
"version": "1.2.0",
"description": (
"Localhost diagnostics for installer progress and "
"SwarmUI / Comfy / Ollama / Assistent. Mostly read-only; "
"POST/GET /assistent/chat-eval is opt-in AssistentChat "
"(may load VRAM / write chat) and is skipped from /snapshot."
"SwarmUI / Comfy / Ollama / Assistent. Mostly read-only. "
"Assistent multi-turn: POST /assistent/session + "
"/assistent/session/{id}/chat (trace object). "
"One-shot: POST/GET /assistent/chat-eval. "
+ _WARN_VRAM
),
},
"servers": [{"url": "http://127.0.0.1:17821"}],
@@ -95,7 +104,9 @@ _OPENAPI: dict[str, Any] = {
}
},
"/snapshot": {
"get": {"summary": "Cheap aggregate (no full diag, no chat-eval)"}
"get": {
"summary": "Cheap aggregate (no full diag, no Assistent chat/session)"
}
},
"/status": {"get": {"summary": "JSON mirror of gpu-rent status"}},
"/state": {"get": {"summary": "state.json without secrets"}},
@@ -142,6 +153,27 @@ _OPENAPI: dict[str, Any] = {
],
}
},
"/assistent/diagnose": {
"get": {
"summary": (
"Full static+live Assistent health (logs on by default). "
+ _WARN_VRAM
),
"parameters": [
{
"name": "chat_smoke",
"in": "query",
"schema": {"type": "boolean"},
},
{
"name": "logs",
"in": "query",
"schema": {"type": "boolean", "default": True},
"description": "Pass logs=0 to skip journal",
},
],
}
},
"/assistent/extension": {
"get": {"summary": "DLL/Sqlite deps/Tab assets under Extensions/swarm-assistent"}
},
@@ -171,18 +203,92 @@ _OPENAPI: dict[str, Any] = {
"/assistent/logs": {
"get": {"summary": "journalctl swarmui filtered for Assistent/Sqlite/build"}
},
"/assistent/session": {
"post": {
"summary": (
"Start multi-turn Assistent debug session "
"(GetNewSession + optional GetConfig). " + _WARN_VRAM
),
"requestBody": {
"required": False,
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"persona": {"type": "string"},
"pack": {"type": "string"},
"model": {"type": "string"},
"skills": {
"type": "array",
"items": {"type": "string"},
},
"context": {
"type": "object",
"description": (
"Synthetic compactContext fields "
"(krea_profile, checkpoint, "
"recommended_params, …)"
),
},
"probe_config": {
"type": "boolean",
"default": True,
},
},
}
}
},
},
}
},
"/assistent/session/{id}": {
"get": {"summary": "In-memory session summary + last trace"},
"delete": {"summary": "Drop in-memory debug session"},
},
"/assistent/session/{id}/chat": {
"post": {
"summary": (
"Send user message; returns reply + trace "
"(patch, Exact merge, compact_context sizes, system_layers). "
+ _WARN_VRAM
),
"requestBody": {
"required": True,
"content": {
"application/json": {
"schema": {
"type": "object",
"required": ["message"],
"properties": {
"message": {"type": "string"},
"timeout": {"type": "number"},
"persona": {"type": "string"},
"pack": {"type": "string"},
"model": {"type": "string"},
"skills": {
"type": "array",
"items": {"type": "string"},
},
"context": {"type": "object"},
},
}
}
},
},
}
},
"/assistent/chat-eval": {
"get": {
"summary": (
"Opt-in AssistentChat eval (GetNewSession→AssistentChat); "
"may load VRAM / write chat — not in /snapshot"
"One-shot AssistentChat (session+chat+delete). " + _WARN_VRAM
),
"parameters": _CHAT_EVAL_PARAMS,
},
"post": {
"summary": (
"Same as GET; prefer JSON body "
"{message,persona,pack,model,timeout}"
"{message,persona,pack,model,timeout,context}"
),
"requestBody": {
"required": False,
@@ -196,6 +302,7 @@ _OPENAPI: dict[str, Any] = {
"pack": {"type": "string"},
"model": {"type": "string"},
"timeout": {"type": "number"},
"context": {"type": "object"},
},
}
}
@@ -207,6 +314,21 @@ _OPENAPI: dict[str, Any] = {
}
def _is_assistent_session_chat(path: str) -> bool:
parts = [p for p in path.split("/") if p]
return (
len(parts) == 4
and parts[0] == "assistent"
and parts[1] == "session"
and parts[3] == "chat"
)
def _is_assistent_session_id(path: str) -> bool:
parts = [p for p in path.split("/") if p]
return len(parts) == 3 and parts[0] == "assistent" and parts[1] == "session"
@dataclass
class DebugHub:
"""Shared state for the sidecar process."""
@@ -397,7 +519,9 @@ def _make_handler(hub: DebugHub):
f"<p>base <code>{hub.base_url}</code> · mostly read-only · 127.0.0.1</p>"
f"<p>Agent: start at <a href='/openapi.json'>/openapi.json</a> "
f"then <a href='/snapshot'>/snapshot</a>. "
f"Opt-in chat: <code>/assistent/chat-eval</code> (not in snapshot).</p>"
f"Opt-in Assistent: <code>/assistent/session</code> (multi-turn) · "
f"<code>/assistent/chat-eval</code> (one-shot) — not in snapshot; "
f"may load VRAM.</p>"
f"<ul>{rows}</ul></body></html>"
)
self._send(200, html.encode("utf-8"), "text/html; charset=utf-8")
@@ -445,6 +569,19 @@ def _make_handler(hub: DebugHub):
},
)
def do_DELETE(self) -> None: # noqa: N802
try:
self._dispatch("DELETE")
except Exception as exc:
self._json(
500,
{
"ok": False,
"error": str(exc)[:300],
"trace": traceback.format_exc()[-800:],
},
)
def _dispatch(self, method: str = "GET") -> None:
parsed = urlparse(self.path)
path = parsed.path.rstrip("/") or "/"
@@ -464,12 +601,27 @@ def _make_handler(hub: DebugHub):
doc["servers"] = [{"url": hub.base_url}]
self._json(200, doc)
return
if method == "POST" and path != "/assistent/chat-eval":
if method == "POST" and path not in {
"/assistent/chat-eval",
"/assistent/session",
} and not _is_assistent_session_chat(path):
self._json(
405,
{
"ok": False,
"error": "POST only allowed for /assistent/chat-eval",
"error": (
"POST only for /assistent/chat-eval, "
"/assistent/session, /assistent/session/{id}/chat"
),
},
)
return
if method == "DELETE" and not _is_assistent_session_id(path):
self._json(
405,
{
"ok": False,
"error": "DELETE only for /assistent/session/{id}",
},
)
return
@@ -592,16 +744,38 @@ def _make_handler(hub: DebugHub):
return
if path == "/assistent" or path.startswith("/assistent/"):
from gpu_rent import debug_assistent
from gpu_rent import debug_assistent_session as das
def _flag(name: str) -> bool:
raw = (qs.get(name) or ["0"])[0].strip().lower()
def _flag(name: str, *, default: bool = False) -> bool:
vals = qs.get(name)
if not vals:
return default
raw = vals[0].strip().lower()
return raw in {"1", "true", "yes", "on"}
chat_smoke = _flag("chat_smoke")
include_logs = _flag("logs")
sub = path[len("/assistent") :].lstrip("/") or ""
try_list = [
"/assistent",
"/assistent/diagnose",
"/assistent/extension",
"/assistent/overlay",
"/assistent/roles",
"/assistent/memory",
"/assistent/api",
"/assistent/wanted",
"/assistent/logs",
"/assistent/session",
"/assistent/session/{id}",
"/assistent/session/{id}/chat",
"/assistent/chat-eval",
]
if sub == "":
if method != "GET":
self._json(405, {"ok": False, "error": "GET only"})
return
cache_key = f"assistent:{int(chat_smoke)}:{int(include_logs)}"
payload = hub.cached(
cache_key,
@@ -612,6 +786,29 @@ def _make_handler(hub: DebugHub):
include_logs=include_logs,
),
)
elif sub == "diagnose":
if method != "GET":
self._json(405, {"ok": False, "error": "GET only"})
return
# logs default on for diagnose; logs=0 to skip
diag_logs = _flag("logs", default=True)
if qs.get("logs") and qs["logs"][0].strip().lower() in {
"0",
"false",
"no",
"off",
}:
diag_logs = False
cache_key = f"assistent/diagnose:{int(chat_smoke)}:{int(diag_logs)}"
payload = hub.cached(
cache_key,
CACHE_TTL_DEFAULT,
lambda: das.collect_assistent_diagnose(
hub.cfg,
chat_smoke=chat_smoke,
include_logs=diag_logs,
),
)
elif sub == "extension":
payload = hub.cached(
"assistent/extension",
@@ -657,8 +854,78 @@ def _make_handler(hub: DebugHub):
CACHE_TTL_LOGS,
lambda: debug_assistent.collect_assistent_logs(hub.cfg),
)
elif sub == "session" and method == "POST":
try:
body = self._read_json_body()
except ValueError as exc:
self._json(400, {"ok": False, "error": str(exc)})
return
skills = body.get("skills")
if skills is not None and not isinstance(skills, list):
self._json(
400, {"ok": False, "error": "skills must be an array"}
)
return
ctx = body.get("context")
if ctx is not None and not isinstance(ctx, dict):
self._json(
400, {"ok": False, "error": "context must be an object"}
)
return
probe = body.get("probe_config", True)
payload = das.create_debug_session(
hub.cfg,
persona=body.get("persona"),
pack=body.get("pack"),
model=body.get("model"),
context=ctx,
skills=skills,
probe_config=bool(probe),
)
elif _is_assistent_session_id(path):
sid = sub.split("/", 1)[1]
if method == "GET":
payload = das.get_debug_session(sid)
elif method == "DELETE":
payload = das.delete_debug_session(sid)
else:
self._json(405, {"ok": False, "error": "GET or DELETE"})
return
elif _is_assistent_session_chat(path):
if method != "POST":
self._json(405, {"ok": False, "error": "POST only"})
return
sid = sub.split("/")[1]
try:
body = self._read_json_body()
except ValueError as exc:
self._json(400, {"ok": False, "error": str(exc)})
return
skills = body.get("skills")
if skills is not None and not isinstance(skills, list):
self._json(
400, {"ok": False, "error": "skills must be an array"}
)
return
ctx = body.get("context")
if ctx is not None and not isinstance(ctx, dict):
self._json(
400, {"ok": False, "error": "context must be an object"}
)
return
payload = das.chat_debug_session(
hub.cfg,
sid,
message=body.get("message"),
timeout=body.get("timeout"),
context=ctx,
pack=body.get("pack"),
model=body.get("model"),
skills=skills,
persona=body.get("persona"),
)
elif sub == "chat-eval":
body: dict[str, Any] = {}
body = {}
if method == "POST":
try:
body = self._read_json_body()
@@ -666,12 +933,21 @@ def _make_handler(hub: DebugHub):
self._json(400, {"ok": False, "error": str(exc)})
return
def _q(name: str) -> str | None:
def _q(name: str) -> Any:
if name in body and body[name] is not None:
return body[name]
vals = qs.get(name)
return vals[0] if vals else None
ctx = _q("context")
if isinstance(ctx, str):
try:
ctx = json.loads(ctx)
except json.JSONDecodeError:
ctx = None
if ctx is not None and not isinstance(ctx, dict):
ctx = None
# Never cache — live AssistentChat / VRAM side effects.
payload = debug_assistent.run_assistent_chat_eval(
hub.cfg,
@@ -680,6 +956,7 @@ def _make_handler(hub: DebugHub):
pack=_q("pack"),
model=_q("model"),
timeout=_q("timeout"),
context=ctx,
)
else:
self._json(
@@ -687,17 +964,7 @@ def _make_handler(hub: DebugHub):
{
"ok": False,
"error": f"unknown path {path}",
"try": [
"/assistent",
"/assistent/extension",
"/assistent/overlay",
"/assistent/roles",
"/assistent/memory",
"/assistent/api",
"/assistent/wanted",
"/assistent/logs",
"/assistent/chat-eval",
],
"try": try_list,
},
)
return
@@ -759,7 +1026,9 @@ def start_debug_server(
f" {base}/openapi.json",
f" {base}/snapshot",
f" {base}/assistent",
f" {base}/assistent/chat-eval (opt-in; not in snapshot)",
f" {base}/assistent/diagnose",
f" {base}/assistent/session (multi-turn; not in snapshot)",
f" {base}/assistent/chat-eval (one-shot; not in snapshot)",
]
if log:
for line in lines:
@@ -796,7 +1065,8 @@ def run_debug_blocking(
try:
if log:
log("Debug API слушает (Ctrl+C — выход). "
"Мутаций конфига/GPU нет; /assistent/chat-eval — opt-in AssistentChat.")
"Мутаций конфига/GPU нет; "
"/assistent/session + /chat-eval — opt-in AssistentChat.")
while True:
time.sleep(3600)
except KeyboardInterrupt:
+23 -248
View File
@@ -2,7 +2,8 @@
Probes extension compile/load markers, overlay personas, ollama-roles,
sqlite, live SwarmUI Assistent* APIs, optional 1-token Ollama chat smoke,
and opt-in AssistentChat evaluation (/assistent/chat-eval).
opt-in AssistentChat evaluation (/assistent/chat-eval), and multi-turn
debug sessions (/assistent/session*).
"""
from __future__ import annotations
@@ -437,107 +438,23 @@ def run_assistent_chat_eval(
pack: str | None = None,
model: str | None = None,
timeout: float | int | str | None = None,
context: dict | None = None,
) -> dict[str, Any]:
"""Opt-in AssistentChat round-trip via local Swarm tunnel, else SSH :7801.
"""Opt-in AssistentChat round-trip (session create → one chat → delete).
May load the chat model into VRAM and write chat history if Sqlite works.
Not part of /snapshot.
Not part of /snapshot. Prefer POST /assistent/session for multi-turn.
"""
text = (message or DEFAULT_CHAT_EVAL_MESSAGE).strip() or DEFAULT_CHAT_EVAL_MESSAGE
persona_id = (persona or "").strip() or "neutral"
pack_name = (pack or "").strip() or "ordinary"
t_chat = _clamp_chat_eval_timeout(timeout)
hints: list[str] = [
"opt-in AssistentChat eval — may load VRAM; may SaveChat if Sqlite works",
"not included in /snapshot",
]
errors: list[str] = []
t0 = time.perf_counter()
from gpu_rent.debug_assistent_session import run_assistent_chat_eval_via_session
base, via = _swarm_base(cfg)
if not base:
return _assistent_chat_eval_via_ssh(
return run_assistent_chat_eval_via_session(
cfg,
message=text,
persona=persona_id,
pack=pack_name,
model=(model or "").strip() or None,
timeout=t_chat,
hints=hints,
t0=t0,
)
sid, err, ms_sess = _session_id(base)
if not sid:
return {
"ok": False,
"via": via,
"ms": round((time.perf_counter() - t0) * 1000, 1),
"session_ms": round(ms_sess, 1),
"error": f"GetNewSession failed: {err}",
"errors": [f"GetNewSession: {err}"],
"hints": hints,
"message": text,
"persona": persona_id,
"pack": pack_name,
}
chosen, preferred = _resolve_chat_model(cfg, base, sid, model=model)
if not chosen:
return {
"ok": False,
"via": via,
"ms": round((time.perf_counter() - t0) * 1000, 1),
"session_ms": round(ms_sess, 1),
"error": "no model (AssistentListModels.preferred / default_chat)",
"errors": ["model required"],
"hints": hints + ["GET /assistent/roles + /ollama — нет preferred chat model"],
"message": text,
"persona": persona_id,
"pack": pack_name,
"preferred": preferred,
}
payload = {
"session_id": sid,
"baseUrl": "http://127.0.0.1:11434",
"model": chosen,
"pack": pack_name,
"persona": persona_id,
"includeBase": True,
"messages": [{"role": "user", "content": text}],
"context_json": json.dumps(
{
"persona": persona_id,
"debug_eval": True,
"has_vision_image": False,
"images_in_request": False,
},
ensure_ascii=False,
),
"skills": [],
}
ok, data, ms_call = _http_json(
f"{base}/API/AssistentChat",
method="POST",
body=payload,
timeout=t_chat,
)
return _finish_chat_eval(
ok=ok,
data=data,
ms_call=ms_call,
ms_total=(time.perf_counter() - t0) * 1000,
via=via,
session_ms=ms_sess,
message=text,
persona=persona_id,
pack=pack_name,
model=chosen,
preferred=preferred,
hints=hints,
errors=errors,
timeout=t_chat,
message=message,
persona=persona,
pack=pack,
model=model,
timeout=timeout,
context=context,
)
@@ -623,157 +540,6 @@ def _finish_chat_eval(
return out
def _assistent_chat_eval_via_ssh(
cfg: Config,
*,
message: str,
persona: str,
pack: str,
model: str | None,
timeout: float,
hints: list[str],
t0: float,
) -> dict[str, Any]:
state = load_state()
if not debug_checks.ssh_ready(cfg, state):
return {
**debug_checks._ssh_fail(state.phase),
"ok": False,
"ms": round((time.perf_counter() - t0) * 1000, 1),
"message": message,
"persona": persona,
"pack": pack,
"model": model,
"errors": [debug_checks.SSH_UNAVAILABLE],
"hints": hints + ["туннель SwarmUI закрыт и SSH нет — gpu-rent tunnel / up"],
}
host = debug_checks.ssh_host(state)
assert host is not None
# Resolve model on laptop if possible (roles via SSH fs), else let remote ListModels.
chosen = model
preferred = None
if not chosen:
roles = collect_assistent_roles(cfg)
preferred = (roles.get("roles") or {}).get("default_chat")
if isinstance(preferred, str) and preferred.strip():
chosen = preferred.strip()
preferred = chosen
payload_model = chosen or ""
# Remote script: GetNewSession → optional ListModels → AssistentChat
script = f'''
import json, urllib.request, time
MSG = {json.dumps(message, ensure_ascii=False)}
PERSONA = {json.dumps(persona, ensure_ascii=False)}
PACK = {json.dumps(pack, ensure_ascii=False)}
MODEL = {json.dumps(payload_model, ensure_ascii=False)}
TIMEOUT = {float(timeout)}
def post(path, payload, timeout=TIMEOUT):
t0 = time.time()
req = urllib.request.Request(
"http://127.0.0.1:7801" + path,
data=json.dumps(payload).encode(),
headers={{"Content-Type": "application/json"}},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read().decode("utf-8", "replace")
ms = (time.time() - t0) * 1000
try:
data = json.loads(raw)
except Exception:
data = raw[:500]
return True, data, ms
except Exception as e:
return False, str(e), (time.time() - t0) * 1000
ok, sess, ms = post("/API/GetNewSession", {{}}, timeout=20)
if not ok or not isinstance(sess, dict) or not sess.get("session_id"):
print(json.dumps({{"ok": False, "error": sess, "session_ms": ms, "via": "ssh"}}))
raise SystemExit(0)
sid = sess["session_id"]
preferred = None
model = MODEL.strip()
if not model:
cok, mdata, _ = post("/API/AssistentListModels", {{
"session_id": sid, "baseUrl": "http://127.0.0.1:11434"
}}, timeout=25)
if cok and isinstance(mdata, dict):
preferred = mdata.get("preferred")
model = (preferred or "").strip()
if not model:
print(json.dumps({{
"ok": False, "error": "no model", "session_ms": ms, "via": "ssh",
"preferred": preferred
}}))
raise SystemExit(0)
payload = {{
"session_id": sid,
"baseUrl": "http://127.0.0.1:11434",
"model": model,
"pack": PACK,
"persona": PERSONA,
"includeBase": True,
"messages": [{{"role": "user", "content": MSG}}],
"context_json": json.dumps({{
"persona": PERSONA, "debug_eval": True,
"has_vision_image": False, "images_in_request": False
}}, ensure_ascii=False),
"skills": [],
}}
cok, data, cms = post("/API/AssistentChat", payload, timeout=TIMEOUT)
print(json.dumps({{
"ok": cok, "data": data, "chat_ms": cms, "session_ms": ms,
"via": "ssh", "model": model, "preferred": preferred or model
}}, ensure_ascii=False))
'''
try:
from gpu_rent.ssh_ops import run_ssh
out = run_ssh(
cfg,
host,
"python3 - <<'PY'\n" + script + "\nPY",
check=False,
timeout=int(timeout) + 40,
).strip()
remote = json.loads(out.splitlines()[-1])
except Exception as exc:
return {
"ok": False,
"via": "ssh",
"ms": round((time.perf_counter() - t0) * 1000, 1),
"message": message,
"persona": persona,
"pack": pack,
"model": model,
"error": str(exc)[:240],
"errors": [str(exc)[:240]],
"hints": hints,
}
data = remote.get("data")
ok = bool(remote.get("ok"))
return _finish_chat_eval(
ok=ok,
data=data if ok or isinstance(data, dict) else remote.get("error") or data,
ms_call=float(remote.get("chat_ms") or 0),
ms_total=(time.perf_counter() - t0) * 1000,
via="ssh",
session_ms=float(remote.get("session_ms") or 0),
message=message,
persona=persona,
pack=pack,
model=remote.get("model") or chosen,
preferred=remote.get("preferred") or preferred,
hints=hints,
errors=[],
timeout=timeout,
)
_FS_CACHE: tuple[float, str, dict[str, Any]] | None = None
_FS_TTL = 8.0
@@ -1366,6 +1132,15 @@ def collect_assistent_deep(
"memory_broken": "GET /assistent/memory + /assistent/api (ListMemory)",
"personas_missing": "GET /assistent/overlay + ListPersonas in /assistent/api",
"chat_smoke": "GET /assistent/api?chat_smoke=1",
"chat_eval": "POST /assistent/chat-eval (opt-in AssistentChat; not in /snapshot)",
"chat_eval": "POST /assistent/chat-eval (one-shot; not in /snapshot)",
"session": (
"POST /assistent/session → POST .../chat (multi-turn + trace) "
"→ DELETE /assistent/session/{id}"
),
"diagnose": "GET /assistent/diagnose or /assistent?logs=1",
"compact_context_gap": (
"AssistentChat does not return compactContext; "
"trace fills from GetConfig Exact + synthetic context_json (see trace.gaps)"
),
},
}
File diff suppressed because it is too large Load Diff
+419
View File
@@ -0,0 +1,419 @@
"""Unit tests for Assistent patch extract + in-memory session store (no GPU)."""
from __future__ import annotations
import time
from gpu_rent.debug_assistent import extract_assistent_patch
from gpu_rent import debug_assistent_session as das
def test_extract_assistent_patch_last_fence():
text = (
"Сначала prose.\n"
"```json\n"
'{"steps": 8, "cfg": 1}\n'
"```\n"
"ещё текст\n"
"```json\n"
'{"prompt": "a cat", "actions": ["generate"], "aspect": "16:9"}\n'
"```\n"
)
out = extract_assistent_patch(text)
assert out["patch"] is not None
assert out["patch"]["prompt"] == "a cat"
assert out["patch"]["generate"] is True
assert out["patch"]["aspect"] == "16:9"
assert "Сначала prose" in out["prose"]
def test_extract_assistent_patch_ignores_non_patch_json():
text = 'hello\n```json\n{"foo": 1}\n```\n'
out = extract_assistent_patch(text)
assert out["patch"] is None
def test_analyze_exact_merge_fills_omitted_params():
patch = {"actions": ["generate"], "prompt": "x"}
exact = {
"generation": {"profile": "turbo", "steps": 8, "cfg": 1, "sigma_shift": 1.15},
"profiles": {
"turbo": {"steps": 8, "cfg": 1, "sigma_shift": 1.15},
"raw": {"steps": 28, "cfg": 4.5, "sigma_shift": 1.15},
},
}
result = das.analyze_exact_merge(
patch,
krea_profile="turbo",
exact=exact,
)
assert result["wants_generate"] is True
assert result["would_fill"]["steps"] == 8
assert result["would_fill"]["cfg"] == 1
assert "steps omitted" in " ".join(result["hints"])
def test_analyze_exact_merge_forces_foreign_leftovers():
patch = {"generate": True, "steps": 20, "cfg": 7}
exact = {
"profiles": {"turbo": {"steps": 8, "cfg": 1, "sigma_shift": 1.15}},
"generation": {"profile": "turbo"},
}
result = das.analyze_exact_merge(
patch, krea_profile="turbo", exact=exact, user_param_intent=False
)
assert result["would_force_to_exact"]["steps"]["to"] == 8
assert result["would_force_to_exact"]["cfg"]["to"] == 1
def test_summarize_compact_context_sizes():
summary = das.summarize_compact_context(
{"persona": "neutral", "krea_profile": "turbo", "recommended_params": {"steps": 8}}
)
assert "persona" in summary["keys"]
assert summary["chars"] > 0
assert summary["token_ish"] >= 1
assert "does not return compactContext" in summary["note"]
def test_session_store_create_get_delete_without_swarm(monkeypatch):
store = das.reset_session_store_for_tests()
class Cfg:
swarmui_local_port = 17801
# No local tunnel
monkeypatch.setattr(das, "_swarm_base", lambda cfg: (None, "none"))
monkeypatch.setattr(
das, "_resolve_chat_model", lambda cfg, base, sid, model=None: (None, None)
)
created = das.create_debug_session(
Cfg(), # type: ignore[arg-type]
persona="neutral",
pack="ordinary",
probe_config=False,
)
assert created["ok"] is True
sid = created["debug_session_id"]
assert sid in store.list_ids()
got = das.get_debug_session(sid)
assert got["ok"] is True
assert got["session"]["persona"] == "neutral"
assert got["session"]["turn_count"] == 0
deleted = das.delete_debug_session(sid)
assert deleted["ok"] is True
assert das.get_debug_session(sid)["ok"] is False
def test_session_store_ttl_eviction(monkeypatch):
store = das.reset_session_store_for_tests()
store.ttl_sec = 0.05
class Cfg:
swarmui_local_port = 17801
monkeypatch.setattr(das, "_swarm_base", lambda cfg: (None, "none"))
monkeypatch.setattr(
das, "_resolve_chat_model", lambda cfg, base, sid, model=None: ("m", "m")
)
created = das.create_debug_session(Cfg(), probe_config=False) # type: ignore[arg-type]
sid = created["debug_session_id"]
time.sleep(0.08)
assert das.get_debug_session(sid)["ok"] is False
def test_session_store_max_cap(monkeypatch):
store = das.reset_session_store_for_tests()
store.max_sessions = 2
class Cfg:
swarmui_local_port = 17801
monkeypatch.setattr(das, "_swarm_base", lambda cfg: (None, "none"))
monkeypatch.setattr(
das, "_resolve_chat_model", lambda cfg, base, sid, model=None: ("m", "m")
)
ids = []
for _ in range(3):
created = das.create_debug_session(Cfg(), probe_config=False) # type: ignore[arg-type]
ids.append(created["debug_session_id"])
time.sleep(0.01)
assert len(store.list_ids()) == 2
assert ids[0] not in store.list_ids()
def test_clamp_message():
short, trunc = das.clamp_message("hi", max_chars=10)
assert short == "hi" and trunc is False
long, trunc = das.clamp_message("x" * 20, max_chars=10)
assert len(long) == 10 and trunc is True
def test_multi_turn_chat_keeps_history(monkeypatch):
store = das.reset_session_store_for_tests()
class Cfg:
swarmui_local_port = 17801
monkeypatch.setattr(
das, "_swarm_base", lambda cfg: ("http://127.0.0.1:17801", "local")
)
monkeypatch.setattr(das, "_session_id", lambda base: ("swarm-1", None, 3.0))
monkeypatch.setattr(
das,
"_resolve_chat_model",
lambda cfg, base, sid, model=None: ("qwen", "qwen"),
)
payloads: list[dict] = []
def fake_http(url, *, method="GET", body=None, timeout=12.0):
if url.endswith("/API/AssistentGetConfig"):
return True, {
"success": True,
"persona": "neutral",
"exact": {
"profiles": {"turbo": {"steps": 8, "cfg": 1, "sigma_shift": 1.15}},
"generation": {"profile": "turbo"},
},
"skills": [],
"enabled_skills": [],
}, 5.0
if url.endswith("/API/AssistentChat"):
payloads.append(body or {})
n = len(payloads)
reply = f'turn{n}\n```json\n{{"steps": {4 + n}, "actions": ["generate"]}}\n```'
return True, {"success": True, "reply": reply, "model": "qwen"}, 20.0
return False, "unexpected " + url, 1.0
monkeypatch.setattr(das, "_http_json", fake_http)
created = das.create_debug_session(Cfg(), probe_config=True) # type: ignore[arg-type]
sid = created["debug_session_id"]
t1 = das.chat_debug_session(Cfg(), sid, message="first") # type: ignore[arg-type]
t2 = das.chat_debug_session(Cfg(), sid, message="second") # type: ignore[arg-type]
assert t1["ok"] and t2["ok"]
assert len(payloads) == 2
assert len(payloads[0]["messages"]) == 1
assert len(payloads[1]["messages"]) == 3 # user, assistant, user
assert payloads[1]["messages"][0]["content"] == "first"
assert payloads[1]["messages"][2]["content"] == "second"
assert t2["trace"]["patch"]["steps"] == 6
assert t2["trace"]["exact_merge"]["wants_generate"] is True
assert "gaps" in t2["trace"]
assert store.get(sid).turn_count == 2
das.delete_debug_session(sid)
def test_build_chat_trace_soft_sqlite():
reply = 'ok\n```json\n{"steps": 8, "actions": ["generate"]}\n```'
trace = das.build_chat_trace(
ok=True,
timings_ms={"chat": 12.0, "total": 20.0},
model="qwen",
preferred="qwen",
persona="neutral",
pack="ordinary",
context={"krea_profile": "turbo", "recommended_params": {"steps": 8, "cfg": 1}},
skills=[],
config_probe={
"ok": True,
"_exact": {
"profiles": {"turbo": {"steps": 8, "cfg": 1, "sigma_shift": 1.15}}
},
"data": {"enabled_skills": []},
},
reply=reply,
response={
"reply": reply,
"system_chars": 1000,
"system_layers": {"core": 100, "pack": 50},
"error": "SaveChat sqlite fail",
},
errors=[],
warnings=[],
hints=[],
)
assert trace["ok"] is True
assert trace["patch"]["steps"] == 8
assert trace["exact_merge"]["wants_generate"] is True
assert any("Sqlite" in e or "sqlite" in e.lower() for e in trace["errors"])
assert "gaps" in trace and len(trace["gaps"]) >= 3
def test_openapi_lists_session_paths(monkeypatch, tmp_path):
import socket
import urllib.request
from gpu_rent import debug_api
from gpu_rent.config import load_config
# Reuse same env helper pattern as test_debug_api
monkeypatch.setenv("HOME", str(tmp_path))
monkeypatch.setenv("USERPROFILE", str(tmp_path))
monkeypatch.chdir(tmp_path)
for key, val in {
"OS_AUTH_URL": "https://example/v3",
"OS_USER_DOMAIN_NAME": "default",
"OS_USERNAME": "u",
"OS_PASSWORD": "secret-password",
"OS_PROJECT_ID": "proj",
"OS_REGION_NAME": "ru-7",
"GPU_RENT_AZ": "ru-7a",
"CIVITAI_API_TOKEN": "civ",
"HF_TOKEN": "hf",
"GIT_TOKEN": "git",
"SELECTEL_API_TOKEN": "sel",
}.items():
monkeypatch.setenv(key, val)
cfg = load_config(require_auth=True)
sock = socket.socket()
sock.bind(("127.0.0.1", 0))
port = sock.getsockname()[1]
sock.close()
srv = debug_api.start_debug_server(cfg, port=port, print_urls=False)
assert srv is not None
try:
with urllib.request.urlopen(
f"http://127.0.0.1:{port}/openapi.json", timeout=5
) as resp:
doc = __import__("json").loads(resp.read().decode())
assert "/assistent/session" in doc["paths"]
assert "/assistent/session/{id}/chat" in doc["paths"]
assert "/assistent/diagnose" in doc["paths"]
assert "VRAM" in doc["info"]["description"]
finally:
debug_api.stop_debug_server(srv)
def test_http_session_chat_mocked(monkeypatch, tmp_path):
"""POST session → chat → GET → DELETE via Debug API (mocked Swarm)."""
import json
import socket
import urllib.request
from gpu_rent import debug_api
from gpu_rent.config import load_config
monkeypatch.setenv("HOME", str(tmp_path))
monkeypatch.setenv("USERPROFILE", str(tmp_path))
monkeypatch.chdir(tmp_path)
for key, val in {
"OS_AUTH_URL": "https://example/v3",
"OS_USER_DOMAIN_NAME": "default",
"OS_USERNAME": "u",
"OS_PASSWORD": "secret",
"OS_PROJECT_ID": "proj",
"OS_REGION_NAME": "ru-7",
"GPU_RENT_AZ": "ru-7a",
"CIVITAI_API_TOKEN": "civ",
"HF_TOKEN": "hf",
"GIT_TOKEN": "git",
"SELECTEL_API_TOKEN": "sel",
}.items():
monkeypatch.setenv(key, val)
cfg = load_config(require_auth=True)
das.reset_session_store_for_tests()
monkeypatch.setattr(
das, "_swarm_base", lambda cfg: ("http://127.0.0.1:17801", "local")
)
monkeypatch.setattr(das, "_session_id", lambda base: ("swarm-sid", None, 3.0))
monkeypatch.setattr(
das,
"_resolve_chat_model",
lambda cfg, base, sid, model=None: ("qwen3-vl:8b", "qwen3-vl:8b"),
)
def fake_http(url, *, method="GET", body=None, timeout=12.0):
if url.endswith("/API/AssistentGetConfig"):
return (
True,
{
"exact": {
"profiles": {"turbo": {"steps": 8, "cfg": 1, "sigma_shift": 1.15}},
"generation": {"profile": "turbo"},
},
"enabled_skills": [],
},
4.0,
)
if url.endswith("/API/AssistentChat"):
n = len((body or {}).get("messages") or [])
reply = (
f"turn-{n}\n"
'```json\n{"prompt":"cat","steps":8,"cfg":1,"actions":["generate"]}\n```'
)
return True, {"reply": reply, "system_chars": 1200, "model": "qwen3-vl:8b"}, 20.0
return False, "unexpected " + url, 1.0
monkeypatch.setattr(das, "_http_json", fake_http)
sock = socket.socket()
sock.bind(("127.0.0.1", 0))
port = sock.getsockname()[1]
sock.close()
srv = debug_api.start_debug_server(cfg, port=port, print_urls=False)
assert srv is not None
base = f"http://127.0.0.1:{port}"
def post(path: str, payload: dict) -> dict:
req = urllib.request.Request(
base + path,
data=json.dumps(payload).encode(),
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req, timeout=10) as resp:
return json.loads(resp.read().decode())
def get(path: str) -> dict:
with urllib.request.urlopen(base + path, timeout=10) as resp:
return json.loads(resp.read().decode())
def delete(path: str) -> dict:
req = urllib.request.Request(base + path, method="DELETE")
with urllib.request.urlopen(req, timeout=10) as resp:
return json.loads(resp.read().decode())
try:
created = post(
"/assistent/session",
{
"persona": "neutral",
"pack": "ordinary",
"context": {"krea_profile": "turbo"},
},
)
assert created["ok"] is True
sid = created["debug_session_id"]
turn1 = post(f"/assistent/session/{sid}/chat", {"message": "какие steps?"})
assert turn1["ok"] is True
assert turn1["trace"]["patch"]["steps"] == 8
assert turn1["trace"]["exact_merge"]["wants_generate"] is True
assert "compact_context" in turn1["trace"]
assert turn1["turn_count"] == 1
turn2 = post(f"/assistent/session/{sid}/chat", {"message": "ещё раз"})
assert turn2["ok"] is True
assert turn2["turn_count"] == 2
assert turn2["session"]["turn_count"] == 2
got = get(f"/assistent/session/{sid}")
assert got["ok"] is True
assert got["session"]["turn_count"] == 2
assert got["last_trace"]["ok"] is True
deleted = delete(f"/assistent/session/{sid}")
assert deleted["ok"] is True
gone = get(f"/assistent/session/{sid}")
assert gone["ok"] is False
finally:
debug_api.stop_debug_server(srv)
+21 -3
View File
@@ -193,24 +193,41 @@ def test_extract_assistent_patch():
def test_chat_eval_mocked_http(monkeypatch, tmp_path):
from gpu_rent import debug_assistent
from gpu_rent import debug_assistent_session as das
_auth_env(monkeypatch, tmp_path)
cfg = load_config(require_auth=True)
das.reset_session_store_for_tests()
monkeypatch.setattr(
debug_assistent,
das,
"_swarm_base",
lambda cfg: ("http://127.0.0.1:17801", "local"),
)
monkeypatch.setattr(
debug_assistent,
das,
"_session_id",
lambda base: ("sess-1", None, 5.0),
)
monkeypatch.setattr(
das,
"_resolve_chat_model",
lambda cfg, base, sid, model=None: ("qwen3-vl:8b", "qwen3-vl:8b"),
)
calls: list[tuple[str, dict]] = []
def fake_http(url, *, method="GET", body=None, timeout=12.0):
if url.endswith("/API/AssistentGetConfig"):
return True, {
"success": True,
"persona": "leonid",
"exact": {
"profiles": {"turbo": {"steps": 8, "cfg": 1, "sigma_shift": 1.15}}
},
"skills": [],
"enabled_skills": [],
}, 8.0
if url.endswith("/API/AssistentListModels"):
return True, {"preferred": "qwen3-vl:8b", "models": ["qwen3-vl:8b"]}, 10.0
if url.endswith("/API/AssistentChat"):
@@ -223,7 +240,7 @@ def test_chat_eval_mocked_http(monkeypatch, tmp_path):
return True, {"success": True, "reply": reply, "model": "qwen3-vl:8b"}, 42.0
return False, "unexpected " + url, 1.0
monkeypatch.setattr(debug_assistent, "_http_json", fake_http)
monkeypatch.setattr(das, "_http_json", fake_http)
out = debug_assistent.run_assistent_chat_eval(
cfg,
@@ -240,6 +257,7 @@ def test_chat_eval_mocked_http(monkeypatch, tmp_path):
assert out["patch"]["cfg"] == 1
assert out["patch"]["aspect"] == "3:4"
assert "turbo" in (out["reply_prose"] or "").lower() or "turbo" in (out["reply"] or "").lower()
assert out.get("trace") and out["trace"].get("exact_merge")
assert calls and calls[0][1]["session_id"] == "sess-1"
assert calls[0][1]["messages"][0]["content"] == "какой checkpoint?"
assert calls[0][1]["includeBase"] is True