Add diagnostics command and enhance error handling for backend states

- Introduced a new CLI command `diag` to collect diagnostics from SwarmUI/Comfy, including API, journal, and paths.
- Enhanced the `ensure_swarm_comfy_installed` function to include diagnostic script handling for errored backends.
- Updated `wait_backend_idle` to trigger diagnostics when backends are in an errored state, improving error recovery.
- Implemented fallback mechanisms for diagnostics in the `run_diagnostics` function, ensuring better visibility into backend issues.
- Added tests to validate the new diagnostic functionality and error handling, ensuring robustness in backend management.
This commit is contained in:
Leonid Pershin
2026-08-21 10:17:47 +03:00
parent 1785ab369c
commit ec4663c04f
7 changed files with 443 additions and 11 deletions
+22
View File
@@ -712,6 +712,28 @@ def ssh() -> None:
_die(exc)
@app.command("diag")
def diag_cmd(
lines: int = typer.Option(80, "--lines", "-n", help="Строк journal в отчёте"),
) -> None:
"""Снять диагностику SwarmUI/Comfy с VM (API + journal + paths)."""
try:
cfg = load_config(require_auth=True)
state = load_state()
if not state.floating_ip:
raise GpuRentError("нет IP — VM не поднята")
from gpu_rent.ready import collect_swarm_diagnostics
# lines reserved for future; swarm_diag uses fixed 80 for now
_ = lines
collect_swarm_diagnostics(cfg, state.floating_ip, console.print)
console.print(
"[dim]/mnt/swarm_data/.gpu-rent-last-diag.txt на VM[/dim]"
)
except GpuRentError as exc:
_die(exc)
@app.command()
def logs(
unit: Optional[str] = typer.Option(