Implement idle-killer enhancements and swarm management improvements

- Marked critical bugs as resolved in the review documentation, including changes to the `arm_idle_killer` function to raise errors on credential creation failures and ensure proper file permissions for JSON credentials.
- Introduced a new `_try_arm_idle_killer` function in `provision.py` to manage idle-killer state more effectively, ensuring it arms correctly during provisioning.
- Updated the `swarm_busy` function in `remote/idle_killer.py` to allow idle state after a specified duration of Swarm unavailability, preventing unnecessary billing.
- Enhanced performance tuning logic in `tune_swarm_perf.py` to ensure proper handling of pip installation success before applying extra arguments.
- Added tests to validate the new idle-killer behavior and swarm management logic, ensuring robustness in handling idle states and error conditions.
This commit is contained in:
Leonid Pershin
2026-08-21 07:04:09 +03:00
parent d409e2e154
commit 1ec615c03e
7 changed files with 268 additions and 104 deletions
+9 -8
View File
@@ -146,14 +146,15 @@ def arm_idle_killer(
log: Log,
) -> None:
if not server_id:
log("idle-killer: нет server_id — пропуск")
return
try:
creds = create_application_credential(conn, cfg, server_id, log)
except CloudError as exc:
log(f"idle-killer слеп: {exc}")
return
put_text(cfg, host, "/tmp/gpu-rent-idle-killer.json", json.dumps(creds, indent=2) + "\n")
raise CloudError("idle-killer: нет server_id")
creds = create_application_credential(conn, cfg, server_id, log)
put_text(
cfg,
host,
"/tmp/gpu-rent-idle-killer.json",
json.dumps(creds, indent=2) + "\n",
mode=0o600,
)
run_ssh(
cfg,
host,