mrleo1nid/gpu-rent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Warm Ollama into VRAM on up and tunnel so Assistent chat is not cold.

Browse Source 
A 1-token /api/chat after tags (and again if /api/ps is empty) loads VL weights before the first message. Mid KEEP_ALIVE is 15m so a short image-gen burst does not unload the model.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Leonid Pershin
2026-08-21 20:47:14 +03:00
co-authored by Cursor
parent 79cb0a7e25
commit ffe5a031b4
12 changed files with 465 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_tunnel_watch.py
+1
View File
@@ -141,6 +141,7 @@ def _stub_tunnel(monkeypatch) -> None:
monkeypatch.setattr("gpu_rent.tunnel._ssh_tunnel_forwarder", lambda: object)
monkeypatch.setattr("gpu_rent.tunnel._start_forwarder", lambda *a, **k: _Fwd())
monkeypatch.setattr("gpu_rent.ready.verify_stack_local", lambda *a, **k: [])
monkeypatch.setattr("gpu_rent.llm_runtime.maybe_warmup_ollama_local", lambda *a, **k: None)
monkeypatch.setattr("gpu_rent.tunnel.load_state", lambda: st)
monkeypatch.setattr("gpu_rent.tunnel.save_state", lambda s: None)
monkeypatch.setattr("gpu_rent.access_card.print_access_card", lambda *a, **k: None)