Warm Ollama into VRAM on up and tunnel so Assistent chat is not cold.

A 1-token /api/chat after tags (and again if /api/ps is empty) loads VL weights before the first message. Mid KEEP_ALIVE is 15m so a short image-gen burst does not unload the model.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Leonid Pershin
2026-08-21 20:47:14 +03:00
co-authored by Cursor
parent 79cb0a7e25
commit ffe5a031b4
12 changed files with 465 additions and 4 deletions
+2
View File
@@ -86,6 +86,7 @@ def test_install_ollama_skips_restart_when_unit_unchanged():
text = files("gpu_rent.remote").joinpath("install_ollama.sh").read_text(encoding="utf-8")
assert "cmp -s" in text
assert "skip restart" in text
assert '"mid", 10 * 1024**3, "15m"' in text
def test_provision_llm_skips_on_api_tags_not_cli_list():
@@ -97,6 +98,7 @@ def test_provision_llm_skips_on_api_tags_not_cli_list():
assert "_ollama_api_tags" in text
assert "awk 'NR>1" not in text
assert "GPU не гасим" in text
assert "ollama_warmup.py" in text
assert "без моделей из ollama-models.yaml" not in text