Enhance GPU probing and performance tuning in provisioning

- Introduced GPU probing functionality to gather and store GPU specifications in `/mnt/swarm_data/.gpu-rent-gpu.json`, aiding in performance tuning.
- Updated `install_ollama.sh` and `install_llamacpp.sh` to utilize GPU information for configuring optimal runtime parameters.
- Enhanced `provision.py` to include GPU probing and performance tuning logic, ensuring better resource allocation for LLM operations.
- Improved documentation in `decisions.md`, `llm.md`, and `swarmui.md` to reflect changes in GPU handling and performance tuning processes.
- Added new tests to validate the GPU probing and model resolution logic, ensuring robustness in handling various GPU configurations.
This commit is contained in:
Leonid Pershin
2026-08-21 06:10:24 +03:00
parent 603165a4ba
commit 2ccb03f7d2
16 changed files with 1270 additions and 138 deletions
+14 -2
View File
@@ -746,7 +746,13 @@ def capture_models_cmd(
kind_filter=kind,
log=lambda m: console.print(m),
)
print_report(report, lambda m: console.print(m), dry_run=dry_run)
print_report(
report,
lambda m: console.print(m),
dry_run=dry_run,
show_models=True,
show_extensions=False,
)
except GpuRentError as exc:
_die(exc)
@@ -767,7 +773,13 @@ def capture_extensions_cmd(
dry_run=dry_run,
log=lambda m: console.print(m),
)
print_report(report, lambda m: console.print(m), dry_run=dry_run)
print_report(
report,
lambda m: console.print(m),
dry_run=dry_run,
show_models=False,
show_extensions=True,
)
except GpuRentError as exc:
_die(exc)