Enhance GPU probing and performance tuning in provisioning
- Introduced GPU probing functionality to gather and store GPU specifications in `/mnt/swarm_data/.gpu-rent-gpu.json`, aiding in performance tuning. - Updated `install_ollama.sh` and `install_llamacpp.sh` to utilize GPU information for configuring optimal runtime parameters. - Enhanced `provision.py` to include GPU probing and performance tuning logic, ensuring better resource allocation for LLM operations. - Improved documentation in `decisions.md`, `llm.md`, and `swarmui.md` to reflect changes in GPU handling and performance tuning processes. - Added new tests to validate the GPU probing and model resolution logic, ensuring robustness in handling various GPU configurations.
This commit is contained in:
+14
-2
@@ -746,7 +746,13 @@ def capture_models_cmd(
|
||||
kind_filter=kind,
|
||||
log=lambda m: console.print(m),
|
||||
)
|
||||
print_report(report, lambda m: console.print(m), dry_run=dry_run)
|
||||
print_report(
|
||||
report,
|
||||
lambda m: console.print(m),
|
||||
dry_run=dry_run,
|
||||
show_models=True,
|
||||
show_extensions=False,
|
||||
)
|
||||
except GpuRentError as exc:
|
||||
_die(exc)
|
||||
|
||||
@@ -767,7 +773,13 @@ def capture_extensions_cmd(
|
||||
dry_run=dry_run,
|
||||
log=lambda m: console.print(m),
|
||||
)
|
||||
print_report(report, lambda m: console.print(m), dry_run=dry_run)
|
||||
print_report(
|
||||
report,
|
||||
lambda m: console.print(m),
|
||||
dry_run=dry_run,
|
||||
show_models=False,
|
||||
show_extensions=True,
|
||||
)
|
||||
except GpuRentError as exc:
|
||||
_die(exc)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user