mrleo1nid/gpu-rent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update model configurations and enhance LLM runtime logic

Browse Source 
- Revised model URLs and descriptions in `llamacpp-models.example.yaml` and `ollama-models.example.yaml` to reflect new recommendations and vision capabilities.
- Updated the LLM runtime logic to support vision projectors and improved model resolution handling.
- Enhanced the installation script to conditionally include vision projectors when available.
- Added tests to validate the inclusion of vision projectors in model presets and ensure proper URL remapping for deprecated models.
- Improved documentation to clarify model usage and configuration options.
This commit is contained in:
Leonid Pershin
2026-08-21 07:40:47 +03:00
parent 618e6e4806
commit 15b95f04c7
7 changed files with 161 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/llm.md
+11 -9
View File
@@ -84,11 +84,12 @@ $env:OLLAMA_HOST = "http://127.0.0.1:17811"
| # | ключ | tag / смысл |
| --- | --- | --- |
| 1 | **recommended** | `huihui_ai/qwen2.5-abliterate:7b` — RU/EN, ~5 GB |
| 2 | light | `qwen2.5:3b` |
| 3 | stock | `qwen2.5:7b` |
| 4 | alt | другой abliterate 7B |
| 5 | empty | только runtime |
| 1 | **recommended** | `huihui_ai/qwen2.5-vl-abliterated:7b` — vision+RU, ~6 GB |
| 2 | light | VL 3B abliterate (~3 GB) |
| 3 | text | text-only `qwen2.5-abliterate:7b` |
| 4 | stock | `qwen2.5:7b` |
| 5 | alt | другой text abliterate 7B |
| 6 | empty | только runtime |
| — | keep | не трогать yaml (если уже спросили повторно) |
`default: true` в yaml — preferred в логе; pull идёт по всему списку.
@@ -123,10 +124,11 @@ Unit `gpu-rent-ollama` читает `/mnt/swarm_data/.gpu-rent-gpu.json`:
| # | ключ | что |
| --- | --- | --- |
| 1 | **recommended** | Qwen2.5 7B abliterate Q4_K_M (~4.7 GB) |
| 2 | light | Qwen2.5 3B Instruct Q4_K_M |
| 3 | stock | официальный 7B Instruct Q4_K_M |
| 4 | empty | только runtime |
| 1 | **recommended** | Qwen2.5-VL 7B abliterate + mmproj (~4.7+0.8 GB, картинки+RU) |
| 2 | light | Qwen2.5 3B Instruct text Q4_K_M |
| 3 | text | Qwen2.5 7B abliterate text-only |
| 4 | stock | официальный 7B Instruct Q4_K_M |
| 5 | empty | только runtime |
| — | keep | не трогать yaml |
`-ngl` / `-c` — по GPU probe. Для recommended abliterate GGUF нужен **`HF_TOKEN`** в `.env` (иначе 401). Вручную: положи GGUF в models и `systemctl restart gpu-rent-llamacpp`.