mrleo1nid/gpu-rent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update model configurations and enhance LLM runtime logic

Browse Source 
- Revised model URLs and descriptions in `llamacpp-models.example.yaml` and `ollama-models.example.yaml` to reflect new recommendations and vision capabilities.
- Updated the LLM runtime logic to support vision projectors and improved model resolution handling.
- Enhanced the installation script to conditionally include vision projectors when available.
- Added tests to validate the inclusion of vision projectors in model presets and ensure proper URL remapping for deprecated models.
- Improved documentation to clarify model usage and configuration options.
This commit is contained in:
Leonid Pershin
2026-08-21 07:40:47 +03:00
parent 618e6e4806
commit 15b95f04c7
7 changed files with 161 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
llamacpp-models.example.yaml
+10 -9
View File
@@ -1,15 +1,16 @@
# Copy to llamacpp-models.yaml (gitignored). Used when LLM_RUNTIME=llamacpp.
# url = direct HTTPS link to a .gguf (Hugging Face resolve/main/…).
# Empty models: [] → only llama-server, GGUF клади вручную на VM.
# Purpose: prompt-help beside SwarmUI (RU/EN).
# url = direct HTTPS link to a .gguf; mmproj = vision projector (Qwen2.5-VL).
# Empty models: [] → only llama-server, GGUF клади вручную.
# Purpose: prompt-help / vision beside SwarmUI (RU/EN, low refusal).
models:
# Recommended: Qwen2.5 7B abliterate Q4_K_M (~4.7 GB)
- url: https://huggingface.co/bartowski/huihui-ai_Qwen2.5-7B-Instruct-abliterated-GGUF/resolve/main/huihui-ai_Qwen2.5-7B-Instruct-abliterated-Q4_K_M.gguf
# Recommended: Qwen2.5-VL 7B abliterate Q4_K_M + mmproj (~4.7 + 0.8 GB)
- url: https://huggingface.co/mradermacher/Qwen2.5-VL-7B-Instruct-abliterated-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-abliterated.Q4_K_M.gguf
mmproj: https://huggingface.co/mradermacher/Qwen2.5-VL-7B-Instruct-abliterated-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-abliterated.mmproj-Q8_0.gguf
default: true
# Lighter (~2 GB):
# - url: https://huggingface.co/bartowski/Qwen2.5-3B-Instruct-GGUF/resolve/main/Qwen2.5-3B-Instruct-Q4_K_M.gguf
# Text-only abliterate 7B (~4.7 GB):
# - url: https://huggingface.co/RichardErkhov/huihui-ai_-_Qwen2.5-7B-Instruct-abliterated-gguf/resolve/main/Qwen2.5-7B-Instruct-abliterated.Q4_K_M.gguf
# Official stock 7B (more refusals):
# - url: https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF/resolve/main/Qwen2.5-7B-Instruct-Q4_K_M.gguf
# Lighter text (~2 GB):
# - url: https://huggingface.co/bartowski/Qwen2.5-3B-Instruct-GGUF/resolve/main/Qwen2.5-3B-Instruct-Q4_K_M.gguf