Leonid Pershin
64f93b4bf6
- Added support for `llamacpp-models.yaml` in `.gitignore` and implemented logic to copy it in `gpu-rent.ps1` and `gpu-rent.sh`. - Enhanced CLI to prompt for llama.cpp model presets during setup and execution, improving user experience. - Updated configuration handling to include `llamacpp_models_manifest` and related functions for managing llama.cpp models. - Improved documentation in `cli.md` and `llm.md` to reflect changes in llama.cpp integration and model management. - Refactored provisioning logic to handle llama.cpp model downloads and configurations effectively.
16 lines
820 B
YAML
16 lines
820 B
YAML
# Copy to llamacpp-models.yaml (gitignored). Used when LLM_RUNTIME=llamacpp.
|
|
# url = direct HTTPS link to a .gguf (Hugging Face resolve/main/…).
|
|
# Empty models: [] → only llama-server, GGUF клади вручную на VM.
|
|
# Purpose: prompt-help beside SwarmUI (RU/EN).
|
|
|
|
models:
|
|
# Recommended: Qwen2.5 7B abliterate Q4_K_M (~4.7 GB)
|
|
- url: https://huggingface.co/bartowski/huihui-ai_Qwen2.5-7B-Instruct-abliterated-GGUF/resolve/main/huihui-ai_Qwen2.5-7B-Instruct-abliterated-Q4_K_M.gguf
|
|
default: true
|
|
|
|
# Lighter (~2 GB):
|
|
# - url: https://huggingface.co/bartowski/Qwen2.5-3B-Instruct-GGUF/resolve/main/Qwen2.5-3B-Instruct-Q4_K_M.gguf
|
|
|
|
# Official stock 7B (more refusals):
|
|
# - url: https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF/resolve/main/Qwen2.5-7B-Instruct-Q4_K_M.gguf
|