Enhance LLM configuration and installation scripts for improved flexibility

- Added new environment variables in `env.example` and `gpu-rent.vars.example` for fine-tuning LLM settings, including CUDA build options and model version pinning.
- Updated `llm.md` documentation to include detailed descriptions of new configuration options and usage cases for LLM setups.
- Enhanced the `provision.py` script to forward new environment variables during remote installations, improving the installation process for LLM components.
- Modified the `install_llamacpp.sh` script to support conditional CUDA builds and asset URL overrides, ensuring better compatibility with various environments.
- Improved logging in the installation scripts to provide clearer feedback during the setup process.
This commit is contained in:
Leonid Pershin
2026-08-21 08:19:37 +03:00
parent ccba40a228
commit 91d2ce0fab
7 changed files with 311 additions and 79 deletions
+11
View File
@@ -46,6 +46,17 @@ OLLAMA_LOCAL_PORT=17811
LLAMACPP_LOCAL_PORT=17812
# OLLAMA_MODELS_MANIFEST=
# LLAMACPP_MODELS_MANIFEST=
# Pin / тонкая настройка LLM (несecреты; удобнее в gpu-rent.vars — см. кейсы AH):
# LLAMACPP_TAG=b10545
# LLAMACPP_BUILD_CUDA=1
# LLAMACPP_FORCE_REINSTALL=1
# LLAMACPP_ASSET_URL=
# LLAMACPP_SHA256=
# LLAMACPP_NGL=40
# LLAMACPP_CTX=4096
# LLAMACPP_EXTRA_ARGS=--flash-attn on
# OLLAMA_VERSION=0.6.5
# OLLAMA_SHA256=
# CIVITAI_API_TOKEN=
# CIVITAI_API_HOST=civitai.red
# Hugging Face (GGUF / gated HF URLs / capture fallback metadata):