Enhance LLM configuration and installation scripts for improved flexibility
- Added new environment variables in `env.example` and `gpu-rent.vars.example` for fine-tuning LLM settings, including CUDA build options and model version pinning. - Updated `llm.md` documentation to include detailed descriptions of new configuration options and usage cases for LLM setups. - Enhanced the `provision.py` script to forward new environment variables during remote installations, improving the installation process for LLM components. - Modified the `install_llamacpp.sh` script to support conditional CUDA builds and asset URL overrides, ensuring better compatibility with various environments. - Improved logging in the installation scripts to provide clearer feedback during the setup process.
This commit is contained in:
+11
@@ -46,6 +46,17 @@ OLLAMA_LOCAL_PORT=17811
|
||||
LLAMACPP_LOCAL_PORT=17812
|
||||
# OLLAMA_MODELS_MANIFEST=
|
||||
# LLAMACPP_MODELS_MANIFEST=
|
||||
# Pin / тонкая настройка LLM (несecреты; удобнее в gpu-rent.vars — см. кейсы A–H):
|
||||
# LLAMACPP_TAG=b10545
|
||||
# LLAMACPP_BUILD_CUDA=1
|
||||
# LLAMACPP_FORCE_REINSTALL=1
|
||||
# LLAMACPP_ASSET_URL=
|
||||
# LLAMACPP_SHA256=
|
||||
# LLAMACPP_NGL=40
|
||||
# LLAMACPP_CTX=4096
|
||||
# LLAMACPP_EXTRA_ARGS=--flash-attn on
|
||||
# OLLAMA_VERSION=0.6.5
|
||||
# OLLAMA_SHA256=
|
||||
# CIVITAI_API_TOKEN=
|
||||
# CIVITAI_API_HOST=civitai.red
|
||||
# Hugging Face (GGUF / gated HF URLs / capture fallback metadata):
|
||||
|
||||
Reference in New Issue
Block a user