Refactor LLM configuration to remove llamacpp support
- Removed references to llamacpp from configuration files, scripts, and documentation, streamlining the LLM setup process to focus solely on Ollama. - Updated environment variables and paths to eliminate llamacpp-related entries, ensuring clarity in the configuration. - Adjusted CLI commands and help messages to reflect the removal of llamacpp, enhancing user experience and reducing confusion. - Revised documentation to provide clear guidance on using Ollama exclusively, including updates to setup instructions and runtime options.
This commit is contained in:
+8
-43
@@ -31,59 +31,24 @@
|
||||
# Подробнее: docs/llm.md
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# --- A) Default: SwarmUI + llama.cpp (auto: CUDA если nvcc на VM, иначе Vulkan) ---
|
||||
# LLM_RUNTIME=llamacpp
|
||||
# --- A) Default: SwarmUI + Ollama ---
|
||||
# LLM_RUNTIME=ollama
|
||||
# ENABLE_SWARMUI=true
|
||||
# LLAMACPP_TAG=b10545
|
||||
# LLAMACPP_BACKEND=auto
|
||||
# OLLAMA_LOCAL_PORT=17811
|
||||
|
||||
# --- B) Только LLM, без SwarmUI ---
|
||||
# WORKLOAD=llm
|
||||
# LLM_RUNTIME=llamacpp
|
||||
# LLAMACPP_TAG=b10545
|
||||
# LLM_RUNTIME=ollama
|
||||
|
||||
# --- C) Явно CUDA (и снести старый Vulkan-бинарь) ---
|
||||
# LLM_RUNTIME=llamacpp
|
||||
# LLAMACPP_TAG=b10545
|
||||
# LLAMACPP_BACKEND=cuda
|
||||
# LLAMACPP_FORCE_REINSTALL=1
|
||||
|
||||
# --- C2) Явно быстрый Vulkan, без compile ---
|
||||
# LLM_RUNTIME=llamacpp
|
||||
# LLAMACPP_BACKEND=vulkan
|
||||
# LLAMACPP_BUILD_CUDA=0
|
||||
|
||||
# --- D) Свой бинарь / pin URL (supply-chain) ---
|
||||
# LLM_RUNTIME=llamacpp
|
||||
# LLAMACPP_ASSET_URL=https://github.com/ggml-org/llama.cpp/releases/download/b10545/llama-b10545-bin-ubuntu-vulkan-x64.tar.gz
|
||||
# LLAMACPP_SHA256=<sha256 архива>
|
||||
# LLAMACPP_FORCE_REINSTALL=1
|
||||
|
||||
# --- E) Делим 4090 со SwarmUI: меньше слоёв / короче контекст ---
|
||||
# LLM_RUNTIME=llamacpp
|
||||
# LLAMACPP_NGL=40
|
||||
# LLAMACPP_CTX=4096
|
||||
# LLAMACPP_EXTRA_ARGS=--flash-attn on
|
||||
|
||||
# --- F) Длинный контекст / почти весь VRAM под LLM (llm-only) ---
|
||||
# WORKLOAD=llm
|
||||
# LLM_RUNTIME=llamacpp
|
||||
# LLAMACPP_NGL=99
|
||||
# LLAMACPP_CTX=32768
|
||||
# LLAMACPP_EXTRA_ARGS=--parallel 1
|
||||
|
||||
# --- G) Ollama вместо llama.cpp + pin версии ---
|
||||
# --- C) Pin версии Ollama (supply-chain) ---
|
||||
# LLM_RUNTIME=ollama
|
||||
# OLLAMA_VERSION=0.6.5
|
||||
# OLLAMA_SHA256=<sha256 of ollama-linux-amd64.tgz>
|
||||
# OLLAMA_LOCAL_PORT=17811
|
||||
|
||||
# --- H) Выключить LLM (гасит unit’ы на следующем up) ---
|
||||
# --- D) Выключить LLM (гасит unit’ы на следующем up) ---
|
||||
# LLM_RUNTIME=none
|
||||
|
||||
# Порты туннеля (localhost):
|
||||
# Порт туннеля (localhost):
|
||||
# OLLAMA_LOCAL_PORT=17811
|
||||
# LLAMACPP_LOCAL_PORT=17812
|
||||
# Свой путь к манифесту GGUF/pull:
|
||||
# LLAMACPP_MODELS_MANIFEST=
|
||||
# Свой путь к манифесту pull:
|
||||
# OLLAMA_MODELS_MANIFEST=
|
||||
|
||||
Reference in New Issue
Block a user