- Introduced a local read-only Debug API accessible at `http://127.0.0.1:17821` for diagnostics and agent interactions.
- Updated CLI commands to include `gpu-rent debug` for launching the Debug API.
- Enhanced documentation to reflect the new Debug API features and usage.
- Modified configuration to include `DEBUG_LOCAL_PORT` for easier customization.
- Added tests to ensure Debug API links are correctly generated in access card outputs.
Write preferred chat tag into Assistent/ollama-roles.json so UI and warmup align with manifest default: true.
Co-authored-by: Cursor <cursoragent@cursor.com>
gpu-rent now writes personas/<id>/ on the VM (not legacy personas.json), adds seed-personas/doctor checks, and shortens mid/high keep-alive now that Assistent parks the LLM before Generate.
Co-authored-by: Cursor <cursoragent@cursor.com>
Scan ru-6 by Nova availability zones (a/b/c). First boot uses FLAVOR_SIZE_PRESET=cheap; reruns stay on the disk segment.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Implemented support for parallel embedding in Ollama, allowing for improved performance in chat and memory functions.
- Updated the `ollama-roles.json` and CPU Modelfile to accommodate new features.
- Increased the maximum loaded models and parallel processing limits to 2, optimizing resource usage.
Co-authored-by: Cursor <cursoragent@cursor.com>
Pull nomic-embed-text for Assistent memory, write ollama-roles.json, CPU Modelfile, and raise MAX_LOADED_MODELS/NUM_PARALLEL to 2.
Co-authored-by: Cursor <cursoragent@cursor.com>
A 1-token /api/chat after tags (and again if /api/ps is empty) loads VL weights before the first message. Mid KEEP_ALIVE is 15m so a short image-gen burst does not unload the model.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Updated the `render_access_panel` function to conditionally hide LLM errors when the Ollama model is operational, improving user experience by reducing unnecessary error visibility.
- Introduced a new `_notes_from_disk` function to streamline the retrieval of notes from disk, enhancing state management during GPU environment checks.
- Refactored error handling in the `_bind_access` function to ensure that stack and GPU environment errors are accurately recorded and managed, improving robustness in session state updates.
- Added tests to validate the new behavior of error handling and state management, ensuring that LLM errors are appropriately suppressed when conditions are met.
- Revised README, architecture, CLI, and other documentation to specify that both `Ctrl+C` and `Ctrl+D` now stop the GPU while preserving disk data, correcting previous inaccuracies.
- Enhanced access card and setup instructions to reflect the updated command behavior for better user understanding.
- Updated tests to ensure the new command behaviors are validated and documented correctly, improving overall clarity in GPU management.
- Enhanced the `OllamaTune` class to include a new `context_length` attribute, improving the configuration for different GPU tiers.
- Updated performance tuning logic to set appropriate context lengths for low, mid, high, and ultra tiers, ensuring optimal resource allocation.
- Modified installation scripts to reflect the new context length settings, enhancing the installation process for Ollama.
- Revised documentation to include context length details in the GPU performance table, providing clearer guidance for users.
- Added tests to validate the correct context length settings in various scenarios, ensuring robustness in performance tuning.
- Updated the `provision_llm` function to utilize the `/api/tags` endpoint for verifying available models, improving accuracy in model management.
- Introduced a new `already_have_ollama_tag` function to ensure exact tag matching, preventing mismatches during model checks.
- Enhanced the `pull_stream` function to require a successful status from the API before proceeding, ensuring reliable model downloads.
- Added logic to handle unwritten blob files, improving the robustness of the model pulling process.
- Updated documentation and tests to reflect these changes, ensuring clarity and reliability in Ollama model operations.
- Clarified the behavior of `Ctrl+C` and `Ctrl+D` in the README and other documentation, specifying that `Ctrl+C` only stops the tunnel while keeping the GPU active, and `Ctrl+D` stops the GPU while preserving disk data.
- Enhanced the CLI documentation to reflect these changes, ensuring users understand the implications of these commands during GPU operations.
- Improved the handling of data bindings and remounting logic in the codebase to prevent issues with empty model tabs in the UI.
- Added tests to validate the new command behaviors and ensure proper documentation alignment.
- Updated the `gpu-rent` logic to ensure the `IsInstalled` flag is accurately set based on the presence of ComfyUI, preventing conflicts during installation.
- Improved the handling of the `/mnt/swarm_data` directory to skip creation if it does not exist, enhancing robustness in provisioning.
- Added tests to validate the new installation state checks and ensure proper diagnostics during the installation process, including scenarios for clearing and setting the `IsInstalled` flag.
- Introduced a new `loading_fail_sec` parameter in the `wait_backend_idle` function to handle prolonged loading states, improving error handling for backend readiness.
- Updated the `ensure_dlbackend_bind` function to stop SwarmUI before remounting, preventing target busy errors and ensuring consistent data mounts.
- Enhanced the `recover_errored_backends` function to account for the new remount logic, improving backend recovery processes.
- Refactored tests to validate the new loading failure conditions and ensure proper handling of backend states during diagnostics.
- Introduced the `clean_extra_args` function to filter out unsafe CLI flags and comment garbage from `ExtraArgs`, improving backend configuration integrity.
- Updated the `repair_extra_args_on_disk` function to rewrite corrupted `ExtraArgs` lines in the FDS file, ensuring cleaner backend settings.
- Enhanced the `recover_errored_backends` function to utilize the new sanitization logic, preserving valid `ExtraArgs` during backend reconfiguration.
- Improved the `sanitize_backends_fds` function to drop comment garbage and retain valid entries, enhancing overall backend management.
- Added tests to validate the new sanitization behavior, ensuring robustness in handling `ExtraArgs` during backend operations.
- Updated the `recover_errored_backends` function to preserve `ExtraArgs` during backend reconfiguration, improving error recovery.
- Refined the `sanitize_backends_fds` function to prevent newline merging issues with bare `\\x` entries, ensuring correct FDS formatting.
- Enhanced the `configure_comfy_backend` function to accept and manage `ExtraArgs`, preventing loss of important configuration during recovery.
- Added tests to validate the new sanitization behavior and ensure that bare `\\x` entries remain intact, improving overall backend management.
- Introduced remounting logic for data directories to ensure correct bindings before restarting services, improving reliability during performance tuning.
- Added functions to sanitize and ensure absolute paths for backend scripts, preventing issues with relative paths in bind mounts.
- Enhanced the `patch_backends_extra_args` function to clean up corrupted entries and ensure proper configuration of backend parameters.
- Updated tests to validate the new sanitization and path handling functionalities, ensuring robustness in backend management.
- Introduced functions to ensure the `IsInstalled` flag is set in `Settings.fds`, preventing the UI from displaying the /Install wizard.
- Added logic to create or patch `Settings.fds` with installation details, including the installation date and version.
- Updated the `install_swarm_comfy.py` script to call the new function, ensuring the installation state is correctly managed during backend operations.
- Added tests to verify the presence of the `IsInstalled` flag in the relevant scripts and ensure proper functionality during installation checks.
- Added a `--update` option to the `up` command in the CLI for forced updates during warm ACTIVE states.
- Improved the idle-killer logic with new functions to check for reusable credentials and refresh settings on existing remote credentials.
- Enhanced the `seed_civitai` function to implement caching for model jobs, optimizing the provisioning process.
- Updated the installation script for Ollama to prevent unnecessary restarts when the service configuration has not changed.
- Added diagnostics for model job caching and improved error handling in various functions.
- Introduced a new `delete_backend` function to facilitate backend removal during recovery processes.
- Improved the `recover_errored_backends` function to handle backend reconfiguration and recreation more effectively, including enhanced logging for diagnostics.
- Added a new `add_comfy_selfstart` function to streamline the addition of new backends.
- Updated the `comfy_start_script` function to prefer absolute paths for better reliability.
- Enhanced diagnostics in `swarm_diag.py` to include detailed backend settings and mount status.
- Added tests to validate the new backend management features and ensure robust error handling.
- Updated the `wait_backend_idle` function to handle disabled backends, introducing a new failure condition and diagnostics collection for better error handling.
- Refactored the `install_swarm_comfy` script to include a new `configure_comfy_backend` function, streamlining backend configuration and enabling better management of backend states.
- Added tests to validate the new functionality for handling disabled backends and ensuring robust diagnostics, improving overall backend management.
- Improved the `install_swarm_comfy` function to handle empty backend states more effectively, introducing recovery mechanisms and enhanced logging for better visibility.
- Updated the `tune_swarm_perf` function to always sanitize backend FDS corruption, ensuring consistent performance tuning.
- Added new tests to validate the functionality of backend recovery and FDS sanitization, ensuring robustness in backend management.
- Introduced the `file_size_bytes` function to convert Civitai model sizes from kilobytes to bytes, improving data handling.
- Updated the `seed_civitai` function to include file size in job definitions, enhancing model processing efficiency.
- Enhanced the `should_skip` function to utilize expected size for faster decision-making during job processing.
- Added tests for new functionality, ensuring robustness in file size handling and job processing logic.
- Enhanced the `with_token` function to support Gitea and Forgejo URLs by constructing the appropriate authentication format for generic HTTPS.
- Updated tests to validate the new functionality, ensuring correct token injection for Gitea URLs while maintaining existing behavior for GitHub and GitLab.
- Improved overall robustness of token handling across different platforms.
- Introduced the `pick_preview_image` function to extract the first usable preview image URL and its suffix from model version data.
- Updated the `seed_civitai` function to include preview image URLs and destinations in job definitions, improving model handling.
- Implemented the `ensure_preview` function to download missing preview images during job processing, enhancing user experience.
- Added tests for `pick_preview_image` to ensure correct functionality across various scenarios, ensuring robustness in image handling.
- Introduced functions `swarm_git_ok`, `clone_swarmui`, and `update_swarmui` to streamline the cloning and updating process of SwarmUI.
- Improved error handling during the update process, ensuring a fallback to cloning if the update fails.
- Enhanced logging messages for better visibility during cloning and updating operations.
- Updated tests to verify the presence of new functions and logging messages in the bootstrap script.
- Added `wget` to the package installation list for improved script functionality.
- Implemented a fallback mechanism using `curl` for .NET installation if the primary script fails, enhancing robustness.
- Updated ownership change for the installation directory to ensure proper permissions for the user.
- Enhanced logging for error scenarios during .NET installation, providing clearer feedback for troubleshooting.
- Added tests to verify the new fallback functionality and ownership changes in the bootstrap script.
- Introduced a new CLI command `diag` to collect diagnostics from SwarmUI/Comfy, including API, journal, and paths.
- Enhanced the `ensure_swarm_comfy_installed` function to include diagnostic script handling for errored backends.
- Updated `wait_backend_idle` to trigger diagnostics when backends are in an errored state, improving error recovery.
- Implemented fallback mechanisms for diagnostics in the `run_diagnostics` function, ensuring better visibility into backend issues.
- Added tests to validate the new diagnostic functionality and error handling, ensuring robustness in backend management.
- Updated the idle-killer logic to treat SwarmUI `empty` and `disabled` states as busy, preventing unnecessary idle time during provisioning.
- Enhanced the `wait_backend_idle` function to recognize suspended backends as ready, improving resource utilization and user feedback.
- Refined the `install_swarm_comfy` script to skip installation when backends are already present, streamlining the setup process.
- Improved the `resolve_llm_runtime` function to prioritize live configuration over stale state notes, ensuring accurate runtime detection.
- Added tests to validate the new backend status handling and idle management logic, ensuring robustness and reliability.
- Replaced direct calls to `venv/bin/pip` with `python -m pip` to avoid execution errors on network mounts and improve compatibility.
- Introduced a new function `find_comfy_python` to locate the appropriate Python executable for ComfyUI installations.
- Updated the `pip_install_sage` function to utilize the new Python handling, enhancing installation reliability.
- Added tests to verify the new behavior of Python environment detection and installation processes, ensuring robustness.
- Enhanced documentation to clarify the transition from 'Idle' to 'ready (running)' for backend states, improving user understanding of system readiness.
- Updated logging messages in the notification system to reflect the new backend status terminology, ensuring accurate feedback during operations.
- Refined access link collection logic to better handle tunneled and non-tunneled scenarios, enhancing user experience.
- Improved tests to validate the new backend status handling and ensure accurate reporting of access links and notifications.
- Updated the `collect_access_links` function to provide clearer user-facing endpoint labels and notes, particularly for non-tunneled scenarios.
- Improved logging messages in the provisioning process to reflect the status of the SwarmUI and Ollama API, enhancing user feedback during setup.
- Added human-readable status messages for backend loading and running states, improving clarity during the waiting process.
- Updated tests to verify the new behavior and ensure accurate reporting of access links and backend statuses.
- Added functions for formatting byte sizes, displaying progress bars, and calculating directory sizes to improve user feedback during installation.
- Implemented a stage detection mechanism to provide real-time updates on the installation process of ComfyUI.
- Enhanced logging to capture recent journal lines related to ComfyUI installation, improving visibility into the installation status.
- Updated tests to verify the presence of new functionality in the installation script, ensuring robustness and reliability.
- Added a new function to ensure headless installation of ComfyUI when backends are empty, enhancing the setup process for SwarmUI.
- Updated documentation to clarify the installation flow and backend readiness checks, ensuring users understand the requirements for a successful setup.
- Enhanced backend status checks to differentiate between 'empty' and 'idle' states, improving error handling and user feedback during provisioning.
- Adjusted logging messages to provide clearer insights into the installation and backend status processes.
- Removed the direct injection of environment variables into the script, preventing issues with `from __future__` imports.
- Updated the `run_python` function to accept an `env` parameter for passing runtime environment variables, enhancing script execution control.
- Adjusted tests to verify the new behavior, ensuring that environment variables are correctly set without modifying the script content.
- Revised model descriptions in `ollama-models.example.yaml` to emphasize uncensored and abliterated requirements, enhancing user understanding.
- Updated documentation in `llm.md` to reflect changes in model tags and their meanings, ensuring accurate guidance for users.
- Modified CLI help messages to clarify the nature of presets, reinforcing that all options are abliterate models with Russian support.
- Enhanced the `llm_runtime.py` file to align preset labels with the updated model descriptions, improving consistency across the codebase.
- Removed references to llamacpp from configuration files, scripts, and documentation, streamlining the LLM setup process to focus solely on Ollama.
- Updated environment variables and paths to eliminate llamacpp-related entries, ensuring clarity in the configuration.
- Adjusted CLI commands and help messages to reflect the removal of llamacpp, enhancing user experience and reducing confusion.
- Revised documentation to provide clear guidance on using Ollama exclusively, including updates to setup instructions and runtime options.
- Added a new configuration option `UP_STOP_ON_FAIL` to control whether the GPU should be stopped automatically if the `up` command fails, enhancing user control over resource management.
- Updated the CLI to include a `--keep-on-fail` flag, allowing users to prevent GPU shutdown during installation errors.
- Enhanced the installation scripts and documentation to reflect these changes, providing clearer guidance on the new behavior and configuration options.
- Improved error handling in the CLI to ensure proper cleanup of resources in case of failure, preventing unexpected billing for unused GPU resources.
- Added new environment variables in `env.example` and `gpu-rent.vars.example` for fine-tuning LLM settings, including CUDA build options and model version pinning.
- Updated `llm.md` documentation to include detailed descriptions of new configuration options and usage cases for LLM setups.
- Enhanced the `provision.py` script to forward new environment variables during remote installations, improving the installation process for LLM components.
- Modified the `install_llamacpp.sh` script to support conditional CUDA builds and asset URL overrides, ensuring better compatibility with various environments.
- Improved logging in the installation scripts to provide clearer feedback during the setup process.
- Updated the `_print_checks` function to replace console prints with logging functions for better traceability.
- Introduced timing functionality in the `doctor`, `dry_run`, and `up` functions to log the duration of preflight checks.
- Modified the `wait_ssh` function to accept a logging callback, improving SSH wait feedback.
- Enhanced the `mark` method in `PhaseTimes` to log phase durations, aiding in performance analysis.
- Updated various remote scripts to ensure error messages are printed to stderr for better error handling.
- Added a new function `pick_llamacpp_linux_asset_url` to select appropriate Linux release assets, prioritizing Ubuntu CUDA and Vulkan options while excluding Windows and macOS binaries.
- Updated the installation script to build `llama-server` from source when Linux CUDA binaries are unavailable, improving compatibility and flexibility.
- Revised documentation to reflect changes in asset handling and installation procedures.
- Added tests to validate the new asset selection logic, ensuring correct behavior in various scenarios.
- Revised model URLs and descriptions in `llamacpp-models.example.yaml` and `ollama-models.example.yaml` to reflect new recommendations and vision capabilities.
- Updated the LLM runtime logic to support vision projectors and improved model resolution handling.
- Enhanced the installation script to conditionally include vision projectors when available.
- Added tests to validate the inclusion of vision projectors in model presets and ensure proper URL remapping for deprecated models.
- Improved documentation to clarify model usage and configuration options.
- Introduced support for Hugging Face API integration, allowing fallback model resolution when Civitai fails.
- Updated configuration to include `HF_TOKEN` and `HF_TOKEN_PATH` for authentication.
- Enhanced model capture logic to differentiate between Civitai and Hugging Face sources.
- Improved error handling for model downloads, providing clearer messages for authentication issues.
- Updated documentation to reflect new environment variables and usage instructions for Hugging Face integration.
- Added tests to validate the new fallback mechanism and ensure robust model resolution.
- Updated the `_log_default` function to handle messages starting with `\r` for in-place updates.
- Introduced `split_ssh_stream` to differentiate between line and progress events in SSH output.
- Implemented `feed_ssh_log` to dispatch log messages based on event type.
- Refactored `_stream_pty_output` to utilize the new logging and event handling functions.
- Enhanced the `log` function in `term.py` to support progress updates, ensuring a cleaner output experience.
- Updated `DownloadProgress` classes in `civitai_fetch.py` and `llamacpp_fetch.py` to print progress in place, improving user feedback during downloads.
- Introduced `fmt_bytes` function for human-readable byte size formatting.
- Added `DownloadProgress` class to track and display download progress with speed and completion percentage.
- Updated `download` functions in `civitai_fetch.py` and `llamacpp_fetch.py` to utilize progress tracking.
- Enhanced `ollama_pull.py` to support streaming progress updates during model pulls.
- Updated tests to validate new formatting and progress tracking functionalities, ensuring accurate output and user feedback.
- Updated CLI documentation to reflect the new handling of `CIVITAI_API_TOKEN`, which is now automatically passed to SwarmUI user settings during startup.
- Improved the `render_access_panel` function to include additional warnings for idle-killer failures and stack errors, enhancing user feedback.
- Introduced a new function `seed_swarmui_api_keys` to manage API key injection into SwarmUI, ensuring seamless integration with the Model Downloader.
- Enhanced GPU environment verification logic to include fail-fast checks for critical components like CUDA, improving error handling and user notifications.
- Updated tests to validate the new API key handling and access panel behavior, ensuring robustness in the integration process.
- Marked critical bugs as resolved in the review documentation, including changes to the `arm_idle_killer` function to raise errors on credential creation failures and ensure proper file permissions for JSON credentials.
- Introduced a new `_try_arm_idle_killer` function in `provision.py` to manage idle-killer state more effectively, ensuring it arms correctly during provisioning.
- Updated the `swarm_busy` function in `remote/idle_killer.py` to allow idle state after a specified duration of Swarm unavailability, preventing unnecessary billing.
- Enhanced performance tuning logic in `tune_swarm_perf.py` to ensure proper handling of pip installation success before applying extra arguments.
- Added tests to validate the new idle-killer behavior and swarm management logic, ensuring robustness in handling idle states and error conditions.
- Rearranged the order of verification steps in the CLI documentation for clarity.
- Enhanced the `test_verify_gpu_env` tests to improve logging and error assertions, ensuring accurate feedback on GPU stack checks.
- Added a new test to skip the torch requirement when only LLM is enabled, reflecting updated behavior in GPU environment verification.
Leonid Pershin
