Write preferred chat tag into Assistent/ollama-roles.json so UI and warmup align with manifest default: true.
Co-authored-by: Cursor <cursoragent@cursor.com>
gpu-rent now writes personas/<id>/ on the VM (not legacy personas.json), adds seed-personas/doctor checks, and shortens mid/high keep-alive now that Assistent parks the LLM before Generate.
Co-authored-by: Cursor <cursoragent@cursor.com>
Scan ru-6 by Nova availability zones (a/b/c). First boot uses FLAVOR_SIZE_PRESET=cheap; reruns stay on the disk segment.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Implemented support for parallel embedding in Ollama, allowing for improved performance in chat and memory functions.
- Updated the `ollama-roles.json` and CPU Modelfile to accommodate new features.
- Increased the maximum loaded models and parallel processing limits to 2, optimizing resource usage.
Co-authored-by: Cursor <cursoragent@cursor.com>
Pull nomic-embed-text for Assistent memory, write ollama-roles.json, CPU Modelfile, and raise MAX_LOADED_MODELS/NUM_PARALLEL to 2.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Introduced `assistent-personas.yaml` to the project and updated the `.gitignore` accordingly.
- Implemented functions to seed and merge assistent personas from local files to the VM.
- Enhanced the model capture process to include merging of wanted models from the VM into `models.yaml`.
- Updated documentation to reflect changes in the assistent personas and model management processes.
Co-authored-by: Cursor <cursoragent@cursor.com>
A 1-token /api/chat after tags (and again if /api/ps is empty) loads VL weights before the first message. Mid KEEP_ALIVE is 15m so a short image-gen burst does not unload the model.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Updated the `render_access_panel` function to conditionally hide LLM errors when the Ollama model is operational, improving user experience by reducing unnecessary error visibility.
- Introduced a new `_notes_from_disk` function to streamline the retrieval of notes from disk, enhancing state management during GPU environment checks.
- Refactored error handling in the `_bind_access` function to ensure that stack and GPU environment errors are accurately recorded and managed, improving robustness in session state updates.
- Added tests to validate the new behavior of error handling and state management, ensuring that LLM errors are appropriately suppressed when conditions are met.
- Revised README, architecture, CLI, and other documentation to specify that both `Ctrl+C` and `Ctrl+D` now stop the GPU while preserving disk data, correcting previous inaccuracies.
- Enhanced access card and setup instructions to reflect the updated command behavior for better user understanding.
- Updated tests to ensure the new command behaviors are validated and documented correctly, improving overall clarity in GPU management.
- Enhanced the `OllamaTune` class to include a new `context_length` attribute, improving the configuration for different GPU tiers.
- Updated performance tuning logic to set appropriate context lengths for low, mid, high, and ultra tiers, ensuring optimal resource allocation.
- Modified installation scripts to reflect the new context length settings, enhancing the installation process for Ollama.
- Revised documentation to include context length details in the GPU performance table, providing clearer guidance for users.
- Added tests to validate the correct context length settings in various scenarios, ensuring robustness in performance tuning.
- Updated the `provision_llm` function to utilize the `/api/tags` endpoint for verifying available models, improving accuracy in model management.
- Introduced a new `already_have_ollama_tag` function to ensure exact tag matching, preventing mismatches during model checks.
- Enhanced the `pull_stream` function to require a successful status from the API before proceeding, ensuring reliable model downloads.
- Added logic to handle unwritten blob files, improving the robustness of the model pulling process.
- Updated documentation and tests to reflect these changes, ensuring clarity and reliability in Ollama model operations.
- Clarified the behavior of `Ctrl+C` and `Ctrl+D` in the README and other documentation, specifying that `Ctrl+C` only stops the tunnel while keeping the GPU active, and `Ctrl+D` stops the GPU while preserving disk data.
- Enhanced the CLI documentation to reflect these changes, ensuring users understand the implications of these commands during GPU operations.
- Improved the handling of data bindings and remounting logic in the codebase to prevent issues with empty model tabs in the UI.
- Added tests to validate the new command behaviors and ensure proper documentation alignment.
- Updated the `gpu-rent` logic to ensure the `IsInstalled` flag is accurately set based on the presence of ComfyUI, preventing conflicts during installation.
- Improved the handling of the `/mnt/swarm_data` directory to skip creation if it does not exist, enhancing robustness in provisioning.
- Added tests to validate the new installation state checks and ensure proper diagnostics during the installation process, including scenarios for clearing and setting the `IsInstalled` flag.
- Added a loop to check for the presence of ComfyUI in specified candidate paths, enhancing the logic for determining if ComfyUI is installed on disk.
- This change improves the accuracy of installation state reporting by expanding the search for ComfyUI binaries.
- Introduced `comfy_on_disk` and `sync_is_installed` functions to manage the `IsInstalled` flag based on the presence of ComfyUI, ensuring accurate installation state reporting.
- Updated `install_swarm_comfy.py` to clear the `IsInstalled` flag when necessary, allowing the installation process to proceed without conflicts.
- Improved logic to prevent setting `IsInstalled` to true on first boot or empty data volumes, addressing issues with the InstallConfirmWS process.
- Enhanced diagnostics and logging for installation state changes, providing clearer feedback during the installation process.
- Introduced a new `loading_fail_sec` parameter in the `wait_backend_idle` function to handle prolonged loading states, improving error handling for backend readiness.
- Updated the `ensure_dlbackend_bind` function to stop SwarmUI before remounting, preventing target busy errors and ensuring consistent data mounts.
- Enhanced the `recover_errored_backends` function to account for the new remount logic, improving backend recovery processes.
- Refactored tests to validate the new loading failure conditions and ensure proper handling of backend states during diagnostics.
- Introduced the `clean_extra_args` function to filter out unsafe CLI flags and comment garbage from `ExtraArgs`, improving backend configuration integrity.
- Updated the `repair_extra_args_on_disk` function to rewrite corrupted `ExtraArgs` lines in the FDS file, ensuring cleaner backend settings.
- Enhanced the `recover_errored_backends` function to utilize the new sanitization logic, preserving valid `ExtraArgs` during backend reconfiguration.
- Improved the `sanitize_backends_fds` function to drop comment garbage and retain valid entries, enhancing overall backend management.
- Added tests to validate the new sanitization behavior, ensuring robustness in handling `ExtraArgs` during backend operations.
- Updated the `recover_errored_backends` function to preserve `ExtraArgs` during backend reconfiguration, improving error recovery.
- Refined the `sanitize_backends_fds` function to prevent newline merging issues with bare `\\x` entries, ensuring correct FDS formatting.
- Enhanced the `configure_comfy_backend` function to accept and manage `ExtraArgs`, preventing loss of important configuration during recovery.
- Added tests to validate the new sanitization behavior and ensure that bare `\\x` entries remain intact, improving overall backend management.
- Introduced remounting logic for data directories to ensure correct bindings before restarting services, improving reliability during performance tuning.
- Added functions to sanitize and ensure absolute paths for backend scripts, preventing issues with relative paths in bind mounts.
- Enhanced the `patch_backends_extra_args` function to clean up corrupted entries and ensure proper configuration of backend parameters.
- Updated tests to validate the new sanitization and path handling functionalities, ensuring robustness in backend management.
- Introduced functions to ensure the `IsInstalled` flag is set in `Settings.fds`, preventing the UI from displaying the /Install wizard.
- Added logic to create or patch `Settings.fds` with installation details, including the installation date and version.
- Updated the `install_swarm_comfy.py` script to call the new function, ensuring the installation state is correctly managed during backend operations.
- Added tests to verify the presence of the `IsInstalled` flag in the relevant scripts and ensure proper functionality during installation checks.
- Added a `--update` option to the `up` command in the CLI for forced updates during warm ACTIVE states.
- Improved the idle-killer logic with new functions to check for reusable credentials and refresh settings on existing remote credentials.
- Enhanced the `seed_civitai` function to implement caching for model jobs, optimizing the provisioning process.
- Updated the installation script for Ollama to prevent unnecessary restarts when the service configuration has not changed.
- Added diagnostics for model job caching and improved error handling in various functions.
- Introduced a new `delete_backend` function to facilitate backend removal during recovery processes.
- Improved the `recover_errored_backends` function to handle backend reconfiguration and recreation more effectively, including enhanced logging for diagnostics.
- Added a new `add_comfy_selfstart` function to streamline the addition of new backends.
- Updated the `comfy_start_script` function to prefer absolute paths for better reliability.
- Enhanced diagnostics in `swarm_diag.py` to include detailed backend settings and mount status.
- Added tests to validate the new backend management features and ensure robust error handling.
- Updated the `wait_backend_idle` function to handle disabled backends, introducing a new failure condition and diagnostics collection for better error handling.
- Refactored the `install_swarm_comfy` script to include a new `configure_comfy_backend` function, streamlining backend configuration and enabling better management of backend states.
- Added tests to validate the new functionality for handling disabled backends and ensuring robust diagnostics, improving overall backend management.
- Improved the `install_swarm_comfy` function to handle empty backend states more effectively, introducing recovery mechanisms and enhanced logging for better visibility.
- Updated the `tune_swarm_perf` function to always sanitize backend FDS corruption, ensuring consistent performance tuning.
- Added new tests to validate the functionality of backend recovery and FDS sanitization, ensuring robustness in backend management.
- Introduced the `file_size_bytes` function to convert Civitai model sizes from kilobytes to bytes, improving data handling.
- Updated the `seed_civitai` function to include file size in job definitions, enhancing model processing efficiency.
- Enhanced the `should_skip` function to utilize expected size for faster decision-making during job processing.
- Added tests for new functionality, ensuring robustness in file size handling and job processing logic.
- Enhanced the `with_token` function to support Gitea and Forgejo URLs by constructing the appropriate authentication format for generic HTTPS.
- Updated tests to validate the new functionality, ensuring correct token injection for Gitea URLs while maintaining existing behavior for GitHub and GitLab.
- Improved overall robustness of token handling across different platforms.
- Introduced the `pick_preview_image` function to extract the first usable preview image URL and its suffix from model version data.
- Updated the `seed_civitai` function to include preview image URLs and destinations in job definitions, improving model handling.
- Implemented the `ensure_preview` function to download missing preview images during job processing, enhancing user experience.
- Added tests for `pick_preview_image` to ensure correct functionality across various scenarios, ensuring robustness in image handling.
- Introduced functions `swarm_git_ok`, `clone_swarmui`, and `update_swarmui` to streamline the cloning and updating process of SwarmUI.
- Improved error handling during the update process, ensuring a fallback to cloning if the update fails.
- Enhanced logging messages for better visibility during cloning and updating operations.
- Updated tests to verify the presence of new functions and logging messages in the bootstrap script.
- Added `wget` to the package installation list for improved script functionality.
- Implemented a fallback mechanism using `curl` for .NET installation if the primary script fails, enhancing robustness.
- Updated ownership change for the installation directory to ensure proper permissions for the user.
- Enhanced logging for error scenarios during .NET installation, providing clearer feedback for troubleshooting.
- Added tests to verify the new fallback functionality and ownership changes in the bootstrap script.
- Introduced a new CLI command `diag` to collect diagnostics from SwarmUI/Comfy, including API, journal, and paths.
- Enhanced the `ensure_swarm_comfy_installed` function to include diagnostic script handling for errored backends.
- Updated `wait_backend_idle` to trigger diagnostics when backends are in an errored state, improving error recovery.
- Implemented fallback mechanisms for diagnostics in the `run_diagnostics` function, ensuring better visibility into backend issues.
- Added tests to validate the new diagnostic functionality and error handling, ensuring robustness in backend management.
- Updated the idle-killer logic to treat SwarmUI `empty` and `disabled` states as busy, preventing unnecessary idle time during provisioning.
- Enhanced the `wait_backend_idle` function to recognize suspended backends as ready, improving resource utilization and user feedback.
- Refined the `install_swarm_comfy` script to skip installation when backends are already present, streamlining the setup process.
- Improved the `resolve_llm_runtime` function to prioritize live configuration over stale state notes, ensuring accurate runtime detection.
- Added tests to validate the new backend status handling and idle management logic, ensuring robustness and reliability.
- Replaced direct calls to `venv/bin/pip` with `python -m pip` to avoid execution errors on network mounts and improve compatibility.
- Introduced a new function `find_comfy_python` to locate the appropriate Python executable for ComfyUI installations.
- Updated the `pip_install_sage` function to utilize the new Python handling, enhancing installation reliability.
- Added tests to verify the new behavior of Python environment detection and installation processes, ensuring robustness.
- Enhanced documentation to clarify the transition from 'Idle' to 'ready (running)' for backend states, improving user understanding of system readiness.
- Updated logging messages in the notification system to reflect the new backend status terminology, ensuring accurate feedback during operations.
- Refined access link collection logic to better handle tunneled and non-tunneled scenarios, enhancing user experience.
- Improved tests to validate the new backend status handling and ensure accurate reporting of access links and notifications.
- Updated the `collect_access_links` function to provide clearer user-facing endpoint labels and notes, particularly for non-tunneled scenarios.
- Improved logging messages in the provisioning process to reflect the status of the SwarmUI and Ollama API, enhancing user feedback during setup.
- Added human-readable status messages for backend loading and running states, improving clarity during the waiting process.
- Updated tests to verify the new behavior and ensure accurate reporting of access links and backend statuses.
- Added functions for formatting byte sizes, displaying progress bars, and calculating directory sizes to improve user feedback during installation.
- Implemented a stage detection mechanism to provide real-time updates on the installation process of ComfyUI.
- Enhanced logging to capture recent journal lines related to ComfyUI installation, improving visibility into the installation status.
- Updated tests to verify the presence of new functionality in the installation script, ensuring robustness and reliability.
- Added a new function to ensure headless installation of ComfyUI when backends are empty, enhancing the setup process for SwarmUI.
- Updated documentation to clarify the installation flow and backend readiness checks, ensuring users understand the requirements for a successful setup.
- Enhanced backend status checks to differentiate between 'empty' and 'idle' states, improving error handling and user feedback during provisioning.
- Adjusted logging messages to provide clearer insights into the installation and backend status processes.
- Modified the `load_config` function to allow launch preferences in `gpu-rent.vars` to override `.env` settings for non-secret variables, enhancing user control.
- Updated the list of Python candidates in `stack_env_probe.py` to include additional paths for ComfyUI, improving the detection of Python environments.
- Added new pip candidates in `tune_swarm_perf.py` to support various ComfyUI installations, ensuring better compatibility with different setups.
- Removed the direct injection of environment variables into the script, preventing issues with `from __future__` imports.
- Updated the `run_python` function to accept an `env` parameter for passing runtime environment variables, enhancing script execution control.
- Adjusted tests to verify the new behavior, ensuring that environment variables are correctly set without modifying the script content.
- Revised model descriptions in `ollama-models.example.yaml` to emphasize uncensored and abliterated requirements, enhancing user understanding.
- Updated documentation in `llm.md` to reflect changes in model tags and their meanings, ensuring accurate guidance for users.
- Modified CLI help messages to clarify the nature of presets, reinforcing that all options are abliterate models with Russian support.
- Enhanced the `llm_runtime.py` file to align preset labels with the updated model descriptions, improving consistency across the codebase.
- Removed references to llamacpp from configuration files, scripts, and documentation, streamlining the LLM setup process to focus solely on Ollama.
- Updated environment variables and paths to eliminate llamacpp-related entries, ensuring clarity in the configuration.
- Adjusted CLI commands and help messages to reflect the removal of llamacpp, enhancing user experience and reducing confusion.
- Revised documentation to provide clear guidance on using Ollama exclusively, including updates to setup instructions and runtime options.
- Reduced default values for IDLE_MINUTES from 60 to 30 and IDLE_GRACE_MINUTES from 90 to 45 across configuration files and documentation.
- Updated related documentation to reflect the new default idle time settings, ensuring consistency in user guidance and system behavior.
- Added a new configuration option `UP_STOP_ON_FAIL` to control whether the GPU should be stopped automatically if the `up` command fails, enhancing user control over resource management.
- Updated the CLI to include a `--keep-on-fail` flag, allowing users to prevent GPU shutdown during installation errors.
- Enhanced the installation scripts and documentation to reflect these changes, providing clearer guidance on the new behavior and configuration options.
- Improved error handling in the CLI to ensure proper cleanup of resources in case of failure, preventing unexpected billing for unused GPU resources.
- Introduced a new variable to determine the type of archive being processed, enhancing the logic for handling different file types.
- Updated the installation process to ensure proper extraction of both zip and tar files, improving compatibility with various archive formats.
- Enhanced logging to provide clearer feedback on the type of archive being processed, aiding in troubleshooting.
- Added error handling to ensure the installation process returns a failure status if the archive does not contain the expected binaries.
Leonid Pershin
