Bump Assistent to 0.6.0: board tabs, Cards Civitai fetch, and chat reliability.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Leonid Pershin
2026-08-21 22:04:37 +03:00
co-authored by Cursor
parent c873894355
commit 46264d7227
9 changed files with 1591 additions and 324 deletions
+39 -83
View File
@@ -1,29 +1,27 @@
# Swarm Assistent
SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat + multi-window board (live Generate + refs), LoRA/trigger awareness, applyable patches, **img2img / inpaint**, slash commands, aspect/seed chips, auto Generate, and Civitai search with Confirm.
SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat + board (Generate | Refs tabs), LoRA chips, personas, model cards with Civitai fetch, img2img/inpaint, slash commands, auto Generate.
**Version 0.5.0**denser Krea 2 knowledge in prompt packs, new patch fields, `/help` slash commands, composer chips.
**Version 0.6.0**board tabs, Cards form + live Civitai meta, Interrupt cancel, slim history, LoRA chips, Turbo/RAW, taste on disk, RU UI.
## Layout
- **Left — Board:** live **Generate** window + **Ref** windows (drop / paste / *Send to Assistent* / Snapshot gen). Per-window **vision** checkbox attaches that image to the next chat.
- **Left — Board tabs:** **Generate** (full-height live view) | **Refs** (reference grid + badge `N · vision M`)
- **Splitter:** drag to resize panes
- **Right (wider):** chat — first visit shows a short how-to. Only the **latest** JSON proposal keeps action buttons; Apply + Generate spins/disables while a generation is running.
- **Chips** above the composer: aspect ratios, Seed lock / random, Vary.
- **Top-right:** prompt pack + settings (Ollama URL, model, auto-apply / auto-generate)
- **Right:** Chat | Cards; persona / pack / Ollama model; settings gear
- **Chips:** aspect, Seed lock/1, Vary, Turbo/RAW; LoRA chip row under them
- Slash autocomplete when the composer starts with `/`
## UX
- **Send to Assistent** under the current Generate image (and History) — adds a Ref window and opens the Assistent tab
- Drag / paste onto a Ref (drop on Generate snapshots into a new Ref)
- **Snapshot gen** copies the live Generate window into a Ref
- **As Init** / **As Mask** / **Clear Init** — selected window → Swarm `Init Image` / `Mask Image`
- Enter sends; Shift+Enter newline
- **Auto-apply** + **Auto-generate** (default on): patch from the LLM is applied and Generate runs when the patch changes prompt/params or includes `actions: ["generate"]`
- Pure Q&A without a patch does **not** start Generate
- **Interrupt** stops Swarm generation / clears busy state
- **Civitai** search cards require **Confirm download** (uses Swarm `DoModelDownloadWS` + stored `civitai_api` key). Auto-download is off by default.
- Pack auto-selects from the user message (critique / inpaint / params / compose / describe / write) unless you changed the dropdown yourself.
- **Send to Assistent** under Generate/History → Ref + Assistent tab
- Drop on Generate → new Ref + switch to Refs tab
- **As Init** / **As Mask** / **Clear Init**
- Enter sends; Shift+Enter newline; Interrupt cancels chat epoch (no late auto-apply)
- Manual **Apply + Generate** / `/gen` always generate; Auto-generate checkbox only for LLM auto-path
- Auto-critique rewrites prompt only (no second Generate)
- Civitai Confirm required (unless auto-download)
- Cards: form fields + previews; **Load Civitai meta** fetches by SHA (`by-hash`); status always explicit
### Slash commands (client-side, no LLM)
@@ -33,24 +31,23 @@ SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat +
| `/gen` | Generate now |
| `/look generate\|refN` | Attach that board window + ask the LLM to look |
| `/init` `/mask` `/clear` | Same as board buttons |
| `/interrupt` | Stop generation |
| `/interrupt` | Stop generation / cancel chat |
| `/aspect 16:9` | Set size from the official 1K table |
| `/seed lock\|random` | Lock or randomize seed |
| `/vary` | New seed, same prompt (+ generate if auto) |
| `/pack write\|critique\|compose\|params\|inpaint\|describe` | Switch pack |
| `/pack write\|critique\|compose\|params\|inpaint\|describe\|card` | Switch pack |
| `/civitai <query>` | Ask LLM to search Civitai (Confirm still required) |
| `/inventory` | Rescan models + refresh LoRA list |
## Requirements
- SwarmUI with a **Krea 2** checkpoint selected
- Ollama on `http://127.0.0.1:11434` **on the GPU VM** (gpu-rent `LLM_RUNTIME=ollama`). The browser talks to SwarmUI; SwarmUI proxies `/api/tags` and `/api/chat`. URL in settings must stay `127.0.0.1:11434`, not the laptop tunnel port 17811.
- At least one pulled model (`ollama pull` / `ollama-models.yaml`). Empty `/api/tags` → empty Model dropdown.
- Optional: Civitai API key in SwarmUI User Settings for search/download.
- Ollama on `http://127.0.0.1:11434` **on the GPU VM** (gpu-rent `LLM_RUNTIME=ollama`)
- At least one pulled model
- Optional: Civitai API key in SwarmUI User Settings
## Install
Clone into SwarmUI `src/Extensions/swarm-assistent` (or let **gpu-rent** seed it from `extensions.yaml` with `requires: ollama`):
```yaml
swarmui:
- url: https://gitea.hsrv.site/mrleo1nid/swarm-assistent.git
@@ -59,81 +56,40 @@ swarmui:
requires: ollama
```
Restart / rebuild SwarmUI after clone.
Restart / rebuild SwarmUI after clone. gpu-rent: `seed-extensions` + restart.
## Prompt packs
| Pack | Role |
| --- | --- |
| `base_krea2` | Always injected: Krea 2 rules + JSON patch / actions contract |
| `write_prompt` | Craft / improve prompts (prose structure) |
| `critique_image` | Vision critique → fixes |
| `compose_scene` | Scene / moodboard via board refs → text |
| `fix_params` | Aspect / steps / CFG / seed / batch |
| `inpaint_edit` | Init Image img2img + Mask inpaint |
| `describe_ref` | Vision → Krea prompt (no generate by default) |
| `catalog_card` | Recommendation card JSON for a checkpoint/LoRA |
| `base_krea2` | Always injected |
| `write_prompt` | Craft / improve prompts |
| `critique_image` | Vision critique |
| `compose_scene` | Scene via board refs |
| `fix_params` | Aspect / steps / CFG / seed (`krea_profile` in live context) |
| `inpaint_edit` | Init + Mask |
| `describe_ref` | Vision → prompt |
| `catalog_card` | Recommendation card JSON |
**Persona** (tone) is a separate dropdown from pack — `base_krea2` → persona → pack. Overlay: `/mnt/swarm_data/Assistent/personas.json` (seeded from gpu-rent `assistent-personas.yaml`).
**Personas:** `neutral`, `lewd`, `aggressive`, `cinema`, `terse`. Overlay: `/mnt/swarm_data/Assistent/personas.json` (gpu-rent seeds from `assistent-personas.yaml` → yaml + json). Assistent reads **json** only.
**Cards** subtab: edit/save `{stem}.assistent.json` next to weights; live chat gets cards for the current checkpoint + enabled LoRAs only. Uninstalled models can enqueue `.gpu-rent-wanted-models.yaml` for the next `up`.
**Taste:** `/mnt/swarm_data/Assistent/taste.json` via `AssistentGetTaste` / `AssistentSaveTaste` (merged with browser localStorage by `updated`).
Live context (checkpoint, server inventory LoRAs + blurbs/triggers, `model_cards`, `taste_profile`, wildcards, current params, persona) is injected every request. Inventory refreshes each chat turn (disk rescan when stale / after download / `/inventory`). After Confirm download Assistent auto-writes a `.assistent.json` card.
**Wanted queue:** Cards → Enqueue wanted → `/mnt/swarm_data/.gpu-rent-wanted-models.yaml`. Laptop: `gpu-rent capture wanted` (also on `capture models` / seed `up`).
**Cloud-only Krea.ai features** (moodboards UI, Generative Sliders, Creativity Raw/Low/Medium/High) are **not** in Swarm. The assistant emulates them with prompt language + board refs. Optional patch fields `creativity` / `intensity` / `complexity` / `movement` guide the LLM only.
### Patch actions
```json
{
"prompt": "...",
"loras": [{"name": "exact", "weight": 0.8, "triggers": ["..."]}],
"aspect": "16:9",
"width": 1376, "height": 768,
"steps": 8, "cfg": 1,
"seed": -1, "images": 1,
"sigma_shift": 1.15,
"creativity": "medium",
"intensity": 0, "complexity": 0, "movement": 0,
"vary": false, "lock_seed": false,
"use_init_image": true,
"init_creativity": 0.45,
"use_mask_image": false,
"clear_prompt_images": false,
"look_at": ["generate"],
"slot_to_init": "generate",
"snapshot_generate": false,
"pack": "critique_image",
"actions": ["generate"],
"search_query": null
}
```
- `aspect` — maps to official 1K sizes (`1:1`, `4:5`, `2:3`, `16:9`, `9:16`, `4:3`, `3:2`, `2.35:1`)
- `vary` / `lock_seed` / `images`|`batch` — seed and batch helpers
- `clear_prompt_images` — strip `<image…>` embeds from the prompt box
- `pack` — switch the active prompt pack for a follow-up
- `generate` — auto-generate after apply (when enabled)
- `look_at` — hop vision from named board windows (`generate`, `ref1`, …)
- `slot_to_init` / `slot_to_mask` — copy that window into Swarm Init / Mask
- `snapshot_generate` — copy live Generate into a Ref
- `search_civitai` + `search_query` — server searches Civitai, second LLM hop, Confirm cards in UI
- `interrupt` — stop current generation
Mask convention: **white = edit**, black = keep. Creativity ≈ denoise (01).
Turbo defaults: steps **8**, CFG **1** (never 0), sigma shift **1.15**.
**Cards:** `{stem}.assistent.json` next to weights; Civitai sidecar `{stem}.civitai.json`. Wanted queue → `.gpu-rent-wanted-models.yaml`.
## API routes
| Route | Role |
| --- | --- |
| `AssistentListModels` | Ollama `/api/tags` |
| `AssistentListInventory` | LoRA / checkpoint / wildcard inventory from Swarm |
| `AssistentSearchCivitai` | Civitai LoRA search |
| `AssistentListInventory` | LoRA / checkpoint / wildcard inventory |
| `AssistentListPersonas` | Bundled + overlay personas |
| `AssistentGetPacks` | Prompt pack texts |
| `AssistentGetCard` / `AssistentSaveCard` | `.assistent.json` cards |
| `AssistentGetCardMeta` | Local sidecar + optional `fetch=true` Civitai by-hash |
| `AssistentEnqueueWanted` | Wanted YAML queue |
| `AssistentGetTaste` / `AssistentSaveTaste` | Persistent taste profile |
| `AssistentSearchCivitai` | Civitai LoRA search |
| `AssistentChat` | HTTP chat (+ Civitai hop) |
| `AssistentChatWS` | Streaming chat WebSocket |