Bump Assistent to 0.6.0: board tabs, Cards Civitai fetch, and chat reliability.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -1,29 +1,27 @@
|
||||
# Swarm Assistent
|
||||
|
||||
SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat + multi-window board (live Generate + refs), LoRA/trigger awareness, applyable patches, **img2img / inpaint**, slash commands, aspect/seed chips, auto Generate, and Civitai search with Confirm.
|
||||
SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat + board (Generate | Refs tabs), LoRA chips, personas, model cards with Civitai fetch, img2img/inpaint, slash commands, auto Generate.
|
||||
|
||||
**Version 0.5.0** — denser Krea 2 knowledge in prompt packs, new patch fields, `/help` slash commands, composer chips.
|
||||
**Version 0.6.0** — board tabs, Cards form + live Civitai meta, Interrupt cancel, slim history, LoRA chips, Turbo/RAW, taste on disk, RU UI.
|
||||
|
||||
## Layout
|
||||
|
||||
- **Left — Board:** live **Generate** window + **Ref** windows (drop / paste / *Send to Assistent* / Snapshot gen). Per-window **vision** checkbox attaches that image to the next chat.
|
||||
- **Left — Board tabs:** **Generate** (full-height live view) | **Refs** (reference grid + badge `N · vision M`)
|
||||
- **Splitter:** drag to resize panes
|
||||
- **Right (wider):** chat — first visit shows a short how-to. Only the **latest** JSON proposal keeps action buttons; Apply + Generate spins/disables while a generation is running.
|
||||
- **Chips** above the composer: aspect ratios, Seed lock / random, Vary.
|
||||
- **Top-right:** prompt pack + settings (Ollama URL, model, auto-apply / auto-generate)
|
||||
- **Right:** Chat | Cards; persona / pack / Ollama model; settings gear
|
||||
- **Chips:** aspect, Seed lock/−1, Vary, Turbo/RAW; LoRA chip row under them
|
||||
- Slash autocomplete when the composer starts with `/`
|
||||
|
||||
## UX
|
||||
|
||||
- **Send to Assistent** under the current Generate image (and History) — adds a Ref window and opens the Assistent tab
|
||||
- Drag / paste onto a Ref (drop on Generate snapshots into a new Ref)
|
||||
- **Snapshot gen** copies the live Generate window into a Ref
|
||||
- **As Init** / **As Mask** / **Clear Init** — selected window → Swarm `Init Image` / `Mask Image`
|
||||
- Enter sends; Shift+Enter newline
|
||||
- **Auto-apply** + **Auto-generate** (default on): patch from the LLM is applied and Generate runs when the patch changes prompt/params or includes `actions: ["generate"]`
|
||||
- Pure Q&A without a patch does **not** start Generate
|
||||
- **Interrupt** stops Swarm generation / clears busy state
|
||||
- **Civitai** search cards require **Confirm download** (uses Swarm `DoModelDownloadWS` + stored `civitai_api` key). Auto-download is off by default.
|
||||
- Pack auto-selects from the user message (critique / inpaint / params / compose / describe / write) unless you changed the dropdown yourself.
|
||||
- **Send to Assistent** under Generate/History → Ref + Assistent tab
|
||||
- Drop on Generate → new Ref + switch to Refs tab
|
||||
- **As Init** / **As Mask** / **Clear Init**
|
||||
- Enter sends; Shift+Enter newline; Interrupt cancels chat epoch (no late auto-apply)
|
||||
- Manual **Apply + Generate** / `/gen` always generate; Auto-generate checkbox only for LLM auto-path
|
||||
- Auto-critique rewrites prompt only (no second Generate)
|
||||
- Civitai Confirm required (unless auto-download)
|
||||
- Cards: form fields + previews; **Load Civitai meta** fetches by SHA (`by-hash`); status always explicit
|
||||
|
||||
### Slash commands (client-side, no LLM)
|
||||
|
||||
@@ -33,24 +31,23 @@ SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat +
|
||||
| `/gen` | Generate now |
|
||||
| `/look generate\|refN` | Attach that board window + ask the LLM to look |
|
||||
| `/init` `/mask` `/clear` | Same as board buttons |
|
||||
| `/interrupt` | Stop generation |
|
||||
| `/interrupt` | Stop generation / cancel chat |
|
||||
| `/aspect 16:9` | Set size from the official 1K table |
|
||||
| `/seed lock\|random` | Lock or randomize seed |
|
||||
| `/vary` | New seed, same prompt (+ generate if auto) |
|
||||
| `/pack write\|critique\|compose\|params\|inpaint\|describe` | Switch pack |
|
||||
| `/pack write\|critique\|compose\|params\|inpaint\|describe\|card` | Switch pack |
|
||||
| `/civitai <query>` | Ask LLM to search Civitai (Confirm still required) |
|
||||
| `/inventory` | Rescan models + refresh LoRA list |
|
||||
|
||||
## Requirements
|
||||
|
||||
- SwarmUI with a **Krea 2** checkpoint selected
|
||||
- Ollama on `http://127.0.0.1:11434` **on the GPU VM** (gpu-rent `LLM_RUNTIME=ollama`). The browser talks to SwarmUI; SwarmUI proxies `/api/tags` and `/api/chat`. URL in settings must stay `127.0.0.1:11434`, not the laptop tunnel port 17811.
|
||||
- At least one pulled model (`ollama pull` / `ollama-models.yaml`). Empty `/api/tags` → empty Model dropdown.
|
||||
- Optional: Civitai API key in SwarmUI User Settings for search/download.
|
||||
- Ollama on `http://127.0.0.1:11434` **on the GPU VM** (gpu-rent `LLM_RUNTIME=ollama`)
|
||||
- At least one pulled model
|
||||
- Optional: Civitai API key in SwarmUI User Settings
|
||||
|
||||
## Install
|
||||
|
||||
Clone into SwarmUI `src/Extensions/swarm-assistent` (or let **gpu-rent** seed it from `extensions.yaml` with `requires: ollama`):
|
||||
|
||||
```yaml
|
||||
swarmui:
|
||||
- url: https://gitea.hsrv.site/mrleo1nid/swarm-assistent.git
|
||||
@@ -59,81 +56,40 @@ swarmui:
|
||||
requires: ollama
|
||||
```
|
||||
|
||||
Restart / rebuild SwarmUI after clone.
|
||||
Restart / rebuild SwarmUI after clone. gpu-rent: `seed-extensions` + restart.
|
||||
|
||||
## Prompt packs
|
||||
|
||||
| Pack | Role |
|
||||
| --- | --- |
|
||||
| `base_krea2` | Always injected: Krea 2 rules + JSON patch / actions contract |
|
||||
| `write_prompt` | Craft / improve prompts (prose structure) |
|
||||
| `critique_image` | Vision critique → fixes |
|
||||
| `compose_scene` | Scene / moodboard via board refs → text |
|
||||
| `fix_params` | Aspect / steps / CFG / seed / batch |
|
||||
| `inpaint_edit` | Init Image img2img + Mask inpaint |
|
||||
| `describe_ref` | Vision → Krea prompt (no generate by default) |
|
||||
| `catalog_card` | Recommendation card JSON for a checkpoint/LoRA |
|
||||
| `base_krea2` | Always injected |
|
||||
| `write_prompt` | Craft / improve prompts |
|
||||
| `critique_image` | Vision critique |
|
||||
| `compose_scene` | Scene via board refs |
|
||||
| `fix_params` | Aspect / steps / CFG / seed (`krea_profile` in live context) |
|
||||
| `inpaint_edit` | Init + Mask |
|
||||
| `describe_ref` | Vision → prompt |
|
||||
| `catalog_card` | Recommendation card JSON |
|
||||
|
||||
**Persona** (tone) is a separate dropdown from pack — `base_krea2` → persona → pack. Overlay: `/mnt/swarm_data/Assistent/personas.json` (seeded from gpu-rent `assistent-personas.yaml`).
|
||||
**Personas:** `neutral`, `lewd`, `aggressive`, `cinema`, `terse`. Overlay: `/mnt/swarm_data/Assistent/personas.json` (gpu-rent seeds from `assistent-personas.yaml` → yaml + json). Assistent reads **json** only.
|
||||
|
||||
**Cards** subtab: edit/save `{stem}.assistent.json` next to weights; live chat gets cards for the current checkpoint + enabled LoRAs only. Uninstalled models can enqueue `.gpu-rent-wanted-models.yaml` for the next `up`.
|
||||
**Taste:** `/mnt/swarm_data/Assistent/taste.json` via `AssistentGetTaste` / `AssistentSaveTaste` (merged with browser localStorage by `updated`).
|
||||
|
||||
Live context (checkpoint, server inventory LoRAs + blurbs/triggers, `model_cards`, `taste_profile`, wildcards, current params, persona) is injected every request. Inventory refreshes each chat turn (disk rescan when stale / after download / `/inventory`). After Confirm download Assistent auto-writes a `.assistent.json` card.
|
||||
|
||||
**Wanted queue:** Cards → Enqueue wanted → `/mnt/swarm_data/.gpu-rent-wanted-models.yaml`. Laptop: `gpu-rent capture wanted` (also on `capture models` / seed `up`).
|
||||
|
||||
**Cloud-only Krea.ai features** (moodboards UI, Generative Sliders, Creativity Raw/Low/Medium/High) are **not** in Swarm. The assistant emulates them with prompt language + board refs. Optional patch fields `creativity` / `intensity` / `complexity` / `movement` guide the LLM only.
|
||||
|
||||
### Patch actions
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt": "...",
|
||||
"loras": [{"name": "exact", "weight": 0.8, "triggers": ["..."]}],
|
||||
"aspect": "16:9",
|
||||
"width": 1376, "height": 768,
|
||||
"steps": 8, "cfg": 1,
|
||||
"seed": -1, "images": 1,
|
||||
"sigma_shift": 1.15,
|
||||
"creativity": "medium",
|
||||
"intensity": 0, "complexity": 0, "movement": 0,
|
||||
"vary": false, "lock_seed": false,
|
||||
"use_init_image": true,
|
||||
"init_creativity": 0.45,
|
||||
"use_mask_image": false,
|
||||
"clear_prompt_images": false,
|
||||
"look_at": ["generate"],
|
||||
"slot_to_init": "generate",
|
||||
"snapshot_generate": false,
|
||||
"pack": "critique_image",
|
||||
"actions": ["generate"],
|
||||
"search_query": null
|
||||
}
|
||||
```
|
||||
|
||||
- `aspect` — maps to official 1K sizes (`1:1`, `4:5`, `2:3`, `16:9`, `9:16`, `4:3`, `3:2`, `2.35:1`)
|
||||
- `vary` / `lock_seed` / `images`|`batch` — seed and batch helpers
|
||||
- `clear_prompt_images` — strip `<image…>` embeds from the prompt box
|
||||
- `pack` — switch the active prompt pack for a follow-up
|
||||
- `generate` — auto-generate after apply (when enabled)
|
||||
- `look_at` — hop vision from named board windows (`generate`, `ref1`, …)
|
||||
- `slot_to_init` / `slot_to_mask` — copy that window into Swarm Init / Mask
|
||||
- `snapshot_generate` — copy live Generate into a Ref
|
||||
- `search_civitai` + `search_query` — server searches Civitai, second LLM hop, Confirm cards in UI
|
||||
- `interrupt` — stop current generation
|
||||
|
||||
Mask convention: **white = edit**, black = keep. Creativity ≈ denoise (0–1).
|
||||
|
||||
Turbo defaults: steps **8**, CFG **1** (never 0), sigma shift **1.15**.
|
||||
**Cards:** `{stem}.assistent.json` next to weights; Civitai sidecar `{stem}.civitai.json`. Wanted queue → `.gpu-rent-wanted-models.yaml`.
|
||||
|
||||
## API routes
|
||||
|
||||
| Route | Role |
|
||||
| --- | --- |
|
||||
| `AssistentListModels` | Ollama `/api/tags` |
|
||||
| `AssistentListInventory` | LoRA / checkpoint / wildcard inventory from Swarm |
|
||||
| `AssistentSearchCivitai` | Civitai LoRA search |
|
||||
| `AssistentListInventory` | LoRA / checkpoint / wildcard inventory |
|
||||
| `AssistentListPersonas` | Bundled + overlay personas |
|
||||
| `AssistentGetPacks` | Prompt pack texts |
|
||||
| `AssistentGetCard` / `AssistentSaveCard` | `.assistent.json` cards |
|
||||
| `AssistentGetCardMeta` | Local sidecar + optional `fetch=true` Civitai by-hash |
|
||||
| `AssistentEnqueueWanted` | Wanted YAML queue |
|
||||
| `AssistentGetTaste` / `AssistentSaveTaste` | Persistent taste profile |
|
||||
| `AssistentSearchCivitai` | Civitai LoRA search |
|
||||
| `AssistentChat` | HTTP chat (+ Civitai hop) |
|
||||
| `AssistentChatWS` | Streaming chat WebSocket |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user