mrleo1nid/swarm-assistent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Expand Assistent to v0.3: Generate loop, Civitai Confirm, img2img/inpaint.

Browse Source 
Server inventory and streaming chat, auto-apply/generate with Interrupt, Civitai search cards (Confirm-only download), plus Init/Mask wiring and inpaint_edit pack.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Leonid Pershin
2026-08-21 13:29:26 +03:00
co-authored by Cursor
parent 06d5df9e93
commit 3380206c6a
11 changed files with 1701 additions and 156 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+48 -7
View File
@@ -1,13 +1,13 @@
# Swarm Assistent
SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat + vision, LoRA/trigger awareness, and applyable prompt/size patches.
SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat + vision, LoRA/trigger awareness, applyable patches, **img2img / inpaint**, auto Generate, and Civitai search with Confirm.
## Layout
- **Left:** vision reference (drop / paste / *Send to Assistent* from Generate)
- **Splitter:** drag to resize panes
- **Right (wider):** chat
- **Top-right:** prompt pack + settings (Ollama URL, model, auto-attach)
- **Top-right:** prompt pack + settings (Ollama URL, model, auto-apply / auto-generate)
## UX
@@ -15,13 +15,19 @@ SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat +
- Drag images from Generate/History or the OS onto the left pane
- Paste (`Ctrl+V`) while the Assistent tab is open
- **Use current** / **Clear** for the vision reference
- **As Init** / **As Mask** / **Clear Init** — wire the vision image into Swarm `Init Image` / `Mask Image` (img2img & inpaint)
- Enter sends; Shift+Enter newline
- **Auto-apply** + **Auto-generate** (default on): patch from the LLM is applied and Generate runs when the patch changes prompt/params or includes `actions: ["generate"]`
- Pure Q&A without a patch does **not** start Generate
- **Interrupt** stops Swarm generation / clears busy state
- **Civitai** search cards require **Confirm download** (uses Swarm `DoModelDownloadWS` + stored `civitai_api` key). Auto-download is off by default.
## Requirements
- SwarmUI with a **Krea 2** checkpoint selected
- Ollama on `http://127.0.0.1:11434` (gpu-rent default when `LLM_RUNTIME=ollama`)
- A chat+vision-capable Ollama model recommended for critique mode
- Ollama on `http://127.0.0.1:11434` **on the GPU VM** (gpu-rent `LLM_RUNTIME=ollama`). The browser talks to SwarmUI; SwarmUI proxies `/api/tags` and `/api/chat`. URL in settings must stay `127.0.0.1:11434`, not the laptop tunnel port 17811.
- At least one pulled model (`ollama pull` / `ollama-models.yaml`). Empty `/api/tags` → empty Model dropdown.
- Optional: Civitai API key in SwarmUI User Settings for search/download.
## Install
@@ -41,13 +47,48 @@ Restart / rebuild SwarmUI after clone.
| Pack | Role |
| --- | --- |
| `base_krea2` | Always injected: Krea 2 rules + JSON patch contract |
| `base_krea2` | Always injected: Krea 2 rules + JSON patch / actions contract |
| `write_prompt` | Craft / improve prompts |
| `critique_image` | Vision critique → fixes |
| `compose_scene` | Scene / moodboard |
| `fix_params` | Width/height/steps/CFG |
| `fix_params` | Width/height/steps/CFG/seed/σ-shift |
| `inpaint_edit` | Init Image img2img + Mask inpaint |
Live context (checkpoint, LoRAs + triggers, current params) is injected every request.
Live context (checkpoint, server inventory LoRAs + triggers, wildcards, current params) is injected every request.
### Patch actions
```json
{
"prompt": "...",
"loras": [{"name": "exact", "weight": 0.8, "triggers": ["..."]}],
"width": 1024, "height": 1280, "steps": 8, "cfg": 1,
"seed": -1, "sigma_shift": 1.15,
"use_init_image": true,
"init_creativity": 0.45,
"use_mask_image": false,
"actions": ["generate"],
"search_query": null
}
```
- `generate` — auto-generate after apply (when enabled)
- `use_init` / `use_mask` — set Assistent vision (or current Generate) as Init / Mask
- `search_civitai` + `search_query` — server searches Civitai, second LLM hop, Confirm cards in UI
- `interrupt` — stop current generation
Mask convention: **white = edit**, black = keep. Creativity ≈ denoise (0–1).
## API routes
| Route | Role |
| --- | --- |
| `AssistentListModels` | Ollama `/api/tags` |
| `AssistentListInventory` | LoRA / checkpoint / wildcard inventory from Swarm |
| `AssistentSearchCivitai` | Civitai LoRA search |
| `AssistentGetPacks` | Prompt pack texts |
| `AssistentChat` | HTTP chat (+ Civitai hop) |
| `AssistentChatWS` | Streaming chat WebSocket |
## License