Files

T

Leonid PershinandCursor 3380206c6a Expand Assistent to v0.3: Generate loop, Civitai Confirm, img2img/inpaint.

Server inventory and streaming chat, auto-apply/generate with Interrupt, Civitai search cards (Confirm-only download), plus Init/Mask wiring and inpaint_edit pack.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-08-21 13:29:26 +03:00

3.8 KiB

Raw Blame History

Swarm Assistent

SwarmUI extension for collaborative Krea 2 prompting via Ollama: chat + vision, LoRA/trigger awareness, applyable patches, img2img / inpaint, auto Generate, and Civitai search with Confirm.

Layout

Left: vision reference (drop / paste / Send to Assistent from Generate)
Splitter: drag to resize panes
Right (wider): chat
Top-right: prompt pack + settings (Ollama URL, model, auto-apply / auto-generate)

UX

Send to Assistent under the current Generate image (and History) — loads vision + opens the Assistent tab
Drag images from Generate/History or the OS onto the left pane
Paste (Ctrl+V) while the Assistent tab is open
Use current / Clear for the vision reference
As Init / As Mask / Clear Init — wire the vision image into Swarm Init Image / Mask Image (img2img & inpaint)
Enter sends; Shift+Enter newline
Auto-apply + Auto-generate (default on): patch from the LLM is applied and Generate runs when the patch changes prompt/params or includes actions: ["generate"]
Pure Q&A without a patch does not start Generate
Interrupt stops Swarm generation / clears busy state
Civitai search cards require Confirm download (uses Swarm DoModelDownloadWS + stored civitai_api key). Auto-download is off by default.

Requirements

SwarmUI with a Krea 2 checkpoint selected
Ollama on http://127.0.0.1:11434 on the GPU VM (gpu-rent LLM_RUNTIME=ollama). The browser talks to SwarmUI; SwarmUI proxies /api/tags and /api/chat. URL in settings must stay 127.0.0.1:11434, not the laptop tunnel port 17811.
At least one pulled model (ollama pull / ollama-models.yaml). Empty /api/tags → empty Model dropdown.
Optional: Civitai API key in SwarmUI User Settings for search/download.

Install

Clone into SwarmUI src/Extensions/swarm-assistent (or let gpu-rent seed it from extensions.yaml with requires: ollama):

swarmui:
  - url: https://gitea.hsrv.site/mrleo1nid/swarm-assistent.git
    ref: main
    dir: swarm-assistent
    requires: ollama

Restart / rebuild SwarmUI after clone.

Prompt packs

Pack	Role
`base_krea2`	Always injected: Krea 2 rules + JSON patch / actions contract
`write_prompt`	Craft / improve prompts
`critique_image`	Vision critique → fixes
`compose_scene`	Scene / moodboard
`fix_params`	Width/height/steps/CFG/seed/σ-shift
`inpaint_edit`	Init Image img2img + Mask inpaint

Live context (checkpoint, server inventory LoRAs + triggers, wildcards, current params) is injected every request.

Patch actions

{
  "prompt": "...",
  "loras": [{"name": "exact", "weight": 0.8, "triggers": ["..."]}],
  "width": 1024, "height": 1280, "steps": 8, "cfg": 1,
  "seed": -1, "sigma_shift": 1.15,
  "use_init_image": true,
  "init_creativity": 0.45,
  "use_mask_image": false,
  "actions": ["generate"],
  "search_query": null
}

generate — auto-generate after apply (when enabled)
use_init / use_mask — set Assistent vision (or current Generate) as Init / Mask
search_civitai + search_query — server searches Civitai, second LLM hop, Confirm cards in UI
interrupt — stop current generation

Mask convention: white = edit, black = keep. Creativity ≈ denoise (0–1).

API routes

Route	Role
`AssistentListModels`	Ollama `/api/tags`
`AssistentListInventory`	LoRA / checkpoint / wildcard inventory from Swarm
`AssistentSearchCivitai`	Civitai LoRA search
`AssistentGetPacks`	Prompt pack texts
`AssistentChat`	HTTP chat (+ Civitai hop)
`AssistentChatWS`	Streaming chat WebSocket

License

MIT

3.8 KiB Raw Blame History Unescape Escape