# Swarm Assistent

SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat + multi-window board (live Generate + refs), LoRA/trigger awareness, applyable patches, **img2img / inpaint**, auto Generate, and Civitai search with Confirm.

## Layout

- **Left — Board:** live **Generate** window + **Ref** windows (drop / paste / *Send to Assistent* / Snapshot gen). Per-window **vision** checkbox attaches that image to the next chat.
- **Splitter:** drag to resize panes
- **Right (wider):** chat — first visit shows a short how-to. Only the **latest** JSON proposal keeps action buttons; Apply + Generate spins/disables while a generation is running.
- **Top-right:** prompt pack + settings (Ollama URL, model, auto-apply / auto-generate)

## UX

- **Send to Assistent** under the current Generate image (and History) — adds a Ref window and opens the Assistent tab
- Drag / paste onto a Ref (drop on Generate snapshots into a new Ref)
- **Snapshot gen** copies the live Generate window into a Ref
- **As Init** / **As Mask** / **Clear Init** — selected window → Swarm `Init Image` / `Mask Image`
- Enter sends; Shift+Enter newline
- **Auto-apply** + **Auto-generate** (default on): patch from the LLM is applied and Generate runs when the patch changes prompt/params or includes `actions: ["generate"]`
- Pure Q&A without a patch does **not** start Generate
- **Interrupt** stops Swarm generation / clears busy state
- **Civitai** search cards require **Confirm download** (uses Swarm `DoModelDownloadWS` + stored `civitai_api` key). Auto-download is off by default.

## Requirements

- SwarmUI with a **Krea 2** checkpoint selected
- Ollama on `http://127.0.0.1:11434` **on the GPU VM** (gpu-rent `LLM_RUNTIME=ollama`). The browser talks to SwarmUI; SwarmUI proxies `/api/tags` and `/api/chat`. URL in settings must stay `127.0.0.1:11434`, not the laptop tunnel port 17811.
- At least one pulled model (`ollama pull` / `ollama-models.yaml`). Empty `/api/tags` → empty Model dropdown.
- Optional: Civitai API key in SwarmUI User Settings for search/download.

## Install

Clone into SwarmUI `src/Extensions/swarm-assistent` (or let **gpu-rent** seed it from `extensions.yaml` with `requires: ollama`):

```yaml
swarmui:
  - url: https://gitea.hsrv.site/mrleo1nid/swarm-assistent.git
    ref: main
    dir: swarm-assistent
    requires: ollama
```

Restart / rebuild SwarmUI after clone.

## Prompt packs

| Pack | Role |
| --- | --- |
| `base_krea2` | Always injected: Krea 2 rules + JSON patch / actions contract |
| `write_prompt` | Craft / improve prompts |
| `critique_image` | Vision critique → fixes |
| `compose_scene` | Scene / moodboard |
| `fix_params` | Width/height/steps/CFG/seed/σ-shift |
| `inpaint_edit` | Init Image img2img + Mask inpaint |

Live context (checkpoint, server inventory LoRAs + triggers, wildcards, current params) is injected every request.

### Patch actions

```json
{
  "prompt": "...",
  "loras": [{"name": "exact", "weight": 0.8, "triggers": ["..."]}],
  "width": 1024, "height": 1280, "steps": 8, "cfg": 1,
  "seed": -1, "sigma_shift": 1.15,
  "use_init_image": true,
  "init_creativity": 0.45,
  "use_mask_image": false,
  "look_at": ["generate"],
  "slot_to_init": "generate",
  "snapshot_generate": false,
  "actions": ["generate"],
  "search_query": null
}
```

- `generate` — auto-generate after apply (when enabled)
- `look_at` — hop vision from named board windows (`generate`, `ref1`, …)
- `slot_to_init` / `slot_to_mask` — copy that window into Swarm Init / Mask
- `snapshot_generate` — copy live Generate into a Ref
- `search_civitai` + `search_query` — server searches Civitai, second LLM hop, Confirm cards in UI
- `interrupt` — stop current generation

Mask convention: **white = edit**, black = keep. Creativity ≈ denoise (0–1).

## API routes

| Route | Role |
| --- | --- |
| `AssistentListModels` | Ollama `/api/tags` |
| `AssistentListInventory` | LoRA / checkpoint / wildcard inventory from Swarm |
| `AssistentSearchCivitai` | Civitai LoRA search |
| `AssistentGetPacks` | Prompt pack texts |
| `AssistentChat` | HTTP chat (+ Civitai hop) |
| `AssistentChatWS` | Streaming chat WebSocket |

## License

MIT