swarm-assistent/README.md

# Swarm Assistent

SwarmUI extension for **collaborative Krea 2** prompting via **Ollama**: chat + board (Generate | Refs tabs), LoRA chips, **persona presets** (`Config/personas/`), **vector memory**, model cards with Civitai fetch, img2img/inpaint, slash commands, auto Generate.

**Version 0.9.0** — Persona **shelves** (short JSON files, nested identity markdown), per-persona **Exact controls** (e.g. Leonid `preference_bias` slider), overlay **clone/author** pack (`/persona new`), UI-only delete for overlay personas. Chats/UI/taste stay in `assistent.sqlite`.

## Layout

- **Left — Board tabs:** **Generate** (full-height live view) | **Refs** (reference grid + badge `N · vision M`); **Посмотри результат** attaches the finished frame and asks for a verdict
- **Splitter:** drag to resize panes
- **Right:** Chat | Cards; persona / pack / Ollama chat model; **Ollama health** badge; settings gear (memory model + skills + **Память**)
- **Chips / slash:** loaded from `Config/_base/ui.json` (persona can override)

## Config (bundled + overlay)

```
Config/
  _base/           # defaults (assistant, ui, models/krea2, exact.json, core, packs, skills, memory-seed, identity)
  personas/<id>/   # sparse shelves: persona/bio/voice/humor/… + optional controls.json / exact.json / memory-seed
```

Disk overlay (wins over bundled): `/mnt/swarm_data/Assistent/` — same folder layout as `Config/`. Drop `_base/…` and `personas/<id>/…` to override. Plus runtime state:

```
Assistent/
  _base/ personas/<id>/    # overlay presets — same names as Config/, sparse
  settings.json            # embed_model, base_url, per-persona skills
  ollama-roles.json        # chat vs memory model tags (gpu-rent writes this)
  memory/assistent.sqlite  # vector memory + tags FTS + chats + ui_state + taste
  _migrated_json/          # one-shot archive of old chats/*.json, ui-state.json, taste.json
```

Copy `personas/leonid/` → new id, edit only differing JSON. See `Config/personas/README.md`.

**Controls:** optional `controls.json` schema + `exact.controls` values. UI shows sliders; LLM may patch `"controls": {…}`. Values persist in overlay Exact (not session_exact).

**Authoring:** pack `author_persona` + `/persona new` clones to overlay only. Delete overlay personas with the ✕ button (never from the model).

## Exact memory (KV)

- `Config/_base/exact.json` — canonical generation defaults, profiles (turbo/raw), aspect table, short facts
- Persona / disk overlays merge via DeepMerge (matching keys overwrite)
- Always injected into the system prompt; UI fills **empty** SwarmUI fields from Exact (no LLM call)
- Chat-session overrides (`session_exact`) last until persona change or clear chat — not written to disk
- Priority: core contract → current user → session_exact → exact (+ persona) → live fields → vector `memory_hits`

## Vector memory

Two layers in `memory/assistent.sqlite` (`persona` column; empty = shared):

- **Shared** — `Config/_base/memory-seed/`, model cards, `scope: "shared"` upserts. Visible to every persona.
- **Personal** — `Config/personas/<id>/memory-seed/` and chat upserts (default). Never copied into shared. Other personas do not retrieve it.
- Retrieve = shared ∪ this persona (and `extends` parents). Hybrid **FTS5 + cosine**, kind quotas (e.g. 3 cards / 3 pitfalls / 4 notes), `min_score`. Same `kind`+`key`: personal overwrites parent overwrites shared.
- Tools: `memory_get`, `memory_search`, `lookup_tags` (Danbooru csv in `Data/Autocompletions`, FTS, **no embeddings**).
- SQLite + Ollama `/api/embed` (default `nomic-embed-text`, pick in ⚙)
- Soft notes only — Exact and the user beat RAG for params
- ⚙ → **Память** lists every row (scope · source · date) with a per-row forget; bundled rows are read-only because reseed brings them back

## Chats and runtime KV

- Every chat (messages + Generate params snapshot) is a row in `assistent.sqlite`, newest **200** kept. History search uses FTS over title + body.
- First launch after 0.8.3 copies `chats/*.json`, `ui-state.json`, and `taste.json` into sqlite, then archives them under `_migrated_json/`.
- localStorage stays as a fast cache; on first run with an empty store the old `swarm_assistent_chats_v1` browser history is migrated up once.
- UI state seeds a **fresh** browser only — anything already in localStorage wins, and `auto_download` is never restored as on
- `settings.json` and persona overlays stay files (layered merge + git). `.assistent.json` cards stay next to weights.
## VRAM handover

- Before every Generate the chat model is unloaded (`keep_alive: 0`) so Krea 2 gets the whole GPU
- Back in the Chat tab it is warmed again with a 1-token request (`keep_alive 15m`, `num_ctx` from `assistant.json`)
- Embed / memory models are never parked — reloading them would stall every retrieve

## UX

- **Send to Assistent** under Generate/History → Ref + Assistent tab
- Enter sends; Shift+Enter newline; Interrupt cancels chat epoch
- Manual **Apply + Generate** / `/gen` always generate; Auto-generate checkbox only for LLM auto-path
- **Посмотри результат** / auto-critique wait for a real Generate frame — model previews and unfinished batches are skipped
- Civitai Confirm required (unless auto-download); queued-but-missing models show a `⏳ wanted` badge in Cards

### Slash commands (client-side, no LLM)

| Command | Effect |
| --- | --- |
| `/help` | List commands |
| `/debug` | Short UI/Exact dump (no LLM) |
| `/debug ask` / `/why` | Dump + short model explanation |
| `/gen` | Generate now |
| `/look generate\|refN` | Attach that board window + ask the LLM to look |
| `/init` `/mask` `/clear` | Same as board buttons |
| `/interrupt` | Stop generation / cancel chat |
| `/aspect 16:9` | Set size from the official 1K table |
| `/seed lock\|random` | Lock or randomize seed |
| `/vary` | New seed, same prompt (+ generate if auto) |
| `/pack write\|critique\|…` | Switch pack |
| `/civitai <query>` | Ask LLM to search Civitai |
| `/inventory` | Rescan models + refresh LoRA list |

## Requirements

- SwarmUI with a **Krea 2** checkpoint selected
- Ollama on `http://127.0.0.1:11434` **on the GPU VM** (gpu-rent `LLM_RUNTIME=ollama`)
- Chat model + memory embed (`use: memory` in `ollama-models.yaml`; gpu-rent creates CPU variant)
- Optional: Civitai API key in SwarmUI User Settings

## Install

```yaml
swarmui:
  - url: https://gitea.hsrv.site/mrleo1nid/swarm-assistent.git
    ref: main
    dir: swarm-assistent
    requires: ollama
```

Restart / rebuild SwarmUI after clone. gpu-rent: `seed-extensions` + restart.

## Packs & skills

**Packs** (one active): `write_prompt`, `critique_image`, `compose_scene`, `fix_params`, `inpaint_edit`, `describe_ref`, `catalog_card`.

**Skills** (checkboxes): `prompting`, `creativity_sliders`, `memory` — procedures; encyclopedia numbers live in Exact, soft notes in memory-seed / RAG.

**Personas:** `neutral`, `lewd`, `aggressive`, `cinema`, `terse`, `leonid` under `Config/personas/`. Overlay clones via `/persona new`.

## API routes

| Route | Role |
| --- | --- |
| `AssistentListModels` | Ollama tags → `models` (chat) + `memory_models` |
| `AssistentGetConfig` | Merged preset for persona (ui, packs, skills, identity, controls) |
| `AssistentSaveControls` | Persist Exact `controls` values for a persona (overlay) |
| `AssistentGetPersonaShelves` | Merged identity shelves + controls |
| `AssistentClonePersona` | Snapshot clone → overlay id |
| `AssistentSavePersona` | Sparse shelf write (overlay only) |
| `AssistentDeletePersona` | UI-only delete of overlay persona |
| `AssistentGetSettings` / `AssistentSaveSettings` | Overlay settings (skills, embed_model) |
| `AssistentListInventory` | LoRA / checkpoint / wildcard inventory |
| `AssistentListPersonas` | Persona catalog |
| `AssistentGetPacks` | Prompt pack texts |
| `AssistentGetCard` / `AssistentSaveCard` | `.assistent.json` cards (+ memory ingest) |
| `AssistentGetCardMeta` | Local sidecar + optional Civitai by-hash |
| `AssistentEnqueueWanted` / `AssistentListWanted` | Wanted YAML queue (write / read + count) |
| `AssistentGetTaste` / `AssistentSaveTaste` | sqlite `kv.taste` |
| `AssistentSearchCivitai` | Civitai LoRA search |
| `AssistentChat` / `AssistentChatWS` | Chat (+ hybrid memory + Civitai/tag hops) |
| `AssistentListMemory` / `AssistentUpsertMemory` / `AssistentForgetMemory` | Vector store (optional `scope` / `persona`) |
| `AssistentSearchMemory` / `AssistentGetMemory` | Hybrid search / exact kind+key |
| `AssistentLookupTags` | Danbooru csv FTS (no embeddings) |
| `AssistentListChats` / `AssistentGetChat` / `AssistentSaveChat` / `AssistentDeleteChat` | sqlite `chats` (optional `q` FTS) |
| `AssistentGetUiState` / `AssistentSaveUiState` | sqlite `kv.ui_state` |
| `AssistentParkLlm` / `AssistentWarmLlm` | Unload / reload the chat model in VRAM |

## License

MIT