Merge branch 'phase/61-portrait-loras-embeds'

This commit is contained in:
Leonid Pershin
2026-08-20 15:14:45 +03:00
18 changed files with 450 additions and 73 deletions
+1
View File
@@ -10,6 +10,7 @@
| [Что нового](whats-new.md) | 54 | | [Что нового](whats-new.md) | 54 |
| [Состояние экрана](view-state.md) | 55 | | [Состояние экрана](view-state.md) | 55 |
| [Модели портретов](portrait-models.md) | 60 | | [Модели портретов](portrait-models.md) | 60 |
| [LoRA и embeddings](portrait-loras-embeds.md) | 61 |
Люди: [`../02-people/people.md`](../02-people/people.md). Штат: [`../03-staffing/staffing.md`](../03-staffing/staffing.md). Люди: [`../02-people/people.md`](../02-people/people.md). Штат: [`../03-staffing/staffing.md`](../03-staffing/staffing.md).
Расписание: [`../04-schedule/schedule.md`](../04-schedule/schedule.md). Присутствие: [`../05-ai/ai.md`](../05-ai/ai.md). Расписание: [`../04-schedule/schedule.md`](../04-schedule/schedule.md). Присутствие: [`../05-ai/ai.md`](../05-ai/ai.md).
@@ -0,0 +1,27 @@
Часть [Вне очереди](README.md). Соседей по папке не читать, если задача не про них.
# LoRA и embeddings на каждом слое
### Зачем
Портрет собирается из модели, пресета и типа кадра. LoRA и textual inversion должны
включаться на любом из этих слоёв, а не только в дефолтах модели.
### Было / Стало / Почему
**Было.** Positive/negative LoRA только на модели и в оверрайде пресета. В Swarm уходила
одна строка `name,weight,…`. Embeddings из Swarm в пикере не было. Теги `<lora:>` в
негативном промпте Swarm не загружают сеть.
**Стало.** LoRA и embeddings (name + weight) на модели, на пресете (добавка) и на каждом
виде кадра. Слои **склеиваются**: модель → пресет → кадр. Все LoRA уходят в
`loras` + `loraweights`. Embeddings — в промпт как `<embed:name>` (вес ≠ 1:
`<embed:name:w>`). Discovery отдаёт список embeddings. По умолчанию списки пустые.
**Почему.** Swarm грузит LoRA параметрами, embeddings — текстом промпта. Кадр (аватар vs
полный рост) может нуждаться в своей LoRA, не меняя модель.
### Что не входит
Whitelist embeddings как у чекпоинтов. ControlNet. Включение PDXL embeds по умолчанию.
Бамп сокета.
@@ -0,0 +1,39 @@
# Фаза 61. LoRA и embeddings
## Зависимости
Фаза 60 (каталог моделей и слои промпта). Ветка от `phase/60-portrait-models`.
## Зачем
LoRA и embeddings можно поставить на модель, на пресет и на тип кадра. Swarm получает
их тем каналом, который реально загружает сеть.
## Задачи
- [x] Discovery: список `embeddings` из Swarm (`models.Embedding`)
- [x] Списки LoRA и embeddings (name + weight) на модели, пресете и avatar/custom/full.
Склейка модель → пресет → кадр. Пустые списки в шаблоне
- [x] Генерация: все LoRA в `loras` и `loraweights`; embeddings в positive/negative как
`<embed:name>`. Не писать `<lora:>` в промпт
- [x] UI: комбобоксы из discovery на модели, в спойлере пресета и в секции кадра. `t(...)`
- [x] HTTP в [`protocol.md`](../../protocol.md), сокет не бампить
## Тесты, без которых фаза не закрыта
- [x] Discovery парсит embeddings; офлайн — пустой список
- [x] Склейка LoRA/embed: модель + пресет + kind
- [x] Промпт содержит `<embed:…>` и не содержит `<lora:`
- [x] Generate шлёт `loras` и `loraweights` раздельно
- [x] Диалог: поля embeddings у модели и у вида кадра
## Критерий готовности
- В диалоге создания школы LoRA и embedding можно добавить на модель, в оверрайд пресета
и на аватар / свой промпт / полный рост
- Список имён — из Swarm, как у LoRA сейчас
- «Показать промпт» показывает embed-теги; LoRA в тексте промпта нет
## Стоп
Не бампить сокет. Не включать конкретные embeds в дефолтный шаблон. Не трогать ControlNet.
+1 -1
View File
@@ -20,4 +20,4 @@
| [54. Что нового](54-whats-new.md) | ✅ | После входа — окно коммитов с прошлого визита | | [54. Что нового](54-whats-new.md) | ✅ | После входа — окно коммитов с прошлого визита |
| [55. Состояние экрана](55-view-state.md) | ✅ | F5 и переходы не сбрасывают школу, вкладки и фильтры | | [55. Состояние экрана](55-view-state.md) | ✅ | F5 и переходы не сбрасывают школу, вкладки и фильтры |
| [60. Модели портретов](60-portrait-models.md) | ✅ | Каталог моделей Swarm ∩ конфиг, слои промпта, спойлер оверрайдов | | [60. Модели портретов](60-portrait-models.md) | ✅ | Каталог моделей Swarm ∩ конфиг, слои промпта, спойлер оверрайдов |
| [61. LoRA и embeddings](61-portrait-loras-embeds.md) | 🔄 | LoRA и embeddings на модели, пресете и типе кадра | | [61. LoRA и embeddings](61-portrait-loras-embeds.md) | | LoRA и embeddings на модели, пресете и типе кадра |
+13 -9
View File
@@ -507,9 +507,10 @@ land under `saves/{id}.portraits/` and survive until the school is deleted.
Returns the positive and negative prompts SwarmUI would receive, without generating an image. Returns the positive and negative prompts SwarmUI would receive, without generating an image.
Same `kind` query as GET portrait. For `kind=custom`, optional query `promptExtra` is appended after Same `kind` query as GET portrait. For `kind=custom`, optional query `promptExtra` is appended after
the shot type; when omitted, the last saved custom prompt is used if one exists. Unknown school the shot type; when omitted, the last saved custom prompt is used if one exists. Configured
is `404` `unknown-school`; unknown person is `404` `unknown-person`. Invalid `kind` is embeddings appear as `<embed:name>` (weight ≠ 1: `<embed:name:w>`); LoRA is not written into the
`400` `invalid-query`; custom without a usable prompt is `400` `invalid-body`. prompt. Unknown school is `404` `unknown-school`; unknown person is `404` `unknown-person`. Invalid
`kind` is `400` `invalid-query`; custom without a usable prompt is `400` `invalid-body`.
```json ```json
{ {
@@ -528,8 +529,10 @@ so the client can disable generate buttons and show reachability without trying
### `GET /api/settings/swarmui` ### `GET /api/settings/swarmui`
Returns the **default** SwarmUI template (`swarmui.json`): a `models` catalog (id, default Returns the **default** SwarmUI template (`swarmui.json`): a `models` catalog (id, default
generation knobs, usually empty base positive/negative), named presets (model id, `style`, extra generation knobs, usually empty base positive/negative, LoRA and embedding lists), named presets
negative, optional generation overrides, per-kind size/`shotType`), `activePresetId` and `ageRules`. (model id, `style`, extra negative, optional generation overrides, per-kind size/`shotType`, LoRA
and embedding lists), `activePresetId` and `ageRules`. LoRA and embeddings on the model, preset and
shot kind concatenate at generate time.
The model picker in the UI is Swarm's list intersected with `models`. A new school copies this into The model picker in the UI is Swarm's list intersected with `models`. A new school copies this into
its save as `portraitSettings`. Living schools generate from that copy, not from this file. Older its save as `portraitSettings`. Living schools generate from that copy, not from this file. Older
saves without `models` lift on load (`positive``style`, kind `positive``shotType`). saves without `models` lift on load (`positive``style`, kind `positive``shotType`).
@@ -542,10 +545,11 @@ return `400` `invalid-body`. Already-created schools keep the copy they were cre
### `GET /api/settings/swarmui/discovery` ### `GET /api/settings/swarmui/discovery`
When SwarmUI is configured and reachable, proxies `ListT2IParams` and returns When SwarmUI is configured and reachable, proxies `ListT2IParams` and returns
`{ connected, models, loras, samplers, schedulers }` for the settings UI. The client hides Swarm `{ connected, models, loras, embeddings, samplers, schedulers }` for the settings UI. LoRA and
models that are missing from the template catalog. When SwarmUI is off or unreachable, `connected` embeddings can be set on the model, the preset, and each shot kind; layers concatenate. The client
is false and the lists are empty; the picker then shows the catalog ids so fields can be filled hides Swarm models that are missing from the template catalog. When SwarmUI is off or unreachable,
manually. `connected` is false and the lists are empty; the picker then shows the catalog ids so fields can
be filled manually.
### `GET /api/schools/{id}/dress-rules` ### `GET /api/schools/{id}/dress-rules`
+6
View File
@@ -53,7 +53,10 @@ const ru = {
settingsOverrides: 'Настройки генерации пресета', settingsOverrides: 'Настройки генерации пресета',
settingsPositiveLoras: 'Positive LoRA', settingsPositiveLoras: 'Positive LoRA',
settingsNegativeLoras: 'Negative LoRA', settingsNegativeLoras: 'Negative LoRA',
settingsPositiveEmbeddings: 'Positive embeddings',
settingsNegativeEmbeddings: 'Negative embeddings',
settingsAddLora: 'Добавить LoRA', settingsAddLora: 'Добавить LoRA',
settingsAddEmbedding: 'Добавить embedding',
settingsAvatarPreset: 'Аватар', settingsAvatarPreset: 'Аватар',
settingsCustomPreset: 'Свой промпт (размеры)', settingsCustomPreset: 'Свой промпт (размеры)',
settingsFullBodyPreset: 'В полный рост', settingsFullBodyPreset: 'В полный рост',
@@ -437,7 +440,10 @@ const en: Messages = {
settingsOverrides: 'Preset generation overrides', settingsOverrides: 'Preset generation overrides',
settingsPositiveLoras: 'Positive LoRA', settingsPositiveLoras: 'Positive LoRA',
settingsNegativeLoras: 'Negative LoRA', settingsNegativeLoras: 'Negative LoRA',
settingsPositiveEmbeddings: 'Positive embeddings',
settingsNegativeEmbeddings: 'Negative embeddings',
settingsAddLora: 'Add LoRA', settingsAddLora: 'Add LoRA',
settingsAddEmbedding: 'Add embedding',
settingsAvatarPreset: 'Avatar', settingsAvatarPreset: 'Avatar',
settingsCustomPreset: 'Custom prompt (size)', settingsCustomPreset: 'Custom prompt (size)',
settingsFullBodyPreset: 'Full body', settingsFullBodyPreset: 'Full body',
+9
View File
@@ -498,6 +498,10 @@ export interface SwarmUiKindPreset {
width: number; width: number;
height: number; height: number;
shotType: string; shotType: string;
positiveLoras?: SwarmUiLoraEntry[] | null;
negativeLoras?: SwarmUiLoraEntry[] | null;
positiveEmbeddings?: SwarmUiLoraEntry[] | null;
negativeEmbeddings?: SwarmUiLoraEntry[] | null;
} }
export interface SwarmUiLoraEntry { export interface SwarmUiLoraEntry {
@@ -518,6 +522,8 @@ export interface SwarmUiModelDefinition {
negative: string; negative: string;
positiveLoras: SwarmUiLoraEntry[]; positiveLoras: SwarmUiLoraEntry[];
negativeLoras: SwarmUiLoraEntry[]; negativeLoras: SwarmUiLoraEntry[];
positiveEmbeddings: SwarmUiLoraEntry[];
negativeEmbeddings: SwarmUiLoraEntry[];
} }
export interface SwarmUiPresetDefinition { export interface SwarmUiPresetDefinition {
@@ -534,6 +540,8 @@ export interface SwarmUiPresetDefinition {
seed?: number | null; seed?: number | null;
positiveLoras?: SwarmUiLoraEntry[] | null; positiveLoras?: SwarmUiLoraEntry[] | null;
negativeLoras?: SwarmUiLoraEntry[] | null; negativeLoras?: SwarmUiLoraEntry[] | null;
positiveEmbeddings?: SwarmUiLoraEntry[] | null;
negativeEmbeddings?: SwarmUiLoraEntry[] | null;
avatar: SwarmUiKindPreset; avatar: SwarmUiKindPreset;
custom: SwarmUiKindPreset; custom: SwarmUiKindPreset;
fullBody: SwarmUiKindPreset; fullBody: SwarmUiKindPreset;
@@ -556,6 +564,7 @@ export interface SwarmUiDiscovery {
connected: boolean; connected: boolean;
models: readonly string[]; models: readonly string[];
loras: readonly string[]; loras: readonly string[];
embeddings: readonly string[];
samplers: readonly string[]; samplers: readonly string[];
schedulers: readonly string[]; schedulers: readonly string[];
} }
@@ -128,6 +128,8 @@ describe('createSchoolDialog', () => {
negative: '', negative: '',
positiveLoras: [], positiveLoras: [],
negativeLoras: [], negativeLoras: [],
positiveEmbeddings: [],
negativeEmbeddings: [],
}, },
], ],
presets: [ presets: [
@@ -43,6 +43,8 @@ function settings(): SwarmUiSettingsFile {
negative: '', negative: '',
positiveLoras: [], positiveLoras: [],
negativeLoras: [], negativeLoras: [],
positiveEmbeddings: [],
negativeEmbeddings: [],
}, },
{ {
id: 'other.safetensors', id: 'other.safetensors',
@@ -57,6 +59,8 @@ function settings(): SwarmUiSettingsFile {
negative: '', negative: '',
positiveLoras: [], positiveLoras: [],
negativeLoras: [], negativeLoras: [],
positiveEmbeddings: [],
negativeEmbeddings: [],
}, },
], ],
presets: [ presets: [
@@ -80,6 +84,7 @@ function discovery(overrides: Partial<SwarmUiDiscovery> = {}): SwarmUiDiscovery
connected: true, connected: true,
models: ['template.safetensors', 'not-in-config.safetensors'], models: ['template.safetensors', 'not-in-config.safetensors'],
loras: [], loras: [],
embeddings: [],
samplers: ['euler'], samplers: ['euler'],
schedulers: ['normal'], schedulers: ['normal'],
...overrides, ...overrides,
@@ -135,7 +140,8 @@ describe('swarmUiSettingsDialog', () => {
expect([...dialog.querySelectorAll('.field__label')].some((node) => node.textContent === t('settingsStyle'))).toBe(true); expect([...dialog.querySelectorAll('.field__label')].some((node) => node.textContent === t('settingsStyle'))).toBe(true);
expect([...dialog.querySelectorAll('.field__label')].some((node) => node.textContent === t('settingsShotType'))).toBe(true); expect([...dialog.querySelectorAll('.field__label')].some((node) => node.textContent === t('settingsShotType'))).toBe(true);
expect(dialog.querySelector('.settings-model')?.textContent).toContain(t('settingsModelDefaults')); expect(dialog.querySelector('.settings-model')?.textContent).toContain(t('settingsPositiveEmbeddings'));
expect([...dialog.querySelectorAll('.settings-kind .settings-subtitle')].some((node) => node.textContent === t('settingsPositiveEmbeddings'))).toBe(true);
const spoiler = dialog.querySelector('details.settings-overrides'); const spoiler = dialog.querySelector('details.settings-overrides');
if (!(spoiler instanceof HTMLDetailsElement)) { if (!(spoiler instanceof HTMLDetailsElement)) {
throw new Error('overrides spoiler is missing'); throw new Error('overrides spoiler is missing');
@@ -25,7 +25,7 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
const ageRulesHost = el('div', { class: 'settings-age-rules' }); const ageRulesHost = el('div', { class: 'settings-age-rules' });
let config: SwarmUiSettingsFile | null = null; let config: SwarmUiSettingsFile | null = null;
let discovery: SwarmUiDiscovery = { connected: false, models: [], loras: [], samplers: [], schedulers: [] }; let discovery: SwarmUiDiscovery = { connected: false, models: [], loras: [], embeddings: [], samplers: [], schedulers: [] };
let editingPresetId = ''; let editingPresetId = '';
const cancelButton = el('button', { class: 'button', type: 'button', onClick: () => modal.close(null) }); const cancelButton = el('button', { class: 'button', type: 'button', onClick: () => modal.close(null) });
@@ -239,6 +239,11 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
return host; return host;
} }
model.positiveLoras ??= [];
model.negativeLoras ??= [];
model.positiveEmbeddings ??= [];
model.negativeEmbeddings ??= [];
host.append( host.append(
textareaField(t('settingsModelPositive'), model.positive, (value) => { textareaField(t('settingsModelPositive'), model.positive, (value) => {
model.positive = value; model.positive = value;
@@ -264,8 +269,10 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
numberField(t('settingsSeed'), model.seed, (value) => { numberField(t('settingsSeed'), model.seed, (value) => {
model.seed = value; model.seed = value;
}), }),
loraSection(t('settingsPositiveLoras'), model.positiveLoras), loraSection(t('settingsPositiveLoras'), model.positiveLoras, discovery.loras, t('settingsAddLora')),
loraSection(t('settingsNegativeLoras'), model.negativeLoras), loraSection(t('settingsNegativeLoras'), model.negativeLoras, discovery.loras, t('settingsAddLora')),
loraSection(t('settingsPositiveEmbeddings'), model.positiveEmbeddings, discovery.embeddings, t('settingsAddEmbedding')),
loraSection(t('settingsNegativeEmbeddings'), model.negativeEmbeddings, discovery.embeddings, t('settingsAddEmbedding')),
); );
return host; return host;
} }
@@ -295,16 +302,33 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
}), }),
); );
const positiveLoras = preset.positiveLoras ?? []; const bindList = (
const negativeLoras = preset.negativeLoras ?? []; title: string,
preset.positiveLoras = preset.positiveLoras ?? null; current: SwarmUiLoraEntry[] | null | undefined,
const positiveHost = loraSection(t('settingsPositiveLoras'), positiveLoras, () => { names: readonly string[],
preset.positiveLoras = positiveLoras; addLabel: string,
assign: (items: SwarmUiLoraEntry[]) => void,
): HTMLElement => {
const items = current ?? [];
return loraSection(title, items, names, addLabel, () => {
assign(items);
}); });
const negativeHost = loraSection(t('settingsNegativeLoras'), negativeLoras, () => { };
preset.negativeLoras = negativeLoras;
}); body.append(
body.append(positiveHost, negativeHost); bindList(t('settingsPositiveLoras'), preset.positiveLoras, discovery.loras, t('settingsAddLora'), (items) => {
preset.positiveLoras = items;
}),
bindList(t('settingsNegativeLoras'), preset.negativeLoras, discovery.loras, t('settingsAddLora'), (items) => {
preset.negativeLoras = items;
}),
bindList(t('settingsPositiveEmbeddings'), preset.positiveEmbeddings, discovery.embeddings, t('settingsAddEmbedding'), (items) => {
preset.positiveEmbeddings = items;
}),
bindList(t('settingsNegativeEmbeddings'), preset.negativeEmbeddings, discovery.embeddings, t('settingsAddEmbedding'), (items) => {
preset.negativeEmbeddings = items;
}),
);
details.append(body); details.append(body);
return details; return details;
} }
@@ -367,32 +391,38 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
} }
} }
function loraSection(title: string, loras: SwarmUiLoraEntry[], onMutate?: () => void): HTMLElement { function loraSection(
title: string,
items: SwarmUiLoraEntry[],
names: readonly string[],
addLabel: string,
onMutate?: () => void,
): HTMLElement {
const host = el('div', { class: 'settings-loras' }); const host = el('div', { class: 'settings-loras' });
const heading = el('h4', { class: 'settings-subtitle', text: title }); const heading = el('h4', { class: 'settings-subtitle', text: title });
const rows = el('div', { class: 'settings-lora-rows' }); const rows = el('div', { class: 'settings-lora-rows' });
const paintRows = (): void => { const paintRows = (): void => {
rows.replaceChildren(); rows.replaceChildren();
for (const lora of loras) { for (const item of items) {
const nameField = choiceField('', lora.name, discovery.loras, (value) => { const nameField = choiceField('', item.name, names, (value) => {
lora.name = value; item.name = value;
}); });
const weightInput = el('input', { class: 'input', type: 'number', value: String(lora.weight) }); const weightInput = el('input', { class: 'input', type: 'number', value: String(item.weight) });
weightInput.step = '0.05'; weightInput.step = '0.05';
weightInput.min = '-4'; weightInput.min = '-4';
weightInput.max = '4'; weightInput.max = '4';
weightInput.addEventListener('input', () => { weightInput.addEventListener('input', () => {
lora.weight = Number(weightInput.value); item.weight = Number(weightInput.value);
}); });
const remove = el('button', { const remove = el('button', {
class: 'button button--small button--danger', class: 'button button--small button--danger',
type: 'button', type: 'button',
text: '×', text: '×',
onClick: () => { onClick: () => {
const index = loras.indexOf(lora); const index = items.indexOf(item);
if (index >= 0) { if (index >= 0) {
loras.splice(index, 1); items.splice(index, 1);
} }
paintRows(); paintRows();
@@ -406,9 +436,9 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
const add = el('button', { const add = el('button', {
class: 'button button--small', class: 'button button--small',
type: 'button', type: 'button',
text: t('settingsAddLora'), text: addLabel,
onClick: () => { onClick: () => {
loras.push({ name: discovery.loras[0] ?? '', weight: 1 }); items.push({ name: names[0] ?? '', weight: 1 });
onMutate?.(); onMutate?.();
paintRows(); paintRows();
}, },
@@ -419,6 +449,10 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
} }
function kindSection(title: string, preset: SwarmUiPresetDefinition['avatar']): HTMLElement { function kindSection(title: string, preset: SwarmUiPresetDefinition['avatar']): HTMLElement {
preset.positiveLoras ??= [];
preset.negativeLoras ??= [];
preset.positiveEmbeddings ??= [];
preset.negativeEmbeddings ??= [];
return el( return el(
'div', 'div',
{ class: 'settings-kind' }, { class: 'settings-kind' },
@@ -432,6 +466,10 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
textareaField(t('settingsShotType'), preset.shotType, (value) => { textareaField(t('settingsShotType'), preset.shotType, (value) => {
preset.shotType = value; preset.shotType = value;
}), }),
loraSection(t('settingsPositiveLoras'), preset.positiveLoras, discovery.loras, t('settingsAddLora')),
loraSection(t('settingsNegativeLoras'), preset.negativeLoras, discovery.loras, t('settingsAddLora')),
loraSection(t('settingsPositiveEmbeddings'), preset.positiveEmbeddings, discovery.embeddings, t('settingsAddEmbedding')),
loraSection(t('settingsNegativeEmbeddings'), preset.negativeEmbeddings, discovery.embeddings, t('settingsAddEmbedding')),
); );
} }
@@ -43,10 +43,12 @@ internal static class PortraitPromptBuilder
} }
} }
var positive = string.Join(", ", parts); var positive = SwarmUiLoraFormatter.AppendEmbedTags(
var negative = SwarmUiLoraFormatter.AppendLoraTags( string.Join(", ", parts),
profile.PositiveEmbeddings);
var negative = SwarmUiLoraFormatter.AppendEmbedTags(
profile.Negative.Trim(), profile.Negative.Trim(),
profile.NegativeLoras); profile.NegativeEmbeddings);
return (positive, negative); return (positive, negative);
} }
+5 -3
View File
@@ -128,10 +128,12 @@ internal sealed class SwarmUiClient
body["clipstopatlayer"] = -profile.ClipSkip; body["clipstopatlayer"] = -profile.ClipSkip;
} }
var loras = SwarmUiLoraFormatter.FormatForApi(profile.PositiveLoras); var (loraNames, loraWeights) = SwarmUiLoraFormatter.FormatForApi(
if (loras is not null) SwarmUiLoraFormatter.Concat(profile.PositiveLoras, profile.NegativeLoras));
if (loraNames is not null)
{ {
body["loras"] = loras; body["loras"] = loraNames;
body["loraweights"] = loraWeights;
} }
using var content = new StringContent(JsonSerializer.Serialize(body), Encoding.UTF8, "application/json"); using var content = new StringContent(JsonSerializer.Serialize(body), Encoding.UTF8, "application/json");
+4 -2
View File
@@ -6,10 +6,11 @@ internal sealed record SwarmUiDiscovery(
bool Connected, bool Connected,
IReadOnlyList<string> Models, IReadOnlyList<string> Models,
IReadOnlyList<string> Loras, IReadOnlyList<string> Loras,
IReadOnlyList<string> Embeddings,
IReadOnlyList<string> Samplers, IReadOnlyList<string> Samplers,
IReadOnlyList<string> Schedulers) IReadOnlyList<string> Schedulers)
{ {
public static SwarmUiDiscovery Offline { get; } = new(false, [], [], [], []); public static SwarmUiDiscovery Offline { get; } = new(false, [], [], [], [], []);
} }
internal static class SwarmUiDiscoveryParser internal static class SwarmUiDiscoveryParser
@@ -18,9 +19,10 @@ internal static class SwarmUiDiscoveryParser
{ {
var models = ReadModelNames(root, "Stable-Diffusion"); var models = ReadModelNames(root, "Stable-Diffusion");
var loras = ReadModelNames(root, "LoRA"); var loras = ReadModelNames(root, "LoRA");
var embeddings = ReadModelNames(root, "Embedding");
var samplers = ReadParamValues(root, "sampler"); var samplers = ReadParamValues(root, "sampler");
var schedulers = ReadParamValues(root, "scheduler"); var schedulers = ReadParamValues(root, "scheduler");
return new SwarmUiDiscovery(true, models, loras, samplers, schedulers); return new SwarmUiDiscovery(true, models, loras, embeddings, samplers, schedulers);
} }
private static IReadOnlyList<string> ReadModelNames(JsonElement root, string subtype) private static IReadOnlyList<string> ReadModelNames(JsonElement root, string subtype)
+55 -22
View File
@@ -2,15 +2,11 @@ namespace HSchool.Server.Game;
internal static class SwarmUiLoraFormatter internal static class SwarmUiLoraFormatter
{ {
/// <summary>SwarmUI comma-separated lora list: name,weight,name,weight,…</summary> /// <summary>SwarmUI comma-separated LoRA names and a matching weights string.</summary>
public static string? FormatForApi(IReadOnlyList<SwarmUiLoraEntry> loras) public static (string? Names, string? Weights) FormatForApi(IReadOnlyList<SwarmUiLoraEntry> loras)
{ {
if (loras.Count == 0) var names = new List<string>();
{ var weights = new List<string>();
return null;
}
var parts = new List<string>(loras.Count * 2);
foreach (var lora in loras) foreach (var lora in loras)
{ {
if (string.IsNullOrWhiteSpace(lora.Name)) if (string.IsNullOrWhiteSpace(lora.Name))
@@ -18,26 +14,21 @@ internal static class SwarmUiLoraFormatter
continue; continue;
} }
parts.Add(lora.Name.Trim()); names.Add(lora.Name.Trim());
parts.Add(lora.Weight.ToString(System.Globalization.CultureInfo.InvariantCulture)); weights.Add(lora.Weight.ToString(System.Globalization.CultureInfo.InvariantCulture));
} }
return parts.Count == 0 ? null : string.Join(',', parts); if (names.Count == 0)
}
public static string AppendLoraTags(string prompt, IReadOnlyList<SwarmUiLoraEntry> loras)
{ {
if (loras.Count == 0) return (null, null);
{
return prompt;
} }
var tags = loras return (string.Join(',', names), string.Join(',', weights));
.Where(lora => !string.IsNullOrWhiteSpace(lora.Name)) }
.Select(lora =>
$"<lora:{lora.Name.Trim()}:{lora.Weight.ToString(System.Globalization.CultureInfo.InvariantCulture)}>")
.ToList();
public static string AppendEmbedTags(string prompt, IReadOnlyList<SwarmUiLoraEntry> embeddings)
{
var tags = Tags(embeddings, "embed");
if (tags.Count == 0) if (tags.Count == 0)
{ {
return prompt; return prompt;
@@ -47,4 +38,46 @@ internal static class SwarmUiLoraFormatter
? string.Join(' ', tags) ? string.Join(' ', tags)
: $"{prompt.TrimEnd()} {string.Join(' ', tags)}"; : $"{prompt.TrimEnd()} {string.Join(' ', tags)}";
} }
public static List<SwarmUiLoraEntry> Concat(params IReadOnlyList<SwarmUiLoraEntry>?[] layers)
{
var result = new List<SwarmUiLoraEntry>();
foreach (var layer in layers)
{
if (layer is null)
{
continue;
}
foreach (var entry in layer)
{
if (!string.IsNullOrWhiteSpace(entry.Name))
{
result.Add(entry);
}
}
}
return result;
}
private static List<string> Tags(IReadOnlyList<SwarmUiLoraEntry> entries, string kind)
{
var tags = new List<string>();
foreach (var entry in entries)
{
if (string.IsNullOrWhiteSpace(entry.Name))
{
continue;
}
var name = entry.Name.Trim();
tags.Add(
entry.Weight == 1
? $"<{kind}:{name}>"
: $"<{kind}:{name}:{entry.Weight.ToString(System.Globalization.CultureInfo.InvariantCulture)}>");
}
return tags;
}
} }
@@ -151,6 +151,8 @@ internal sealed class SwarmUiConfigFile
{ {
model.PositiveLoras ??= []; model.PositiveLoras ??= [];
model.NegativeLoras ??= []; model.NegativeLoras ??= [];
model.PositiveEmbeddings ??= [];
model.NegativeEmbeddings ??= [];
if (string.IsNullOrWhiteSpace(model.Label)) if (string.IsNullOrWhiteSpace(model.Label))
{ {
model.Label = LabelFromId(model.Id); model.Label = LabelFromId(model.Id);
@@ -359,6 +361,16 @@ internal sealed class SwarmUiConfigFile
{ {
preset.NegativeLoras = null; preset.NegativeLoras = null;
} }
if (SameLoras(preset.PositiveEmbeddings, model.PositiveEmbeddings))
{
preset.PositiveEmbeddings = null;
}
if (SameLoras(preset.NegativeEmbeddings, model.NegativeEmbeddings))
{
preset.NegativeEmbeddings = null;
}
} }
public SwarmUiModelDefinition? FindModel(string id) => public SwarmUiModelDefinition? FindModel(string id) =>
@@ -484,6 +496,10 @@ internal sealed class SwarmUiModelDefinition
public List<SwarmUiLoraEntry>? NegativeLoras { get; set; } public List<SwarmUiLoraEntry>? NegativeLoras { get; set; }
public List<SwarmUiLoraEntry>? PositiveEmbeddings { get; set; }
public List<SwarmUiLoraEntry>? NegativeEmbeddings { get; set; }
public void Validate() public void Validate()
{ {
if (Steps is < 1 or > 200) if (Steps is < 1 or > 200)
@@ -501,8 +517,10 @@ internal sealed class SwarmUiModelDefinition
throw new InvalidOperationException($"Model '{Id}' clipSkip must be between 0 and 12."); throw new InvalidOperationException($"Model '{Id}' clipSkip must be between 0 and 12.");
} }
SwarmUiPresetDefinition.ValidateLoras(Id, PositiveLoras, "positive"); SwarmUiPresetDefinition.ValidateLoras(Id, PositiveLoras, "positive LoRA");
SwarmUiPresetDefinition.ValidateLoras(Id, NegativeLoras, "negative"); SwarmUiPresetDefinition.ValidateLoras(Id, NegativeLoras, "negative LoRA");
SwarmUiPresetDefinition.ValidateLoras(Id, PositiveEmbeddings, "positive embedding");
SwarmUiPresetDefinition.ValidateLoras(Id, NegativeEmbeddings, "negative embedding");
} }
public static List<SwarmUiModelDefinition> Catalog() => public static List<SwarmUiModelDefinition> Catalog() =>
@@ -532,6 +550,10 @@ internal sealed class SwarmUiModelDefinition
Sampler = sampler, Sampler = sampler,
Scheduler = scheduler, Scheduler = scheduler,
Seed = seed, Seed = seed,
PositiveLoras = [],
NegativeLoras = [],
PositiveEmbeddings = [],
NegativeEmbeddings = [],
}; };
} }
@@ -566,6 +588,10 @@ internal sealed class SwarmUiPresetDefinition
public List<SwarmUiLoraEntry>? NegativeLoras { get; set; } public List<SwarmUiLoraEntry>? NegativeLoras { get; set; }
public List<SwarmUiLoraEntry>? PositiveEmbeddings { get; set; }
public List<SwarmUiLoraEntry>? NegativeEmbeddings { get; set; }
public SwarmUiKindPreset? Avatar { get; set; } public SwarmUiKindPreset? Avatar { get; set; }
public SwarmUiKindPreset? Custom { get; set; } public SwarmUiKindPreset? Custom { get; set; }
@@ -602,8 +628,26 @@ internal sealed class SwarmUiPresetDefinition
throw new InvalidOperationException($"Preset '{Id}' clipSkip must be between 0 and 12."); throw new InvalidOperationException($"Preset '{Id}' clipSkip must be between 0 and 12.");
} }
ValidateLoras(Id, PositiveLoras, "positive"); ValidateLoras(Id, PositiveLoras, "positive LoRA");
ValidateLoras(Id, NegativeLoras, "negative"); ValidateLoras(Id, NegativeLoras, "negative LoRA");
ValidateLoras(Id, PositiveEmbeddings, "positive embedding");
ValidateLoras(Id, NegativeEmbeddings, "negative embedding");
ValidateKind(Avatar, "avatar");
ValidateKind(Custom, "custom");
ValidateKind(FullBody, "fullBody");
}
private void ValidateKind(SwarmUiKindPreset? kind, string name)
{
if (kind is null)
{
return;
}
ValidateLoras($"{Id}.{name}", kind.PositiveLoras, "positive LoRA");
ValidateLoras($"{Id}.{name}", kind.NegativeLoras, "negative LoRA");
ValidateLoras($"{Id}.{name}", kind.PositiveEmbeddings, "positive embedding");
ValidateLoras($"{Id}.{name}", kind.NegativeEmbeddings, "negative embedding");
} }
internal static void ValidateLoras(string ownerId, IReadOnlyList<SwarmUiLoraEntry>? loras, string side) internal static void ValidateLoras(string ownerId, IReadOnlyList<SwarmUiLoraEntry>? loras, string side)
@@ -617,12 +661,12 @@ internal sealed class SwarmUiPresetDefinition
{ {
if (string.IsNullOrWhiteSpace(lora.Name)) if (string.IsNullOrWhiteSpace(lora.Name))
{ {
throw new InvalidOperationException($"'{ownerId}' has an empty {side} LoRA name."); throw new InvalidOperationException($"'{ownerId}' has an empty {side} name.");
} }
if (lora.Weight is < -4 or > 4) if (lora.Weight is < -4 or > 4)
{ {
throw new InvalidOperationException($"'{ownerId}' LoRA '{lora.Name}' weight is out of range."); throw new InvalidOperationException($"'{ownerId}' {side} '{lora.Name}' weight is out of range.");
} }
} }
} }
@@ -657,8 +701,10 @@ internal sealed class SwarmUiPresetDefinition
kindPreset.ResolvedShotType(), kindPreset.ResolvedShotType(),
"", "",
negative, negative,
PositiveLoras ?? model.PositiveLoras ?? [], SwarmUiLoraFormatter.Concat(model.PositiveLoras, PositiveLoras, kindPreset.PositiveLoras),
NegativeLoras ?? model.NegativeLoras ?? [], SwarmUiLoraFormatter.Concat(model.NegativeLoras, NegativeLoras, kindPreset.NegativeLoras),
SwarmUiLoraFormatter.Concat(model.PositiveEmbeddings, PositiveEmbeddings, kindPreset.PositiveEmbeddings),
SwarmUiLoraFormatter.Concat(model.NegativeEmbeddings, NegativeEmbeddings, kindPreset.NegativeEmbeddings),
kindPreset); kindPreset);
} }
@@ -756,6 +802,14 @@ internal sealed class SwarmUiKindPreset
/// <summary>Legacy kind positive; copied into <see cref="ShotType"/> on load.</summary> /// <summary>Legacy kind positive; copied into <see cref="ShotType"/> on load.</summary>
public string? Positive { get; set; } public string? Positive { get; set; }
public List<SwarmUiLoraEntry>? PositiveLoras { get; set; }
public List<SwarmUiLoraEntry>? NegativeLoras { get; set; }
public List<SwarmUiLoraEntry>? PositiveEmbeddings { get; set; }
public List<SwarmUiLoraEntry>? NegativeEmbeddings { get; set; }
public void LiftShotType() public void LiftShotType()
{ {
if (string.IsNullOrWhiteSpace(ShotType) && !string.IsNullOrWhiteSpace(Positive)) if (string.IsNullOrWhiteSpace(ShotType) && !string.IsNullOrWhiteSpace(Positive))
@@ -794,4 +848,6 @@ internal sealed record SwarmUiResolvedProfile(
string Negative, string Negative,
IReadOnlyList<SwarmUiLoraEntry> PositiveLoras, IReadOnlyList<SwarmUiLoraEntry> PositiveLoras,
IReadOnlyList<SwarmUiLoraEntry> NegativeLoras, IReadOnlyList<SwarmUiLoraEntry> NegativeLoras,
IReadOnlyList<SwarmUiLoraEntry> PositiveEmbeddings,
IReadOnlyList<SwarmUiLoraEntry> NegativeEmbeddings,
SwarmUiKindPreset KindPreset); SwarmUiKindPreset KindPreset);
@@ -86,6 +86,7 @@ public class SwarmUiClientTests
preset.ClipSkip = 2; preset.ClipSkip = 2;
preset.Sampler = "dpmpp_sde"; preset.Sampler = "dpmpp_sde";
preset.Scheduler = "karras"; preset.Scheduler = "karras";
preset.PositiveLoras = [new SwarmUiLoraEntry { Name = "style.safetensors", Weight = 0.75 }];
var profile = preset.ToProfile(PortraitKind.Avatar); var profile = preset.ToProfile(PortraitKind.Avatar);
await client.GenerateAsync("a student", "bad", profile, CancellationToken.None); await client.GenerateAsync("a student", "bad", profile, CancellationToken.None);
@@ -93,6 +94,8 @@ public class SwarmUiClientTests
using var document = JsonDocument.Parse(handler.GenerateBody!); using var document = JsonDocument.Parse(handler.GenerateBody!);
Assert.Equal(-2, document.RootElement.GetProperty("clipstopatlayer").GetInt32()); Assert.Equal(-2, document.RootElement.GetProperty("clipstopatlayer").GetInt32());
Assert.False(document.RootElement.TryGetProperty("clipskip", out _)); Assert.False(document.RootElement.TryGetProperty("clipskip", out _));
Assert.Equal("style.safetensors", document.RootElement.GetProperty("loras").GetString());
Assert.Equal("0.75", document.RootElement.GetProperty("loraweights").GetString());
} }
[Fact] [Fact]
@@ -0,0 +1,146 @@
using System.Text.Json;
using HSchool.Server.Game;
namespace HSchool.Server.Tests;
public class SwarmUiLoraFormatterTests
{
[Fact]
public void FormatForApi_SplitsNamesAndWeights()
{
var (names, weights) = SwarmUiLoraFormatter.FormatForApi(
[
new SwarmUiLoraEntry { Name = "style.safetensors", Weight = 0.8 },
new SwarmUiLoraEntry { Name = "face.safetensors", Weight = 1 },
]);
Assert.Equal("style.safetensors,face.safetensors", names);
Assert.Equal("0.8,1", weights);
}
[Fact]
public void Concat_StacksModelPresetAndKind()
{
var stacked = SwarmUiLoraFormatter.Concat(
[new SwarmUiLoraEntry { Name = "model-lora", Weight = 1 }],
[new SwarmUiLoraEntry { Name = "preset-lora", Weight = 0.5 }],
[new SwarmUiLoraEntry { Name = "kind-lora", Weight = 0.2 }]);
Assert.Equal(["model-lora", "preset-lora", "kind-lora"], stacked.Select(entry => entry.Name));
}
[Fact]
public void AppendEmbedTags_AddsSwarmEmbedSyntax()
{
var prompt = SwarmUiLoraFormatter.AppendEmbedTags(
"a student",
[
new SwarmUiLoraEntry { Name = "pos.safetensors", Weight = 1 },
new SwarmUiLoraEntry { Name = "soft.safetensors", Weight = 0.6 },
]);
Assert.Contains("<embed:pos.safetensors>", prompt, StringComparison.Ordinal);
Assert.Contains("<embed:soft.safetensors:0.6>", prompt, StringComparison.Ordinal);
Assert.DoesNotContain("<lora:", prompt, StringComparison.Ordinal);
}
[Fact]
public void DiscoveryParser_ReadsEmbeddings()
{
using var document = JsonDocument.Parse(
"""
{
"models": {
"Stable-Diffusion": ["base.safetensors"],
"LoRA": ["style.safetensors"],
"Embedding": ["pos.safetensors", "neg.safetensors"]
},
"list": []
}
""");
var discovery = SwarmUiDiscoveryParser.Parse(document.RootElement);
Assert.Equal(["pos.safetensors", "neg.safetensors"], discovery.Embeddings);
Assert.Equal(["style.safetensors"], discovery.Loras);
Assert.Empty(SwarmUiDiscovery.Offline.Embeddings);
}
[Fact]
public void ToProfile_StacksLorasAndEmbeddingsAcrossLayers()
{
var model = new SwarmUiModelDefinition
{
Id = "m.safetensors",
PositiveLoras = [new SwarmUiLoraEntry { Name = "model-lora", Weight = 1 }],
PositiveEmbeddings = [new SwarmUiLoraEntry { Name = "model-embed", Weight = 1 }],
};
var preset = new SwarmUiPresetDefinition
{
Id = "default",
Label = "Default",
Model = "m.safetensors",
PositiveLoras = [new SwarmUiLoraEntry { Name = "preset-lora", Weight = 0.5 }],
Avatar = new SwarmUiKindPreset
{
Width = 512,
Height = 512,
PositiveEmbeddings = [new SwarmUiLoraEntry { Name = "avatar-embed", Weight = 0.8 }],
},
};
var profile = preset.ToProfile(PortraitKind.Avatar, model);
Assert.Equal(["model-lora", "preset-lora"], profile.PositiveLoras.Select(entry => entry.Name));
Assert.Equal(["model-embed", "avatar-embed"], profile.PositiveEmbeddings.Select(entry => entry.Name));
}
[Fact]
public void Build_PutsEmbedsInPromptAndOmitsLoraTags()
{
var model = new SwarmUiModelDefinition
{
Id = "m.safetensors",
PositiveLoras = [new SwarmUiLoraEntry { Name = "style-lora", Weight = 1 }],
NegativeEmbeddings = [new SwarmUiLoraEntry { Name = "neg-embed", Weight = 1 }],
};
var preset = SwarmUiPresetDefinition.CreateDefault();
preset.PositiveEmbeddings = [new SwarmUiLoraEntry { Name = "pos-embed", Weight = 1 }];
var card = new HSchool.Server.Api.PersonCardResponse(
"f0.c0",
"Maria Ivanova",
"Ivanova",
"Maria",
"",
true,
16,
new DateTime(2000, 3, 14, 0, 0, 0, DateTimeKind.Utc),
["student"],
5,
"A",
"class-1",
null,
null,
[],
[],
[],
[],
null,
null,
new HSchool.Server.Api.PersonFamilyResponse([], [], [], []),
[],
[],
0f,
0f);
var (positive, negative) = PortraitPromptBuilder.Build(
card,
preset.ToProfile(PortraitKind.Avatar, model),
PortraitKind.Avatar);
Assert.Contains("<embed:pos-embed>", positive, StringComparison.Ordinal);
Assert.Contains("<embed:neg-embed>", negative, StringComparison.Ordinal);
Assert.DoesNotContain("<lora:", positive, StringComparison.Ordinal);
Assert.DoesNotContain("<lora:", negative, StringComparison.Ordinal);
}
}
@@ -110,6 +110,7 @@ public class SwarmUiSettingsStoreTests
], ],
[], [],
[], [],
[],
[]); []);
var allowed = config.AllowedModelIds(discovery); var allowed = config.AllowedModelIds(discovery);