checkpoint before checking out main

This commit is contained in:
Leonid Pershin
2026-08-20 14:43:00 +03:00
parent 477f8cf6c9
commit 5d46550f12
18 changed files with 429 additions and 64 deletions
+1
View File
@@ -10,6 +10,7 @@
| [Что нового](whats-new.md) | 54 | | [Что нового](whats-new.md) | 54 |
| [Состояние экрана](view-state.md) | 55 | | [Состояние экрана](view-state.md) | 55 |
| [Модели портретов](portrait-models.md) | 60 | | [Модели портретов](portrait-models.md) | 60 |
| [LoRA и embeddings](portrait-loras-embeds.md) | 61 |
Люди: [`../02-people/people.md`](../02-people/people.md). Штат: [`../03-staffing/staffing.md`](../03-staffing/staffing.md). Люди: [`../02-people/people.md`](../02-people/people.md). Штат: [`../03-staffing/staffing.md`](../03-staffing/staffing.md).
Расписание: [`../04-schedule/schedule.md`](../04-schedule/schedule.md). Присутствие: [`../05-ai/ai.md`](../05-ai/ai.md). Расписание: [`../04-schedule/schedule.md`](../04-schedule/schedule.md). Присутствие: [`../05-ai/ai.md`](../05-ai/ai.md).
@@ -0,0 +1,27 @@
Часть [Вне очереди](README.md). Соседей по папке не читать, если задача не про них.
# LoRA и embeddings на каждом слое
### Зачем
Портрет собирается из модели, пресета и типа кадра. LoRA и textual inversion должны
включаться на любом из этих слоёв, а не только в дефолтах модели.
### Было / Стало / Почему
**Было.** Positive/negative LoRA только на модели и в оверрайде пресета. В Swarm уходила
одна строка `name,weight,…`. Embeddings из Swarm в пикере не было. Теги `<lora:>` в
негативном промпте Swarm не загружают сеть.
**Стало.** LoRA и embeddings (name + weight) на модели, на пресете (добавка) и на каждом
виде кадра. Слои **склеиваются**: модель → пресет → кадр. Все LoRA уходят в
`loras` + `loraweights`. Embeddings — в промпт как `<embed:name>` (вес ≠ 1:
`<embed:name:w>`). Discovery отдаёт список embeddings. По умолчанию списки пустые.
**Почему.** Swarm грузит LoRA параметрами, embeddings — текстом промпта. Кадр (аватар vs
полный рост) может нуждаться в своей LoRA, не меняя модель.
### Что не входит
Whitelist embeddings как у чекпоинтов. ControlNet. Включение PDXL embeds по умолчанию.
Бамп сокета.
@@ -0,0 +1,39 @@
# Фаза 61. LoRA и embeddings
## Зависимости
Фаза 60 (каталог моделей и слои промпта). Ветка от `phase/60-portrait-models`.
## Зачем
LoRA и embeddings можно поставить на модель, на пресет и на тип кадра. Swarm получает
их тем каналом, который реально загружает сеть.
## Задачи
- [ ] Discovery: список `embeddings` из Swarm (`models.Embedding`)
- [ ] Списки LoRA и embeddings (name + weight) на модели, пресете и avatar/custom/full.
Склейка модель → пресет → кадр. Пустые списки в шаблоне
- [ ] Генерация: все LoRA в `loras` и `loraweights`; embeddings в positive/negative как
`<embed:name>`. Не писать `<lora:>` в промпт
- [ ] UI: комбобоксы из discovery на модели, в спойлере пресета и в секции кадра. `t(...)`
- [ ] HTTP в [`protocol.md`](../../protocol.md), сокет не бампить
## Тесты, без которых фаза не закрыта
- [ ] Discovery парсит embeddings; офлайн — пустой список
- [ ] Склейка LoRA/embed: модель + пресет + kind
- [ ] Промпт содержит `<embed:…>` и не содержит `<lora:`
- [ ] Generate шлёт `loras` и `loraweights` раздельно
- [ ] Диалог: поля embeddings у модели и у вида кадра
## Критерий готовности
- В диалоге создания школы LoRA и embedding можно добавить на модель, в оверрайд пресета
и на аватар / свой промпт / полный рост
- Список имён — из Swarm, как у LoRA сейчас
- «Показать промпт» показывает embed-теги; LoRA в тексте промпта нет
## Стоп
Не бампить сокет. Не включать конкретные embeds в дефолтный шаблон. Не трогать ControlNet.
+1
View File
@@ -20,3 +20,4 @@
| [54. Что нового](54-whats-new.md) | ✅ | После входа — окно коммитов с прошлого визита | | [54. Что нового](54-whats-new.md) | ✅ | После входа — окно коммитов с прошлого визита |
| [55. Состояние экрана](55-view-state.md) | ✅ | F5 и переходы не сбрасывают школу, вкладки и фильтры | | [55. Состояние экрана](55-view-state.md) | ✅ | F5 и переходы не сбрасывают школу, вкладки и фильтры |
| [60. Модели портретов](60-portrait-models.md) | ✅ | Каталог моделей Swarm ∩ конфиг, слои промпта, спойлер оверрайдов | | [60. Модели портретов](60-portrait-models.md) | ✅ | Каталог моделей Swarm ∩ конфиг, слои промпта, спойлер оверрайдов |
| [61. LoRA и embeddings](61-portrait-loras-embeds.md) | 🔄 | LoRA и embeddings на модели, пресете и типе кадра |
+3 -2
View File
@@ -542,8 +542,9 @@ return `400` `invalid-body`. Already-created schools keep the copy they were cre
### `GET /api/settings/swarmui/discovery` ### `GET /api/settings/swarmui/discovery`
When SwarmUI is configured and reachable, proxies `ListT2IParams` and returns When SwarmUI is configured and reachable, proxies `ListT2IParams` and returns
`{ connected, models, loras, samplers, schedulers }` for the settings UI. The client hides Swarm `{ connected, models, loras, embeddings, samplers, schedulers }` for the settings UI. LoRA and
models that are missing from the template catalog. When SwarmUI is off or unreachable, `connected` embeddings can be set on the model, the preset, and each shot kind; layers concatenate.
The client hides Swarm models that are missing from the template catalog. When SwarmUI is off or unreachable, `connected`
is false and the lists are empty; the picker then shows the catalog ids so fields can be filled is false and the lists are empty; the picker then shows the catalog ids so fields can be filled
manually. manually.
+6
View File
@@ -53,7 +53,10 @@ const ru = {
settingsOverrides: 'Настройки генерации пресета', settingsOverrides: 'Настройки генерации пресета',
settingsPositiveLoras: 'Positive LoRA', settingsPositiveLoras: 'Positive LoRA',
settingsNegativeLoras: 'Negative LoRA', settingsNegativeLoras: 'Negative LoRA',
settingsPositiveEmbeddings: 'Positive embeddings',
settingsNegativeEmbeddings: 'Negative embeddings',
settingsAddLora: 'Добавить LoRA', settingsAddLora: 'Добавить LoRA',
settingsAddEmbedding: 'Добавить embedding',
settingsAvatarPreset: 'Аватар', settingsAvatarPreset: 'Аватар',
settingsCustomPreset: 'Свой промпт (размеры)', settingsCustomPreset: 'Свой промпт (размеры)',
settingsFullBodyPreset: 'В полный рост', settingsFullBodyPreset: 'В полный рост',
@@ -431,7 +434,10 @@ const en: Messages = {
settingsOverrides: 'Preset generation overrides', settingsOverrides: 'Preset generation overrides',
settingsPositiveLoras: 'Positive LoRA', settingsPositiveLoras: 'Positive LoRA',
settingsNegativeLoras: 'Negative LoRA', settingsNegativeLoras: 'Negative LoRA',
settingsPositiveEmbeddings: 'Positive embeddings',
settingsNegativeEmbeddings: 'Negative embeddings',
settingsAddLora: 'Add LoRA', settingsAddLora: 'Add LoRA',
settingsAddEmbedding: 'Add embedding',
settingsAvatarPreset: 'Avatar', settingsAvatarPreset: 'Avatar',
settingsCustomPreset: 'Custom prompt (size)', settingsCustomPreset: 'Custom prompt (size)',
settingsFullBodyPreset: 'Full body', settingsFullBodyPreset: 'Full body',
+9
View File
@@ -498,6 +498,10 @@ export interface SwarmUiKindPreset {
width: number; width: number;
height: number; height: number;
shotType: string; shotType: string;
positiveLoras?: SwarmUiLoraEntry[] | null;
negativeLoras?: SwarmUiLoraEntry[] | null;
positiveEmbeddings?: SwarmUiLoraEntry[] | null;
negativeEmbeddings?: SwarmUiLoraEntry[] | null;
} }
export interface SwarmUiLoraEntry { export interface SwarmUiLoraEntry {
@@ -518,6 +522,8 @@ export interface SwarmUiModelDefinition {
negative: string; negative: string;
positiveLoras: SwarmUiLoraEntry[]; positiveLoras: SwarmUiLoraEntry[];
negativeLoras: SwarmUiLoraEntry[]; negativeLoras: SwarmUiLoraEntry[];
positiveEmbeddings: SwarmUiLoraEntry[];
negativeEmbeddings: SwarmUiLoraEntry[];
} }
export interface SwarmUiPresetDefinition { export interface SwarmUiPresetDefinition {
@@ -534,6 +540,8 @@ export interface SwarmUiPresetDefinition {
seed?: number | null; seed?: number | null;
positiveLoras?: SwarmUiLoraEntry[] | null; positiveLoras?: SwarmUiLoraEntry[] | null;
negativeLoras?: SwarmUiLoraEntry[] | null; negativeLoras?: SwarmUiLoraEntry[] | null;
positiveEmbeddings?: SwarmUiLoraEntry[] | null;
negativeEmbeddings?: SwarmUiLoraEntry[] | null;
avatar: SwarmUiKindPreset; avatar: SwarmUiKindPreset;
custom: SwarmUiKindPreset; custom: SwarmUiKindPreset;
fullBody: SwarmUiKindPreset; fullBody: SwarmUiKindPreset;
@@ -556,6 +564,7 @@ export interface SwarmUiDiscovery {
connected: boolean; connected: boolean;
models: readonly string[]; models: readonly string[];
loras: readonly string[]; loras: readonly string[];
embeddings: readonly string[];
samplers: readonly string[]; samplers: readonly string[];
schedulers: readonly string[]; schedulers: readonly string[];
} }
@@ -128,6 +128,8 @@ describe('createSchoolDialog', () => {
negative: '', negative: '',
positiveLoras: [], positiveLoras: [],
negativeLoras: [], negativeLoras: [],
positiveEmbeddings: [],
negativeEmbeddings: [],
}, },
], ],
presets: [ presets: [
@@ -43,6 +43,8 @@ function settings(): SwarmUiSettingsFile {
negative: '', negative: '',
positiveLoras: [], positiveLoras: [],
negativeLoras: [], negativeLoras: [],
positiveEmbeddings: [],
negativeEmbeddings: [],
}, },
{ {
id: 'other.safetensors', id: 'other.safetensors',
@@ -57,6 +59,8 @@ function settings(): SwarmUiSettingsFile {
negative: '', negative: '',
positiveLoras: [], positiveLoras: [],
negativeLoras: [], negativeLoras: [],
positiveEmbeddings: [],
negativeEmbeddings: [],
}, },
], ],
presets: [ presets: [
@@ -80,6 +84,7 @@ function discovery(overrides: Partial<SwarmUiDiscovery> = {}): SwarmUiDiscovery
connected: true, connected: true,
models: ['template.safetensors', 'not-in-config.safetensors'], models: ['template.safetensors', 'not-in-config.safetensors'],
loras: [], loras: [],
embeddings: [],
samplers: ['euler'], samplers: ['euler'],
schedulers: ['normal'], schedulers: ['normal'],
...overrides, ...overrides,
@@ -135,7 +140,8 @@ describe('swarmUiSettingsDialog', () => {
expect([...dialog.querySelectorAll('.field__label')].some((node) => node.textContent === t('settingsStyle'))).toBe(true); expect([...dialog.querySelectorAll('.field__label')].some((node) => node.textContent === t('settingsStyle'))).toBe(true);
expect([...dialog.querySelectorAll('.field__label')].some((node) => node.textContent === t('settingsShotType'))).toBe(true); expect([...dialog.querySelectorAll('.field__label')].some((node) => node.textContent === t('settingsShotType'))).toBe(true);
expect(dialog.querySelector('.settings-model')?.textContent).toContain(t('settingsModelDefaults')); expect(dialog.querySelector('.settings-model')?.textContent).toContain(t('settingsPositiveEmbeddings'));
expect([...dialog.querySelectorAll('.settings-kind .settings-subtitle')].some((node) => node.textContent === t('settingsPositiveEmbeddings'))).toBe(true);
const spoiler = dialog.querySelector('details.settings-overrides'); const spoiler = dialog.querySelector('details.settings-overrides');
if (!(spoiler instanceof HTMLDetailsElement)) { if (!(spoiler instanceof HTMLDetailsElement)) {
throw new Error('overrides spoiler is missing'); throw new Error('overrides spoiler is missing');
@@ -264,8 +264,10 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
numberField(t('settingsSeed'), model.seed, (value) => { numberField(t('settingsSeed'), model.seed, (value) => {
model.seed = value; model.seed = value;
}), }),
loraSection(t('settingsPositiveLoras'), model.positiveLoras), loraSection(t('settingsPositiveLoras'), model.positiveLoras, discovery.loras, t('settingsAddLora')),
loraSection(t('settingsNegativeLoras'), model.negativeLoras), loraSection(t('settingsNegativeLoras'), model.negativeLoras, discovery.loras, t('settingsAddLora')),
loraSection(t('settingsPositiveEmbeddings'), model.positiveEmbeddings, discovery.embeddings, t('settingsAddEmbedding')),
loraSection(t('settingsNegativeEmbeddings'), model.negativeEmbeddings, discovery.embeddings, t('settingsAddEmbedding')),
); );
return host; return host;
} }
@@ -295,16 +297,33 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
}), }),
); );
const positiveLoras = preset.positiveLoras ?? []; const bindList = (
const negativeLoras = preset.negativeLoras ?? []; title: string,
preset.positiveLoras = preset.positiveLoras ?? null; current: SwarmUiLoraEntry[] | null | undefined,
const positiveHost = loraSection(t('settingsPositiveLoras'), positiveLoras, () => { names: readonly string[],
preset.positiveLoras = positiveLoras; addLabel: string,
}); assign: (items: SwarmUiLoraEntry[]) => void,
const negativeHost = loraSection(t('settingsNegativeLoras'), negativeLoras, () => { ): HTMLElement => {
preset.negativeLoras = negativeLoras; const items = current ?? [];
}); return loraSection(title, items, names, addLabel, () => {
body.append(positiveHost, negativeHost); assign(items);
});
};
body.append(
bindList(t('settingsPositiveLoras'), preset.positiveLoras, discovery.loras, t('settingsAddLora'), (items) => {
preset.positiveLoras = items;
}),
bindList(t('settingsNegativeLoras'), preset.negativeLoras, discovery.loras, t('settingsAddLora'), (items) => {
preset.negativeLoras = items;
}),
bindList(t('settingsPositiveEmbeddings'), preset.positiveEmbeddings, discovery.embeddings, t('settingsAddEmbedding'), (items) => {
preset.positiveEmbeddings = items;
}),
bindList(t('settingsNegativeEmbeddings'), preset.negativeEmbeddings, discovery.embeddings, t('settingsAddEmbedding'), (items) => {
preset.negativeEmbeddings = items;
}),
);
details.append(body); details.append(body);
return details; return details;
} }
@@ -367,32 +386,38 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
} }
} }
function loraSection(title: string, loras: SwarmUiLoraEntry[], onMutate?: () => void): HTMLElement { function loraSection(
title: string,
items: SwarmUiLoraEntry[],
names: readonly string[],
addLabel: string,
onMutate?: () => void,
): HTMLElement {
const host = el('div', { class: 'settings-loras' }); const host = el('div', { class: 'settings-loras' });
const heading = el('h4', { class: 'settings-subtitle', text: title }); const heading = el('h4', { class: 'settings-subtitle', text: title });
const rows = el('div', { class: 'settings-lora-rows' }); const rows = el('div', { class: 'settings-lora-rows' });
const paintRows = (): void => { const paintRows = (): void => {
rows.replaceChildren(); rows.replaceChildren();
for (const lora of loras) { for (const item of items) {
const nameField = choiceField('', lora.name, discovery.loras, (value) => { const nameField = choiceField('', item.name, names, (value) => {
lora.name = value; item.name = value;
}); });
const weightInput = el('input', { class: 'input', type: 'number', value: String(lora.weight) }); const weightInput = el('input', { class: 'input', type: 'number', value: String(item.weight) });
weightInput.step = '0.05'; weightInput.step = '0.05';
weightInput.min = '-4'; weightInput.min = '-4';
weightInput.max = '4'; weightInput.max = '4';
weightInput.addEventListener('input', () => { weightInput.addEventListener('input', () => {
lora.weight = Number(weightInput.value); item.weight = Number(weightInput.value);
}); });
const remove = el('button', { const remove = el('button', {
class: 'button button--small button--danger', class: 'button button--small button--danger',
type: 'button', type: 'button',
text: '×', text: '×',
onClick: () => { onClick: () => {
const index = loras.indexOf(lora); const index = items.indexOf(item);
if (index >= 0) { if (index >= 0) {
loras.splice(index, 1); items.splice(index, 1);
} }
paintRows(); paintRows();
@@ -406,9 +431,9 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
const add = el('button', { const add = el('button', {
class: 'button button--small', class: 'button button--small',
type: 'button', type: 'button',
text: t('settingsAddLora'), text: addLabel,
onClick: () => { onClick: () => {
loras.push({ name: discovery.loras[0] ?? '', weight: 1 }); items.push({ name: names[0] ?? '', weight: 1 });
onMutate?.(); onMutate?.();
paintRows(); paintRows();
}, },
@@ -419,6 +444,10 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
} }
function kindSection(title: string, preset: SwarmUiPresetDefinition['avatar']): HTMLElement { function kindSection(title: string, preset: SwarmUiPresetDefinition['avatar']): HTMLElement {
preset.positiveLoras ??= [];
preset.negativeLoras ??= [];
preset.positiveEmbeddings ??= [];
preset.negativeEmbeddings ??= [];
return el( return el(
'div', 'div',
{ class: 'settings-kind' }, { class: 'settings-kind' },
@@ -432,6 +461,10 @@ export function swarmUiSettingsDialog(initial?: SwarmUiSettingsFile): Promise<Sw
textareaField(t('settingsShotType'), preset.shotType, (value) => { textareaField(t('settingsShotType'), preset.shotType, (value) => {
preset.shotType = value; preset.shotType = value;
}), }),
loraSection(t('settingsPositiveLoras'), preset.positiveLoras, discovery.loras, t('settingsAddLora')),
loraSection(t('settingsNegativeLoras'), preset.negativeLoras, discovery.loras, t('settingsAddLora')),
loraSection(t('settingsPositiveEmbeddings'), preset.positiveEmbeddings, discovery.embeddings, t('settingsAddEmbedding')),
loraSection(t('settingsNegativeEmbeddings'), preset.negativeEmbeddings, discovery.embeddings, t('settingsAddEmbedding')),
); );
} }
@@ -43,10 +43,12 @@ internal static class PortraitPromptBuilder
} }
} }
var positive = string.Join(", ", parts); var positive = SwarmUiLoraFormatter.AppendEmbedTags(
var negative = SwarmUiLoraFormatter.AppendLoraTags( string.Join(", ", parts),
profile.PositiveEmbeddings);
var negative = SwarmUiLoraFormatter.AppendEmbedTags(
profile.Negative.Trim(), profile.Negative.Trim(),
profile.NegativeLoras); profile.NegativeEmbeddings);
return (positive, negative); return (positive, negative);
} }
+5 -3
View File
@@ -128,10 +128,12 @@ internal sealed class SwarmUiClient
body["clipstopatlayer"] = -profile.ClipSkip; body["clipstopatlayer"] = -profile.ClipSkip;
} }
var loras = SwarmUiLoraFormatter.FormatForApi(profile.PositiveLoras); var (loraNames, loraWeights) = SwarmUiLoraFormatter.FormatForApi(
if (loras is not null) SwarmUiLoraFormatter.Concat(profile.PositiveLoras, profile.NegativeLoras));
if (loraNames is not null)
{ {
body["loras"] = loras; body["loras"] = loraNames;
body["loraweights"] = loraWeights;
} }
using var content = new StringContent(JsonSerializer.Serialize(body), Encoding.UTF8, "application/json"); using var content = new StringContent(JsonSerializer.Serialize(body), Encoding.UTF8, "application/json");
+4 -2
View File
@@ -6,10 +6,11 @@ internal sealed record SwarmUiDiscovery(
bool Connected, bool Connected,
IReadOnlyList<string> Models, IReadOnlyList<string> Models,
IReadOnlyList<string> Loras, IReadOnlyList<string> Loras,
IReadOnlyList<string> Embeddings,
IReadOnlyList<string> Samplers, IReadOnlyList<string> Samplers,
IReadOnlyList<string> Schedulers) IReadOnlyList<string> Schedulers)
{ {
public static SwarmUiDiscovery Offline { get; } = new(false, [], [], [], []); public static SwarmUiDiscovery Offline { get; } = new(false, [], [], [], [], []);
} }
internal static class SwarmUiDiscoveryParser internal static class SwarmUiDiscoveryParser
@@ -18,9 +19,10 @@ internal static class SwarmUiDiscoveryParser
{ {
var models = ReadModelNames(root, "Stable-Diffusion"); var models = ReadModelNames(root, "Stable-Diffusion");
var loras = ReadModelNames(root, "LoRA"); var loras = ReadModelNames(root, "LoRA");
var embeddings = ReadModelNames(root, "Embedding");
var samplers = ReadParamValues(root, "sampler"); var samplers = ReadParamValues(root, "sampler");
var schedulers = ReadParamValues(root, "scheduler"); var schedulers = ReadParamValues(root, "scheduler");
return new SwarmUiDiscovery(true, models, loras, samplers, schedulers); return new SwarmUiDiscovery(true, models, loras, embeddings, samplers, schedulers);
} }
private static IReadOnlyList<string> ReadModelNames(JsonElement root, string subtype) private static IReadOnlyList<string> ReadModelNames(JsonElement root, string subtype)
+56 -23
View File
@@ -2,15 +2,11 @@ namespace HSchool.Server.Game;
internal static class SwarmUiLoraFormatter internal static class SwarmUiLoraFormatter
{ {
/// <summary>SwarmUI comma-separated lora list: name,weight,name,weight,…</summary> /// <summary>SwarmUI comma-separated LoRA names and a matching weights string.</summary>
public static string? FormatForApi(IReadOnlyList<SwarmUiLoraEntry> loras) public static (string? Names, string? Weights) FormatForApi(IReadOnlyList<SwarmUiLoraEntry> loras)
{ {
if (loras.Count == 0) var names = new List<string>();
{ var weights = new List<string>();
return null;
}
var parts = new List<string>(loras.Count * 2);
foreach (var lora in loras) foreach (var lora in loras)
{ {
if (string.IsNullOrWhiteSpace(lora.Name)) if (string.IsNullOrWhiteSpace(lora.Name))
@@ -18,26 +14,21 @@ internal static class SwarmUiLoraFormatter
continue; continue;
} }
parts.Add(lora.Name.Trim()); names.Add(lora.Name.Trim());
parts.Add(lora.Weight.ToString(System.Globalization.CultureInfo.InvariantCulture)); weights.Add(lora.Weight.ToString(System.Globalization.CultureInfo.InvariantCulture));
} }
return parts.Count == 0 ? null : string.Join(',', parts); if (names.Count == 0)
{
return (null, null);
}
return (string.Join(',', names), string.Join(',', weights));
} }
public static string AppendLoraTags(string prompt, IReadOnlyList<SwarmUiLoraEntry> loras) public static string AppendEmbedTags(string prompt, IReadOnlyList<SwarmUiLoraEntry> embeddings)
{ {
if (loras.Count == 0) var tags = Tags(embeddings, "embed");
{
return prompt;
}
var tags = loras
.Where(lora => !string.IsNullOrWhiteSpace(lora.Name))
.Select(lora =>
$"<lora:{lora.Name.Trim()}:{lora.Weight.ToString(System.Globalization.CultureInfo.InvariantCulture)}>")
.ToList();
if (tags.Count == 0) if (tags.Count == 0)
{ {
return prompt; return prompt;
@@ -47,4 +38,46 @@ internal static class SwarmUiLoraFormatter
? string.Join(' ', tags) ? string.Join(' ', tags)
: $"{prompt.TrimEnd()} {string.Join(' ', tags)}"; : $"{prompt.TrimEnd()} {string.Join(' ', tags)}";
} }
public static List<SwarmUiLoraEntry> Concat(params IReadOnlyList<SwarmUiLoraEntry>?[] layers)
{
var result = new List<SwarmUiLoraEntry>();
foreach (var layer in layers)
{
if (layer is null)
{
continue;
}
foreach (var entry in layer)
{
if (!string.IsNullOrWhiteSpace(entry.Name))
{
result.Add(entry);
}
}
}
return result;
}
private static List<string> Tags(IReadOnlyList<SwarmUiLoraEntry> entries, string kind)
{
var tags = new List<string>();
foreach (var entry in entries)
{
if (string.IsNullOrWhiteSpace(entry.Name))
{
continue;
}
var name = entry.Name.Trim();
tags.Add(
entry.Weight == 1
? $"<{kind}:{name}>"
: $"<{kind}:{name}:{entry.Weight.ToString(System.Globalization.CultureInfo.InvariantCulture)}>");
}
return tags;
}
} }
@@ -151,6 +151,8 @@ internal sealed class SwarmUiConfigFile
{ {
model.PositiveLoras ??= []; model.PositiveLoras ??= [];
model.NegativeLoras ??= []; model.NegativeLoras ??= [];
model.PositiveEmbeddings ??= [];
model.NegativeEmbeddings ??= [];
if (string.IsNullOrWhiteSpace(model.Label)) if (string.IsNullOrWhiteSpace(model.Label))
{ {
model.Label = LabelFromId(model.Id); model.Label = LabelFromId(model.Id);
@@ -359,6 +361,16 @@ internal sealed class SwarmUiConfigFile
{ {
preset.NegativeLoras = null; preset.NegativeLoras = null;
} }
if (SameLoras(preset.PositiveEmbeddings, model.PositiveEmbeddings))
{
preset.PositiveEmbeddings = null;
}
if (SameLoras(preset.NegativeEmbeddings, model.NegativeEmbeddings))
{
preset.NegativeEmbeddings = null;
}
} }
public SwarmUiModelDefinition? FindModel(string id) => public SwarmUiModelDefinition? FindModel(string id) =>
@@ -484,6 +496,10 @@ internal sealed class SwarmUiModelDefinition
public List<SwarmUiLoraEntry>? NegativeLoras { get; set; } public List<SwarmUiLoraEntry>? NegativeLoras { get; set; }
public List<SwarmUiLoraEntry>? PositiveEmbeddings { get; set; }
public List<SwarmUiLoraEntry>? NegativeEmbeddings { get; set; }
public void Validate() public void Validate()
{ {
if (Steps is < 1 or > 200) if (Steps is < 1 or > 200)
@@ -501,8 +517,10 @@ internal sealed class SwarmUiModelDefinition
throw new InvalidOperationException($"Model '{Id}' clipSkip must be between 0 and 12."); throw new InvalidOperationException($"Model '{Id}' clipSkip must be between 0 and 12.");
} }
SwarmUiPresetDefinition.ValidateLoras(Id, PositiveLoras, "positive"); SwarmUiPresetDefinition.ValidateLoras(Id, PositiveLoras, "positive LoRA");
SwarmUiPresetDefinition.ValidateLoras(Id, NegativeLoras, "negative"); SwarmUiPresetDefinition.ValidateLoras(Id, NegativeLoras, "negative LoRA");
SwarmUiPresetDefinition.ValidateLoras(Id, PositiveEmbeddings, "positive embedding");
SwarmUiPresetDefinition.ValidateLoras(Id, NegativeEmbeddings, "negative embedding");
} }
public static List<SwarmUiModelDefinition> Catalog() => public static List<SwarmUiModelDefinition> Catalog() =>
@@ -566,6 +584,10 @@ internal sealed class SwarmUiPresetDefinition
public List<SwarmUiLoraEntry>? NegativeLoras { get; set; } public List<SwarmUiLoraEntry>? NegativeLoras { get; set; }
public List<SwarmUiLoraEntry>? PositiveEmbeddings { get; set; }
public List<SwarmUiLoraEntry>? NegativeEmbeddings { get; set; }
public SwarmUiKindPreset? Avatar { get; set; } public SwarmUiKindPreset? Avatar { get; set; }
public SwarmUiKindPreset? Custom { get; set; } public SwarmUiKindPreset? Custom { get; set; }
@@ -602,8 +624,26 @@ internal sealed class SwarmUiPresetDefinition
throw new InvalidOperationException($"Preset '{Id}' clipSkip must be between 0 and 12."); throw new InvalidOperationException($"Preset '{Id}' clipSkip must be between 0 and 12.");
} }
ValidateLoras(Id, PositiveLoras, "positive"); ValidateLoras(Id, PositiveLoras, "positive LoRA");
ValidateLoras(Id, NegativeLoras, "negative"); ValidateLoras(Id, NegativeLoras, "negative LoRA");
ValidateLoras(Id, PositiveEmbeddings, "positive embedding");
ValidateLoras(Id, NegativeEmbeddings, "negative embedding");
ValidateKind(Avatar, "avatar");
ValidateKind(Custom, "custom");
ValidateKind(FullBody, "fullBody");
}
private void ValidateKind(SwarmUiKindPreset? kind, string name)
{
if (kind is null)
{
return;
}
ValidateLoras($"{Id}.{name}", kind.PositiveLoras, "positive LoRA");
ValidateLoras($"{Id}.{name}", kind.NegativeLoras, "negative LoRA");
ValidateLoras($"{Id}.{name}", kind.PositiveEmbeddings, "positive embedding");
ValidateLoras($"{Id}.{name}", kind.NegativeEmbeddings, "negative embedding");
} }
internal static void ValidateLoras(string ownerId, IReadOnlyList<SwarmUiLoraEntry>? loras, string side) internal static void ValidateLoras(string ownerId, IReadOnlyList<SwarmUiLoraEntry>? loras, string side)
@@ -617,12 +657,12 @@ internal sealed class SwarmUiPresetDefinition
{ {
if (string.IsNullOrWhiteSpace(lora.Name)) if (string.IsNullOrWhiteSpace(lora.Name))
{ {
throw new InvalidOperationException($"'{ownerId}' has an empty {side} LoRA name."); throw new InvalidOperationException($"'{ownerId}' has an empty {side} name.");
} }
if (lora.Weight is < -4 or > 4) if (lora.Weight is < -4 or > 4)
{ {
throw new InvalidOperationException($"'{ownerId}' LoRA '{lora.Name}' weight is out of range."); throw new InvalidOperationException($"'{ownerId}' {side} '{lora.Name}' weight is out of range.");
} }
} }
} }
@@ -657,8 +697,10 @@ internal sealed class SwarmUiPresetDefinition
kindPreset.ResolvedShotType(), kindPreset.ResolvedShotType(),
"", "",
negative, negative,
PositiveLoras ?? model.PositiveLoras ?? [], SwarmUiLoraFormatter.Concat(model.PositiveLoras, PositiveLoras, kindPreset.PositiveLoras),
NegativeLoras ?? model.NegativeLoras ?? [], SwarmUiLoraFormatter.Concat(model.NegativeLoras, NegativeLoras, kindPreset.NegativeLoras),
SwarmUiLoraFormatter.Concat(model.PositiveEmbeddings, PositiveEmbeddings, kindPreset.PositiveEmbeddings),
SwarmUiLoraFormatter.Concat(model.NegativeEmbeddings, NegativeEmbeddings, kindPreset.NegativeEmbeddings),
kindPreset); kindPreset);
} }
@@ -756,6 +798,14 @@ internal sealed class SwarmUiKindPreset
/// <summary>Legacy kind positive; copied into <see cref="ShotType"/> on load.</summary> /// <summary>Legacy kind positive; copied into <see cref="ShotType"/> on load.</summary>
public string? Positive { get; set; } public string? Positive { get; set; }
public List<SwarmUiLoraEntry>? PositiveLoras { get; set; }
public List<SwarmUiLoraEntry>? NegativeLoras { get; set; }
public List<SwarmUiLoraEntry>? PositiveEmbeddings { get; set; }
public List<SwarmUiLoraEntry>? NegativeEmbeddings { get; set; }
public void LiftShotType() public void LiftShotType()
{ {
if (string.IsNullOrWhiteSpace(ShotType) && !string.IsNullOrWhiteSpace(Positive)) if (string.IsNullOrWhiteSpace(ShotType) && !string.IsNullOrWhiteSpace(Positive))
@@ -794,4 +844,6 @@ internal sealed record SwarmUiResolvedProfile(
string Negative, string Negative,
IReadOnlyList<SwarmUiLoraEntry> PositiveLoras, IReadOnlyList<SwarmUiLoraEntry> PositiveLoras,
IReadOnlyList<SwarmUiLoraEntry> NegativeLoras, IReadOnlyList<SwarmUiLoraEntry> NegativeLoras,
IReadOnlyList<SwarmUiLoraEntry> PositiveEmbeddings,
IReadOnlyList<SwarmUiLoraEntry> NegativeEmbeddings,
SwarmUiKindPreset KindPreset); SwarmUiKindPreset KindPreset);
@@ -86,6 +86,7 @@ public class SwarmUiClientTests
preset.ClipSkip = 2; preset.ClipSkip = 2;
preset.Sampler = "dpmpp_sde"; preset.Sampler = "dpmpp_sde";
preset.Scheduler = "karras"; preset.Scheduler = "karras";
preset.PositiveLoras = [new SwarmUiLoraEntry { Name = "style.safetensors", Weight = 0.75 }];
var profile = preset.ToProfile(PortraitKind.Avatar); var profile = preset.ToProfile(PortraitKind.Avatar);
await client.GenerateAsync("a student", "bad", profile, CancellationToken.None); await client.GenerateAsync("a student", "bad", profile, CancellationToken.None);
@@ -93,6 +94,8 @@ public class SwarmUiClientTests
using var document = JsonDocument.Parse(handler.GenerateBody!); using var document = JsonDocument.Parse(handler.GenerateBody!);
Assert.Equal(-2, document.RootElement.GetProperty("clipstopatlayer").GetInt32()); Assert.Equal(-2, document.RootElement.GetProperty("clipstopatlayer").GetInt32());
Assert.False(document.RootElement.TryGetProperty("clipskip", out _)); Assert.False(document.RootElement.TryGetProperty("clipskip", out _));
Assert.Equal("style.safetensors", document.RootElement.GetProperty("loras").GetString());
Assert.Equal("0.75", document.RootElement.GetProperty("loraweights").GetString());
} }
[Fact] [Fact]
@@ -0,0 +1,145 @@
using System.Text.Json;
using HSchool.Server.Game;
namespace HSchool.Server.Tests;
public class SwarmUiLoraFormatterTests
{
[Fact]
public void FormatForApi_SplitsNamesAndWeights()
{
var (names, weights) = SwarmUiLoraFormatter.FormatForApi(
[
new SwarmUiLoraEntry { Name = "style.safetensors", Weight = 0.8 },
new SwarmUiLoraEntry { Name = "face.safetensors", Weight = 1 },
]);
Assert.Equal("style.safetensors,face.safetensors", names);
Assert.Equal("0.8,1", weights);
}
[Fact]
public void Concat_StacksModelPresetAndKind()
{
var stacked = SwarmUiLoraFormatter.Concat(
[new SwarmUiLoraEntry { Name = "model-lora", Weight = 1 }],
[new SwarmUiLoraEntry { Name = "preset-lora", Weight = 0.5 }],
[new SwarmUiLoraEntry { Name = "kind-lora", Weight = 0.2 }]);
Assert.Equal(["model-lora", "preset-lora", "kind-lora"], stacked.Select(entry => entry.Name));
}
[Fact]
public void AppendEmbedTags_AddsSwarmEmbedSyntax()
{
var prompt = SwarmUiLoraFormatter.AppendEmbedTags(
"a student",
[
new SwarmUiLoraEntry { Name = "pos.safetensors", Weight = 1 },
new SwarmUiLoraEntry { Name = "soft.safetensors", Weight = 0.6 },
]);
Assert.Contains("<embed:pos.safetensors>", prompt, StringComparison.Ordinal);
Assert.Contains("<embed:soft.safetensors:0.6>", prompt, StringComparison.Ordinal);
Assert.DoesNotContain("<lora:", prompt, StringComparison.Ordinal);
}
[Fact]
public void DiscoveryParser_ReadsEmbeddings()
{
using var document = JsonDocument.Parse(
"""
{
"models": {
"Stable-Diffusion": ["base.safetensors"],
"LoRA": ["style.safetensors"],
"Embedding": ["pos.safetensors", "neg.safetensors"]
},
"list": []
}
""");
var discovery = SwarmUiDiscoveryParser.Parse(document.RootElement);
Assert.Equal(["pos.safetensors", "neg.safetensors"], discovery.Embeddings);
Assert.Equal(["style.safetensors"], discovery.Loras);
}
[Fact]
public void ToProfile_StacksLorasAndEmbeddingsAcrossLayers()
{
var model = new SwarmUiModelDefinition
{
Id = "m.safetensors",
PositiveLoras = [new SwarmUiLoraEntry { Name = "model-lora", Weight = 1 }],
PositiveEmbeddings = [new SwarmUiLoraEntry { Name = "model-embed", Weight = 1 }],
};
var preset = new SwarmUiPresetDefinition
{
Id = "default",
Label = "Default",
Model = "m.safetensors",
PositiveLoras = [new SwarmUiLoraEntry { Name = "preset-lora", Weight = 0.5 }],
Avatar = new SwarmUiKindPreset
{
Width = 512,
Height = 512,
PositiveEmbeddings = [new SwarmUiLoraEntry { Name = "avatar-embed", Weight = 0.8 }],
},
};
var profile = preset.ToProfile(PortraitKind.Avatar, model);
Assert.Equal(["model-lora", "preset-lora"], profile.PositiveLoras.Select(entry => entry.Name));
Assert.Equal(["model-embed", "avatar-embed"], profile.PositiveEmbeddings.Select(entry => entry.Name));
}
[Fact]
public void Build_PutsEmbedsInPromptAndOmitsLoraTags()
{
var model = new SwarmUiModelDefinition
{
Id = "m.safetensors",
PositiveLoras = [new SwarmUiLoraEntry { Name = "style-lora", Weight = 1 }],
NegativeEmbeddings = [new SwarmUiLoraEntry { Name = "neg-embed", Weight = 1 }],
};
var preset = SwarmUiPresetDefinition.CreateDefault();
preset.PositiveEmbeddings = [new SwarmUiLoraEntry { Name = "pos-embed", Weight = 1 }];
var card = new HSchool.Server.Api.PersonCardResponse(
"f0.c0",
"Maria Ivanova",
"Ivanova",
"Maria",
"",
true,
16,
new DateTime(2000, 3, 14, 0, 0, 0, DateTimeKind.Utc),
["student"],
5,
"A",
"class-1",
null,
null,
[],
[],
[],
[],
null,
null,
new HSchool.Server.Api.PersonFamilyResponse([], [], [], []),
[],
[],
0f,
0f);
var (positive, negative) = PortraitPromptBuilder.Build(
card,
preset.ToProfile(PortraitKind.Avatar, model),
PortraitKind.Avatar);
Assert.Contains("<embed:pos-embed>", positive, StringComparison.Ordinal);
Assert.Contains("<embed:neg-embed>", negative, StringComparison.Ordinal);
Assert.DoesNotContain("<lora:", positive, StringComparison.Ordinal);
Assert.DoesNotContain("<lora:", negative, StringComparison.Ordinal);
}
}
@@ -110,6 +110,7 @@ public class SwarmUiSettingsStoreTests
], ],
[], [],
[], [],
[],
[]); []);
var allowed = config.AllowedModelIds(discovery); var allowed = config.AllowedModelIds(discovery);