Files

T

Leonid PershinandCursor 07bb521b70 Add Assistent reference books and remove training dataset output.

Introduce books/ with civitai-krea2 and HF fictext builders, sha-diff seed to VM, and drop train.jsonl from the civitai scrape pipeline.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-08-23 22:56:07 +03:00

1.0 KiB

Raw Blame History

Assistent reference books (local)

FTS search.jsonl indexes for Swarm Assistent Knowledge Hub. Not in Typer main CLI — use:

books-dataset.bat              # python -m gpu_rent.books
python -m gpu_rent.books list
python -m gpu_rent.books build civitai
python -m gpu_rent.books download fictext --variant both
gpu-rent seed-books            # push changed books → VM

Layout

Path	Role
`civitai-krea2/`	From `civitai-dataset split` → `book.yaml` + `search.jsonl` + `meta.json`
`ru-fictext-rplus/`	HF `ru-fictext-nsfw-data-r+.parquet` → chunked search rows
`ru-fictext-full/`	HF full parquet

On up / seed-personas / seed-books: changed books (by meta.content_sha) → /mnt/swarm_data/Assistent/books/<id>/.

Civitai scrape: manual on laptop; up pushes when local book changed.

Fictext: pip install 'gpu-rent[books]' (pyarrow) for build.

See docs/local-folders.md, datasets/README.md.

1.0 KiB Raw Blame History

Assistent reference books (local)

Layout

1.0 KiB

Raw Blame History