feat: local image generation - #79
Merged
Merged
Conversation
Add a desktop-only local image runtime that discovers upstream releases dynamically, lets users select compatible CPU and accelerator builds, verifies model and runtime artifacts, and registers curated image models without fixed database IDs. Coordinate the sidecar with llama.cpp so only one local workload owns the GPU, support multi-reference and structured LoRA requests, and replace guessed VRAM sizing with an exact one-step runtime probe. Expose the curated installer and download state in image-generation settings, localize the new controls across all supported locales, and hide synced stable-diffusion.cpp models and providers on mobile.
Detect compatible stable-diffusion.cpp builds and recommend a GPU-specific runtime variant while keeping manual variant selection available. Add installation, switching, deletion, and persistent active-version handling through a dedicated settings page. Reuse the Hugging Face download card and seed expected asset sizes so GitHub runtime downloads report progress correctly.
Add Stable Diffusion model variant installation and removal through the shared download queue, including live progress and runtime pairing. Estimate sd.cpp component placement before download from the selected engine's devices and live memory budgets, then retain real execution probes for installed models. Surface GPU, CPU-offload, RAM, and component reserve details in a bottom menu so users can compare variants before installing.
Manage versioned stable-diffusion.cpp runtimes and variants, select compatible backends from detected hardware, and persist per-device multi-GPU budgets and split policies for planning and launch. Gate model downloads on an installed engine, recalculate runnability after hardware refresh, and expose fast non-blocking SHA verification so completed transfers no longer appear stuck.
Add larger FLUX.2 Klein variants and the Krea 2 series to the local model catalog with component paths and hardware-aware requirements. Persist model-level LoRAs in the model editor, merge them with request adapters in stable-diffusion.cpp, load them authoritatively from the database, and expose filtered runtime application diagnostics. Local avatar generation now sends concise prompts directly so adapter trigger words are not diluted by template prose. Add the associated model fields, commands, parameter descriptions, and localized copy across every supported locale.
Replace hardcoded sd.cpp sampling defaults with persisted model-aware sampler, scheduler, guidance, VAE tiling, reference, and high-resolution controls. Expose model-level pre-prompts, apply them once before provider dispatch, add payload regression tests, and translate the new UI across every supported locale.
…ecks Persist discovered and manually edited LoRA keywords, file identity, and architecture in SQLite so every model and generation request uses the shared library record. Detect architecture from SafeTensors metadata or exact-hash Civitai data, warn on unknown compatibility, block known mismatches, and repair partial schemas through migrations v80 and v81. Add the localized bottom-sheet keyword editor, required-keyword validation, prompt activation, and identical-file import reuse across all supported locales.
… and support custom model files
…and step caching to local diffusion
Split the scene writer template by image backend so existing remote workflows keep their reference-driven format while local models receive concise LoRA-aware instructions. Resolve saved character and persona LoRAs with trigger keywords, generate local scenes without appearance prose or reference images, and avoid duplicating trigger keywords in the final image prompt.
… files in picker, add unfiltered browser search
…nents, forward sd-server logs
Avatar editing remained exposed whenever an avatar generation model was available, including when the selected model used the local sdcpp provider. Hide local edit and refine entry points, guard stale edit actions, and explain the restriction in every supported locale while preserving generation, regeneration, repositioning, and upscaling.
Convert Flux/Klein LoRA tensor aliases before stable-diffusion.cpp inference so all compatible tensors are applied, while preserving manually keywordless LoRAs as always-active styles. Keep transient WIDTHxHEIGHT input text during editing so vertical image sizes can be saved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Local Image Generation: stable-diffusion.cpp engine, Playground, LoRA ecosystem, and guided tours
This PR ships fully local, on-device image generation end to end: a managed stable-diffusion.cpp runtime, a curated hardware-aware model catalog, a dedicated Playground for freeform generation, a LoRA library with an integrated CivitAI browser, deep integration into the existing avatar/scene flows, guided tours across every new surface, and full localization. Local generation is desktop-only by design; mobile keeps the existing provider-based flows and never touches sd.cpp.
Stable Diffusion Engine
sd-serversidecar with a native JSON API, forwarded logs, and lifecycle handling.Local Model Catalog
Playground
A new desktop-only
/playgroundroute reachable from Settings → Image Generation:LoRA Library + CivitAI
directly in the library).
Chat / Avatar / Scene Integration
Settings & Discoverability UX
Guided Tours
desktopOnlyflag (steps are filtered on mobile instead of stalling on missing targets) andoptionalflag (condip in ~400ms instead of 4s).window.__debug.resetTour(id)/resetAllTours().window.__debug.resetTour(id)/resetAllTours().Desktop-only Gating
/playgroundand the LoRA Library render a desktop-only fallback if reached directly; sd.cpp commands are never invoked from mobile code paths.i18n
en.tsand translated across all 21 locales in each file's established register and terminology; zh-Hans/zh-Hant written independently. Brand terms (CivitAI, LoRA, Playground, Stable Diffusion, ESRGAN) untranslated.Verification
bun run check(tsc + cargo check) green throughout.