feat: add SenseNova U1.5 support - #1935
Open
Maphist0 wants to merge 8 commits into
Open
Conversation
Support root-level sharded Safetensors directories while preserving Diffusers directory precedence. Detect SenseNova U1.5 using independent tensor markers for its MoT generation branch, vision input path, and pixel-flow output head. Register the architecture as a DiT model and ignore checkpoint tensors that are unused by the text-to-image runtime.
Add a dedicated SenseNova tokenizer built on the Qwen2 BPE implementation.
Add model-local configuration detection, checkpoint-layout convolutions, timestep and vision embeddings, and the progressive PixelShuffle RGB decoder.
Add separate understanding and generation transformer branches with three-axis RoPE and grouped-query attention.
Register the U1 conditioner, MoT runner, and full-resolution RGB path. Add shifted-flow Euler sampling, pass prompt token IDs to the diffusion runner, and route CFG negatives through the U1-specific unconditional prompt builder.
Copy per-layer prefix K/V tensors into dedicated graph outputs so ggml keeps their buffers alive until the runner migrates them into persistent cache storage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add SenseNova U1.5 8B MoT support for non-thinking text-to-image generation.
SenseNova U1.5 performs flow matching directly in RGB pixel space, so it does not require a separate text encoder or VAE. This PR adds:
The implementation supports non-thinking text-to-image generation only. Thinking-mode prompt expansion, image editing, visual understanding, and interleaved generation are outside this PR.
Related Issue / Discussion
Related #1841 #1894
Additional Information
Native generation was compared with the pinned official implementation using the same checkpoint, prompt, empty negative prompt, seed, dimensions, Euler sampler, CFG, flow shift, and BF16 precision.
The one-step output is nearly pixel-identical. Differences accumulate over the full 50-step run because the official pipeline uses SDPA while the sdcpp invocation uses the ggml CUDA flash-attention path.
Sample output:
The (long) prompt that generates the following image
Checklist