dew.nn.backbones.qwen_image
Qwen-Image 2.1’s transformer, as Diffusers’ QwenImage21Transformer2DModel
runs it at commit 6256aa76.
One residual stream holds the image and then the text. The text is the
vision-language encoder’s states passed through a zero-centred RMS norm and
a GELU MLP, and the image is one linear map per latent position. Every block
modulates both with scales and tanh gates from one shared projection of the
time embedding, attends, and runs a SwiGLU. Attention is block-causal: the
text is causal, and the image attends to all the text and to itself. Under
causal_condition the text is modulated from time zero, so its activations
do not change from one sampling step to the next. The rotary table’s three
axes advance a text token on all three, and place the image at the frame
after the text, on a zero-centred height and width grid. The source starts
the image’s frame after the call’s longest prompt. Here each row starts
after its own text, which matches the source for a prompt alone or for a
batch of prompts of one length, and keeps each row independent of the rest
of its batch.
| Name | Summary |
|---|---|
QwenImageTransformer | Runs QwenImage21Transformer2DModel behind Dew’s model interface. |
QwenImageTransformer
Section titled “QwenImageTransformer”class QwenImageTransformer( in_channels: int = 64, out_channels: int = 64, num_layers: int = 32, heads: int = 32, head_dim: int = 128, context_in_dim: int = 4096, mlp_ratio: int = 3, axes_dims_rope: Sequence[int] = (16, 56, 56), eps: float = 1e-06, causal_condition: bool = True, dtype: Dtype | None = None, precision: PrecisionLike = None, attention_impl: AttentionImpl = 'auto',)Runs QwenImage21Transformer2DModel behind Dew’s model interface.
__call__ takes NHWC latents with one token per position, the model time
the schedule supplies, and a DenoisingCondition, and returns the
prediction at the image’s tokens. The model time is the sigma times the
training count; the source reaches the same product by dividing its
timestep by a thousand and multiplying it back. The condition’s context
is the encoder’s text states after the system prompt, right-padded, and
its mask marks the real tokens. Each row’s real tokens come first in
its text, which is the layout that both the rotary positions and the
attention’s key lengths assume.
text_keyword: str-
Every call takes the text as
conditioning, which shares the image’s residual stream.
QwenImageTransformer.condition_lengths
Section titled “QwenImageTransformer.condition_lengths”def condition_lengths(lengths: np.ndarray, spans: np.ndarray) -> np.ndarrayQwenImageTransformer.projections
Section titled “QwenImageTransformer.projections”def projections(input_channels: int) -> dict[str, tuple[int, int]]QwenImageTransformer.model_flops
Section titled “QwenImageTransformer.model_flops”def model_flops(lengths) -> floatQwenImageTransformer.image_lengths
Section titled “QwenImageTransformer.image_lengths”def image_lengths(sample_shape: tuple[int, ...], rows: int) -> np.ndarrayQwenImageTransformer.multisequence_flops
Section titled “QwenImageTransformer.multisequence_flops”def multisequence_flops( lengths, context_lengths, *, input_channels: int, text_features: int, pooled_features: int | None = None,) -> floatText’s two input maps and the shared time-zero row are included.