Skip to content

dew.nn.backbones.qwen_image

Qwen-Image 2.1’s transformer, as Diffusers’ QwenImage21Transformer2DModel runs it at commit 6256aa76.

One residual stream holds the image and then the text. The text is the vision-language encoder’s states passed through a zero-centred RMS norm and a GELU MLP, and the image is one linear map per latent position. Every block modulates both with scales and tanh gates from one shared projection of the time embedding, attends, and runs a SwiGLU. Attention is block-causal: the text is causal, and the image attends to all the text and to itself. Under causal_condition the text is modulated from time zero, so its activations do not change from one sampling step to the next. The rotary table’s three axes advance a text token on all three, and place the image at the frame after the text, on a zero-centred height and width grid. The source starts the image’s frame after the call’s longest prompt. Here each row starts after its own text, which matches the source for a prompt alone or for a batch of prompts of one length, and keeps each row independent of the rest of its batch.

NameSummary
QwenImageTransformerRuns QwenImage21Transformer2DModel behind Dew’s model interface.

Flax module source

class QwenImageTransformer(
in_channels: int = 64,
out_channels: int = 64,
num_layers: int = 32,
heads: int = 32,
head_dim: int = 128,
context_in_dim: int = 4096,
mlp_ratio: int = 3,
axes_dims_rope: Sequence[int] = (16, 56, 56),
eps: float = 1e-06,
causal_condition: bool = True,
dtype: Dtype | None = None,
precision: PrecisionLike = None,
attention_impl: AttentionImpl = 'auto',
)

Runs QwenImage21Transformer2DModel behind Dew’s model interface.

__call__ takes NHWC latents with one token per position, the model time the schedule supplies, and a DenoisingCondition, and returns the prediction at the image’s tokens. The model time is the sigma times the training count; the source reaches the same product by dividing its timestep by a thousand and multiplying it back. The condition’s context is the encoder’s text states after the system prompt, right-padded, and its mask marks the real tokens. Each row’s real tokens come first in its text, which is the layout that both the rotary positions and the attention’s key lengths assume.

text_keyword: str

Every call takes the text as conditioning, which shares the image’s residual stream.

def condition_lengths(lengths: np.ndarray, spans: np.ndarray) -> np.ndarray
def projections(input_channels: int) -> dict[str, tuple[int, int]]
def model_flops(lengths) -> float
def image_lengths(sample_shape: tuple[int, ...], rows: int) -> np.ndarray
def multisequence_flops(
lengths,
context_lengths,
*,
input_channels: int,
text_features: int,
pooled_features: int | None = None,
) -> float

Text’s two input maps and the shared time-zero row are included.