dew.nn.backbones.z_image
Z-Image’s single-stream transformer (S3-DiT), as Diffusers 0.40.0’s
ZImageTransformer2DModel runs it for text-to-image.
Each 2x2 latent patch is one token. Each stream is padded to a multiple of 32 tokens with a learned pad token, and the image’s pad tokens sit at position (0, 0, 0). Two time-modulated refiner blocks run over the image and two unmodulated ones over the prompt, then the main blocks run over both, image first. Every block RMS-norms before and after the attention and the SwiGLU, and the tanh of the time modulation gates its residuals. The three-axis rotary places prompt token i at (1 + i, 0, 0) and patch (h, w) at (1 + P, h, w), where P is the padded prompt length. The angles are looked up in a float64 table rounded to float32, as the source does. The source pads a batch to its longest row and masks; here the prompt region is the conditioner’s budget rounded up to 32, masked the same way. The source takes time 1 - sigma and returns the negated flow, while this module takes Dew’s model time and returns Dew’s flow.
| Name | Summary |
|---|---|
ZImageBlock | Runs one ZImageTransformerBlock. |
ZImageTransformer | Runs Diffusers 0.40.0’s ZImageTransformer2DModel behind Dew’s model interface. |
ZImageBlock
Section titled “ZImageBlock”class ZImageBlock( epsilon: float = 1e-05, modulation: bool = True, dtype: Dtype | None = None, precision: PrecisionLike = None, attention_impl: AttentionImpl = 'auto',)Runs one ZImageTransformerBlock.
With modulation, one projection of the time embedding gives the input
scales (one plus the projection) and the output gates (its tanh) of the
attention and the feed-forward. Without it, as in the prompt’s refiner
blocks, the block has no time modulation.
ZImageBlock.model_flops
Section titled “ZImageBlock.model_flops”def model_flops(lengths) -> floatReal-token QKV/output and gated MLP; pad rows stay outside MFU.
ZImageTransformer
Section titled “ZImageTransformer”class ZImageTransformer( in_channels: int = 16, dim: int = 3840, n_layers: int = 30, n_refiner_layers: int = 2, n_heads: int = 30, norm_eps: float = 1e-05, cap_feat_dim: int = 2560, rope_theta: float = 256.0, t_scale: float = 1000.0, axes_dims: Sequence[int] = (32, 48, 48), axes_lens: Sequence[int] = (1536, 512, 512), dtype: Dtype | None = None, precision: PrecisionLike = None, attention_impl: AttentionImpl = 'auto',)Runs Diffusers 0.40.0’s ZImageTransformer2DModel behind Dew’s model interface.
__call__ takes NHWC latents, the model time the schedule supplies (the
sigma times the training count) and a DenoisingCondition, and returns
the flow. The condition’s context is the prompt states, padded on the
right to a fixed budget, and its mask marks the real tokens.
text_keyword: str-
Every call takes the caption as
conditioning, which joins the image’s single stream.
ZImageTransformer.projections
Section titled “ZImageTransformer.projections”def projections(input_channels: int, text_features: int) -> dict[str, tuple[int, int]]ZImageTransformer.condition_lengths
Section titled “ZImageTransformer.condition_lengths”def condition_lengths(lengths: np.ndarray, spans: np.ndarray) -> np.ndarrayZImageTransformer.model_flops
Section titled “ZImageTransformer.model_flops”def model_flops(lengths) -> floatZImageTransformer.image_lengths
Section titled “ZImageTransformer.image_lengths”def image_lengths(sample_shape: tuple[int, ...], rows: int) -> np.ndarrayZImageTransformer.multimodal_flops
Section titled “ZImageTransformer.multimodal_flops”def multimodal_flops( lengths, context_lengths, input_channels: int, text_features: int,) -> floatStatic patch/text maps, both refiner stacks and the joined stack.