Skip to content

dew.nn.inputs

Numeric model inputs shared by preprocessing, training and generation.

NameSummary
CONTEXTThe ModelInputs.conditioning entry a decoder’s cross-attention layers read, [B, C, features] (dew.nn.cross_attention).
CONTEXT_LENGTHSThe ModelInputs.conditioning entry counting each row’s real context tokens, [B] integers; a row’s padding comes after them.
RESPONSE_FIELDSHow each token field Dew’s processors write continues into a response.
AttentionMetadataPer-token data the layers read alongside their inputs, independent of physical cache-slot addresses.
CrossContextThe sequence a call’s cross-attention layers read: embeddings [B, C, features] and each row’s count of real tokens, lengths [B] int32, its padding after them, and where the row packs several documents, each context token’s document, segments [B, C] (0 for padding, CONTEXT_SEGMENTS), whose tokens the call’s tokens of that document alone read; and the call’s own tokens’ documents, queries [B, S], which the decoder sets from its coordinates.
FieldExtension
LayerInputsInputs each decoder layer reads for itself, as [B, S, layers, ...] leaves.
ModelInputsToken rows with sequence-aligned fields and row-aligned conditioning.
ModelKwarg
RowPlanWhere one request’s rows sit while a task runs.

attribute source

CONTEXT = 'context'

The ModelInputs.conditioning entry a decoder’s cross-attention layers read, [B, C, features] (dew.nn.cross_attention).

attribute source

CONTEXT_LENGTHS = 'context_lengths'

The ModelInputs.conditioning entry counting each row’s real context tokens, [B] integers; a row’s padding comes after them. Without it every context token is real.

attribute source

RESPONSE_FIELDS: Mapping[str, FieldExtension]

How each token field Dew’s processors write continues into a response.

dataclass source

class AttentionMetadata(
coordinates: Coordinates | None = None,
image_groups: jax.Array | None = None,
block_groups: jax.Array | None = None,
rotary_positions: jax.Array | None = None,
pairwise_mask: jax.Array | None = None,
key_positions: jax.Array | None = None,
key_runs: jax.Array | None = None,
token_ids: jax.Array | None = None,
engram_ids: jax.Array | None = None,
media: jax.Array | None = None,
admitted: Admitted | None = None,
position_bias: jax.Array | None = None,
blocks: jax.Array | None = None,
context: CrossContext | None = None,
progress: jax.Array | None = None,
routing_block: int | None = struct.field(pytree_node=False, default=None),
whole_blocks: bool = struct.field(pytree_node=False, default=False),
)

Per-token data the layers read alongside their inputs, independent of physical cache-slot addresses.

token_ids are the rows’ vocabulary ids, for a layer that routes by them (DeepSeek V4’s hash router); the model sets them when it has such a layer. engram_ids are every engram layer’s n-gram bucket ids, [B, S, layers, columns]. The model hashes them from the token ids once for the whole stack (dew.nn.engram), the same way it gives a hash router the ids. media [B, S] marks the positions a media encoder fills; DeepSeek-V4.1 routes those by its image bias and keeps them out of every n-gram. admitted turns a cached call into a serving step’s mixed call (Admitted). position_bias is added to every attention logit, [B or 1, heads, S, keys]: T5’s relative position table, which its encoder computes once for its layers. blocks [B, S] number each row’s routing blocks of routing_block places for a block router (numbered_blocks), which the model computes once for its layers. context is the sequence its cross-attention layers read (CrossContext). progress [B] is the updates the run committed before a training step, its declared input (dew.nn.progress), one row’s apiece, None at the run’s end; a layer whose computation changes over a run reads it. pairwise_mask [B, S, keys], or key_runs [B, S, R, 2], each query’s key intervals whose union it reads, replaces a call’s visibility, a windowed layer windowing it over key_positions [B, keys].

image_groups and block_groups [B, S] int32 number the tokens’ blocks, each a run of contiguous tokens, an image’s or a prompt’s, and -1 for a token outside any, padding among them; the model calls take them as image_groups= and block_groups=, and so does dew.nn.attention.scaled_dot_product_attention. A causal attention layer that reads them (AttentionMixer.bidirectional_images, bidirectional_blocks) lets each token of an image see the image’s later tokens within its window, and each token of a block its whole block, within its document; a layer reading both sees the union, and every other layer stays causal.

coordinates are the tokens’ documents, validity, positions and cache slots (Coordinates). Their right_padded says each row’s real tokens come first, its padding after them, as a training batch pads its rows. A real token then reads no padding through a causal layer, so those layers skip what would keep it out: causal attention takes no key mask and stays on its fused kernel, and a conv reads the row as it stands (dew.nn.linear.held_conv1d). A padding slot’s output is then unspecified.

whole_blocks: bool

Whether each block of block_groups is its document’s whole run of real tokens, as its producer states it (a decision row read whole, QuestionLayout.split ‘whole’): a layer that reads blocks whole then reads each document both ways, which is non-causal attention by documents, and takes it so (CausalSelfAttention._masking), its blocks unread.

valid: jax.Array | None

The real tokens (Coordinates.valid), None where every token is.

dataclass source

class CrossContext(segments: jax.Array | None = None, queries: jax.Array | None = None)

The sequence a call’s cross-attention layers read: embeddings [B, C, features] and each row’s count of real tokens, lengths [B] int32, its padding after them, and where the row packs several documents, each context token’s document, segments [B, C] (0 for padding, CONTEXT_SEGMENTS), whose tokens the call’s tokens of that document alone read; and the call’s own tokens’ documents, queries [B, S], which the decoder sets from its coordinates.

def parted(
conditioning: Mapping[str, jax.Array] | None,
) -> tuple[dict[str, jax.Array] | None, dict[str, jax.Array] | None]

conditioning’s context entries (CONTEXT, CONTEXT_LENGTHS), which a decoder reads, and the rest, which a module around the decoder reads itself (a multimodal model’s media); each None where it holds nothing.

def from_conditioning(conditioning: Mapping[str, jax.Array]) -> CrossContext

The context a call’s conditioning carries under CONTEXT, CONTEXT_LENGTHS and CONTEXT_SEGMENTS, as JAX arrays (checked).

def checked(
conditioning: Mapping[str, jax.typing.ArrayLike],
xp=np,
) -> tuple[jax.typing.ArrayLike, jax.typing.ArrayLike, jax.typing.ArrayLike | None]

The context, its lengths and its tokens’ documents a call’s conditioning carries under CONTEXT, CONTEXT_LENGTHS and CONTEXT_SEGMENTS, as arrays of xp (NumPy’s on the host, where a server admits a request; JAX’s), each row’s length its context’s width where none is given, the documents None where none are given, each in the dtype given.

Raises ValueError for any other entry, for a context that is not [B, C, features] floats, for lengths that are not one integer per row, and for documents that are not one integer per context token.

attribute source

dataclass source

class LayerInputs(
embeddings: jax.Array | None = None,
experts: jax.Array | Mapping[str, jax.Array] | None = None,
routed: jax.Array | None = None,
depth: jax.Array | None = None,
)

Inputs each decoder layer reads for itself, as [B, S, layers, ...] leaves.

The stack slices every leaf on axis 2 the same way. So each layer gets its own [B, S, ...] slice whether the stack runs as a scan, reads its weights from a bank one layer at a time, or runs as a pipeline stage. embeddings is Gemma 3n/4’s per-layer input signal. experts and routed are a routing replay’s [..., top_k] expert ids and coverage (dew.nn.protocols.Routes), with routed broadcast over the layer axis: one record for a layer’s one routed site, or, for layers that route at several, one a routed site keyed by its path: FEEDFORWARD_ROUTES, MIXER_ROUTES, and a composed mixer’s sites below it (mixer/<branch>, nested by /). Each layer reads the paths of its own sites; a stack whose layers route at different sites holds every path. depth is each layer’s index in the stack, for a mixer that computes by it (MixerBase.reads_depth, MiniMax’s lightning rates).

def span(first: int, count: int) -> LayerInputs

Return layers first through first + count - 1, keeping the layer axis.

def layer(index) -> LayerInputs

Return one layer’s slice without the layer axis; index may be traced.

dataclass source

class ModelInputs(
token_fields: Mapping[str, jax.Array | Coordinates] = struct.field(default_factory=dict),
conditioning: Mapping[str, jax.Array] = struct.field(default_factory=dict),
)

Token rows with sequence-aligned fields and row-aligned conditioning.

Every leaf has the batch on axis 0, and each of token_fields also has the sequence on axis 1, at the length of tokens. conditioning holds media payloads and their valid lengths. A token field such as image_indices gives the index of the media feature read at each text slot (-1 for a text token), so a sliced prompt still points at the same features. The processor validates the fields on the host. take_rows, slice_tokens and align_left also work inside JIT.

def from_value(
value: ModelInputs | jax.typing.ArrayLike | Sequence[Sequence[int]],
) -> ModelInputs

Return value as validated ModelInputs, without lossy id conversion or moving resident arrays.

A ModelInputs is validated as it is, and a JAX array is wrapped where it sits. Anything else is read on the host as an integer [B, S] array, and ids that int32 cannot represent raise ValueError. Rows given as a sequence are each read by token_id’s law first (_token_row), so a bool among them is refused rather than promoted into an id.

Its token layout is one coordinates field past here (laid): flat segment_ids, attention_mask and positions fields are converted once.

Distributed algorithms call this inside their agreed validation phase. It runs no collectives and no model.

def laid() -> ModelInputs

These inputs with their token layout as one coordinates field (COORDINATES_FIELD): their Coordinates as they are, else built once from their flat segment_ids, attention_mask and positions fields, which it replaces (field_coordinates); fields carrying both are refused. Every model method past __call__ reads a layout only as Coordinates (ledger 1122).

def validate() -> None

Check the numeric layout on the host before dispatching a model.

Raises ValueError when tokens is not an integer [B, S] array, when a token field has a reserved name or does not start with the tokens’ shape, or when a conditioning entry has another batch size.

def take_rows(indices: jax.Array) -> ModelInputs

Select the same rows from tokens, sequence fields and media.

def slice_tokens(start: int | None = None, stop: int | None = None) -> ModelInputs

Slice the tokens and token fields to slots start:stop; media features keep their indices.

def align_left(left_padding: jax.Array) -> ModelInputs

Rotate each row so its real tokens start at slot 0 and its left_padding slots move to the end.

Positions are token fields, so they move with the tokens and keep their values; they are not recomputed from the new slot numbers. Raises ValueError unless left_padding has one count per row.

def kwargs() -> dict[str, jax.Array | Mapping[str, jax.Array] | Coordinates]

Return the keyword arguments for a model call, without the tokens.

Each token field is its own keyword, the token layout one coordinates keyword (laid), and conditioning is included only when it is not empty.

def extended(
tokens: jax.Array,
*,
rules: Mapping[str, FieldExtension] | None = None,
) -> ModelInputs

These inputs with a response’s [B, W] tokens after the prompt’s slots, each token field continued by its rule in rules or RESPONSE_FIELDS.

A field with no rule raises ValueError rather than take a guessed value; the conditioning stays as it is. The rules read the prompt’s real slots (real_slots), so a packed row’s padding continues nothing. Works inside JIT.

attribute source

dataclass source

class RowPlan(
mesh: jax.sharding.Mesh | None,
rows: int,
count: int,
process: int,
processes: int,
)

Where one request’s rows sit while a task runs.

Without a mesh every array stays on the default device. On a mesh, rows split over its request axes (request_axes). Each process contributes count rows, its rows real ones followed by repeats that pad them to a multiple of its devices on those axes, so every device holds the same shape and every collective lines up. Results keep that sharding, and host reads a process’s real rows back.

global_rows: int

The number of rows of a placed array, over all processes.

padding: np.ndarray

A mask over the count placed rows, True for the repeats and False for real rows (padded_rows).

def over(mesh: jax.sharding.Mesh | None, rows: int) -> RowPlan
def pad(tree)

Repeat rows up to count, cycling through the real ones (padded_rows); device arrays stay on device.

def place(tree)

Place padded host or device rows, sharded by row on a mesh.

Without a mesh this is jax.device_put. Raises ValueError for a leaf without count rows on axis 0.

def keys(key: jax.Array) -> jax.Array

Return one key per placed row, folded from key by global row index.

A pool of processes therefore draws the same values as one process does for the same rows. On a mesh, key may be replicated over several processes; the keys are folded from this process’s copy on one device and then sharded to their devices.

def host(leaf) -> np.ndarray

Return this process’s real rows of a result leaf as a host array.