dew.nn.inputs
Numeric model inputs shared by preprocessing, training and generation.
| Name | Summary |
|---|---|
CONTEXT | The ModelInputs.conditioning entry a decoder’s cross-attention layers read, [B, C, features] (dew.nn.cross_attention). |
CONTEXT_LENGTHS | The ModelInputs.conditioning entry counting each row’s real context tokens, [B] integers; a row’s padding comes after them. |
RESPONSE_FIELDS | How each token field Dew’s processors write continues into a response. |
AttentionMetadata | Per-token data the layers read alongside their inputs, independent of physical cache-slot addresses. |
CrossContext | The sequence a call’s cross-attention layers read: embeddings [B, C, features] and each row’s count of real tokens, lengths [B] int32, its padding after them, and where the row packs several documents, each context token’s document, segments [B, C] (0 for padding, CONTEXT_SEGMENTS), whose tokens the call’s tokens of that document alone read; and the call’s own tokens’ documents, queries [B, S], which the decoder sets from its coordinates. |
FieldExtension | |
LayerInputs | Inputs each decoder layer reads for itself, as [B, S, layers, ...] leaves. |
ModelInputs | Token rows with sequence-aligned fields and row-aligned conditioning. |
ModelKwarg | |
RowPlan | Where one request’s rows sit while a task runs. |
CONTEXT
Section titled “CONTEXT”CONTEXT = 'context'The ModelInputs.conditioning entry a decoder’s cross-attention layers read,
[B, C, features] (dew.nn.cross_attention).
CONTEXT_LENGTHS
Section titled “CONTEXT_LENGTHS”CONTEXT_LENGTHS = 'context_lengths'The ModelInputs.conditioning entry counting each row’s real context tokens, [B]
integers; a row’s padding comes after them. Without it every context token is real.
RESPONSE_FIELDS
Section titled “RESPONSE_FIELDS”RESPONSE_FIELDS: Mapping[str, FieldExtension]How each token field Dew’s processors write continues into a response.
AttentionMetadata
Section titled “AttentionMetadata”class AttentionMetadata( coordinates: Coordinates | None = None, image_groups: jax.Array | None = None, block_groups: jax.Array | None = None, rotary_positions: jax.Array | None = None, pairwise_mask: jax.Array | None = None, key_positions: jax.Array | None = None, key_runs: jax.Array | None = None, token_ids: jax.Array | None = None, engram_ids: jax.Array | None = None, media: jax.Array | None = None, admitted: Admitted | None = None, position_bias: jax.Array | None = None, blocks: jax.Array | None = None, context: CrossContext | None = None, progress: jax.Array | None = None, routing_block: int | None = struct.field(pytree_node=False, default=None), whole_blocks: bool = struct.field(pytree_node=False, default=False),)Per-token data the layers read alongside their inputs, independent of physical cache-slot addresses.
token_ids are the rows’ vocabulary ids, for a layer that routes by them
(DeepSeek V4’s hash router); the model sets them when it has such a layer.
engram_ids are every engram layer’s n-gram bucket ids,
[B, S, layers, columns]. The model hashes them from the token ids once
for the whole stack (dew.nn.engram), the same way it gives a hash router
the ids. media [B, S] marks the positions a media encoder fills;
DeepSeek-V4.1 routes those by its image bias and keeps them out of every
n-gram. admitted turns a cached call into a serving step’s mixed call
(Admitted). position_bias is added to every attention logit,
[B or 1, heads, S, keys]: T5’s relative position table, which its
encoder computes once for its layers. blocks [B, S] number each row’s
routing blocks of routing_block places for a block router
(numbered_blocks), which the model computes once for its layers.
context is the sequence its cross-attention layers read (CrossContext).
progress [B] is the updates the run committed before a training step, its declared input
(dew.nn.progress), one row’s apiece, None at the run’s end; a layer whose computation changes over
a run reads it.
pairwise_mask [B, S, keys], or key_runs [B, S, R, 2], each query’s
key intervals whose union it reads, replaces a call’s visibility, a
windowed layer windowing it over key_positions [B, keys].
image_groups and block_groups [B, S] int32 number the tokens’ blocks,
each a run of contiguous tokens, an image’s or a prompt’s, and -1 for a
token outside any, padding among them; the model calls take them as
image_groups= and block_groups=, and so does
dew.nn.attention.scaled_dot_product_attention. A causal attention layer that reads
them (AttentionMixer.bidirectional_images, bidirectional_blocks) lets
each token of an image see the image’s later tokens within its window,
and each token of a block its whole block, within its document; a layer
reading both sees the union, and every other layer stays causal.
coordinates are the tokens’ documents, validity, positions and cache
slots (Coordinates). Their right_padded says each row’s real tokens
come first, its padding after them, as a training batch pads its rows. A
real token then reads no padding through a causal layer, so those layers
skip what would keep it out: causal attention takes no key mask and stays
on its fused kernel, and a conv reads the row as it stands
(dew.nn.linear.held_conv1d). A padding slot’s output is then
unspecified.
whole_blocks: bool-
Whether each block of
block_groupsis its document’s whole run of real tokens, as its producer states it (a decision row read whole,QuestionLayout.split‘whole’): a layer that reads blocks whole then reads each document both ways, which is non-causal attention by documents, and takes it so (CausalSelfAttention._masking), its blocks unread. valid: jax.Array | None-
The real tokens (
Coordinates.valid), None where every token is.
CrossContext
Section titled “CrossContext”class CrossContext(segments: jax.Array | None = None, queries: jax.Array | None = None)The sequence a call’s cross-attention layers read: embeddings [B, C, features]
and each row’s count of real tokens, lengths [B] int32, its padding after them, and where the
row packs several documents, each context token’s document, segments [B, C] (0 for padding,
CONTEXT_SEGMENTS), whose tokens the call’s tokens of that document alone read; and the call’s own
tokens’ documents, queries [B, S], which the decoder sets from its coordinates.
CrossContext.parted
Section titled “CrossContext.parted”def parted( conditioning: Mapping[str, jax.Array] | None,) -> tuple[dict[str, jax.Array] | None, dict[str, jax.Array] | None]conditioning’s context entries (CONTEXT, CONTEXT_LENGTHS), which a decoder reads, and
the rest, which a module around the decoder reads itself (a multimodal model’s media); each
None where it holds nothing.
CrossContext.from_conditioning
Section titled “CrossContext.from_conditioning”def from_conditioning(conditioning: Mapping[str, jax.Array]) -> CrossContextThe context a call’s conditioning carries under CONTEXT, CONTEXT_LENGTHS and
CONTEXT_SEGMENTS, as JAX arrays (checked).
CrossContext.checked
Section titled “CrossContext.checked”def checked( conditioning: Mapping[str, jax.typing.ArrayLike], xp=np,) -> tuple[jax.typing.ArrayLike, jax.typing.ArrayLike, jax.typing.ArrayLike | None]The context, its lengths and its tokens’ documents a call’s conditioning carries under
CONTEXT, CONTEXT_LENGTHS and CONTEXT_SEGMENTS, as arrays of xp (NumPy’s on the host,
where a server admits a request; JAX’s), each row’s length its context’s width where none is
given, the documents None where none are given, each in the dtype given.
Raises ValueError for any other entry, for a context that is not
[B, C, features] floats, for lengths that are not one integer
per row, and for documents that are not one integer per context token.
FieldExtension
Section titled “FieldExtension”LayerInputs
Section titled “LayerInputs”class LayerInputs( embeddings: jax.Array | None = None, experts: jax.Array | Mapping[str, jax.Array] | None = None, routed: jax.Array | None = None, depth: jax.Array | None = None,)Inputs each decoder layer reads for itself, as [B, S, layers, ...] leaves.
The stack slices every leaf on axis 2 the same way. So each layer gets its
own [B, S, ...] slice whether the stack runs as a scan, reads its weights
from a bank one layer at a time, or runs as a pipeline stage. embeddings
is Gemma 3n/4’s per-layer input signal. experts and routed are a
routing replay’s [..., top_k] expert ids and coverage
(dew.nn.protocols.Routes), with routed broadcast over the layer axis:
one record for a layer’s one routed site, or, for layers that route at
several, one a routed site keyed by its path: FEEDFORWARD_ROUTES,
MIXER_ROUTES, and a composed mixer’s sites below it
(mixer/<branch>, nested by /). Each layer reads the paths of its own
sites; a stack whose layers route at different sites holds every path.
depth is each layer’s index in the stack, for a mixer that computes by
it (MixerBase.reads_depth, MiniMax’s lightning rates).
LayerInputs.span
Section titled “LayerInputs.span”def span(first: int, count: int) -> LayerInputsReturn layers first through first + count - 1, keeping the layer axis.
LayerInputs.layer
Section titled “LayerInputs.layer”def layer(index) -> LayerInputsReturn one layer’s slice without the layer axis; index may be traced.
ModelInputs
Section titled “ModelInputs”class ModelInputs( token_fields: Mapping[str, jax.Array | Coordinates] = struct.field(default_factory=dict), conditioning: Mapping[str, jax.Array] = struct.field(default_factory=dict),)Token rows with sequence-aligned fields and row-aligned conditioning.
Every leaf has the batch on axis 0, and each of token_fields also has
the sequence on axis 1, at the length of tokens. conditioning holds
media payloads and their valid lengths. A token field such as
image_indices gives the index of the media feature read at each text
slot (-1 for a text token), so a sliced prompt still points at the same
features. The processor validates the fields on the host. take_rows,
slice_tokens and align_left also work inside JIT.
ModelInputs.from_value
Section titled “ModelInputs.from_value”def from_value( value: ModelInputs | jax.typing.ArrayLike | Sequence[Sequence[int]],) -> ModelInputsReturn value as validated ModelInputs, without lossy id conversion or moving resident arrays.
A ModelInputs is validated as it is, and a JAX array is wrapped where
it sits. Anything else is read on the host as an integer [B, S]
array, and ids that int32 cannot represent raise ValueError. Rows given
as a sequence are each read by token_id’s law first (_token_row), so
a bool among them is refused rather than promoted into an id.
Its token layout is one coordinates field past here (laid): flat segment_ids,
attention_mask and positions fields are converted once.
Distributed algorithms call this inside their agreed validation phase. It runs no collectives and no model.
ModelInputs.laid
Section titled “ModelInputs.laid”def laid() -> ModelInputsThese inputs with their token layout as one coordinates field (COORDINATES_FIELD): their
Coordinates as they are, else built once from their flat segment_ids, attention_mask and
positions fields, which it replaces (field_coordinates); fields carrying both are refused.
Every model method past __call__ reads a layout only as Coordinates (ledger 1122).
ModelInputs.validate
Section titled “ModelInputs.validate”def validate() -> NoneCheck the numeric layout on the host before dispatching a model.
Raises ValueError when tokens is not an integer [B, S] array, when
a token field has a reserved name or does not start with the tokens’
shape, or when a conditioning entry has another batch size.
ModelInputs.take_rows
Section titled “ModelInputs.take_rows”def take_rows(indices: jax.Array) -> ModelInputsSelect the same rows from tokens, sequence fields and media.
ModelInputs.slice_tokens
Section titled “ModelInputs.slice_tokens”def slice_tokens(start: int | None = None, stop: int | None = None) -> ModelInputsSlice the tokens and token fields to slots start:stop; media features keep their indices.
ModelInputs.align_left
Section titled “ModelInputs.align_left”def align_left(left_padding: jax.Array) -> ModelInputsRotate each row so its real tokens start at slot 0 and its left_padding slots move to the end.
Positions are token fields, so they move with the tokens and keep their
values; they are not recomputed from the new slot numbers. Raises
ValueError unless left_padding has one count per row.
ModelInputs.kwargs
Section titled “ModelInputs.kwargs”def kwargs() -> dict[str, jax.Array | Mapping[str, jax.Array] | Coordinates]Return the keyword arguments for a model call, without the tokens.
Each token field is its own keyword, the token layout one coordinates keyword (laid), and
conditioning is included only when it is not empty.
ModelInputs.extended
Section titled “ModelInputs.extended”def extended( tokens: jax.Array, *, rules: Mapping[str, FieldExtension] | None = None,) -> ModelInputsThese inputs with a response’s [B, W] tokens after the prompt’s slots,
each token field continued by its rule in rules or RESPONSE_FIELDS.
A field with no rule raises ValueError rather than take a guessed
value; the conditioning stays as it is. The rules read the prompt’s
real slots (real_slots), so a packed row’s padding continues
nothing. Works inside JIT.
ModelKwarg
Section titled “ModelKwarg”RowPlan
Section titled “RowPlan”class RowPlan( mesh: jax.sharding.Mesh | None, rows: int, count: int, process: int, processes: int,)Where one request’s rows sit while a task runs.
Without a mesh every array stays on the default device. On a mesh, rows
split over its request axes (request_axes). Each process contributes count rows, its
rows real ones followed by repeats that pad them to a multiple of its
devices on those axes, so every device holds the same shape and every
collective lines up. Results keep that sharding, and host reads a
process’s real rows back.
global_rows: int-
The number of rows of a placed array, over all processes.
padding: np.ndarray-
A mask over the
countplaced rows, True for the repeats and False for real rows (padded_rows).
RowPlan.over
Section titled “RowPlan.over”def over(mesh: jax.sharding.Mesh | None, rows: int) -> RowPlanRowPlan.pad
Section titled “RowPlan.pad”def pad(tree)Repeat rows up to count, cycling through the real ones (padded_rows); device arrays
stay on device.
RowPlan.place
Section titled “RowPlan.place”def place(tree)Place padded host or device rows, sharded by row on a mesh.
Without a mesh this is jax.device_put. Raises ValueError for a leaf
without count rows on axis 0.
RowPlan.keys
Section titled “RowPlan.keys”def keys(key: jax.Array) -> jax.ArrayReturn one key per placed row, folded from key by global row index.
A pool of processes therefore draws the same values as one process does
for the same rows. On a mesh, key may be replicated over several
processes; the keys are folded from this process’s copy on one device
and then sharded to their devices.
RowPlan.host
Section titled “RowPlan.host”def host(leaf) -> np.ndarrayReturn this process’s real rows of a result leaf as a host array.