dew.decision.layout
How a state and its questions become the token rows a decision model reads.
A layout places each question’s instructions and each of its options at a
span of tokens, which the head reads. MarkerLayout is Laya’s (laya/common.py
build_sequence, build_head and parallel_layout at NandhaKishorM/laya
a4a8921), one row per question: the question, then one marker token before
each option’s text, then the state,
[CLS] <type> question: <instructions> [SEP] [MASK] opt0 [MASK] opt1 ... [SEP] state [SEP]and the model scores each option at its marker. The option texts share a
token budget, head_max_len, and the state takes what is left of max_len.
StateFirstLayout puts the state first and each marker after its option,
for a causal backbone. JointLayout lays every question of a request in
one row, each option a span of its own.
| Name | Summary |
|---|---|
TokenizerSpecials | What Specials.of reads off a tokenizer, as a transformers tokenizer holds it: its special tokens’ ids, each None where it has none, and an id’s token text. |
Specials | The special tokens a layout places. |
Laid | One question as a row lays it out. |
Encoded | One row: its tokens and the questions laid out in it. |
OPTIONS | A question field’s axis of one entry per option slot (QuestionField.shape). |
QuestionField | One array a row holds along its questions: its name, the shape each question slot holds (OPTIONS for an axis of one entry per option slot), its dtype, its value in a slot that holds no question, and when a row carries it: always, where its options stand under labels (labelled), in training (trained), or where its questions stand in consistency groups (grouped). |
QUESTION_FIELDS | The fields along a row’s questions a model reads (DecisionInputs): kinds, questions (the real ones), spans, option_spans, options (the real ones), and where its options stand under labels (Labels) readout, each question’s answer cue. |
question_fields | row’s questions as fixed-shape arrays (QUESTION_FIELDS), count question slots of width option slots each, readout where its options stand under labels (labelled). |
MEDIA_MARK | The mark of a row’s images (Encoded.media), which no question’s mark, a (name, part) pair, is. |
INSTRUCTIONS_MARK | The part of a question’s mark that its instructions take, beside its options’ slots. |
render | Return a value as the model reads it: text as it is, and anything structured as JSON. |
context_tokens | A context’s tokens as a row carries them: its text’s (render), without specials, and none for no context. |
with_context | rows with the context a backbone’s cross-attention of length context tokens reads beside each (None: a backbone with none): a schema laid out as the rows’ context (Schema.stream), read whole, or else context, a request’s own, its first length tokens (context_tokens). |
Bounds | The most rows (segments), questions and options one packed window holds; options is None where the layout trims a question’s options to share a budget rather than giving each tokens of its own (QuestionLayout). |
Layout | How a state and its questions become rows of tokens that fit max_len (fits). |
LETTERS | |
LABEL_NAMES | Every label, in order: A to Z, then AA, AB, …, ZZ, as a release’s slots name them. |
label_names | The first count labels of LABEL_NAMES. |
Labels | Options rendered as lettered lines, read at the row’s answer cue by a readout of the labels’ logits (dew.decision.head.LabelReadout, SlotReadout): slot s as `{name}. |
QuestionLayout | A layout of one row per question, scored at one marker token per option. |
MarkerLayout | Laya’s layout, for a bidirectional backbone: the question, a marker before each option, then the state. |
StateFirstLayout | The layout for a causal backbone: the state, the question, then each option followed by its marker. |
JOINT_SYSTEM | The joint layout’s system prompt. |
JOINT_PREFILL | The joint layout’s opening of the assistant’s turn, after the turn’s own markup. |
Framing | How a joint row frames its text as a conversation: a system message, the user’s turn that holds the state and the schema, and the assistant’s turn opened with prefill. |
JOINT_NOUL | A noul option’s description where its question gives none. |
Text | Text a joint row writes as given, at its place in the user’s turn. |
Media | Where a joint row places a request’s images, each as the backbone’s processor writes one (Processor.image_text) and expands it; a request without images places nothing here. |
Render | How a segment writes a value as text (rendered). |
State | The state, as compact JSON with sorted keys, its start kept: cut to its own budget (max_tokens, or a joint layout’s max_state_tokens where it gives none) and to its share of what the rest of the row leaves of its max_len (dew.data.tokens.share_tokens). |
allocated | How many of its own tokens each copy of the state keeps of room (dew.data.tokens.share_tokens): a copy kept whole (keep="all") all of its lengths, whatever its cap, and the others what is left, each at most its own cap (max_tokens, or default_cap where it gives none). |
RowOrder | The order a question layout’s own row writes its parts in (QuestionLayout.order). |
built_in | The built-in question row as segments (QuestionLayout.written): the begin token, the question’s header and instructions as one piece ("{kind} question: {instructions}"), the option lines and the state, each part after a separator, in order: the state first, the question first, or the state, the question and the state again; Laya’s markers before each option where the question comes first, a causal row’s after, or lettered lines read at the labels’ cue, which ends the row in place of its last separator. |
How | How a stream reads another (Reads): all of its tokens, or those up to the reader’s own. |
HOWS | Each How as the model reads it (DecisionInputs.reads); 0 is reading none. |
Reads | Which of a row’s streams each stream’s tokens attend to: its attention record (row 2319). |
streamed_attention | What a row of segments reads (Layout.attention): its reads over its streams (Reads.lowered) and the most runs of one stream it lays (fragment_count); None for a row read in order. |
segment_stream | The stream a segment of a row’s segments stands in (Reads): its own where it names one, a question’s option lines its instructions’ (Instructions.stream), else the row’s. |
stream_names | A row’s streams (Reads): the row’s, then each other one its segments name in their order, the context its cross-attention reads not among them. |
fragment_count | The most runs of one stream a row of segments lays (the layout’s framing in the row’s stream): the spans the model lowers its reads over (DecisionInputs.fragments). |
stream_point | Append to chunks the boundary where a row’s stream turns to stream from current, a chunk of no ids at which its text is encoded apart (dew.decision.renderer.render), and return the stream now written; none where it stays. |
streamed_chunks | chunks, each in its stream of streams, with the boundaries where the row turns to another (stream_point), the row’s own again after them. |
token_streams | Each of a row’s length tokens’ stream, as its index in its segments’ (stream_names), from the boundaries its rendering kept (stream_point): the row’s own before the first. |
Schema | Every question of the request as a field of the schema, never cut: its id, type and instructions, then each allowed option, each rendered as compact JSON with sorted keys, {"description": ..., "option_id": ...}. |
Segment | A part of a joint row’s user turn (JointLayout.segments). |
Special | A special token the row writes as its id: the tokenizer’s begin token (none where it has none) or the layout’s separator (Specials). |
Instructions | A question’s instructions under a header of its kind (headers, {} the rendered instructions), rendered by render (rendered); defaults are a kind’s instructions where its question’s render empty, as instructions of whitespace alone do under a release’s entry. |
OptionLines | A question’s options, one line each, joined by join: line filled with the option’s label (QuestionLayout.labels), its name and its description rendered by render, bare where that renders empty, and {option} its whole text as the built-in rows write it (QuestionLayout.options: key: description, level i: description, false: ...); each option’s span is its whole line. |
AnswerCue | The cue a readout answers after: template filled with the options’ labels joined by join, many past at options. |
QuestionSegment | A part of a question row’s text (QuestionLayout.segments). |
QUESTION_KINDS | The parts a question row renders (QuestionSegment), which its layout refuses any other of. |
JOINT_KINDS | The parts a joint row renders (Segment), which its layout refuses any other of: its questions, their instructions and options, are its schema’s, and it reads no cue. |
RowSegment | A part of either layout’s row (Segment, QuestionSegment), as a run names its row’s parts (DecisionRunConfig.segments); each layout refuses the parts it has none of. |
rendered | value as a segment writes it: question, a question layout’s text (render) without the marker’s text; compact, a joint row’s sorted compact JSON; entry, a release’s entry (a string stripped, any other value indented JSON, None nothing); state, a release’s state (a string as it is, any other value indented JSON, None nothing). |
JOINT_SEGMENTS | The joint row’s user turn as the Laya joint release lays it out: the state’s heading, the images, the state, the schema, and a line break before the turn’s end. |
JointLayout | Every question of a request in one row, each option a span of its own. |
default_layout | The layout a head reads where none is given: every question of a request in one row (JointLayout) for a head that reads several a row (Head.one_question false), else the one the backbone’s order asks for, the state first for a causal backbone (StateFirstLayout) and Laya’s markers for a bidirectional one (MarkerLayout), each choice option written as its description, the option’s text (QuestionLayout.option_text). |
question_owners | The [B, Q] row each question of packed windows belongs to, 0 for a padding slot: the row the packer placed it in (rows, PackedRows’ question_rows), or where none is given the row its span starts in, and where the spans are the context’s (DecisionInputs.spans_in_context) the window’s one row (DecisionInputs.collate). |
DecisionInputs | A batch of laid-out rows: packed windows of whole rows, or rows padded to the longest. |
labelled_rows | Whether rows’ options stand under labels (Labels), read at an answer cue; one batch reads its rows’ options one way. |
OPTION_TILE | What a batch’s option slots round up to when its caller fixes none (DecisionInputs.collate). |
compacted | A row’s real options, options [Q, K], as slots flat indices into its grid in row order, padded with 0, and which of them are real. |
TokenizerSpecials
Section titled “TokenizerSpecials”class TokenizerSpecials(Protocol)What Specials.of reads off a tokenizer, as a transformers tokenizer
holds it: its special tokens’ ids, each None where it has none, and an id’s
token text.
TokenizerSpecials.convert_ids_to_tokens
Section titled “TokenizerSpecials.convert_ids_to_tokens”def convert_ids_to_tokens(ids: int, /) -> str | list[str]Specials
Section titled “Specials”class Specials( begin: int | None, separator: int, marker: int, marker_text: str, pad: int,)The special tokens a layout places.
They are the row’s start token (None for a vocabulary without one), the separator, the option marker (and its text, which is removed from anything a caller writes), and the padding id.
Specials.of
Section titled “Specials.of”def of(tokenizer: Tokenizer | TokenizerSpecials) -> SpecialsReturn a tokenizer’s own special tokens, read off any tokenizer that
holds them as TokenizerSpecials declares, a transformers tokenizer or
the one an HFTokenizer loads among them.
That is [CLS] or its start token, [SEP] or its end token, [MASK] (or else its end token) as the marker, and its padding token or else its end token.
class Laid( name: str, kind: int, span: tuple[int, int], options: tuple[tuple[int, int], ...], order: tuple[int, ...], readout: int | None = None, label_ids: tuple[tuple[int, ...], ...] | None = None,)One question as a row lays it out.
span holds the question’s own tokens, its instructions, and options each
option slot’s tokens, as (start, end) positions in the row. Slot s shows the
question’s option order[s].
readout: int | None-
Where a readout of labels reads the question’s answer (
Labels): the position of the row’s answer cue’s last token, or None where the options are read at their own tokens. label_ids: tuple[tuple[int, ...], ...] | None-
Each option slot’s label as the token ids a readout reads at
readout, one a variant (Labels.variants), or None where no label stands for the options. kind: int-
The question’s type, numbered as
KINDSnumbers it.
Encoded
Section titled “Encoded”class Encoded( tokens: tuple[int, ...], questions: tuple[Laid, ...], positions: tuple[int, ...] | None, slots: tuple[int, ...] | None, state_tokens: int, state_kept: int, media: tuple[int, int] | None = None, instructions_cut: int = 0, options_cut: int = 0, state_cut: int = 0, state_characters_cut: int = 0, options_alike: bool = False, context: tuple[int, ...] | None = None, spans_in_context: bool = False, blocks: tuple[int, ...] | None = None, streams: tuple[int, ...] | None = None,)One row: its tokens and the questions laid out in it.
In the parallel layout it also holds each token’s position and option slot (0 for the tokens every option shares).
media: tuple[int, int] | None-
Where the row holds a request’s images, as a [start, end) span of its tokens.
instructions_cut: int-
The question’s instructions’ tokens its layout’s declared budget cut (
QuestionLayout.head_max_len); none where they are kept whole. options_cut: int-
The options’ tokens its layout’s declared budgets cut (
QuestionLayout.head_max_len,option_tokens), over every option; none where each is kept whole. state_cut: int-
The state’s tokens the row’s cut removed, over every copy of it the row holds, the whole state where the row holds none of it.
state_characters_cut: int-
The state’s characters a cut removed where the row’s text is encoded whole, over every copy: a cut text encodes anew, so its removed tokens are not a count to subtract.
options_alike: bool-
Whether a cut left two of a question’s options as the same tokens where their whole texts differ, which no reading tells apart: such a row is refused (
Layout.refusal). context: tuple[int, ...] | None-
The tokens of the context the row reads beside its state, through a backbone’s cross-attention, or None for none.
spans_in_context: bool-
Whether the questions’ spans are positions in the context, which holds the schema (
Schema.stream), rather than in the row. blocks: tuple[int, ...] | None-
Each token’s block (
QuestionLayout.blocks): 0 on the row before its question, 1 from the question to the row’s end, which a backbone that reads blocks attends to whole within each (AttentionMetadata.block_groups); None for a row read in order. streams: tuple[int, ...] | None-
Each token’s stream, its index in the layout’s (
stream_names), which its reads say what each token attends to (Reads,Layout.attention); None for a row read in order. state_tokens: int-
The state’s length in tokens, before the layout cut it to fit.
trimmed: bool-
Whether its layout’s budgets cut the question’s instructions or options.
cuts: dict[str, int]-
The tokens each cut removed from the row, by what it cut (
instructions,options,state), or the characters of a state its row encodes whole (state_characters), andoptions_alikewhere its options’ cut left two alike; empty where the row holds its state and its questions whole. Training counts them in its data counters and a server in its own (dew.data.mixing.CorpusCounts), alike.
OPTIONS
Section titled “OPTIONS”OPTIONS: Literal['options'] = 'options'A question field’s axis of one entry per option slot (QuestionField.shape).
QuestionField
Section titled “QuestionField”class QuestionField( name: str, shape: tuple[int | Literal['options'], ...], dtype: type, fill: int = 0, when: Literal['always', 'labelled', 'trained', 'grouped'] = 'always',)One array a row holds along its questions: its name, the shape each question slot holds
(OPTIONS for an axis of one entry per option slot), its dtype, its value in a slot that holds
no question, and when a row carries it: always, where its options stand under labels
(labelled), in training (trained), or where its questions stand in consistency groups
(grouped).
QuestionField.empty
Section titled “QuestionField.empty”def empty(questions: int, width: int) -> np.ndarrayThe field over questions slots of width options each, every slot at its fill.
QUESTION_FIELDS
Section titled “QUESTION_FIELDS”QUESTION_FIELDSThe fields along a row’s questions a model reads (DecisionInputs): kinds, questions (the
real ones), spans, option_spans, options (the real ones), and where its options stand under
labels (Labels) readout, each question’s answer cue. Every producer, packer and consumer of a
row’s questions reads their names, shapes and fills here; a training row extends them
(dew.decision.rows.TRAINING_FIELDS).
question_fields
Section titled “question_fields”def question_fields( row: Encoded, count: int, width: int, labelled: bool = False,) -> dict[str, np.ndarray]row’s questions as fixed-shape arrays (QUESTION_FIELDS), count question slots of width
option slots each, readout where its options stand under labels (labelled). A training row
(Encoding) and a served batch (DecisionInputs.collate) both read a row through this.
MEDIA_MARK
Section titled “MEDIA_MARK”MEDIA_MARK = ('media',)The mark of a row’s images (Encoded.media), which no question’s mark, a (name, part) pair, is.
INSTRUCTIONS_MARK
Section titled “INSTRUCTIONS_MARK”INSTRUCTIONS_MARK = 'instructions'The part of a question’s mark that its instructions take, beside its options’ slots.
render
Section titled “render”def render(value: JSON) -> strReturn a value as the model reads it: text as it is, and anything structured as JSON.
context_tokens
Section titled “context_tokens”def context_tokens( tokenizer: Tokenizer, context: JSON | None, length: int | None,) -> np.ndarrayA context’s tokens as a row carries them: its text’s (render), without specials, and none for
no context. A context of more than length tokens, which the cross-attention reads no more of,
is refused (ValueError), never cut: a cut would answer from part of what the caller gave; None
reads it whole, as a count of a context no cross-attention reads does.
with_context
Section titled “with_context”def with_context( rows: Sequence[Encoded], tokenizer: Tokenizer, context: JSON | None, length: int | None,) -> list[Encoded]rows with the context a backbone’s cross-attention of length context tokens reads beside
each (None: a backbone with none): a schema laid out as the rows’ context (Schema.stream), read
whole, or else context, a request’s own, its first length tokens (context_tokens). The one
resolution of a row’s context, which a run’s encoding and a task’s admission both read. Refused
(ValueError): a request’s context beside a schema read as the context, a context with no
cross-attention to read it, and a schema longer than length.
Bounds
Section titled “Bounds”class Bounds(segments: int, questions: int, options: int | None)The most rows (segments), questions and options one packed window holds;
options is None where the layout trims a question’s options to share a
budget rather than giving each tokens of its own (QuestionLayout).
Layout
Section titled “Layout”class Layout(max_len: int = 512)How a state and its questions become rows of tokens that fit max_len (fits).
joint: bool-
Whether every question of a request shares one row, rather than one row each.
images: bool-
Whether a row holds a request’s images, each as the backbone’s processor writes and expands one (
Processor.image_text). spans_in_context: bool-
Whether its rows write their questions to their contexts, which their cross-attention reads, their spans the contexts’ (
Encoded.spans_in_context). attention: tuple[tuple[tuple[int, ...], ...], int] | None-
What each of its rows’ tokens reads as the model lowers it (
DecisionInputs.reads,fragments), where its rows stand in streams (Reads); None for rows read in order.
Layout.state
Section titled “Layout.state”def state(encoder: Tokenizer, specials: Specials, state: JSON) -> tuple[int, ...]Return the state’s tokens, which every question of a request shares.
Layout.rows
Section titled “Layout.rows”def rows( encoder: Tokenizer, specials: Specials, state: JSON, questions: Mapping[str, Question], *, orders: Mapping[str, Sequence[int]] | None = None, media: Sequence[int] = (), tokens: Sequence[int] | None = None,) -> list[Encoded]Lay out questions about state, each question’s options in its orders entry.
Slot s of a question shows its option order[s]; a question orders does not
name keeps its own order (or the layout’s). media are the tokens the
backbone’s processor gave a request’s images, which a layout with an
image text places in the row. tokens are the state’s (state),
where the caller has them already.
Layout.fits
Section titled “Layout.fits”def fits(row: Encoded) -> boolWhether row is within max_len. A layout cuts the state to fit
and keeps a schema whole, so a row whose schema alone passes
max_len does not fit: training leaves its example out (Fit),
counted as unfit, and a task refuses it.
Layout.refusal
Section titled “Layout.refusal”def refusal(row: Encoded, context_length: int | None = None) -> str | NoneWhy training leaves row out, or None where it trains: unfit, past max_len (fits), or,
where its questions are its context’s (Encoded.spans_in_context), past the context_length
tokens its backbone’s cross-attention reads (with_context); or options_alike, where a
declared cut left two options the same tokens, which no reading tells apart. Training counts it
under the reason (dew.data.mixing.CorpusCounts); a server refuses an unfit request under the
same reason, and answers one whose options its release’s budgets cut alike as the release does,
the cut in its usage.
Layout.shortest_row
Section titled “Layout.shortest_row”def shortest_row(encoder: Tokenizer, specials: Specials) -> tuple[int, int]The tokens of the row bounds counts by, one one-option question about an empty state laid
out with room to spare, and its context’s (its schema where the row reads it there): no window
shorter, nor a context, holds its bounds.
Layout.row_questions
Section titled “Layout.row_questions”def row_questions(questions: int) -> intThe most questions a row of this layout holds where an example asks questions: all of them,
a row an example.
Layout.bounds
Section titled “Layout.bounds”def bounds( encoder: Tokenizer, specials: Specials, context_length: int | None = None, segments: int | None = None, context_window: int | None = None,) -> BoundsThe most rows, at most segments, questions and options a window of max_len tokens
packs, from the fewest tokens each takes in this layout: a row of one
one-option question about an empty state, and what one more option or
one more one-option question adds to it, laid out with room to spare
so that nothing is cut. A window of m rows then holds at most m
questions and m options in m rows’ tokens, and each further question
or option takes its own. The packer holds every window to these
bounds, so a row shorter still starts a new window rather than
outgrowing one. Where the questions are laid out in the row’s context
(Encoded.spans_in_context), each row reads a context of its own of at most context_length
tokens (DecisionInputs.context_segments), the window’s contexts at most context_window (one
row’s where none is given): a window holds as many rows as its tokens hold, and as many questions
and options as its contexts do, a context’s schema of one question taking the place a row’s
tokens take along them.
LETTERS
Section titled “LETTERS”LETTERS = tuple(chr(ord('A') + index) for index in range(26))LABEL_NAMES
Section titled “LABEL_NAMES”LABEL_NAMES = (*LETTERS, *(first + second for first in LETTERS for second in LETTERS))Every label, in order: A to Z, then AA, AB, …, ZZ, as a release’s slots name them.
label_names
Section titled “label_names”def label_names(count: int) -> tuple[str, ...]The first count labels of LABEL_NAMES.
Labels
Section titled “Labels”class Labels( cue: str = '\nAnswer:', variants: tuple[str, ...] = (' {}',), names: tuple[str, ...] | None = None,)Options rendered as lettered lines, read at the row’s answer cue by a readout of the labels’
logits (dew.decision.head.LabelReadout, SlotReadout): slot s as {name}. {text} on a line of
its own, its span the whole line, and after the options the cue, whose last token the readout
reads. Each label is read as the token id of each of variants, a template of its name ({}): a
release’s slot is one variant, its name after a space; a readout groups several as one. A variant
that spells as other than one token is refused.
names are the labels in slot order, a release’s own list where it names its slots otherwise
(one that leaves out the pairs its tokenizer spells as two tokens), or label_names’ where unset;
the lines and the ids both read them (names_for), and a question with more options than they
name is refused.
Labels.names_for
Section titled “Labels.names_for”def names_for(count: int) -> tuple[str, ...]The first count labels.
Labels.capacity
Section titled “Labels.capacity”def capacity(encoder: Tokenizer) -> intHow many options these labels letter under encoder: their own names, or every name of
LABEL_NAMES whose every variant it spells as one token (spelled).
Labels.spelled
Section titled “Labels.spelled”def spelled(encoder: Tokenizer, count: int) -> LabelsThese labels named for encoder: themselves where they name their own, else the first count of
LABEL_NAMES whose every variant encoder spells as one token, skipping those it spells as
two (Qwen3.5 spells ” BQ” as two, and 570 of the 702 as one), so a run letters as many options
as its tokenizer can. Fewer than count such names is refused.
Labels.ids
Section titled “Labels.ids”def ids(encoder: Tokenizer, count: int) -> tuple[tuple[int, ...], ...]Each of the first count labels’ ids, one a variant.
QuestionLayout
Section titled “QuestionLayout”class QuestionLayout( max_len: int = 512, head_max_len: int | None = None, option_tokens: int | None = None, parallel: bool = False, labels: Labels | None = None, segments: tuple[QuestionSegment, ...] | None = None, unit: Unit = 'pieces', framing: Framing | None = None, blocks: bool = False, split: Literal['part', 'question', 'whole'] | None = None, order: RowOrder | None = None, reads: Reads | None = None, option_text: Literal['named', 'described'] = 'named',)A layout of one row per question, scored at one marker token per option.
Each option is rendered as text (option_text: key: description for a choice, or its
description alone, level i: description for a score, false: ... and true: ... for a noul).
The question, its instructions and options, is kept whole, and the state
fills the rest of max_len; a list state (a conversation) keeps its end, and
any other state keeps its start. A question that alone passes max_len is a
row that does not fit, which training leaves out and a task refuses, each
counting it (Layout.refusal); its text is never cut silently.
A layout may declare budgets for the question, as Laya’s release does: each
option cut to option_tokens, and where the options exceed head_max_len
minus 16 tokens every option cut evenly to fit and the instructions given
what the options leave, at least 8 tokens. Each cut is counted
(Encoded.instructions_cut, options_cut), and a row whose cut leaves two options alike is
refused (Encoded.options_alike).
parallel starts every option at the same position and lets no option attend
to another, so the options’ order does not affect their scores; whatever
follows the options continues after the longest one.
head_max_len: int | None-
The tokens the question, its instructions and its options, may take, as a release declares it (Laya’s 192), or None to keep the question whole.
option_tokens: int | None-
The most tokens one option keeps, as a release declares it (Laya’s 48), or None to keep each whole.
labels: Labels | None-
Options rendered as lettered lines and read at an answer cue (
Labels), or None for a marker after each option, read at the marker. segments: tuple[QuestionSegment, ...] | None-
A row’s text as segments of its own (
dew.decision.renderer), read at its labels’ readout at the row’s last token, or None for the layout’s own text. unit: Unit-
How
segments’ text is encoded: piece by piece, or whole (dew.decision.renderer). framing: Framing | None-
The conversation
segments’ text is the user’s turn of, or None for none. blocks: bool-
Whether each row splits into two blocks (
Encoded.blocks), its parts assegmentsorder them: a state-first row’s state, and its question from its header to the row’s end, a published slot release’s prefix and suffix (serve.patchblock_ids); a question-first row’s question, and its state from the state’s first token to the row’s end (theTextbetween them keeps to the part it follows). A backbone that reads blocks attends to each whole, and the second to the first in order. Off, every row is read in order. split: Literal['part', 'question', 'whole'] | None-
Where a row in
blockssplits: at its second part (part, a row of segments’ own law: a state-first row’s question from its header on, a question-first row’s state from its first token on), at its question’s first token (question, the built-in rows’), or nowhere, the whole row one block its backbone reads both ways (whole); None takes the layout’s own law. A row read in order (blocksoff) splits nowhere, so it states none. order: RowOrder | None-
The order the layout’s own row writes its parts in (
built_in), each part as the built-in row writes it, its begin token, separators, header, budgets and a conversation’s kept end alike: the state first, the question first, or the state, the question and the state again (re_read, each copy its share of the room); None takes the layout’s own (StateFirstLayout’s state first,MarkerLayout’s question first). A row of its ownsegmentsstates its own order. reads: Reads | None-
Which of the row’s streams each reads (
Reads), where its question’s segments stand in a stream of the row’s own (Instructions.stream); None for a row read in order. option_text: Literal['named', 'described']-
How a choice’s option is written (
options):named, its key and its description (key: description), as Laya’s release reads its rows; ordescribed, its description alone, the option’s text, which a fresh head reads (default_layout), so a key that only identifies the option (option_0) writes nothing the row reads. An option with no description is its key. written: tuple[QuestionSegment, ...]-
The row’s parts in order: its own
segments, or the layout’s built-in row (built_in). spans_in_context: bool-
Whether its row writes its question to its context (
Instructions.stream). attention: tuple[tuple[tuple[int, ...], ...], int] | None-
Its rows’ reads over their streams (
streamed_attention).
QuestionLayout.state
Section titled “QuestionLayout.state”def state(encoder: Tokenizer, specials: Specials, state: JSON) -> tuple[int, ...]Return the state’s tokens, which every question of a request shares.
QuestionLayout.row_questions
Section titled “QuestionLayout.row_questions”def row_questions(questions: int) -> intOne: a question layout writes a row a question.
QuestionLayout.rows
Section titled “QuestionLayout.rows”def rows( encoder: Tokenizer, specials: Specials, state: JSON, questions: Mapping[str, Question], *, orders: Mapping[str, Sequence[int]] | None = None, media: Sequence[int] = (), tokens: Sequence[int] | None = None,) -> list[Encoded]QuestionLayout.encode
Section titled “QuestionLayout.encode”def encode( encoder: Tokenizer, specials: Specials, question: Question, state: Sequence[int], *, conversation: bool = False, order: Sequence[int] | None = None, name: str = '',) -> EncodedLay out question over the state’s tokens state (state, a row of one copy of the state
rendered as the layout renders it), with its options in order (rows).
Slot s shows option order[s]; None keeps the question’s order. A conversation keeps the end of
the state instead of its start.
QuestionLayout.options
Section titled “QuestionLayout.options”def options(question: Question) -> tuple[str, ...]Return each option’s text, in the question’s order (option_text).
MarkerLayout
Section titled “MarkerLayout”class MarkerLayout( max_len: int = 512, head_max_len: int | None = None, option_tokens: int | None = None, parallel: bool = False, labels: Labels | None = None, segments: tuple[QuestionSegment, ...] | None = None, unit: Unit = 'pieces', framing: Framing | None = None, blocks: bool = False, split: Literal['part', 'question', 'whole'] | None = None, order: RowOrder | None = None, reads: Reads | None = None, option_text: Literal['named', 'described'] = 'named',)Laya’s layout, for a bidirectional backbone: the question, a marker before each option, then the state.
parallel is Laya’s option_layout="parallel"; the published checkpoints
are sequential.
StateFirstLayout
Section titled “StateFirstLayout”class StateFirstLayout( max_len: int = 512, head_max_len: int | None = None, option_tokens: int | None = None, parallel: bool = False, labels: Labels | None = None, segments: tuple[QuestionSegment, ...] | None = None, unit: Unit = 'pieces', framing: Framing | None = None, blocks: bool = False, split: Literal['part', 'question', 'whole'] | None = None, order: RowOrder | None = None, reads: Reads | None = None, option_text: Literal['named', 'described'] = 'named',)The layout for a causal backbone: the state, the question, then each option followed by its marker.
A causal backbone reads each token only after the tokens before it:
[start] state [SEP] <type> question: <instructions> [SEP] opt0 [MARK] opt1 [MARK] ... [SEP]so the hidden state at each marker has read the state, the question and that option. In the parallel layout an option reads no other option.
JOINT_SYSTEM
Section titled “JOINT_SYSTEM”JOINT_SYSTEM = "Read the complete state and schema. Decide every field jointly. Each answer must be exactly one of that field's allowed options."The joint layout’s system prompt.
JOINT_PREFILL
Section titled “JOINT_PREFILL”JOINT_PREFILL = 'JOINT SCHEMA DECISIONS:'The joint layout’s opening of the assistant’s turn, after the turn’s own markup.
Framing
Section titled “Framing”class Framing( system: str = JOINT_SYSTEM, prefill: str = JOINT_PREFILL, thinking: bool | None = False, template: bool = True, chat_template: str | None = None, options: Mapping[str, JSON] = dict(),)How a joint row frames its text as a conversation: a system message, the user’s turn that
holds the state and the schema, and the assistant’s turn opened with prefill.
The turns’ markup is the backbone’s own: where the tokenizer has a chat template and template
holds, its template renders the system message, the user’s turn around its content and the
generation prompt (dew.data.chat.framed), thinking its reasoning switch (False: a reasoning
model’s empty thought, None: the template’s own default), so every token of the markup is the
tokenizer’s own special token, never ChatML spelled out for a vocabulary without it.
chat_template, a Jinja chat template, renders the turns in place of the tokenizer’s own (a base
model’s, which has none, or a framing of an experiment’s), and options are the other variables
a template reads beside the messages; both are recorded with the layout, so a run reloads its
framing. A tokenizer with no chat template and none given, or template False, frames the turns
as plain text: the system message, then the user’s content, each after a blank line.
Framing.around
Section titled “Framing.around”def around(encoder: Tokenizer) -> tuple[str, str]The text before the user’s turn’s content and after it, to the assistant’s prefill.
JOINT_NOUL
Section titled “JOINT_NOUL”JOINT_NOUL = {'true': 'The proposition is true or the answer is yes.', 'false': 'The proposition is false or the answer is no.'}A noul option’s description where its question gives none.
class Text(text: str)Text a joint row writes as given, at its place in the user’s turn. Text beside the turn’s markup, or beside other text, is read with it as one piece, as the turn writes it.
class Media()Where a joint row places a request’s images, each as the backbone’s processor writes one
(Processor.image_text) and expands it; a request without images places nothing here.
Render
Section titled “Render”How a segment writes a value as text (rendered).
class State( render: Render = 'compact', keep: Literal['start', 'end', 'all', 'end_if_conversation'] = 'start', max_tokens: int | None = None,)The state, as compact JSON with sorted keys, its start kept: cut to its own budget
(max_tokens, or a joint layout’s max_state_tokens where it gives none) and to its share of
what the rest of the row leaves of its max_len (dew.data.tokens.share_tokens). A row may hold
the state more than once, each copy its own segment, render, budget and keep. A
question row’s segments say how it renders (render, rendered) and what a cut keeps of it
(keep: its start, its end, or all of it, a row that it leaves too long then unfit); a joint row
renders it compact.
keep: Literal['start', 'end', 'all', 'end_if_conversation']-
What a cut keeps of the state: its start, its end, all of it, or its end where it is a conversation (a list state, its last turns) and its start otherwise, as the built-in rows keep it.
allocated
Section titled “allocated”def allocated( lengths: Sequence[int], copies: Sequence[State], room: int, default_cap: int | None = None,) -> list[int]How many of its own tokens each copy of the state keeps of room (dew.data.tokens.share_tokens):
a copy kept whole (keep="all") all of its lengths, whatever its cap, and the others what is
left, each at most its own cap (max_tokens, or default_cap where it gives none).
RowOrder
Section titled “RowOrder”The order a question layout’s own row writes its parts in (QuestionLayout.order).
built_in
Section titled “built_in”def built_in(labels: Labels | None, order: RowOrder) -> tuple[QuestionSegment, ...]The built-in question row as segments (QuestionLayout.written): the begin token, the question’s
header and instructions as one piece ("{kind} question: {instructions}"), the option lines and the
state, each part after a separator, in order: the state first, the question first, or the state,
the question and the state again; Laya’s markers before each option where the question comes first,
a causal row’s after, or lettered lines read at the labels’ cue, which ends the row in place of its
last separator. A conversation keeps its end in every copy.
How a stream reads another (Reads): all of its tokens, or those up to the reader’s own.
HOWS: dict[str, int] = {'whole': 1, 'causal': 2}Each How as the model reads it (DecisionInputs.reads); 0 is reading none.
class Reads(table: tuple[tuple[str, str, How], ...])Which of a row’s streams each stream’s tokens attend to: its attention record (row 2319).
A row’s tokens stand in streams: "row", and each other stream a segment names (Schema.stream,
Instructions.stream, its option lines with it), every other segment in the row’s own. table
holds (reader, read, how): a token of reader reads the tokens of read ("self", its own
stream), all of them ("whole") or those up to itself ("causal"); a pair the table omits reads
none. Two streams in one pass where the state reads the schema and the schema itself alone:
Reads((("row", "self", "causal"), ("row", "schema", "whole"), ("schema", "self", "causal"))),
the schema in Schema(stream="schema"), wherever the row lays it. Each stream reads itself, so
no token reads nothing; positions run along the row as it lays its tokens.
Reads.lowered
Section titled “Reads.lowered”def lowered(streams: Sequence[str]) -> tuple[tuple[int, ...], ...]The record as the model reads it, over a row’s streams in their order: each stream’s
reading of itself, then of each stream (HOWS, 0 for none).
streamed_attention
Section titled “streamed_attention”def streamed_attention( reads: Reads | None, segments: Sequence[Segment | QuestionSegment],) -> tuple[tuple[tuple[int, ...], ...], int] | NoneWhat a row of segments reads (Layout.attention): its reads over its streams (Reads.lowered)
and the most runs of one stream it lays (fragment_count); None for a row read in order.
segment_stream
Section titled “segment_stream”def segment_stream( segment: Segment | QuestionSegment, segments: Sequence[Segment | QuestionSegment],) -> strThe stream a segment of a row’s segments stands in (Reads): its own where it names one, a
question’s option lines its instructions’ (Instructions.stream), else the row’s.
stream_names
Section titled “stream_names”def stream_names(segments: Sequence[Segment | QuestionSegment]) -> tuple[str, ...]A row’s streams (Reads): the row’s, then each other one its segments name in their order, the
context its cross-attention reads not among them.
fragment_count
Section titled “fragment_count”def fragment_count(segments: Sequence[Segment | QuestionSegment]) -> intThe most runs of one stream a row of segments lays (the layout’s framing in the row’s stream):
the spans the model lowers its reads over (DecisionInputs.fragments).
stream_point
Section titled “stream_point”def stream_point( chunks: list[Chunk] | list[Chunk | State], stream: str, current: str,) -> strAppend to chunks the boundary where a row’s stream turns to stream from current, a chunk of no
ids at which its text is encoded apart (dew.decision.renderer.render), and return the stream now
written; none where it stays.
streamed_chunks
Section titled “streamed_chunks”def streamed_chunks(chunks: Sequence[Chunk], streams: Sequence[str]) -> list[Chunk]chunks, each in its stream of streams, with the boundaries where the row turns to another
(stream_point), the row’s own again after them.
token_streams
Section titled “token_streams”def token_streams( points: Mapping[Hashable, int], length: int, segments: Sequence[Segment | QuestionSegment],) -> tuple[int, ...]Each of a row’s length tokens’ stream, as its index in its segments’ (stream_names), from the
boundaries its rendering kept (stream_point): the row’s own before the first. It lays at most the
runs of one stream its segments do (fragment_count), which the model numbers them by.
Schema
Section titled “Schema”class Schema(stream: str = 'row')Every question of the request as a field of the schema, never cut: its id, type and
instructions, then each allowed option, each rendered as compact JSON with sorted keys,
{"description": ..., "option_id": ...}. The head pools each option’s tokens and each
question’s instructions’.
stream is where the schema is read: in the row ("row"), or as the row’s context
("context"), which the backbone reads in a pass of its own, with its own weights, and its
cross-attention layers read beside the row (DecisionModel.states), the head pooling the
questions and options from that pass; or as a stream of the row of its own name, which the
layout’s Reads says what it reads and what reads it.
Segment
Section titled “Segment”A part of a joint row’s user turn (JointLayout.segments).
Special
Section titled “Special”class Special(token: Literal['begin', 'separator'])A special token the row writes as its id: the tokenizer’s begin token (none where it has
none) or the layout’s separator (Specials).
Instructions
Section titled “Instructions”class Instructions( headers: Mapping[str, str], render: Render = 'question', defaults: Mapping[str, str] = dict(), span: Literal['instructions', 'header'] = 'instructions', stream: str = 'row',)A question’s instructions under a header of its kind (headers, {} the rendered
instructions), rendered by render (rendered); defaults are a kind’s instructions where its
question’s render empty, as instructions of whitespace alone do under a release’s entry. The
question’s span is its instructions’ (span="instructions"), or the header and its instructions
written as one piece (span="header", the built-in rows’ "{kind} question: {instructions}").
stream is where they are read: in the row ("row"), or as the row’s context ("context"), which
its cross-attention reads, the question’s spans the context’s (Encoded.spans_in_context), its
options’ lines streaming with it; or as a stream of the row of its own name (Reads).
OptionLines
Section titled “OptionLines”class OptionLines( line: str = '{label}. {name}: {text}', bare: str = '{label}. {name}', join: str = '\n', render: Render = 'entry', level: str = 'level {}', noul: tuple[str, str] = ('yes', 'no'), criteria: Mapping[str, str] = dict(), marker: Literal['none', 'before', 'after'] = 'none', noul_first: Literal['true', 'false'] = 'true',)A question’s options, one line each, joined by join: line filled with the option’s label
(QuestionLayout.labels), its name and its description rendered by render, bare where that
renders empty, and {option} its whole text as the built-in rows write it (QuestionLayout.options:
key: description, level i: description, false: ...); each option’s span is its whole line. A
choice’s names are its keys, a score’s levels level, and a noul’s noul, its true option first
(noul_first), criteria its options’ descriptions where its question gives none.
marker writes the layout’s marker token before or after each line, where the option is read at
its marker (Laya’s markers before, a causal row’s after) and its span is the marker’s; none reads
the options under labels at the answer cue. They are read where their question’s instructions are
(Instructions.stream).
AnswerCue
Section titled “AnswerCue”class AnswerCue( template: str = 'Answer with one letter: {labels}.', many: str = 'Answer with one label: {labels}.', at: int = 26, join: str = ', ',)The cue a readout answers after: template filled with the options’ labels joined by join,
many past at options.
QuestionSegment
Section titled “QuestionSegment”A part of a question row’s text (QuestionLayout.segments).
QUESTION_KINDS
Section titled “QUESTION_KINDS”QUESTION_KINDS: tuple[type, ...] = get_args(QuestionSegment.__value__)The parts a question row renders (QuestionSegment), which its layout refuses any other of.
JOINT_KINDS
Section titled “JOINT_KINDS”JOINT_KINDS: tuple[type, ...] = get_args(Segment.__value__)The parts a joint row renders (Segment), which its layout refuses any other of: its questions,
their instructions and options, are its schema’s, and it reads no cue.
RowSegment
Section titled “RowSegment”A part of either layout’s row (Segment, QuestionSegment), as a run names its row’s parts
(DecisionRunConfig.segments); each layout refuses the parts it has none of.
rendered
Section titled “rendered”def rendered(value: JSON, policy: Render, specials: Specials) -> strvalue as a segment writes it: question, a question layout’s text (render) without the
marker’s text; compact, a joint row’s sorted compact JSON; entry, a release’s entry (a string
stripped, any other value indented JSON, None nothing); state, a release’s state (a string as
it is, any other value indented JSON, None nothing).
JOINT_SEGMENTS
Section titled “JOINT_SEGMENTS”JOINT_SEGMENTS: tuple[Segment, ...] = (Text('STATE:\n'), Media(), State(), Schema(), Text('\n'))The joint row’s user turn as the Laya joint release lays it out: the state’s heading, the images, the state, the schema, and a line break before the turn’s end.
JointLayout
Section titled “JointLayout”class JointLayout( max_len: int = 16384, max_state_tokens: int | None = None, framing: Framing = Framing(), segments: tuple[Segment, ...] = JOINT_SEGMENTS, reads: Reads | None = None,)Every question of a request in one row, each option a span of its own.
The row is a chat prompt: the system prompt (JOINT_SYSTEM), then the user’s turn, its
segments in order (JOINT_SEGMENTS: the state’s heading, a request’s images, the state,
then a schema of numbered fields, each with its id, type, instructions and allowed options),
and the opening of the assistant’s turn. A layout of segments of its own writes the turn
otherwise: the schema first, or text of its own between the parts; it holds the schema once,
the state and the images at most once each. A choice’s options come in the order of their
keys, and a noul’s as true, then false, with JOINT_NOUL’s descriptions where the question
gives none. The state is cut to max_state_tokens, and to what the rest of the row leaves of
max_len; the schema itself is never cut, so a schema longer than max_len leaves no state
and a row that does not fit (fits). The row’s conversation markup is the backbone’s own chat
template’s (framing).
reads: Reads | None-
Which of the row’s streams each reads (
Reads), where its schema stands in a stream of the row’s own (Schema.stream); None for a row read in order. spans_in_context: bool-
Whether the row’s schema is its context (
Schema.stream), its spans the context’s. attention: tuple[tuple[tuple[int, ...], ...], int] | None-
Its rows’ reads over their streams (
streamed_attention).
JointLayout.state
Section titled “JointLayout.state”def state(encoder: Tokenizer, specials: Specials, state: JSON) -> tuple[int, ...]JointLayout.rows
Section titled “JointLayout.rows”def rows( encoder: Tokenizer, specials: Specials, state: JSON, questions: Mapping[str, Question], *, orders: Mapping[str, Sequence[int]] | None = None, media: Sequence[int] = (), tokens: Sequence[int] | None = None,) -> list[Encoded]The request’s one row, rendered as chunks (dew.decision.renderer, piece by piece as the
joint release encodes it): each run of text between the turn’s other parts one chunk, the
images, each copy of the state and the schema chunks of ids, and each question’s and option’s
tokens the marks its chunks carry.
JointLayout.options
Section titled “JointLayout.options”def options(question: Question) -> tuple[tuple[str, JSON], ...]Return each option’s key and description, in the question’s order.
JointLayout.order
Section titled “JointLayout.order”def order(question: Question) -> list[int]Return the question’s options in the joint order: a choice’s by key, a noul’s true first.
default_layout
Section titled “default_layout”def default_layout(causal: bool, one_question: bool) -> LayoutThe layout a head reads where none is given: every question of a
request in one row (JointLayout) for a head that reads several a row
(Head.one_question false), else the one the backbone’s order asks for,
the state first for a causal backbone (StateFirstLayout) and Laya’s
markers for a bidirectional one (MarkerLayout), each choice option written as its description,
the option’s text (QuestionLayout.option_text).
question_owners
Section titled “question_owners”def question_owners( segments, spans, questions, spans_in_context: bool, xp=np, rows=None,)The [B, Q] row each question of packed windows belongs to, 0 for a padding slot: the row the
packer placed it in (rows, PackedRows’ question_rows), or where none is given the row its
span starts in, and where the spans are the context’s (DecisionInputs.spans_in_context) the
window’s one row (DecisionInputs.collate). xp is the array namespace, NumPy for host geometry
and jax.numpy in a trace, so the model’s question grouping and the counted FLOPs read one owner.
DecisionInputs
Section titled “DecisionInputs”class DecisionInputs( positions: jnp.ndarray | None = None, slots: jnp.ndarray | None = None, option_slots: jnp.ndarray | None = None, slotted: jnp.ndarray | None = None, context: jnp.ndarray | None = None, context_lengths: jnp.ndarray | None = None, context_segments: jnp.ndarray | None = None, question_rows: jnp.ndarray | None = None, readout: jnp.ndarray | None = None, blocks: jnp.ndarray | None = None, streams: jnp.ndarray | None = None, reads: tuple[tuple[int, ...], ...] | None = struct.field(pytree_node=False, default=None), fragments: int = struct.field(pytree_node=False, default=1), max_segments: int = struct.field(pytree_node=False, default=1), spans_in_context: bool = struct.field(pytree_node=False, default=False), whole_blocks: bool = struct.field(pytree_node=False, default=False),)A batch of laid-out rows: packed windows of whole rows, or rows padded to the longest.
tokens[B, L]ids,segments[B, L]each token’s row in its window, numbered from 1 in window order and 0 on the padding after the last (Pack); a padded batch is windows of one row each (collate);positions[B, L]each token’s place in its row (None: counted from 0 along it), and in the parallel layoutslots[B, L]each token’s option slot, 0 for the tokens every option shares;kinds[B, Q]each question’s type (KINDS),questions[B, Q]the real questions, in their rows’ window order,spans[B, Q, 2]their instructions’ (start, end) in the window;option_spans[B, Q, K, 2]each option slot’s (start, end),options[B, Q, K]the real slots;option_slots[B, P]each window’s real options as flat indicesq * K + kinto its[Q, K]grid, in that order (compacted), andslotted[B, P]which of them are real, or None for the grid itself;context[B, C]each window’s rows’ context tokens end to end, which a backbone’s cross-attention reads,context_lengths[B]how many are real, the padding after them, andcontext_segments[B, C]each one’s row in the window assegmentsnumbers it (0 for padding), which its row’s tokens alone read; or None for rows that carry none;question_rows[B, Q]each question’s row in its window as the packer placed it, numbered assegmentsnumbers its tokens (0 for a padding slot), or None where a question’s row is the one its span starts in.
A row of one question with one marker per option, Laya’s, has spans one
token long. A head that reads option_slots computes the real options
alone, P rather than Q x K of them: a window holds a few of its grid’s.
max_segments bounds the rows a window holds (Layout.bounds).
readout: jnp.ndarray | None-
[B, Q]where a readout of labels reads each question’s answer (Laid.readout), or None; the labels’ ids are the readout head’s own (LabelReadout.label_ids). blocks: jnp.ndarray | None-
[B, L]each token’s block in its row (Encoded.blocks), -1 on the padding, or None for rows read in order. streams: jnp.ndarray | None-
[B, L]each token’s stream in its row (Encoded.streams), -1 on the padding, or None for rows read in order. reads: tuple[tuple[int, ...], ...] | None-
What each stream’s tokens read, as the layout lowers its
Reads(Reads.lowered), where the rows stand instreams. fragments: int-
The most runs of one stream a row lays (
fragment_count), which its tokens’ key runs number. spans_in_context: bool-
Whether the questions’ spans are positions in
context, which holds the schema (Schema.stream), rather than intokens. whole_blocks: bool-
Whether each row’s one block is the whole row (
QuestionLayout.split‘whole’), which a backbone reads as non-causal attention by rows (AttentionMetadata.whole_blocks). valid: jnp.ndarray-
Return the
[B, L]real tokens. question_segments: jnp.ndarray-
Return the
[B, Q]row each question belongs to, 0 for a padding slot (question_owners). markers: jnp.ndarray-
Return the
[B, Q, K]first token of each option slot, where a marker layout scores it.
DecisionInputs.coordinates
Section titled “DecisionInputs.coordinates”def coordinates() -> CoordinatesThe tokens’ Coordinates: each row a document, real tokens first, at
most max_segments a window. The padding after them runs as one more
sequence, which a sequence kernel adds to the documents itself
(dew.nn.attention.flash_attention).
DecisionInputs.collate
Section titled “DecisionInputs.collate”def collate( rows: Sequence[Encoded], pad: int, *, questions: int | None = None, options: int | None = None, option_slots: int | None = None, context_length: int | None = None, attention: tuple[tuple[tuple[int, ...], ...], int] | None = None,) -> DecisionInputsRight-pad rows into one batch (pad_token_rows), one row a window, its question
and option slots to the widest.
questions and options widen those slots further, to shapes a caller
keeps fixed. option_slots is the real options a row may hold, a shape
a caller keeps fixed too (compacted); without it, the batch’s most,
rounded up to OPTION_TILE, where that is fewer than the grid’s slots.
With a context_length, each row’s context (Encoded.context, none
where it holds none) is padded to that many tokens. Rows in streams are read as their layout’s
attention says (Layout.attention).
labelled_rows
Section titled “labelled_rows”def labelled_rows(rows: Sequence[Encoded]) -> boolWhether rows’ options stand under labels (Labels), read at an answer cue; one batch reads
its rows’ options one way.
OPTION_TILE
Section titled “OPTION_TILE”OPTION_TILE = 128What a batch’s option slots round up to when its caller fixes none (DecisionInputs.collate).
compacted
Section titled “compacted”def compacted(options: np.ndarray, slots: int) -> tuple[np.ndarray, np.ndarray]A row’s real options, options [Q, K], as slots flat indices into
its grid in row order, padded with 0, and which of them are real.
Raises ValueError for a row of more real options than slots: its shape
is the caller’s to keep, and widening it would compile the step again.