Skip to content

dew.decision.layout

How a state and its questions become the token rows a decision model reads.

A layout places each question’s instructions and each of its options at a span of tokens, which the head reads. MarkerLayout is Laya’s (laya/common.py build_sequence, build_head and parallel_layout at NandhaKishorM/laya a4a8921), one row per question: the question, then one marker token before each option’s text, then the state,

[CLS] <type> question: <instructions> [SEP] [MASK] opt0 [MASK] opt1 ... [SEP] state [SEP]

and the model scores each option at its marker. The option texts share a token budget, head_max_len, and the state takes what is left of max_len. StateFirstLayout puts the state first and each marker after its option, for a causal backbone. JointLayout lays every question of a request in one row, each option a span of its own.

NameSummary
TokenizerSpecialsWhat Specials.of reads off a tokenizer, as a transformers tokenizer holds it: its special tokens’ ids, each None where it has none, and an id’s token text.
SpecialsThe special tokens a layout places.
LaidOne question as a row lays it out.
EncodedOne row: its tokens and the questions laid out in it.
OPTIONSA question field’s axis of one entry per option slot (QuestionField.shape).
QuestionFieldOne array a row holds along its questions: its name, the shape each question slot holds (OPTIONS for an axis of one entry per option slot), its dtype, its value in a slot that holds no question, and when a row carries it: always, where its options stand under labels (labelled), in training (trained), or where its questions stand in consistency groups (grouped).
QUESTION_FIELDSThe fields along a row’s questions a model reads (DecisionInputs): kinds, questions (the real ones), spans, option_spans, options (the real ones), and where its options stand under labels (Labels) readout, each question’s answer cue.
question_fieldsrow’s questions as fixed-shape arrays (QUESTION_FIELDS), count question slots of width option slots each, readout where its options stand under labels (labelled).
MEDIA_MARKThe mark of a row’s images (Encoded.media), which no question’s mark, a (name, part) pair, is.
INSTRUCTIONS_MARKThe part of a question’s mark that its instructions take, beside its options’ slots.
renderReturn a value as the model reads it: text as it is, and anything structured as JSON.
context_tokensA context’s tokens as a row carries them: its text’s (render), without specials, and none for no context.
with_contextrows with the context a backbone’s cross-attention of length context tokens reads beside each (None: a backbone with none): a schema laid out as the rows’ context (Schema.stream), read whole, or else context, a request’s own, its first length tokens (context_tokens).
BoundsThe most rows (segments), questions and options one packed window holds; options is None where the layout trims a question’s options to share a budget rather than giving each tokens of its own (QuestionLayout).
LayoutHow a state and its questions become rows of tokens that fit max_len (fits).
LETTERS
LABEL_NAMESEvery label, in order: A to Z, then AA, AB, …, ZZ, as a release’s slots name them.
label_namesThe first count labels of LABEL_NAMES.
LabelsOptions rendered as lettered lines, read at the row’s answer cue by a readout of the labels’ logits (dew.decision.head.LabelReadout, SlotReadout): slot s as `{name}.
QuestionLayoutA layout of one row per question, scored at one marker token per option.
MarkerLayoutLaya’s layout, for a bidirectional backbone: the question, a marker before each option, then the state.
StateFirstLayoutThe layout for a causal backbone: the state, the question, then each option followed by its marker.
JOINT_SYSTEMThe joint layout’s system prompt.
JOINT_PREFILLThe joint layout’s opening of the assistant’s turn, after the turn’s own markup.
FramingHow a joint row frames its text as a conversation: a system message, the user’s turn that holds the state and the schema, and the assistant’s turn opened with prefill.
JOINT_NOULA noul option’s description where its question gives none.
TextText a joint row writes as given, at its place in the user’s turn.
MediaWhere a joint row places a request’s images, each as the backbone’s processor writes one (Processor.image_text) and expands it; a request without images places nothing here.
RenderHow a segment writes a value as text (rendered).
StateThe state, as compact JSON with sorted keys, its start kept: cut to its own budget (max_tokens, or a joint layout’s max_state_tokens where it gives none) and to its share of what the rest of the row leaves of its max_len (dew.data.tokens.share_tokens).
allocatedHow many of its own tokens each copy of the state keeps of room (dew.data.tokens.share_tokens): a copy kept whole (keep="all") all of its lengths, whatever its cap, and the others what is left, each at most its own cap (max_tokens, or default_cap where it gives none).
RowOrderThe order a question layout’s own row writes its parts in (QuestionLayout.order).
built_inThe built-in question row as segments (QuestionLayout.written): the begin token, the question’s header and instructions as one piece ("{kind} question: {instructions}"), the option lines and the state, each part after a separator, in order: the state first, the question first, or the state, the question and the state again; Laya’s markers before each option where the question comes first, a causal row’s after, or lettered lines read at the labels’ cue, which ends the row in place of its last separator.
HowHow a stream reads another (Reads): all of its tokens, or those up to the reader’s own.
HOWSEach How as the model reads it (DecisionInputs.reads); 0 is reading none.
ReadsWhich of a row’s streams each stream’s tokens attend to: its attention record (row 2319).
streamed_attentionWhat a row of segments reads (Layout.attention): its reads over its streams (Reads.lowered) and the most runs of one stream it lays (fragment_count); None for a row read in order.
segment_streamThe stream a segment of a row’s segments stands in (Reads): its own where it names one, a question’s option lines its instructions’ (Instructions.stream), else the row’s.
stream_namesA row’s streams (Reads): the row’s, then each other one its segments name in their order, the context its cross-attention reads not among them.
fragment_countThe most runs of one stream a row of segments lays (the layout’s framing in the row’s stream): the spans the model lowers its reads over (DecisionInputs.fragments).
stream_pointAppend to chunks the boundary where a row’s stream turns to stream from current, a chunk of no ids at which its text is encoded apart (dew.decision.renderer.render), and return the stream now written; none where it stays.
streamed_chunkschunks, each in its stream of streams, with the boundaries where the row turns to another (stream_point), the row’s own again after them.
token_streamsEach of a row’s length tokens’ stream, as its index in its segments’ (stream_names), from the boundaries its rendering kept (stream_point): the row’s own before the first.
SchemaEvery question of the request as a field of the schema, never cut: its id, type and instructions, then each allowed option, each rendered as compact JSON with sorted keys, {"description": ..., "option_id": ...}.
SegmentA part of a joint row’s user turn (JointLayout.segments).
SpecialA special token the row writes as its id: the tokenizer’s begin token (none where it has none) or the layout’s separator (Specials).
InstructionsA question’s instructions under a header of its kind (headers, {} the rendered instructions), rendered by render (rendered); defaults are a kind’s instructions where its question’s render empty, as instructions of whitespace alone do under a release’s entry.
OptionLinesA question’s options, one line each, joined by join: line filled with the option’s label (QuestionLayout.labels), its name and its description rendered by render, bare where that renders empty, and {option} its whole text as the built-in rows write it (QuestionLayout.options: key: description, level i: description, false: ...); each option’s span is its whole line.
AnswerCueThe cue a readout answers after: template filled with the options’ labels joined by join, many past at options.
QuestionSegmentA part of a question row’s text (QuestionLayout.segments).
QUESTION_KINDSThe parts a question row renders (QuestionSegment), which its layout refuses any other of.
JOINT_KINDSThe parts a joint row renders (Segment), which its layout refuses any other of: its questions, their instructions and options, are its schema’s, and it reads no cue.
RowSegmentA part of either layout’s row (Segment, QuestionSegment), as a run names its row’s parts (DecisionRunConfig.segments); each layout refuses the parts it has none of.
renderedvalue as a segment writes it: question, a question layout’s text (render) without the marker’s text; compact, a joint row’s sorted compact JSON; entry, a release’s entry (a string stripped, any other value indented JSON, None nothing); state, a release’s state (a string as it is, any other value indented JSON, None nothing).
JOINT_SEGMENTSThe joint row’s user turn as the Laya joint release lays it out: the state’s heading, the images, the state, the schema, and a line break before the turn’s end.
JointLayoutEvery question of a request in one row, each option a span of its own.
default_layoutThe layout a head reads where none is given: every question of a request in one row (JointLayout) for a head that reads several a row (Head.one_question false), else the one the backbone’s order asks for, the state first for a causal backbone (StateFirstLayout) and Laya’s markers for a bidirectional one (MarkerLayout), each choice option written as its description, the option’s text (QuestionLayout.option_text).
question_ownersThe [B, Q] row each question of packed windows belongs to, 0 for a padding slot: the row the packer placed it in (rows, PackedRows’ question_rows), or where none is given the row its span starts in, and where the spans are the context’s (DecisionInputs.spans_in_context) the window’s one row (DecisionInputs.collate).
DecisionInputsA batch of laid-out rows: packed windows of whole rows, or rows padded to the longest.
labelled_rowsWhether rows’ options stand under labels (Labels), read at an answer cue; one batch reads its rows’ options one way.
OPTION_TILEWhat a batch’s option slots round up to when its caller fixes none (DecisionInputs.collate).
compactedA row’s real options, options [Q, K], as slots flat indices into its grid in row order, padded with 0, and which of them are real.

class source

class TokenizerSpecials(Protocol)

What Specials.of reads off a tokenizer, as a transformers tokenizer holds it: its special tokens’ ids, each None where it has none, and an id’s token text.

def convert_ids_to_tokens(ids: int, /) -> str | list[str]

dataclass source

class Specials(
begin: int | None,
separator: int,
marker: int,
marker_text: str,
pad: int,
)

The special tokens a layout places.

They are the row’s start token (None for a vocabulary without one), the separator, the option marker (and its text, which is removed from anything a caller writes), and the padding id.

def of(tokenizer: Tokenizer | TokenizerSpecials) -> Specials

Return a tokenizer’s own special tokens, read off any tokenizer that holds them as TokenizerSpecials declares, a transformers tokenizer or the one an HFTokenizer loads among them.

That is [CLS] or its start token, [SEP] or its end token, [MASK] (or else its end token) as the marker, and its padding token or else its end token.

dataclass source

class Laid(
name: str,
kind: int,
span: tuple[int, int],
options: tuple[tuple[int, int], ...],
order: tuple[int, ...],
readout: int | None = None,
label_ids: tuple[tuple[int, ...], ...] | None = None,
)

One question as a row lays it out.

span holds the question’s own tokens, its instructions, and options each option slot’s tokens, as (start, end) positions in the row. Slot s shows the question’s option order[s].

readout: int | None

Where a readout of labels reads the question’s answer (Labels): the position of the row’s answer cue’s last token, or None where the options are read at their own tokens.

label_ids: tuple[tuple[int, ...], ...] | None

Each option slot’s label as the token ids a readout reads at readout, one a variant (Labels.variants), or None where no label stands for the options.

kind: int

The question’s type, numbered as KINDS numbers it.

dataclass source

class Encoded(
tokens: tuple[int, ...],
questions: tuple[Laid, ...],
positions: tuple[int, ...] | None,
slots: tuple[int, ...] | None,
state_tokens: int,
state_kept: int,
media: tuple[int, int] | None = None,
instructions_cut: int = 0,
options_cut: int = 0,
state_cut: int = 0,
state_characters_cut: int = 0,
options_alike: bool = False,
context: tuple[int, ...] | None = None,
spans_in_context: bool = False,
blocks: tuple[int, ...] | None = None,
streams: tuple[int, ...] | None = None,
)

One row: its tokens and the questions laid out in it.

In the parallel layout it also holds each token’s position and option slot (0 for the tokens every option shares).

media: tuple[int, int] | None

Where the row holds a request’s images, as a [start, end) span of its tokens.

instructions_cut: int

The question’s instructions’ tokens its layout’s declared budget cut (QuestionLayout.head_max_len); none where they are kept whole.

options_cut: int

The options’ tokens its layout’s declared budgets cut (QuestionLayout.head_max_len, option_tokens), over every option; none where each is kept whole.

state_cut: int

The state’s tokens the row’s cut removed, over every copy of it the row holds, the whole state where the row holds none of it.

state_characters_cut: int

The state’s characters a cut removed where the row’s text is encoded whole, over every copy: a cut text encodes anew, so its removed tokens are not a count to subtract.

options_alike: bool

Whether a cut left two of a question’s options as the same tokens where their whole texts differ, which no reading tells apart: such a row is refused (Layout.refusal).

context: tuple[int, ...] | None

The tokens of the context the row reads beside its state, through a backbone’s cross-attention, or None for none.

spans_in_context: bool

Whether the questions’ spans are positions in the context, which holds the schema (Schema.stream), rather than in the row.

blocks: tuple[int, ...] | None

Each token’s block (QuestionLayout.blocks): 0 on the row before its question, 1 from the question to the row’s end, which a backbone that reads blocks attends to whole within each (AttentionMetadata.block_groups); None for a row read in order.

streams: tuple[int, ...] | None

Each token’s stream, its index in the layout’s (stream_names), which its reads say what each token attends to (Reads, Layout.attention); None for a row read in order.

state_tokens: int

The state’s length in tokens, before the layout cut it to fit.

trimmed: bool

Whether its layout’s budgets cut the question’s instructions or options.

cuts: dict[str, int]

The tokens each cut removed from the row, by what it cut (instructions, options, state), or the characters of a state its row encodes whole (state_characters), and options_alike where its options’ cut left two alike; empty where the row holds its state and its questions whole. Training counts them in its data counters and a server in its own (dew.data.mixing.CorpusCounts), alike.

attribute source

OPTIONS: Literal['options'] = 'options'

A question field’s axis of one entry per option slot (QuestionField.shape).

dataclass source

class QuestionField(
name: str,
shape: tuple[int | Literal['options'], ...],
dtype: type,
fill: int = 0,
when: Literal['always', 'labelled', 'trained', 'grouped'] = 'always',
)

One array a row holds along its questions: its name, the shape each question slot holds (OPTIONS for an axis of one entry per option slot), its dtype, its value in a slot that holds no question, and when a row carries it: always, where its options stand under labels (labelled), in training (trained), or where its questions stand in consistency groups (grouped).

def empty(questions: int, width: int) -> np.ndarray

The field over questions slots of width options each, every slot at its fill.

attribute source

QUESTION_FIELDS

The fields along a row’s questions a model reads (DecisionInputs): kinds, questions (the real ones), spans, option_spans, options (the real ones), and where its options stand under labels (Labels) readout, each question’s answer cue. Every producer, packer and consumer of a row’s questions reads their names, shapes and fills here; a training row extends them (dew.decision.rows.TRAINING_FIELDS).

function source

def question_fields(
row: Encoded,
count: int,
width: int,
labelled: bool = False,
) -> dict[str, np.ndarray]

row’s questions as fixed-shape arrays (QUESTION_FIELDS), count question slots of width option slots each, readout where its options stand under labels (labelled). A training row (Encoding) and a served batch (DecisionInputs.collate) both read a row through this.

attribute source

MEDIA_MARK = ('media',)

The mark of a row’s images (Encoded.media), which no question’s mark, a (name, part) pair, is.

attribute source

INSTRUCTIONS_MARK = 'instructions'

The part of a question’s mark that its instructions take, beside its options’ slots.

function source

def render(value: JSON) -> str

Return a value as the model reads it: text as it is, and anything structured as JSON.

function source

def context_tokens(
tokenizer: Tokenizer,
context: JSON | None,
length: int | None,
) -> np.ndarray

A context’s tokens as a row carries them: its text’s (render), without specials, and none for no context. A context of more than length tokens, which the cross-attention reads no more of, is refused (ValueError), never cut: a cut would answer from part of what the caller gave; None reads it whole, as a count of a context no cross-attention reads does.

function source

def with_context(
rows: Sequence[Encoded],
tokenizer: Tokenizer,
context: JSON | None,
length: int | None,
) -> list[Encoded]

rows with the context a backbone’s cross-attention of length context tokens reads beside each (None: a backbone with none): a schema laid out as the rows’ context (Schema.stream), read whole, or else context, a request’s own, its first length tokens (context_tokens). The one resolution of a row’s context, which a run’s encoding and a task’s admission both read. Refused (ValueError): a request’s context beside a schema read as the context, a context with no cross-attention to read it, and a schema longer than length.

dataclass source

class Bounds(segments: int, questions: int, options: int | None)

The most rows (segments), questions and options one packed window holds; options is None where the layout trims a question’s options to share a budget rather than giving each tokens of its own (QuestionLayout).

dataclass source

class Layout(max_len: int = 512)

How a state and its questions become rows of tokens that fit max_len (fits).

joint: bool

Whether every question of a request shares one row, rather than one row each.

images: bool

Whether a row holds a request’s images, each as the backbone’s processor writes and expands one (Processor.image_text).

spans_in_context: bool

Whether its rows write their questions to their contexts, which their cross-attention reads, their spans the contexts’ (Encoded.spans_in_context).

attention: tuple[tuple[tuple[int, ...], ...], int] | None

What each of its rows’ tokens reads as the model lowers it (DecisionInputs.reads, fragments), where its rows stand in streams (Reads); None for rows read in order.

def state(encoder: Tokenizer, specials: Specials, state: JSON) -> tuple[int, ...]

Return the state’s tokens, which every question of a request shares.

def rows(
encoder: Tokenizer,
specials: Specials,
state: JSON,
questions: Mapping[str, Question],
*,
orders: Mapping[str, Sequence[int]] | None = None,
media: Sequence[int] = (),
tokens: Sequence[int] | None = None,
) -> list[Encoded]

Lay out questions about state, each question’s options in its orders entry.

Slot s of a question shows its option order[s]; a question orders does not name keeps its own order (or the layout’s). media are the tokens the backbone’s processor gave a request’s images, which a layout with an image text places in the row. tokens are the state’s (state), where the caller has them already.

def fits(row: Encoded) -> bool

Whether row is within max_len. A layout cuts the state to fit and keeps a schema whole, so a row whose schema alone passes max_len does not fit: training leaves its example out (Fit), counted as unfit, and a task refuses it.

def refusal(row: Encoded, context_length: int | None = None) -> str | None

Why training leaves row out, or None where it trains: unfit, past max_len (fits), or, where its questions are its context’s (Encoded.spans_in_context), past the context_length tokens its backbone’s cross-attention reads (with_context); or options_alike, where a declared cut left two options the same tokens, which no reading tells apart. Training counts it under the reason (dew.data.mixing.CorpusCounts); a server refuses an unfit request under the same reason, and answers one whose options its release’s budgets cut alike as the release does, the cut in its usage.

def shortest_row(encoder: Tokenizer, specials: Specials) -> tuple[int, int]

The tokens of the row bounds counts by, one one-option question about an empty state laid out with room to spare, and its context’s (its schema where the row reads it there): no window shorter, nor a context, holds its bounds.

def row_questions(questions: int) -> int

The most questions a row of this layout holds where an example asks questions: all of them, a row an example.

def bounds(
encoder: Tokenizer,
specials: Specials,
context_length: int | None = None,
segments: int | None = None,
context_window: int | None = None,
) -> Bounds

The most rows, at most segments, questions and options a window of max_len tokens packs, from the fewest tokens each takes in this layout: a row of one one-option question about an empty state, and what one more option or one more one-option question adds to it, laid out with room to spare so that nothing is cut. A window of m rows then holds at most m questions and m options in m rows’ tokens, and each further question or option takes its own. The packer holds every window to these bounds, so a row shorter still starts a new window rather than outgrowing one. Where the questions are laid out in the row’s context (Encoded.spans_in_context), each row reads a context of its own of at most context_length tokens (DecisionInputs.context_segments), the window’s contexts at most context_window (one row’s where none is given): a window holds as many rows as its tokens hold, and as many questions and options as its contexts do, a context’s schema of one question taking the place a row’s tokens take along them.

attribute source

LETTERS = tuple(chr(ord('A') + index) for index in range(26))

attribute source

LABEL_NAMES = (*LETTERS, *(first + second for first in LETTERS for second in LETTERS))

Every label, in order: A to Z, then AA, AB, …, ZZ, as a release’s slots name them.

function source

def label_names(count: int) -> tuple[str, ...]

The first count labels of LABEL_NAMES.

dataclass source

class Labels(
cue: str = '\nAnswer:',
variants: tuple[str, ...] = (' {}',),
names: tuple[str, ...] | None = None,
)

Options rendered as lettered lines, read at the row’s answer cue by a readout of the labels’ logits (dew.decision.head.LabelReadout, SlotReadout): slot s as {name}. {text} on a line of its own, its span the whole line, and after the options the cue, whose last token the readout reads. Each label is read as the token id of each of variants, a template of its name ({}): a release’s slot is one variant, its name after a space; a readout groups several as one. A variant that spells as other than one token is refused.

names are the labels in slot order, a release’s own list where it names its slots otherwise (one that leaves out the pairs its tokenizer spells as two tokens), or label_names’ where unset; the lines and the ids both read them (names_for), and a question with more options than they name is refused.

def names_for(count: int) -> tuple[str, ...]

The first count labels.

def capacity(encoder: Tokenizer) -> int

How many options these labels letter under encoder: their own names, or every name of LABEL_NAMES whose every variant it spells as one token (spelled).

def spelled(encoder: Tokenizer, count: int) -> Labels

These labels named for encoder: themselves where they name their own, else the first count of LABEL_NAMES whose every variant encoder spells as one token, skipping those it spells as two (Qwen3.5 spells ” BQ” as two, and 570 of the 702 as one), so a run letters as many options as its tokenizer can. Fewer than count such names is refused.

def ids(encoder: Tokenizer, count: int) -> tuple[tuple[int, ...], ...]

Each of the first count labels’ ids, one a variant.

dataclass source

class QuestionLayout(
max_len: int = 512,
head_max_len: int | None = None,
option_tokens: int | None = None,
parallel: bool = False,
labels: Labels | None = None,
segments: tuple[QuestionSegment, ...] | None = None,
unit: Unit = 'pieces',
framing: Framing | None = None,
blocks: bool = False,
split: Literal['part', 'question', 'whole'] | None = None,
order: RowOrder | None = None,
reads: Reads | None = None,
option_text: Literal['named', 'described'] = 'named',
)

A layout of one row per question, scored at one marker token per option.

Each option is rendered as text (option_text: key: description for a choice, or its description alone, level i: description for a score, false: ... and true: ... for a noul). The question, its instructions and options, is kept whole, and the state fills the rest of max_len; a list state (a conversation) keeps its end, and any other state keeps its start. A question that alone passes max_len is a row that does not fit, which training leaves out and a task refuses, each counting it (Layout.refusal); its text is never cut silently.

A layout may declare budgets for the question, as Laya’s release does: each option cut to option_tokens, and where the options exceed head_max_len minus 16 tokens every option cut evenly to fit and the instructions given what the options leave, at least 8 tokens. Each cut is counted (Encoded.instructions_cut, options_cut), and a row whose cut leaves two options alike is refused (Encoded.options_alike).

parallel starts every option at the same position and lets no option attend to another, so the options’ order does not affect their scores; whatever follows the options continues after the longest one.

head_max_len: int | None

The tokens the question, its instructions and its options, may take, as a release declares it (Laya’s 192), or None to keep the question whole.

option_tokens: int | None

The most tokens one option keeps, as a release declares it (Laya’s 48), or None to keep each whole.

labels: Labels | None

Options rendered as lettered lines and read at an answer cue (Labels), or None for a marker after each option, read at the marker.

segments: tuple[QuestionSegment, ...] | None

A row’s text as segments of its own (dew.decision.renderer), read at its labels’ readout at the row’s last token, or None for the layout’s own text.

unit: Unit

How segments’ text is encoded: piece by piece, or whole (dew.decision.renderer).

framing: Framing | None

The conversation segments’ text is the user’s turn of, or None for none.

blocks: bool

Whether each row splits into two blocks (Encoded.blocks), its parts as segments order them: a state-first row’s state, and its question from its header to the row’s end, a published slot release’s prefix and suffix (serve.patch block_ids); a question-first row’s question, and its state from the state’s first token to the row’s end (the Text between them keeps to the part it follows). A backbone that reads blocks attends to each whole, and the second to the first in order. Off, every row is read in order.

split: Literal['part', 'question', 'whole'] | None

Where a row in blocks splits: at its second part (part, a row of segments’ own law: a state-first row’s question from its header on, a question-first row’s state from its first token on), at its question’s first token (question, the built-in rows’), or nowhere, the whole row one block its backbone reads both ways (whole); None takes the layout’s own law. A row read in order (blocks off) splits nowhere, so it states none.

order: RowOrder | None

The order the layout’s own row writes its parts in (built_in), each part as the built-in row writes it, its begin token, separators, header, budgets and a conversation’s kept end alike: the state first, the question first, or the state, the question and the state again (re_read, each copy its share of the room); None takes the layout’s own (StateFirstLayout’s state first, MarkerLayout’s question first). A row of its own segments states its own order.

reads: Reads | None

Which of the row’s streams each reads (Reads), where its question’s segments stand in a stream of the row’s own (Instructions.stream); None for a row read in order.

option_text: Literal['named', 'described']

How a choice’s option is written (options): named, its key and its description (key: description), as Laya’s release reads its rows; or described, its description alone, the option’s text, which a fresh head reads (default_layout), so a key that only identifies the option (option_0) writes nothing the row reads. An option with no description is its key.

written: tuple[QuestionSegment, ...]

The row’s parts in order: its own segments, or the layout’s built-in row (built_in).

spans_in_context: bool

Whether its row writes its question to its context (Instructions.stream).

attention: tuple[tuple[tuple[int, ...], ...], int] | None

Its rows’ reads over their streams (streamed_attention).

def state(encoder: Tokenizer, specials: Specials, state: JSON) -> tuple[int, ...]

Return the state’s tokens, which every question of a request shares.

def row_questions(questions: int) -> int

One: a question layout writes a row a question.

def rows(
encoder: Tokenizer,
specials: Specials,
state: JSON,
questions: Mapping[str, Question],
*,
orders: Mapping[str, Sequence[int]] | None = None,
media: Sequence[int] = (),
tokens: Sequence[int] | None = None,
) -> list[Encoded]
def encode(
encoder: Tokenizer,
specials: Specials,
question: Question,
state: Sequence[int],
*,
conversation: bool = False,
order: Sequence[int] | None = None,
name: str = '',
) -> Encoded

Lay out question over the state’s tokens state (state, a row of one copy of the state rendered as the layout renders it), with its options in order (rows).

Slot s shows option order[s]; None keeps the question’s order. A conversation keeps the end of the state instead of its start.

def options(question: Question) -> tuple[str, ...]

Return each option’s text, in the question’s order (option_text).

dataclass source

class MarkerLayout(
max_len: int = 512,
head_max_len: int | None = None,
option_tokens: int | None = None,
parallel: bool = False,
labels: Labels | None = None,
segments: tuple[QuestionSegment, ...] | None = None,
unit: Unit = 'pieces',
framing: Framing | None = None,
blocks: bool = False,
split: Literal['part', 'question', 'whole'] | None = None,
order: RowOrder | None = None,
reads: Reads | None = None,
option_text: Literal['named', 'described'] = 'named',
)

Laya’s layout, for a bidirectional backbone: the question, a marker before each option, then the state.

parallel is Laya’s option_layout="parallel"; the published checkpoints are sequential.

dataclass source

class StateFirstLayout(
max_len: int = 512,
head_max_len: int | None = None,
option_tokens: int | None = None,
parallel: bool = False,
labels: Labels | None = None,
segments: tuple[QuestionSegment, ...] | None = None,
unit: Unit = 'pieces',
framing: Framing | None = None,
blocks: bool = False,
split: Literal['part', 'question', 'whole'] | None = None,
order: RowOrder | None = None,
reads: Reads | None = None,
option_text: Literal['named', 'described'] = 'named',
)

The layout for a causal backbone: the state, the question, then each option followed by its marker.

A causal backbone reads each token only after the tokens before it:

[start] state [SEP] <type> question: <instructions> [SEP] opt0 [MARK] opt1 [MARK] ... [SEP]

so the hidden state at each marker has read the state, the question and that option. In the parallel layout an option reads no other option.

attribute source

JOINT_SYSTEM = "Read the complete state and schema. Decide every field jointly. Each answer must be exactly one of that field's allowed options."

The joint layout’s system prompt.

attribute source

JOINT_PREFILL = 'JOINT SCHEMA DECISIONS:'

The joint layout’s opening of the assistant’s turn, after the turn’s own markup.

dataclass source

class Framing(
system: str = JOINT_SYSTEM,
prefill: str = JOINT_PREFILL,
thinking: bool | None = False,
template: bool = True,
chat_template: str | None = None,
options: Mapping[str, JSON] = dict(),
)

How a joint row frames its text as a conversation: a system message, the user’s turn that holds the state and the schema, and the assistant’s turn opened with prefill.

The turns’ markup is the backbone’s own: where the tokenizer has a chat template and template holds, its template renders the system message, the user’s turn around its content and the generation prompt (dew.data.chat.framed), thinking its reasoning switch (False: a reasoning model’s empty thought, None: the template’s own default), so every token of the markup is the tokenizer’s own special token, never ChatML spelled out for a vocabulary without it. chat_template, a Jinja chat template, renders the turns in place of the tokenizer’s own (a base model’s, which has none, or a framing of an experiment’s), and options are the other variables a template reads beside the messages; both are recorded with the layout, so a run reloads its framing. A tokenizer with no chat template and none given, or template False, frames the turns as plain text: the system message, then the user’s content, each after a blank line.

def around(encoder: Tokenizer) -> tuple[str, str]

The text before the user’s turn’s content and after it, to the assistant’s prefill.

attribute source

JOINT_NOUL = {'true': 'The proposition is true or the answer is yes.', 'false': 'The proposition is false or the answer is no.'}

A noul option’s description where its question gives none.

dataclass source

class Text(text: str)

Text a joint row writes as given, at its place in the user’s turn. Text beside the turn’s markup, or beside other text, is read with it as one piece, as the turn writes it.

dataclass source

class Media()

Where a joint row places a request’s images, each as the backbone’s processor writes one (Processor.image_text) and expands it; a request without images places nothing here.

attribute source

How a segment writes a value as text (rendered).

dataclass source

class State(
render: Render = 'compact',
keep: Literal['start', 'end', 'all', 'end_if_conversation'] = 'start',
max_tokens: int | None = None,
)

The state, as compact JSON with sorted keys, its start kept: cut to its own budget (max_tokens, or a joint layout’s max_state_tokens where it gives none) and to its share of what the rest of the row leaves of its max_len (dew.data.tokens.share_tokens). A row may hold the state more than once, each copy its own segment, render, budget and keep. A question row’s segments say how it renders (render, rendered) and what a cut keeps of it (keep: its start, its end, or all of it, a row that it leaves too long then unfit); a joint row renders it compact.

keep: Literal['start', 'end', 'all', 'end_if_conversation']

What a cut keeps of the state: its start, its end, all of it, or its end where it is a conversation (a list state, its last turns) and its start otherwise, as the built-in rows keep it.

function source

def allocated(
lengths: Sequence[int],
copies: Sequence[State],
room: int,
default_cap: int | None = None,
) -> list[int]

How many of its own tokens each copy of the state keeps of room (dew.data.tokens.share_tokens): a copy kept whole (keep="all") all of its lengths, whatever its cap, and the others what is left, each at most its own cap (max_tokens, or default_cap where it gives none).

attribute source

The order a question layout’s own row writes its parts in (QuestionLayout.order).

function source

def built_in(labels: Labels | None, order: RowOrder) -> tuple[QuestionSegment, ...]

The built-in question row as segments (QuestionLayout.written): the begin token, the question’s header and instructions as one piece ("{kind} question: {instructions}"), the option lines and the state, each part after a separator, in order: the state first, the question first, or the state, the question and the state again; Laya’s markers before each option where the question comes first, a causal row’s after, or lettered lines read at the labels’ cue, which ends the row in place of its last separator. A conversation keeps its end in every copy.

attribute source

How a stream reads another (Reads): all of its tokens, or those up to the reader’s own.

attribute source

HOWS: dict[str, int] = {'whole': 1, 'causal': 2}

Each How as the model reads it (DecisionInputs.reads); 0 is reading none.

dataclass source

class Reads(table: tuple[tuple[str, str, How], ...])

Which of a row’s streams each stream’s tokens attend to: its attention record (row 2319).

A row’s tokens stand in streams: "row", and each other stream a segment names (Schema.stream, Instructions.stream, its option lines with it), every other segment in the row’s own. table holds (reader, read, how): a token of reader reads the tokens of read ("self", its own stream), all of them ("whole") or those up to itself ("causal"); a pair the table omits reads none. Two streams in one pass where the state reads the schema and the schema itself alone: Reads((("row", "self", "causal"), ("row", "schema", "whole"), ("schema", "self", "causal"))), the schema in Schema(stream="schema"), wherever the row lays it. Each stream reads itself, so no token reads nothing; positions run along the row as it lays its tokens.

def lowered(streams: Sequence[str]) -> tuple[tuple[int, ...], ...]

The record as the model reads it, over a row’s streams in their order: each stream’s reading of itself, then of each stream (HOWS, 0 for none).

function source

def streamed_attention(
reads: Reads | None,
segments: Sequence[Segment | QuestionSegment],
) -> tuple[tuple[tuple[int, ...], ...], int] | None

What a row of segments reads (Layout.attention): its reads over its streams (Reads.lowered) and the most runs of one stream it lays (fragment_count); None for a row read in order.

function source

def segment_stream(
segment: Segment | QuestionSegment,
segments: Sequence[Segment | QuestionSegment],
) -> str

The stream a segment of a row’s segments stands in (Reads): its own where it names one, a question’s option lines its instructions’ (Instructions.stream), else the row’s.

function source

def stream_names(segments: Sequence[Segment | QuestionSegment]) -> tuple[str, ...]

A row’s streams (Reads): the row’s, then each other one its segments name in their order, the context its cross-attention reads not among them.

function source

def fragment_count(segments: Sequence[Segment | QuestionSegment]) -> int

The most runs of one stream a row of segments lays (the layout’s framing in the row’s stream): the spans the model lowers its reads over (DecisionInputs.fragments).

function source

def stream_point(
chunks: list[Chunk] | list[Chunk | State],
stream: str,
current: str,
) -> str

Append to chunks the boundary where a row’s stream turns to stream from current, a chunk of no ids at which its text is encoded apart (dew.decision.renderer.render), and return the stream now written; none where it stays.

function source

def streamed_chunks(chunks: Sequence[Chunk], streams: Sequence[str]) -> list[Chunk]

chunks, each in its stream of streams, with the boundaries where the row turns to another (stream_point), the row’s own again after them.

function source

def token_streams(
points: Mapping[Hashable, int],
length: int,
segments: Sequence[Segment | QuestionSegment],
) -> tuple[int, ...]

Each of a row’s length tokens’ stream, as its index in its segments’ (stream_names), from the boundaries its rendering kept (stream_point): the row’s own before the first. It lays at most the runs of one stream its segments do (fragment_count), which the model numbers them by.

dataclass source

class Schema(stream: str = 'row')

Every question of the request as a field of the schema, never cut: its id, type and instructions, then each allowed option, each rendered as compact JSON with sorted keys, {"description": ..., "option_id": ...}. The head pools each option’s tokens and each question’s instructions’.

stream is where the schema is read: in the row ("row"), or as the row’s context ("context"), which the backbone reads in a pass of its own, with its own weights, and its cross-attention layers read beside the row (DecisionModel.states), the head pooling the questions and options from that pass; or as a stream of the row of its own name, which the layout’s Reads says what it reads and what reads it.

attribute source

A part of a joint row’s user turn (JointLayout.segments).

dataclass source

class Special(token: Literal['begin', 'separator'])

A special token the row writes as its id: the tokenizer’s begin token (none where it has none) or the layout’s separator (Specials).

dataclass source

class Instructions(
headers: Mapping[str, str],
render: Render = 'question',
defaults: Mapping[str, str] = dict(),
span: Literal['instructions', 'header'] = 'instructions',
stream: str = 'row',
)

A question’s instructions under a header of its kind (headers, {} the rendered instructions), rendered by render (rendered); defaults are a kind’s instructions where its question’s render empty, as instructions of whitespace alone do under a release’s entry. The question’s span is its instructions’ (span="instructions"), or the header and its instructions written as one piece (span="header", the built-in rows’ "{kind} question: {instructions}"). stream is where they are read: in the row ("row"), or as the row’s context ("context"), which its cross-attention reads, the question’s spans the context’s (Encoded.spans_in_context), its options’ lines streaming with it; or as a stream of the row of its own name (Reads).

dataclass source

class OptionLines(
line: str = '{label}. {name}: {text}',
bare: str = '{label}. {name}',
join: str = '\n',
render: Render = 'entry',
level: str = 'level {}',
noul: tuple[str, str] = ('yes', 'no'),
criteria: Mapping[str, str] = dict(),
marker: Literal['none', 'before', 'after'] = 'none',
noul_first: Literal['true', 'false'] = 'true',
)

A question’s options, one line each, joined by join: line filled with the option’s label (QuestionLayout.labels), its name and its description rendered by render, bare where that renders empty, and {option} its whole text as the built-in rows write it (QuestionLayout.options: key: description, level i: description, false: ...); each option’s span is its whole line. A choice’s names are its keys, a score’s levels level, and a noul’s noul, its true option first (noul_first), criteria its options’ descriptions where its question gives none.

marker writes the layout’s marker token before or after each line, where the option is read at its marker (Laya’s markers before, a causal row’s after) and its span is the marker’s; none reads the options under labels at the answer cue. They are read where their question’s instructions are (Instructions.stream).

dataclass source

class AnswerCue(
template: str = 'Answer with one letter: {labels}.',
many: str = 'Answer with one label: {labels}.',
at: int = 26,
join: str = ', ',
)

The cue a readout answers after: template filled with the options’ labels joined by join, many past at options.

attribute source

A part of a question row’s text (QuestionLayout.segments).

attribute source

QUESTION_KINDS: tuple[type, ...] = get_args(QuestionSegment.__value__)

The parts a question row renders (QuestionSegment), which its layout refuses any other of.

attribute source

JOINT_KINDS: tuple[type, ...] = get_args(Segment.__value__)

The parts a joint row renders (Segment), which its layout refuses any other of: its questions, their instructions and options, are its schema’s, and it reads no cue.

attribute source

A part of either layout’s row (Segment, QuestionSegment), as a run names its row’s parts (DecisionRunConfig.segments); each layout refuses the parts it has none of.

function source

def rendered(value: JSON, policy: Render, specials: Specials) -> str

value as a segment writes it: question, a question layout’s text (render) without the marker’s text; compact, a joint row’s sorted compact JSON; entry, a release’s entry (a string stripped, any other value indented JSON, None nothing); state, a release’s state (a string as it is, any other value indented JSON, None nothing).

attribute source

JOINT_SEGMENTS: tuple[Segment, ...] = (Text('STATE:\n'), Media(), State(), Schema(), Text('\n'))

The joint row’s user turn as the Laya joint release lays it out: the state’s heading, the images, the state, the schema, and a line break before the turn’s end.

dataclass source

class JointLayout(
max_len: int = 16384,
max_state_tokens: int | None = None,
framing: Framing = Framing(),
segments: tuple[Segment, ...] = JOINT_SEGMENTS,
reads: Reads | None = None,
)

Every question of a request in one row, each option a span of its own.

The row is a chat prompt: the system prompt (JOINT_SYSTEM), then the user’s turn, its segments in order (JOINT_SEGMENTS: the state’s heading, a request’s images, the state, then a schema of numbered fields, each with its id, type, instructions and allowed options), and the opening of the assistant’s turn. A layout of segments of its own writes the turn otherwise: the schema first, or text of its own between the parts; it holds the schema once, the state and the images at most once each. A choice’s options come in the order of their keys, and a noul’s as true, then false, with JOINT_NOUL’s descriptions where the question gives none. The state is cut to max_state_tokens, and to what the rest of the row leaves of max_len; the schema itself is never cut, so a schema longer than max_len leaves no state and a row that does not fit (fits). The row’s conversation markup is the backbone’s own chat template’s (framing).

reads: Reads | None

Which of the row’s streams each reads (Reads), where its schema stands in a stream of the row’s own (Schema.stream); None for a row read in order.

spans_in_context: bool

Whether the row’s schema is its context (Schema.stream), its spans the context’s.

attention: tuple[tuple[tuple[int, ...], ...], int] | None

Its rows’ reads over their streams (streamed_attention).

def state(encoder: Tokenizer, specials: Specials, state: JSON) -> tuple[int, ...]
def rows(
encoder: Tokenizer,
specials: Specials,
state: JSON,
questions: Mapping[str, Question],
*,
orders: Mapping[str, Sequence[int]] | None = None,
media: Sequence[int] = (),
tokens: Sequence[int] | None = None,
) -> list[Encoded]

The request’s one row, rendered as chunks (dew.decision.renderer, piece by piece as the joint release encodes it): each run of text between the turn’s other parts one chunk, the images, each copy of the state and the schema chunks of ids, and each question’s and option’s tokens the marks its chunks carry.

def options(question: Question) -> tuple[tuple[str, JSON], ...]

Return each option’s key and description, in the question’s order.

def order(question: Question) -> list[int]

Return the question’s options in the joint order: a choice’s by key, a noul’s true first.

function source

def default_layout(causal: bool, one_question: bool) -> Layout

The layout a head reads where none is given: every question of a request in one row (JointLayout) for a head that reads several a row (Head.one_question false), else the one the backbone’s order asks for, the state first for a causal backbone (StateFirstLayout) and Laya’s markers for a bidirectional one (MarkerLayout), each choice option written as its description, the option’s text (QuestionLayout.option_text).

function source

def question_owners(
segments,
spans,
questions,
spans_in_context: bool,
xp=np,
rows=None,
)

The [B, Q] row each question of packed windows belongs to, 0 for a padding slot: the row the packer placed it in (rows, PackedRows’ question_rows), or where none is given the row its span starts in, and where the spans are the context’s (DecisionInputs.spans_in_context) the window’s one row (DecisionInputs.collate). xp is the array namespace, NumPy for host geometry and jax.numpy in a trace, so the model’s question grouping and the counted FLOPs read one owner.

dataclass source

class DecisionInputs(
positions: jnp.ndarray | None = None,
slots: jnp.ndarray | None = None,
option_slots: jnp.ndarray | None = None,
slotted: jnp.ndarray | None = None,
context: jnp.ndarray | None = None,
context_lengths: jnp.ndarray | None = None,
context_segments: jnp.ndarray | None = None,
question_rows: jnp.ndarray | None = None,
readout: jnp.ndarray | None = None,
blocks: jnp.ndarray | None = None,
streams: jnp.ndarray | None = None,
reads: tuple[tuple[int, ...], ...] | None = struct.field(pytree_node=False, default=None),
fragments: int = struct.field(pytree_node=False, default=1),
max_segments: int = struct.field(pytree_node=False, default=1),
spans_in_context: bool = struct.field(pytree_node=False, default=False),
whole_blocks: bool = struct.field(pytree_node=False, default=False),
)

A batch of laid-out rows: packed windows of whole rows, or rows padded to the longest.

  • tokens [B, L] ids, segments [B, L] each token’s row in its window, numbered from 1 in window order and 0 on the padding after the last (Pack); a padded batch is windows of one row each (collate);
  • positions [B, L] each token’s place in its row (None: counted from 0 along it), and in the parallel layout slots [B, L] each token’s option slot, 0 for the tokens every option shares;
  • kinds [B, Q] each question’s type (KINDS), questions [B, Q] the real questions, in their rows’ window order, spans [B, Q, 2] their instructions’ (start, end) in the window;
  • option_spans [B, Q, K, 2] each option slot’s (start, end), options [B, Q, K] the real slots;
  • option_slots [B, P] each window’s real options as flat indices q * K + k into its [Q, K] grid, in that order (compacted), and slotted [B, P] which of them are real, or None for the grid itself;
  • context [B, C] each window’s rows’ context tokens end to end, which a backbone’s cross-attention reads, context_lengths [B] how many are real, the padding after them, and context_segments [B, C] each one’s row in the window as segments numbers it (0 for padding), which its row’s tokens alone read; or None for rows that carry none;
  • question_rows [B, Q] each question’s row in its window as the packer placed it, numbered as segments numbers its tokens (0 for a padding slot), or None where a question’s row is the one its span starts in.

A row of one question with one marker per option, Laya’s, has spans one token long. A head that reads option_slots computes the real options alone, P rather than Q x K of them: a window holds a few of its grid’s. max_segments bounds the rows a window holds (Layout.bounds).

readout: jnp.ndarray | None

[B, Q] where a readout of labels reads each question’s answer (Laid.readout), or None; the labels’ ids are the readout head’s own (LabelReadout.label_ids).

blocks: jnp.ndarray | None

[B, L] each token’s block in its row (Encoded.blocks), -1 on the padding, or None for rows read in order.

streams: jnp.ndarray | None

[B, L] each token’s stream in its row (Encoded.streams), -1 on the padding, or None for rows read in order.

reads: tuple[tuple[int, ...], ...] | None

What each stream’s tokens read, as the layout lowers its Reads (Reads.lowered), where the rows stand in streams.

fragments: int

The most runs of one stream a row lays (fragment_count), which its tokens’ key runs number.

spans_in_context: bool

Whether the questions’ spans are positions in context, which holds the schema (Schema.stream), rather than in tokens.

whole_blocks: bool

Whether each row’s one block is the whole row (QuestionLayout.split ‘whole’), which a backbone reads as non-causal attention by rows (AttentionMetadata.whole_blocks).

valid: jnp.ndarray

Return the [B, L] real tokens.

question_segments: jnp.ndarray

Return the [B, Q] row each question belongs to, 0 for a padding slot (question_owners).

markers: jnp.ndarray

Return the [B, Q, K] first token of each option slot, where a marker layout scores it.

def coordinates() -> Coordinates

The tokens’ Coordinates: each row a document, real tokens first, at most max_segments a window. The padding after them runs as one more sequence, which a sequence kernel adds to the documents itself (dew.nn.attention.flash_attention).

def collate(
rows: Sequence[Encoded],
pad: int,
*,
questions: int | None = None,
options: int | None = None,
option_slots: int | None = None,
context_length: int | None = None,
attention: tuple[tuple[tuple[int, ...], ...], int] | None = None,
) -> DecisionInputs

Right-pad rows into one batch (pad_token_rows), one row a window, its question and option slots to the widest.

questions and options widen those slots further, to shapes a caller keeps fixed. option_slots is the real options a row may hold, a shape a caller keeps fixed too (compacted); without it, the batch’s most, rounded up to OPTION_TILE, where that is fewer than the grid’s slots. With a context_length, each row’s context (Encoded.context, none where it holds none) is padded to that many tokens. Rows in streams are read as their layout’s attention says (Layout.attention).

function source

def labelled_rows(rows: Sequence[Encoded]) -> bool

Whether rows’ options stand under labels (Labels), read at an answer cue; one batch reads its rows’ options one way.

attribute source

OPTION_TILE = 128

What a batch’s option slots round up to when its caller fixes none (DecisionInputs.collate).

function source

def compacted(options: np.ndarray, slots: int) -> tuple[np.ndarray, np.ndarray]

A row’s real options, options [Q, K], as slots flat indices into its grid in row order, padded with 0, and which of them are real.

Raises ValueError for a row of more real options than slots: its shape is the caller’s to keep, and widening it would compile the step again.