dew.decision
Decision models: one forward pass answers typed questions about a state.
A request is a state (text, or any JSON) and named questions, each a
Noul, a Choice or a Score (TypeSafe Jev’s three, docs.typesafe.ai).
The model lays the questions out with the state (Laya’s MarkerLayout, the
StateFirstLayout for a causal backbone, or the JointLayout, every
question of a request in one row), reads the rows with a backbone, scores
every option with a head (Laya’s DecisionHead, or the JointSchemaHead,
which decides a row’s questions together, or a readout of options written under
Labels: LabelReadout, the backbone’s own logits at them, and a slot release’s
SlotReadout), and answers with a distribution
over exactly the options each question names. Decide is the task that
answers requests, and DecisionObjective trains one on proper scoring
rules. Pretrained.load reads released Laya and joint-head sources, and
Decide.of turns their declared settings into a decision task.
| Name | Summary |
|---|---|
AURC | The area under the risk-coverage curve. |
ECE | Expected calibration error: the gap between the top probability and the accuracy. |
KINDS | The question types in the order a laid-out row numbers them, Laya’s (laya/common.py QTYPES: choice 0, score 1, noul 2). |
NONE_OF_THE_ABOVE | Documented in dew.decision.rows. |
TASK_FILE | The file Decide.save writes into a run: its calibration, budget and name. Documented in dew.decision.task. |
Abstention | The lowest calibrated top probability an answer needs, per bucket, or else default. |
Accuracy | The share of answered questions whose most likely option is the right one. |
Answer | |
AnswerCue | The cue a readout answers after: template filled with the options’ labels joined by join, many past at options. Documented in dew.decision.layout. |
Answered | The questions of a Decisions with a known answer, one row each: their probabilities [M, K], real options [M, K], right options [M], gold distributions [M, K] and whether [M] they ask a score. |
Binning | Histogram binning of the calibrated top probability, per bucket. |
Brier | The quadratic score, sum_k (p_k - t_k)^2, between 0 and 2. |
Budget | How much one forward pass may hold: at most tokens tokens in at most rows windows. Documented in dew.decision.task. |
Calibration | How a decision model’s raw logits become reported answers. |
Choice | A choice among named options, 1 to 255 of them. |
ChoiceAnswer | One probability per option, and the most likely option. |
Combined | A weighted sum of rules, itself proper when every weight is positive. |
Confidence | A statistic of an answer’s distribution, between 0 and 1. |
Decide | Answers typed questions about a state, with one forward pass per batch. Documented in dew.decision.task. |
DecisionHead | Scores each option of a question at its marker in the backbone’s states. |
DecisionInputs | A batch of laid-out rows: packed windows of whole rows, or rows padded to the longest. Documented in dew.decision.layout. |
DecisionModel | Any backbone with final states, read by a decision head (HeadedModel). |
DecisionObjective | Trains a decision model on questions with known answers. Documented in dew.decision.objective. |
Encoded | One row: its tokens and the questions laid out in it. Documented in dew.decision.layout. |
Encoding | How a labelled example becomes the row a step packs and trains on. Documented in dew.decision.rows. |
EntropyConfidence | Laya’s confidence: one minus the distribution’s entropy over log n (laya/common.py, confidence_from_probs), for every type but a Noul, which Laya gives its top probability. |
Example | One state, its questions, and the right answer to some of them. Documented in dew.decision.data. |
Framing | How a joint row frames its text as a conversation: a system message, the user’s turn that holds the state and the schema, and the assistant’s turn opened with prefill. Documented in dew.decision.layout. |
Group | Questions of one example that ask the same thing (negated, paraphrased, its options re-ordered), named name: members maps each one’s every option key onto a canonical key the members share, so a consistency term (DecisionObjective.consistency) can hold their answers to one distribution over the canonical keys. Documented in dew.decision.data. |
GroupedExamples | Examples whose rows train together, so a consistency term (DecisionObjective.consistency) holds questions asked across them to one answer: the same question negated, paraphrased or its options re-ordered in rows of their own, under any layout. Documented in dew.decision.data. |
Head | What every decision head offers: its call, whether it reads the output table, and the record a run keeps of it. |
Instructions | A question’s instructions under a header of its kind (headers, {} the rendered instructions), rendered by render (rendered); defaults are a kind’s instructions where its question’s render empty, as instructions of whitespace alone do under a release’s entry. Documented in dew.decision.layout. |
JevConfidence | TypeSafe Jev’s confidence (docs.typesafe.ai/confidence). |
JointLayout | Every question of a request in one row, each option a span of its own. Documented in dew.decision.layout. |
JointSchemaHead | The joint schema head: decides every question of a row together. |
LabelReadout | Dew’s readout of labels: each option slot’s logit is the backbone’s own logit for its label at the question’s answer cue (dew.decision.layout.Labels), through the output table’s input QDQ, scale, bias and softcap, computed in fp32 (DecisionModel.readout_logits, OutputTable.columns). |
Labels | Options rendered as lettered lines, read at the row’s answer cue by a readout of the labels’ logits (dew.decision.head.LabelReadout, SlotReadout): slot s as {name}. Documented in [dew.decision.layout`](/api/dew.decision.layout/). |
Laid | One question as a row lays it out. Documented in dew.decision.layout. |
LayaCheckpoint | One released Laya checkpoint: its model, variables, layout and tokenizer, and its agent’s temperatures. |
Layout | How a state and its questions become rows of tokens that fit max_len (fits). Documented in dew.decision.layout. |
LogLoss | The cross entropy, -sum_k t_k log p_k: the logarithmic score. |
MacroF1 | The mean over classes of each class’s F1, the classes those the split’s answered questions are or are answered as (sklearn’s f1_score(average="macro")): a class is a question’s option (Decisions.classes), the same in every batch and split that asks it, so each class’s true and false positives and false negatives add over the whole pass before its F1 is taken. |
MarkerLayout | Laya’s layout, for a bidirectional backbone: the question, a marker before each option, then the state. Documented in dew.decision.layout. |
Media | Where a joint row places a request’s images, each as the backbone’s processor writes one (Processor.image_text) and expands it; a request without images places nothing here. Documented in dew.decision.layout. |
Noul | A yes-or-no question. |
NoulAnswer | The answer to a Noul: P(false) and P(true), and a confidence computed from them. |
OptionLines | A question’s options, one line each, joined by join: line filled with the option’s label (QuestionLayout.labels), its name and its description rendered by render, bare where that renders empty, and {option} its whole text as the built-in rows write it (QuestionLayout.options: key: description, level i: description, false: ...); each option’s span is its whole line. Documented in dew.decision.layout. |
Question | One question about a state, with its options in the order the model scores them. |
QuestionLayout | A layout of one row per question, scored at one marker token per option. Documented in dew.decision.layout. |
QuestionSegment | A part of a question row’s text (QuestionLayout.segments). Documented in dew.decision.layout. |
RankedProbability | The ranked probability score of ordered levels. |
Schema | Every question of the request as a field of the schema, never cut: its id, type and instructions, then each allowed option, each rendered as compact JSON with sorted keys, {"description": ..., "option_id": ...}. Documented in dew.decision.layout. |
Score | A rating on ordered levels, 2 to 10 of them, lowest first. |
ScoreAnswer | One probability per level, lowest first, and the expected level. |
Scored | One held-out answer, with its question’s type and the model’s raw logits. |
ScoredQuestions | How many questions with a known answer the split scored, the count every other metric of it is taken over. |
ScoringRule | A proper scoring rule, lower is better. |
Segment | A part of a joint row’s user turn (JointLayout.segments). Documented in dew.decision.layout. |
SlotReadout | A published slot release’s head, its law as the release’s runtime computes it: slot i’s score is <slot_i, h> + <q(h), k(m_i)> / sqrt(head_dim), h the question’s answer state (DecisionInputs.readout), m_i the mean of the final states over option i’s whole rendered line (its span), q and k each a [head_dim, features] projection, with no slot bias and no softcap on the sum. |
Specials | The special tokens a layout places. Documented in dew.decision.layout. |
Spherical | The spherical score as a charge, 1 - (t . |
State | The state, as compact JSON with sorted keys, its start kept: cut to its own budget (max_tokens, or a joint layout’s max_state_tokens where it gives none) and to its share of what the rest of the row leaves of its max_len (dew.data.tokens.share_tokens). Documented in dew.decision.layout. |
StateFirstLayout | The layout for a causal backbone: the state, the question, then each option followed by its marker. Documented in dew.decision.layout. |
Temperatures | The temperature each question’s logits are divided by. |
Text | Text a joint row writes as given, at its place in the user’s turn. Documented in dew.decision.layout. |
TopProbability | The answer’s own probability, the largest one. |
Usage | What a request cost: the tokens every question row read, how much of the state the rows kept, the smallest share any question kept, and each cut by what it cut, the rows it shortened and the tokens it removed, as training counts a row’s cuts (cut_counts). Documented in dew.decision.task. |
Weights | The checkpoint a task’s weights came from: its step, and whether they are the run’s average. Documented in dew.decision.task. |
bucket | Return the calibration bucket of question: its type and its option count. |
corpus | Corpus name’s split of the mixture directory root_url as decision loading reads it, from the shards and index its manifest names (or the corpus’s one JSONL file), each row an Example under example; None when the corpus has no such split. Documented in dew.decision.data. |
decisions | dataset’s rows as decisions, each an Example under example, for a chain to batch: a flat table’s text and label columns read as one question over options (its ClassLabel names, or read whole in memory, where none are given), its context column, where named, each example’s context, or rows that hold their examples already. Documented in dew.decision.data. |
kind_of | Return the number a laid-out row gives question’s type (KINDS). |
mixture | The decision corpora of the mixture directory root_url (its mixture.json): every corpus of a positive weight, the manifest’s weights over weights, mixed by them for training, and their held rows together, or None where none has any. Documented in dew.decision.data. |
render | Return a value as the model reads it: text as it is, and anything structured as JSON. Documented in dew.decision.layout. |
class AURC()The area under the risk-coverage curve.
It is the error rate among the rows at or above each confidence, weighted by
how many rows that confidence adds, so a group of equal confidences counts as
a whole in any row order (Laya’s aurc, laya/evals.py). Lower is better; it
rewards a confidence that ranks right answers above wrong ones.
AURC.merge
Section titled “AURC.merge”def merge( accumulated: tuple[np.ndarray, np.ndarray], contribution: tuple[np.ndarray, np.ndarray],) -> tuple[np.ndarray, np.ndarray]AURC.finalize
Section titled “AURC.finalize”def finalize(accumulated: tuple[np.ndarray, np.ndarray]) -> floatclass ECE(bins: int = 15)Expected calibration error: the gap between the top probability and the accuracy.
It is taken over bins equal bins on (0, 1], closed on the right, and
weighted by how many rows each bin holds (Laya’s ece_score, 15 bins).
ECE.binned
Section titled “ECE.binned”def binned(tops: np.ndarray, correct: np.ndarray) -> np.ndarrayPer bin, the row count, the summed top probability tops and the
number correct, for answers however they were drawn: finalize
reads these, from a run’s validation pass or a served benchmark’s (reliability_bins).
ECE.merge
Section titled “ECE.merge”def merge(accumulated: np.ndarray, contribution: np.ndarray) -> np.ndarrayECE.finalize
Section titled “ECE.finalize”def finalize(accumulated: np.ndarray) -> floatKINDS: tuple[type[Question], ...] = (Choice, Score, Noul)The question types in the order a laid-out row numbers them, Laya’s (laya/common.py QTYPES: choice 0, score 1, noul 2). A head whose type embedding orders them otherwise maps these numbers to its own rows.
Abstention
Section titled “Abstention”class Abstention(thresholds: Mapping[str, float] = dict(), default: float | None = None)The lowest calibrated top probability an answer needs, per bucket, or else default.
An answer below it abstains. None gates nothing.
Abstention.threshold
Section titled “Abstention.threshold”def threshold(kind: str, count: int) -> float | NoneAbstention.fit
Section titled “Abstention.fit”def fit( scored: Sequence[Scored], temperatures: Temperatures, binning: Binning | None = None, *, target_error: float = 0.1, bucket_minimum: int = 100, conservative: bool = True,) -> AbstentionReturn the smallest threshold that meets target_error, per bucket.
Only buckets with at least bucket_minimum answers get one. A threshold
meets the target when the accepted answers err at most target_error,
counting a group of tied answers as a whole; with conservative, the error
is estimated as (errors + 1) / (accepted + 1). A bucket that no threshold
satisfies gets 1.0 (Laya’s fit_abstention_thresholds).
Accuracy
Section titled “Accuracy”class Accuracy()The share of answered questions whose most likely option is the right one.
Answer
Section titled “Answer”Answered
Section titled “Answered”class Answered( probabilities: np.ndarray, options: np.ndarray, labels: np.ndarray, targets: np.ndarray, ordinal: np.ndarray,)The questions of a Decisions with a known answer, one row each: their
probabilities [M, K], real options [M, K], right options [M], gold
distributions [M, K] and whether [M] they ask a score.
Answered.of
Section titled “Answered.of”def of(artifact: Artifact) -> AnsweredBinning
Section titled “Binning”class Binning(maps: Mapping[str, tuple[float, ...]] = dict())Histogram binning of the calibrated top probability, per bucket.
bins equal bins on [0, 1] each map to the held-out accuracy of the answers
that fell in them (Laya’s fit_binning_map). A bucket without a map keeps
the probability.
Binning.apply
Section titled “Binning.apply”def apply(kind: str, count: int, top: float) -> floatBinning.fit
Section titled “Binning.fit”def fit( scored: Sequence[Scored], temperatures: Temperatures, *, bins: int = 15, bucket_minimum: int = 200,) -> BinningReturn a map for every bucket with at least bucket_minimum answers.
An empty bin keeps its midpoint.
class Brier()The quadratic score, sum_k (p_k - t_k)^2, between 0 and 2.
Brier.charge
Section titled “Brier.charge”def charge(logits, target, options, ordinal)Calibration
Section titled “Calibration”class Calibration( temperatures: Temperatures = Temperatures(), binning: Binning | None = None, abstention: Abstention | None = None,)How a decision model’s raw logits become reported answers.
Calibration.probabilities
Section titled “Calibration.probabilities”def probabilities(question: Question, logits: np.ndarray) -> np.ndarrayCalibration.abstains
Section titled “Calibration.abstains”def abstains(question: Question, probabilities: np.ndarray) -> boolReturn whether the answer’s calibrated top probability is below its threshold.
Choice
Section titled “Choice”class Choice(instructions: JSON, criteria: Mapping[str, JSON] | Sequence[str])A choice among named options, 1 to 255 of them.
criteria maps each option’s key to what it means, or to None where the
key says enough; a list of keys describes none of them.
Choice.answer
Section titled “Choice.answer”def answer(probabilities: np.ndarray, confidence: Confidence) -> ChoiceAnswerChoice.wire
Section titled “Choice.wire”def wire() -> dict[str, JSON]ChoiceAnswer
Section titled “ChoiceAnswer”class ChoiceAnswer( options: tuple[str, ...], probabilities: np.ndarray, confidence: float, abstained: bool = False,)One probability per option, and the most likely option.
choice: str-
The most likely option; the first of several equally likely ones.
Combined
Section titled “Combined”class Combined(terms: tuple[tuple[float, ScoringRule], ...])A weighted sum of rules, itself proper when every weight is positive.
Combined.charge
Section titled “Combined.charge”def charge(logits, target, options, ordinal)Confidence
Section titled “Confidence”class Confidence(ABC)A statistic of an answer’s distribution, between 0 and 1.
DecisionHead
Section titled “DecisionHead”class DecisionHead(Head)Scores each option of a question at its marker in the backbone’s states.
This is Laya’s head (laya/common.py DecisionModel at NandhaKishorM/laya
a4a8921). The question type’s embedding is added to every state, then
layers pre-norm transformer blocks run over the row, as
nn.TransformerEncoderLayer(norm_first=True) builds them: LayerNorm with
bias at 1e-5, biased attention with features // 64 heads and no position
encoding, a biased ReLU feed-forward four times as wide, and padding keys
masked. Then a LayerNorm, Dense, erf GELU, Dense scorer reads the state at
each marker. The blocks are Dew’s DecoderBlock. Laya’s action head is left
out, because its own model card reports that it carries no signal.
It reads one question per row (QuestionLayout), the first token of each
option slot being its marker, and a window’s rows apart from each other.
DecisionHead.decision_flops
Section titled “DecisionHead.decision_flops”def decision_flops(lengths, questions, options) -> floatFull-row encoder layers and the scorer at real option markers; it pools no spans.
DecisionHead.setup
Section titled “DecisionHead.setup”def setup()DecisionModel
Section titled “DecisionModel”class DecisionModel( backbone: nn.Module, head: Head, context_encoder: ContextEncoder = 'table',)Any backbone with final states, read by a decision head (HeadedModel).
The rows a window packs reach the backbone as its documents (Coordinates).
In the parallel layout an option attends to no other option’s tokens, and the
backbone’s attention windows measure distance in positions
(attention_pairwise_mask, attention_key_positions).
context_encoder: ContextEncoder-
How a row’s context becomes the states its cross-attention reads (
encode_context):table, its tokens through the backbone’s own embedding table, orbackbone, the backbone’s final states over it in a pass of its own, as a schema read in context always is.
DecisionModel.reads_progress
Section titled “DecisionModel.reads_progress”def reads_progress() -> boolWhether its backbone’s computation changes over a run (dew.nn.progress.ReadsProgress).
DecisionModel.initialize_head
Section titled “DecisionModel.initialize_head”def initialize_head(key: jax.Array, backbone: Variables, tokens: jax.Array) -> VariablesDraw the head over a held backbone, at tokens’ shape, without running the backbone.
Readouts draw from its output table or vocabulary rows as their own contract declares; a tied readout has no local parameters. The token values are only initialization geometry.
DecisionModel.states
Section titled “DecisionModel.states”def states( variables: Variables, inputs: DecisionInputs, *, media: ModelInputs | None = None, train: bool = False, rngs: dict[str, jax.Array] | None = None, progress: Progress = END,) -> jax.ArrayReturn the backbone’s [B, L, features] final states, each packed
row read apart from the others in its window.
media are the backbone’s own inputs for images a request’s one row
holds (Images.inputs), with which it reads them. Rows that carry a
context (DecisionInputs.context) hand the backbone’s cross-attention
each row’s context, embedded by the backbone’s own table (stream_embeddings),
or, where the context holds the schema (DecisionInputs.spans_in_context),
the backbone’s own final states over it (encode_context). The backbone reads the run’s
progress where it reads it (dew.nn.progress), the run’s end by default.
DecisionModel.logits
Section titled “DecisionModel.logits”def logits( variables: Variables, inputs: DecisionInputs, *, media: ModelInputs | None = None, train: bool = False, rngs: dict[str, jax.Array] | None = None, progress: Progress = END,) -> jax.ArrayReturn the [B, Q, K] fp32 option logits, with -inf past each question’s options, the
backbone at the run’s progress (states).
DecisionModel.observed_logits
Section titled “DecisionModel.observed_logits”def observed_logits( variables: Variables, inputs: DecisionInputs, *, media: ModelInputs | None = None, train: bool = False, rngs: dict[str, jax.Array] | None = None, progress: Progress = END, opened: Sequence[str] = (),) -> tuple[jax.Array, Variables]The option logits (logits), and what the row’s pass sowed into the opened collections
(dew.objectives.base.Step.sown).
DecisionModel.readout_logits
Section titled “DecisionModel.readout_logits”def readout_logits( variables: Variables, states: jax.Array, inputs: DecisionInputs, labels: tuple[tuple[int, ...], ...],) -> jax.ArrayThe backbone’s own logits [B, Q, K, V] at each label’s ids of each question’s answer
state (DecisionInputs.readout), its first K labels as many as its option slots
(HeadedModel.label_logits).
EntropyConfidence
Section titled “EntropyConfidence”class EntropyConfidence(Confidence)Laya’s confidence: one minus the distribution’s entropy over log n
(laya/common.py, confidence_from_probs), for every type but a Noul,
which Laya gives its top probability.
class Head( reads_table: bool = False, reads_labels: bool = False, draws_from_rows: bool = False, pools_spans: bool = False, one_question: bool = False, width_field: str = 'features',)What every decision head offers: its call, whether it reads the output table, and the record a run keeps of it.
A run records a head’s fields, its numerics among them, but not its width, which is the backbone’s.
reads_labels: bool-
Whether the head reads the answer state of each question at its row’s answer cue (
DecisionInputs.readout) against labels (Labels), the backbone’s output table itstable(OutputTable, None where no matrix alone is the backbone’s head). draws_from_rows: bool-
Whether the head draws its parameters from the backbone’s vocabulary rows (
VocabularyRows), itstablewhen it is drawn, and reads no table after. pools_spans: bool-
Whether the head pools states over the questions’ spans (
DecisionInputs.spans,option_spans), from the pass that holds them: the context’s states (spanned) where the spans are the row’s context’s (DecisionInputs.spans_in_context), its pooling then counted over the context’s tokens (decision_flops(pooled=)). one_question: bool-
Whether the head reads one question a row, refusing a joint row’s several questions at the host boundary (
validate_rows). width_field: str-
The field holding the width of the states the head reads.
Head.tied_labels
Section titled “Head.tied_labels”def tied_labels() -> tuple[tuple[int, ...], ...] | NoneThe label ids, [K][V], at which the head reads the backbone’s own logits (tied), or
None where it reads none.
Head.labelled
Section titled “Head.labelled”def labelled(labels: Labels, tokenizer: Tokenizer) -> tuple[Head, Labels]The head reading labels as tokenizer spells them, and the labels as named, where it reads
labels and was drawn before its tokenizer was known, with no ids (a run’s letter readout,
LabelReadout); itself and labels otherwise.
Head.validate_rows
Section titled “Head.validate_rows”def validate_rows(rows: Sequence[Encoded]) -> NoneRefuse encoded host rows that ask more questions than this head reads.
Head.record
Section titled “Head.record”def record() -> RecordReturn the head as a run records it, the registry’s class record (dew.registry.component):
its class by import path and its fields but the width, which the backbone it reads gives, so
any head reloads, one of a package the run trusts too.
Head.from_record
Section titled “Head.from_record”def from_record(record: JSON | Mapping[str, object], width: int) -> HeadRebuild the head a run recorded, reading states width wide, in the numerics it
recorded. An earlier run’s record is read through dew.inference.tasks.earlier_shapes.
JevConfidence
Section titled “JevConfidence”class JevConfidence(Confidence)TypeSafe Jev’s confidence (docs.typesafe.ai/confidence).
For a Choice of n options, it is how far the top probability sits above an even split, (p_max - 1/n) / (1 - 1/n). For a Score, it weighs each level’s distance from the most likely level m, 1 - sum_i p_i |i - m| / MAD, where MAD is the same average distance for an even spread measured from the middle level, floored at 0. Jev reports none for a Noul; this gives the Choice formula over its two options, |2 p - 1|, as Jev’s page suggests. A choice with one option is certain.
JointSchemaHead
Section titled “JointSchemaHead”class JointSchemaHead(Head)The joint schema head: decides every question of a row together.
Over the backbone’s normed states it pools each question’s
instructions and each option’s tokens, and each option’s output-table rows
(the lexical vector). The options attend to the row in routing_layers
routing layers; each question becomes a field from its own pooled tokens,
an attention-weighted summary of its options, the row’s last token and its
type; the fields attend to each other and to the row in layers decoder
layers. An option’s logit is a lexical prior (the cosine of its lexical
vector with its question and the last token) plus a gated joint score (the
cosine of field and option, and a residual MLP over the pair). The three
scales are learned in log space and held to 100 at most.
It reads every question of a row (JointLayout), and the backbone’s output
table. In a window of packed rows each question and option reads its own
row alone, and the row’s last token is its own. Each option is computed on
its own but for its weight in its question’s summary, so where the inputs
compact each window’s real options (DecisionInputs.option_slots) it
computes those alone, and lays their
routed states (for the summaries) and their logits back on the grid.
JointSchemaHead.decision_flops
Section titled “JointSchemaHead.decision_flops”def decision_flops(lengths, questions, options, *, pooled=None) -> floatMemory/option projections, evidence O-by-L and field Q-by-L attention, and the span
pooling over each row, or over pooled tokens a row where the spans are a context’s.
Each attention has query/output maps on its queries and key/value maps on memory. The field layers also have Q-by-Q self-attention. The residual scorer reads four width-sized features per option. Span pooling is a masked matrix product and is counted. Lookup embeddings and pointwise normalization/gates are excluded.
JointSchemaHead.setup
Section titled “JointSchemaHead.setup”def setup()LabelReadout
Section titled “LabelReadout”class LabelReadout(Head)Dew’s readout of labels: each option slot’s logit is the backbone’s own logit for its label at
the question’s answer cue (dew.decision.layout.Labels), through the output table’s input QDQ,
scale, bias and softcap, computed in fp32 (DecisionModel.readout_logits, OutputTable.columns).
A label read as several token variants (label_ids[k], one id a variant) is their grouped logit,
the logsumexp of theirs. Untrained, the head is the backbone’s own readout.
untied gives each slot’s every variant a trainable row and bias of its own, drawn from its
variant’s output row (scale folded in) and bias, read through the same input QDQ and softcap and
grouped alike, so at its draw it is the tied readout at every state; it needs the backbone’s
output table to draw from (table). Its rows and their product compute in at least float32, its
table’s or dtype where wider.
A row’s labels are the head’s, its first as many as the question has options (validate_rows);
option slots past the labels, a batch’s padding, read -inf. A head drawn with no ids reads count
labels, named where its tokenizer is known: the layout’s own names, or where they name none, the
first count its tokenizer spells as one token each (labelled, Labels.spelled); with no count,
every label it letters (Labels.capacity).
LabelReadout.tied_labels
Section titled “LabelReadout.tied_labels”def tied_labels() -> tuple[tuple[int, ...], ...] | NoneLabelReadout.labelled
Section titled “LabelReadout.labelled”def labelled(labels: Labels, tokenizer: Tokenizer) -> tuple[LabelReadout, Labels]LabelReadout.validate_rows
Section titled “LabelReadout.validate_rows”def validate_rows(rows: Sequence[Encoded]) -> NoneRefuse rows whose labels are not the head’s (_labels_of).
LabelReadout.decision_flops
Section titled “LabelReadout.decision_flops”def decision_flops(lengths, questions, options) -> floatThe labels’ product over each answer state: its slots’ variants, a row each.
LayaCheckpoint
Section titled “LayaCheckpoint”class LayaCheckpoint( model: DecisionModel, variables: Variables, layout: MarkerLayout, tokenizer: HFTokenizer, specials: Specials, temperatures: tuple[float, float, float], bucket_temperatures: dict[str, float],)One released Laya checkpoint: its model, variables, layout and tokenizer, and its agent’s temperatures.
temperatures is rl_agent_config.json’s per-type list (choice, score,
noul), and bucket_temperatures its per-(type, option count) map, as the file
states them.
LayaCheckpoint.save
Section titled “LayaCheckpoint.save”def save(directory: str | Path, *, max_shard_size: int | str = MAX_SHARD_SIZE) -> NoneWrite this checkpoint in Laya’s layout, which Pretrained.load and llama.cpp’s converter read.
That is model.safetensors (the encoder’s ModernBertModel tensors under
encoder. and the head’s under Laya’s names), encoder/config.json, the
tokenizer under tokenizer/, and rl_agent_config.json with the head’s
depth, the layout’s budgets and the temperatures. The backbone must be a
ModernBERT encoder and the head Laya’s. A backbone under LoRA is written
with its factors merged into their kernels (Pretrained.bound). Laya’s config states a layout’s
window, question budgets and option layout alone, so a layout that chooses more (an order, labels,
segments, blocks) is refused before anything is written; a run saved by Decide.save keeps it.
LogLoss
Section titled “LogLoss”class LogLoss()The cross entropy, -sum_k t_k log p_k: the logarithmic score.
LogLoss.charge
Section titled “LogLoss.charge”def charge(logits, target, options, ordinal)MacroF1
Section titled “MacroF1”class MacroF1()The mean over classes of each class’s F1, the classes those the split’s answered questions are
or are answered as (sklearn’s f1_score(average="macro")): a class is a question’s option
(Decisions.classes), the same in every batch and split that asks it, so each class’s true and
false positives and false negatives add over the whole pass before its F1 is taken.
MacroF1.merge
Section titled “MacroF1.merge”def merge( accumulated: dict[int, np.ndarray], contribution: dict[int, np.ndarray],) -> dict[int, np.ndarray]MacroF1.finalize
Section titled “MacroF1.finalize”def finalize(accumulated: dict[int, np.ndarray]) -> floatclass Noul(instructions: JSON, criteria: Mapping[str, JSON] = dict())A yes-or-no question. criteria may say what true and false mean.
Its options are false then true, so an answer’s second probability is
the probability of yes.
Noul.answer
Section titled “Noul.answer”def answer(probabilities: np.ndarray, confidence: Confidence) -> NoulAnswerNoul.wire
Section titled “Noul.wire”def wire() -> dict[str, JSON]NoulAnswer
Section titled “NoulAnswer”class NoulAnswer(probabilities: np.ndarray, confidence: float, abstained: bool = False)The answer to a Noul: P(false) and P(true), and a confidence computed from them.
abstained: bool-
Whether a calibration’s abstention threshold gated the answer out.
noul: float-
The probability that the answer is yes.
Question
Section titled “Question”class Question(ABC)One question about a state, with its options in the order the model scores them.
kind: str-
Jev’s name for the question type: noul, choice or score.
options: tuple[str, ...]-
Return the answer keys, one per option, in the order the model scores them.
descriptions: tuple[JSON, ...]-
Return what each option means, as the request gave it, with None where it gave nothing.
Question.answer
Section titled “Question.answer”def answer(probabilities: np.ndarray, confidence: Confidence) -> AnswerReturn this question’s answer for probabilities, one per option.
Question.wire
Section titled “Question.wire”def wire() -> dict[str, JSON]Return the question in a Jev request’s format.
Question.from_wire
Section titled “Question.from_wire”def from_wire(request: Mapping[str, object]) -> QuestionBuild the question from a Jev request’s type, instructions and criteria.
Question.index
Section titled “Question.index”def index(label: object) -> intReturn the position of the option that label names, given as its key or as an int index.
RankedProbability
Section titled “RankedProbability”class RankedProbability()The ranked probability score of ordered levels.
It sums the squared gaps between the forecast’s and the target’s cumulative distributions over the K - 1 cuts between K levels, sum_k (P_k - T_k)^2 / (K - 1). It reads only a score question’s levels and charges any other row nothing, since its options have no order. As a metric it averages over the scored rows.
RankedProbability.charge
Section titled “RankedProbability.charge”def charge(logits, target, options, ordinal)class Score(instructions: JSON, criteria: Sequence[JSON])A rating on ordered levels, 2 to 10 of them, lowest first.
criteria describes each level; the answer’s score is the expected
level, so it can land between two.
Score.answer
Section titled “Score.answer”def answer(probabilities: np.ndarray, confidence: Confidence) -> ScoreAnswerScore.wire
Section titled “Score.wire”def wire() -> dict[str, JSON]ScoreAnswer
Section titled “ScoreAnswer”class ScoreAnswer( legend: tuple[JSON, ...], probabilities: np.ndarray, confidence: float, abstained: bool = False,)One probability per level, lowest first, and the expected level.
score: float-
The expected level, which can land between two.
Scored
Section titled “Scored”class Scored(kind: str, logits: np.ndarray, target: np.ndarray)One held-out answer, with its question’s type and the model’s raw logits.
It holds the question’s type (Question.kind), the model’s raw logits over
its options, and the gold distribution over them (Example.distribution):
a target’s, or all of the mass on the answer.
label: int-
The right option: the gold’s most likely one, as
Example.labelsreads a target.
ScoredQuestions
Section titled “ScoredQuestions”class ScoredQuestions()How many questions with a known answer the split scored, the count every other metric of it is taken over.
ScoredQuestions.merge
Section titled “ScoredQuestions.merge”def merge(accumulated: int, contribution: int) -> intScoredQuestions.finalize
Section titled “ScoredQuestions.finalize”def finalize(accumulated: int) -> floatScoringRule
Section titled “ScoringRule”class ScoringRule(ABC)A proper scoring rule, lower is better.
ScoringRule.charge
Section titled “ScoringRule.charge”def charge( logits: jax.Array, target: jax.Array, options: jax.Array, ordinal: jax.Array,) -> jax.ArrayReturn the [B] charge of each row.
The inputs are the [B, K] logits (any value past a row’s options), the
[B, K] target distributions, the [B, K] real options, and the [B] rows
whose options are ordered levels.
SlotReadout
Section titled “SlotReadout”class SlotReadout(Head)A published slot release’s head, its law as the release’s runtime computes it: slot i’s score
is <slot_i, h> + <q(h), k(m_i)> / sqrt(head_dim), h the question’s answer state
(DecisionInputs.readout), m_i the mean of the final states over option i’s whole rendered line
(its span), q and k each a [head_dim, features] projection, with no slot bias and no softcap on
the sum. Its slot rows are drawn from the backbone’s output rows of the slots’ label ids, q from a
normal of std features ** -0.5 and k at zero, so at its draw it scores the labels’ rows alone;
a release’s reader loads all three (readout_head.pt).
slot_rows is the rows the head holds, the labels’ count where unset: a release may hold more
rows than its tokenizer spells labels for, the rows past the labels kept, drawn at zero, and read by
no option. Drawn fresh, its slot rows are the backbone’s vocabulary rows at its labels
(VocabularyRows, its table then), or a normal of std features ** -0.5 over a backbone that
gives none. It computes in at least float32, its states’ or dtype where wider; option slots past
the labels, a batch’s padding, read -inf.
SlotReadout.validate_rows
Section titled “SlotReadout.validate_rows”def validate_rows(rows: Sequence[Encoded]) -> NoneRefuse rows whose labels are not the head’s (_labels_of).
SlotReadout.decision_flops
Section titled “SlotReadout.decision_flops”def decision_flops(lengths, questions, options, *, pooled=None) -> floatEach real option’s slot product and key, its affinity with its question’s query, and its
line’s pooling over the row, or over pooled tokens a row where the lines are a context’s; each
question’s query.
Spherical
Section titled “Spherical”class Spherical()The spherical score as a charge, 1 - (t . p) / |p|, between 0 and 1.
Spherical.charge
Section titled “Spherical.charge”def charge(logits, target, options, ordinal)Temperatures
Section titled “Temperatures”class Temperatures( types: Mapping[str, float] = dict(), buckets: Mapping[str, float] = dict(), lowest: float = 0.5, highest: float = 5.0,)The temperature each question’s logits are divided by.
The temperatures are held within [lowest, highest], as Laya’s agent holds
a checkpoint’s: it refuses to sharpen past 0.5 (its clamp_temperature).
Temperatures.of
Section titled “Temperatures.of”def of(kind: str, count: int) -> floatReturn the temperature for a question of type kind (Question.kind) with count options.
That is its bucket’s temperature, or else its type’s, or else 1.
Temperatures.applied
Section titled “Temperatures.applied”def applied() -> TemperaturesReturn these temperatures as of applies them, each held within [lowest, highest].
A reader that does not hold them itself, as llama.cpp’s server does not, then answers as this does.
Temperatures.fit
Section titled “Temperatures.fit”def fit( scored: Sequence[Scored], *, type_minimum: int = TYPE_MINIMUM, bucket_minimum: int = BUCKET_MINIMUM, lowest: float | None = None, highest: float | None = None,) -> TemperaturesReturn the temperatures that minimize held-out log loss.
There is one per question type with at least type_minimum answers, and one
per bucket with at least bucket_minimum (TYPE_MINIMUM, BUCKET_MINIMUM), each held
within [lowest, highest], the class’s own bounds where not given.
TopProbability
Section titled “TopProbability”class TopProbability(Confidence)The answer’s own probability, the largest one.
This is what a temperature fit calibrates and what expected calibration error measures.
bucket
Section titled “bucket”def bucket(question: Question) -> strReturn the calibration bucket of question: its type and its option count.
The counts are grouped as 2, 3-5, 6-10 or 11+ (Laya’s temp_bucket).
kind_of
Section titled “kind_of”def kind_of(question: Question) -> intReturn the number a laid-out row gives question’s type (KINDS).