Skip to content

dew.artifacts

Typed values that an objective’s evaluation produces.

An objective returns these from its scoring and preview hooks. A metric reads the scoring artifact of the type it expects, and a tracker renders each preview according to its type. The array fields are pytree leaves, so they can pass through jit, while the optional captions and decoded text stay on the host as static metadata.

NameSummary
ArtifactWhat an objective’s scoring and preview hooks return: a dataclass whose per-row fields lead with the batch’s rows, which a validation pass cuts to the real ones.
DecisionsA decision model’s probabilities [N, Q, K] over the options of each row’s questions, the real options [N, Q, K], each question’s right option [N, Q] and its gold distribution over the options [N, Q, K], one-hot where the answer is certain and soft where annotators split, which questions [N, Q] ask a score, whose options are ordered levels, and which [N, Q] have a known answer to score.
ImageGridImages in [-1, 1], [N, H, W, C], with the text each was conditioned on where there was any.
RepresentationsEncoder outputs [N, D] and the labels of the records they came from, for a probe to score.
TextSamplesGenerated token rows, with optional decoded preview text and prompt.
TokenScoresTeacher-forced per-token losses [N, L] and the weight of each target.
VideoGridClips in [-1, 1], [N, T, H, W, C], with the text each was conditioned on where there was any.
uint8_pixelsConvert [-1, 1] pixels, as ImageGrid and VideoGrid hold them, to uint8 in [0, 255].

attribute source

What an objective’s scoring and preview hooks return: a dataclass whose per-row fields lead with the batch’s rows, which a validation pass cuts to the real ones. Dew’s own are the classes above. A package’s objective may score into a dataclass of its own, which its metrics read by type (Metric.reads); a metric picks exactly one scoring artifact, and previews never satisfy metrics. An artifact whose metrics read the rows they score through a mask of its own keeps its batch’s shape through a validation pass (FixedRows). A tracker shows a preview of Dew’s types, so a package shows its own as one of them, a spike raster as an ImageGrid.

dataclass source

class Decisions(classes: jax.Array | None = None)

A decision model’s probabilities [N, Q, K] over the options of each row’s questions, the real options [N, Q, K], each question’s right option [N, Q] and its gold distribution over the options [N, Q, K], one-hot where the answer is certain and soft where annotators split, which questions [N, Q] ask a score, whose options are ordered levels, and which [N, Q] have a known answer to score.

classes: jax.Array | None

Each option slot’s class [N, Q, K, 2], its question and option as a 64-bit key in two int32 lanes (dew.decision.rows.class_id), which a per-class metric reads; None where the rows carry none (a served request’s).

def unscored(padding: np.ndarray) -> Decisions

These decisions at their batch’s shape with the questions of the padding rows unscored (FixedRows): every metric of them reads its rows through scored.

dataclass source

class ImageGrid(captions: tuple[str, ...] = struct.field(pytree_node=False, default=()))

Images in [-1, 1], [N, H, W, C], with the text each was conditioned on where there was any.

dataclass source

class Representations()

Encoder outputs [N, D] and the labels of the records they came from, for a probe to score.

dataclass source

class TextSamples(
prompt: str = struct.field(pytree_node=False, default=''),
texts: tuple[str, ...] = struct.field(pytree_node=False, default=()),
)

Generated token rows, with optional decoded preview text and prompt.

dataclass source

class TokenScores()

Teacher-forced per-token losses [N, L] and the weight of each target.

A weight is 1 where the target counts and 0 where it is padding or a document’s first token. A perplexity is exp of the weighted mean loss over a whole pass, so a batch with no counted target adds nothing to it.

correct: jax.Array

Per-token top-1 correctness, from the same logits that produced the losses.

dataclass source

class VideoGrid(captions: tuple[str, ...] = struct.field(pytree_node=False, default=()))

Clips in [-1, 1], [N, T, H, W, C], with the text each was conditioned on where there was any.

function source

def uint8_pixels(images: ArrayLike) -> NDArray[np.uint8]

Convert [-1, 1] pixels, as ImageGrid and VideoGrid hold them, to uint8 in [0, 255].

Each value maps to its nearest level, computed as (x + 1) * 127.5 in float32 and rounded half to even (np.rint). The result is clipped, because a sample can leave the range. Metrics score these bytes and trackers preview them, so both see the same image.