dew.decision.data
Labelled requests: what a decision model trains and calibrates on.
| Name | Summary |
|---|---|
Group | Questions of one example that ask the same thing (negated, paraphrased, its options re-ordered), named name: members maps each one’s every option key onto a canonical key the members share, so a consistency term (DecisionObjective.consistency) can hold their answers to one distribution over the canonical keys. |
Example | One state, its questions, and the right answer to some of them. |
GroupedExamples | Examples whose rows train together, so a consistency term (DecisionObjective.consistency) holds questions asked across them to one answer: the same question negated, paraphrased or its options re-ordered in rows of their own, under any layout. |
groups_of | A row’s groups, as Example.groups holds them: a mapping of each group’s name to its members, each member’s option keys mapped to canonical keys, or None for none. |
ParseExample | A row of an example’s fields as its Example, a row of grouped examples’ fields (examples) as its GroupedExamples, or a row that holds either already as it is. |
ParseTable | A flat table’s row as a one-question Example. |
decisions | dataset’s rows as decisions, each an Example under example, for a chain to batch: a flat table’s text and label columns read as one question over options (its ClassLabel names, or read whole in memory, where none are given), its context column, where named, each example’s context, or rows that hold their examples already. |
STREAMED_WINDOW | The rows a corpus read as a stream draws its order from (mixture): decision examples are rows of text, so this many hold tens of megabytes. |
mixture | The decision corpora of the mixture directory root_url (its mixture.json): every corpus of a positive weight, the manifest’s weights over weights, mixed by them for training, and their held rows together, or None where none has any. |
corpus | Corpus name’s split of the mixture directory root_url as decision loading reads it, from the shards and index its manifest names (or the corpus’s one JSONL file), each row an Example under example; None when the corpus has no such split. |
made_by | The manifest a decision mixture’s corpora state they were made by (mixture): read again from the root and descriptor they name, or the copy a chain recorded before they named them. |
class Group(name: str, members: Mapping[str, Mapping[str, str]])Questions of one example that ask the same thing (negated, paraphrased, its options re-ordered),
named name: members maps each one’s every option key onto a canonical key the members share,
so a consistency term (DecisionObjective.consistency) can hold their answers to one distribution
over the canonical keys.
Example
Section titled “Example”class Example( state: JSON, questions: Mapping[str, Question], answers: Mapping[str, str | int] = dict(), targets: Mapping[str, tuple[float, ...]] = dict(), context: JSON | None = None, groups: tuple[Group, ...] = (),)One state, its questions, and the right answer to some of them.
An answer names an option by its key or its index (Question.index), so
a dataset’s integer label column can be used as it is. Where the gold is a
distribution, as a teacher’s or a panel’s is, targets gives it instead:
one probability per option, in the question’s own order. A question with
neither is asked but not scored.
context is a sequence the decision reads beside its state, through the
backbone’s cross-attention layers (DecisionObjective’s context), rendered
as text; None, or a backbone without those layers, reads none.
groups name questions that ask the same thing (Group), each question in
one at most; an objective without a consistency term reads none.
Example.of
Section titled “Example.of”def of(row: Example | Mapping[str, object]) -> ExampleReturn an example as it is, or build one from a row with state, questions and answers.
The questions may be Questions or in Jev’s wire format.
Example.group_of
Section titled “Example.group_of”def group_of(name: str) -> tuple[int, Group] | NoneThe index and group question name stands in, or None.
Example.labels
Section titled “Example.labels”def labels() -> dict[str, int]Return each answered question’s right option, as an index: a target’s most likely one.
Example.distribution
Section titled “Example.distribution”def distribution(name: str) -> np.ndarrayReturn question name’s gold as a distribution over its options.
That is its target, or else all of the mass on its answer.
GroupedExamples
Section titled “GroupedExamples”class GroupedExamples(examples: tuple[Example, ...], groups: tuple[Group, ...] = ())Examples whose rows train together, so a consistency term (DecisionObjective.consistency)
holds questions asked across them to one answer: the same question negated, paraphrased or its
options re-ordered in rows of their own, under any layout.
groups names each set of such questions (Group), its members keyed "<example index>.<question>"
by the example’s index in examples and the question’s name there, each mapping its question’s
every option key onto canonical keys the members share, as an example’s own groups do. An example’s
own groups hold too, numbered after these. The examples’ rows train in one packed window, as a unit
the packer does not split (DecisionObjective.data): a unit too long for a window is refused.
GroupedExamples.group_of
Section titled “GroupedExamples.group_of”def group_of(index: int, name: str) -> tuple[int, Mapping[str, str]] | NoneThe unit-wide index of the group example index’s question name stands in, counting each
example’s own groups after these, and the member’s map onto its canonical keys, or None.
GroupedExamples.of
Section titled “GroupedExamples.of”def of(row: GroupedExamples | Mapping[str, object]) -> GroupedExamplesReturn grouped examples as they are, or build them from a row with examples (each as
Example.of reads one) and groups (groups_of), its members named
"<example index>.<question>".
groups_of
Section titled “groups_of”def groups_of(value: JSON) -> tuple[Group, ...]A row’s groups, as Example.groups holds them: a mapping of each group’s name to its members,
each member’s option keys mapped to canonical keys, or None for none.
ParseExample
Section titled “ParseExample”class ParseExample()A row of an example’s fields as its Example, a row of grouped examples’ fields (examples) as
its GroupedExamples, or a row that holds either already as it is. A Grain transform, so a chain
that parses its rows by it is recorded and read back (dew.data.records).
ParseExample.map
Section titled “ParseExample.map”def map(element: Batch) -> BatchParseTable
Section titled “ParseTable”class ParseTable( options: tuple[str, ...], text: str = 'text', label: str = 'label', question: str = 'label', instructions: str = 'Classify the input into the correct category.', by_value: bool = False, context: str | None = None,)A flat table’s row as a one-question Example. A label names its
option by key, or by index where it is an int; where the options are the
label column’s own values (by_value), every label names its option by
value, an int one included. A context column’s value is the example’s
context (Example.context), none where the column holds None. A row’s
groups field reaches the example as Example.of reads it, so a group
over the row’s one question is refused as any group of one member is.
decisions
Section titled “decisions”def decisions( dataset: Dataset, *, options: Sequence[str] = (), text: str = 'text', label: str = 'label', question: str = 'label', instructions: str = 'Classify the input into the correct category.', context: str | None = None,) -> Pipelinedataset’s rows as decisions, each an Example under example, for a chain to batch: a flat
table’s text and label columns read as one question over options (its ClassLabel names, or
read whole in memory, where none are given), its context column, where named, each example’s
context, or rows that hold their examples already. Its stages
before this one, a holdout’s selection among them, stand; a decision chain is a store’s rows, and
a mixture of them mixes decision chains (mixture). Raises ValueError for a mixture or phases
under it.
STREAMED_WINDOW
Section titled “STREAMED_WINDOW”STREAMED_WINDOW = 2 ** 14The rows a corpus read as a stream draws its order from (mixture): decision examples are rows of
text, so this many hold tens of megabytes.
mixture
Section titled “mixture”def mixture( root_url: str, *, weights: Mapping[str, float] | None = None, descriptor: str | None = None, seed: int | None = 0, order: Literal['seeded', 'schedule'] = 'seeded',) -> tuple[Pipeline, Pipeline | None]The decision corpora of the mixture directory root_url (its mixture.json): every corpus of a
positive weight, the manifest’s weights over weights, mixed by them for training, and their held
rows together, or None where none has any. descriptor pins a replaced manifest by its URL.
Each corpus trains on its rows in an order drawn from seed and its name (Pipeline.shuffle): all
of them permuted where the run reads by index, within STREAMED_WINDOW rows where it streams. The
mix takes each corpus’s next row in that order, choosing the corpus each position reads by a
permutation seed draws for each window of the mix (mix(seed=)), so every window holds each
corpus at its share in no fixed order, and a corpus whose file groups its rows by task gives no
window the next rows of its file. order="schedule" chooses the corpora in Grain’s schedule, the
same every window in their manifest’s order, as a mixture read before its corpus order was seeded
(row 3870); seed=None reads each corpus in its file’s order, in that schedule. Stages of the
chain, the order resumes and continues into a later phase as the mix read it
(dew.data.compile).
corpus
Section titled “corpus”def corpus( root_url: str, name: str, split: Literal['train', 'held'] = 'held', descriptor: str | None = None,) -> Pipeline | NoneCorpus name’s split of the mixture directory root_url as decision loading reads it, from the
shards and index its manifest names (or the corpus’s one JSONL file), each row an Example under
example; None when the corpus has no such split. A benchmark reads one corpus’s held rows through
this, whatever its weight.
made_by
Section titled “made_by”def made_by(facts: Mapping[str, object]) -> Mapping[str, object]The manifest a decision mixture’s corpora state they were made by (mixture): read again from
the root and descriptor they name, or the copy a chain recorded before they named them.