dew.data.dataset
The compiled Loader, shared reader partitions and checkpoint contracts.
Every stream is opened for a DataPartition, the share of each global batch
its reader reads. The trainer gets it from the mesh (DataPartition.of),
because processes that a pipeline or a split sequence spans hold the same rows
and so read the same share. A loader reads the share it is given and nothing
else.
An indexed global order resumes from a record count across process counts. Unknown-length, rejecting and token-budget orders instead save Grain’s per-share state and refuse changed topology before a reader opens.
| Name | Summary |
|---|---|
Budgeted | A stream that reports how long stopping it may take. |
Checkpointable | A data stream that can report where it stopped and resume from there. |
Closeable | A stream that holds resources a stopped run must release, such as worker processes, file handles or a shared memory block. |
DataPartition | The share of every global batch a reader reads: the indexth of count equal, disjoint shares. |
Forwarding | Forwards a stream wrapper’s stop signal, stop budget, close, identity and where it ends to its source. |
GlobalStream | Reads a training stream whose saved position is one global record count. |
Identified | A training stream that names the records it reads and says where it ends (Ends). |
Loader | Opens the batches a run trains and validates on. |
Loading | Read throughput settings, as Grain’s four knobs. |
PhasedStream | Reads one global order per phase, switching at step boundaries. |
Ramp | Grows the global batch over the run’s first records. |
RampedStream | Grows a training stream’s step over the run’s first records. |
Reader | Opens a fresh iterator over one share of every global batch, the share the partition names. |
Records | Returns records by index, which is how the loaders read a source. |
Resumable | A batch stream that can resume where it stopped; this is what a batch ramp wraps. |
SourceSlice | Reads source[start:stop] by index. |
Stage | One stage of a batch ramp: the global batch a step reads in it, and the record it starts at. |
Stoppable | A stream that can be asked to stop before it is drained. |
mapped | Return stream with stage applied to each batch, forwarding stop, close and position. |
Budgeted
Section titled “Budgeted”class Budgeted(Protocol)A stream that reports how long stopping it may take.
It is separate from Stoppable because a stream can report a budget
without accepting a stop request. A wrapper forwards each one separately.
Checkpointable
Section titled “Checkpointable”class Checkpointable(Protocol)A data stream that can report where it stopped and resume from there.
Grain’s iterators satisfy this, and so does GlobalStream, and so does
a stream a reader declares to replay (replayed). The state itself is
opaque here; dew.position tells which of its two kinds a checkpoint
holds.
Checkpointable.get_state
Section titled “Checkpointable.get_state”def get_state() -> PositionCheckpointable.set_state
Section titled “Checkpointable.set_state”def set_state(state: Position) -> NoneCloseable
Section titled “Closeable”class Closeable(Protocol)A stream that holds resources a stopped run must release, such as worker processes, file handles or a shared memory block.
Closeable.close
Section titled “Closeable.close”def close() -> NoneDataPartition
Section titled “DataPartition”class DataPartition( index: int = 0, count: int = 1, readers: int = 1, reader: int = 0, row_multiple: int = 1, window: int = 1,)The share of every global batch a reader reads: the indexth of count equal, disjoint shares.
A loader cuts its record order as index :: count, so the shares of
global batch k together hold the same records at every count, and a record
count marks the same place in the stream whatever the count. The trainer
gets the share a process reads from the mesh (DataPartition.of).
Processes whose devices hold the same rows read the same share, because
the axes between them split a sequence or hold a pipeline’s stages, not
rows. DataPartition() is a single reader of every row, which is what a
single process reads.
readers: int-
The number of processes that read this share.
Each reads the same records. A source whose rows can differ between reads of one share, such as a network fetch that drops failed records, must either refuse more than one reader or read on the first reader alone and send its batch to the others (
dew.distributed.batches.first_reader_batch). reader: int-
This process’s place among the share’s readers, in process order (0 for the first).
row_multiple: int-
Number of distinct row shards this reader feeds within its host.
window: int-
The steps an update pools (
Trainer’s accumulation): a batch ramp changes stage only where an update begins (Ramp.stages). It cuts the stream into steps and names no data, so it is no part of what a share reads.
DataPartition.of
Section titled “DataPartition.of”def of(mesh: Mesh, window: int = 1) -> DataPartitionReturn the share of every global batch this process reads on mesh.
A batch’s rows split over the row axes and no others: those the rules
in context give activation_batch (batch_axes). The sequence axis
splits positions, the tensor axis splits widths, and the stage axis
holds a pipeline’s stages, so processes whose devices
hold the same row shards need the same rows. The processes therefore
fall into groups by the rows they hold. Each group reads one share,
numbered by the first row shard it holds, and every process in the
group reads it (readers); reader is this process’s place among
them, in process order.
A hand-built device order can produce groups whose rows overlap without being the same rows. No share can then be read whole by each group, so such a mesh is refused.
DataPartition.bounded
Section titled “DataPartition.bounded”def bounded(examples: int, records: int | None = None) -> intThis share’s examples of the pool’s first examples, where each share reads records of its own
dealt index :: count (cut) from a root of records a pass, each pass over its own: its records
in every whole pass the bound spans, and its records among those the last pass reaches. Where no
pass’s count is known ahead (a step before the bound drops records), the examples themselves are
dealt so, every count-th from its index.
DataPartition.held
Section titled “DataPartition.held”def held(rows: int, offset: int = 0) -> tuple[slice, bool]Which of rows rows a validation read of this share holds, and whether they are real:
every count-th from its index, offset places on as a resume’s count inside a batch leaves
it (GlobalStream); where that holds none of rows that hold some, the first in their place,
not real (FilledBatch marks them so), so every share reads a batch where any does and
none scores another’s row.
DataPartition.cut
Section titled “DataPartition.cut”def cut(offset: int = 0) -> sliceThis share’s rows of an order or a batch: every count-th from its first (first).
DataPartition.first
Section titled “DataPartition.first”def first(offset: int = 0) -> intThis share’s first row of an order or a batch: its index, offset places on as a resume’s
count inside a batch leaves it (GlobalStream).
DataPartition.rows
Section titled “DataPartition.rows”def rows(batch: int) -> intReturn how many rows of a batch-row global batch one share holds.
batch must split evenly into the shares: a remainder would train on
fewer records a step than the run reports, and a batch smaller than
the share count would leave a share with nothing.
Forwarding
Section titled “Forwarding”class ForwardingForwards a stream wrapper’s stop signal, stop budget, close, identity and where it ends to its source.
Subclasses keep the source at _source and override close for their own
cleanup around super().close().
Forwarding.request_stop
Section titled “Forwarding.request_stop”def request_stop() -> NoneForwarding.close
Section titled “Forwarding.close”def close() -> NoneForwarding.identity
Section titled “Forwarding.identity”def identity() -> strGlobalStream
Section titled “GlobalStream”class GlobalStream( open_at: Callable[[int], pygrain.DatasetIterator[Batch]], batch: int, plan: Plan, loading: Loading, *, resume: Literal['exact', 'non-exact'] = 'exact', endless: bool = True,)Reads a training stream whose saved position is one global record count.
The stream is an endless shuffled order over the whole corpus, and each step is cut from it here. Global batch k is records [k * batch, (k + 1) * batch) of that order, and share p of n reads every nth of them starting at p. The position is then the number of records the run has consumed: one number, the same on every reader, tied to no share. A checkpoint written by two processes resumes on one process or four, on the same records in the same steps.
open_at(offset) starts the share’s read at a record offset. A restore
uses it instead of replaying the stream, since an offset is just a slice
bound. It is called on the first batch and again after set_state, so a
stream restored before it is read does not start its workers twice.
The offset alone would resume that many records into whatever order the
run now has, so the plan’s order (order) and the plan are saved with
it and a restore whose plan disagrees is refused (Plan.check).
dew.position defines the format that both this class and
dew.checkpoints read.
Every share steps as many batches of the order, its whole batches where
it drops a remainder, and else a share past the order’s last batch
reading on with batches of no real row (dew.data.lowering._evened), so
where it ends, the shares run out at the same read (ends).
order: str-
The order’s description, which a saved position is compared against.
GlobalStream.get_state
Section titled “GlobalStream.get_state”def get_state() -> bytesGlobalStream.identity
Section titled “GlobalStream.identity”def identity() -> strThe plan’s identity (Plan.identity), which a pool agrees before it
reads: hosts whose copies of a corpus hold other records, or whose
steps make other values, are refused however the sources describe
themselves.
GlobalStream.set_state
Section titled “GlobalStream.set_state”def set_state(state: bytes) -> NoneGlobalStream.close
Section titled “GlobalStream.close”def close() -> NoneIdentified
Section titled “Identified”class Identified(Protocol)A training stream that names the records it reads and says where it
ends (Ends).
Each process of a pool opens its own stream, over its own copy of the
corpus, and Trainer.fit refuses a pool whose streams name different
records before any process reads one. It agrees every read of a stream
that may run out on some processes and not others, so a process whose
stream ran out stops its peers before they step without it; one that
never ends, or ends on every share together, is agreed at its first
read (dew.training._fit.FitRun.agrees).
Identified.identity
Section titled “Identified.identity”def identity() -> strLoader
Section titled “Loader”class Loader( train: Reader, val: Reader | None, records: int | None, batch: int, ramp: Ramp | None = None, held_out: int = 0, counts: Mapping[str, CorpusCounts] = dict(), validation_counts: Mapping[str, CorpusCounts] = dict(), plan: Plan | None = None, statistics: Callable[[], Mapping[str, float]] = _data_statistics, batch_tokens: int | None = None, device: tuple[DeviceStep, ...] = (), widths: Mapping[str, object] = dict(), tail: Tail = 'dropped', order_records: int | None = None, examples: int | None = None, pass_examples: int | None = None, epochs: int | None = None, bound_ends: bool = True, token_ends: tuple[int | None, ...] | None = None,)Opens the batches a run trains and validates on.
train(partition) opens an endless shuffled stream. val(partition)
opens one pass over the held-out records in a fixed order, which ends by
itself; val is None when nothing is held out. Both read the share of
each global batch that partition names (DataPartition). batch is the
global batch and records the number of training records behind it, so
steps_per_epoch is one pass over them. ramped sets ramp when the run
grows its batch over its first records, and batch is then the batch the
ramp ends at. held_out is how many records of the training split a spec
kept back for validation, which fit reports when the run starts; it is 0
when validation is a separate split or there is none.
Each factory call returns a fresh iterator, and the code that opened it
owns closing it. Close it after use when it is Closeable; every wrapper
around a stream (host steps, ramps, phases, the readers) forwards close
to the stream it wraps. Never close the shared dataset or its backing
store. A batch an iterator yields stays valid until its consumer asks for
the next one. A consumer that keeps batches longer copies them, since a
batch made by worker processes lives in shared memory and holds a
descriptor for as long as it is kept. A source’s optional request_stop
is a separate, thread-safe signal; it does not make it safe to call the
final close while another thread is still iterating.
Image and video fields are uint8 in [0, 255], text is the tokenized
{"input_ids", "attention_mask"} dict under “text”, and a token window
is int32 ids under “text”.
Whether a run resumes its data position depends on the iterator.
Grain-backed iterators have get_state and set_state, and host
transforms forward the pair. A stream with neither trains, and a resume
of its checkpoint is refused, unless its reader declares that it reads
the same batches each time it opens (replayed), which positions it by
replay. An indexed global-order position resumes on any
partition, over one corpus or a weighted mixture. A stream that batches
its own records reports its share’s own position, and dew.checkpoints
resumes it only on a reader of that share.
device: tuple[DeviceStep, ...]-
The training batch’s steps the compiled training step runs, on the accelerator, after the reader delivers it (
DeviceStep). widths: Mapping[str, object]-
The run’s static widths its steps bound (
run_widths): the facts its objective’s run-scoped steps read (Configured.scope), such as a decision’sslots, which a saved run of the objective records (Objective.bind_widths); empty where they read none. tail: Tail-
What becomes of the records an epoch leaves short of a whole batch: read, carried into the next epoch by an order that reads on or kept and filled by one that keeps them, so an epoch’s steps count their batch; or dropped, so they count none (
Tail). order_records: int | None-
The records a training read of a global order that ends hands out over all its passes, the same on every share, by which a batch ramp ends its last step (
RampedStream); None where the order never ends or its read counts none ahead. examples: int | None-
The records the training read is bounded to before they are packed (
compile’sexamples), whose steps a run’sexamples=budget takes (example_steps); None where it is not bounded. pass_examples: int | None-
The source examples one pass of a training read with no record count holds (
recordsNone: an objective makes rows of them, or a pack lays them out, after the order, or a filter cuts it per share), as its read states them (dew.data.compile._pass_examples): an epoch of it; None where nothing states them, or the read has a record count. epochs: int | None-
The epochs a training read with no record count is bounded to,
examplestheir passes over its source examples (pass_examples), as a run of that many reads it (dew.data.compile.fitted); None for a read bounded otherwise or not at all. bound_ends: bool-
Whether a read bounded in
examplesends where its bound does, having read every one: no take or slice after the bound ends it first, which a run on the examples clock refuses (dew.training._fit.clocked), whose last batch consumes the examples no batch counted (drained). token_ends: tuple[int | None, ...] | None-
The committed tokens each phase of a schedule ends at (
dew.data.pipeline.schedule), the last where the data ends or None where it runs on, which the trainer reads its commits against and ends the phase at (PhasedStream.ending); None for a read on no token clock.
Loader.epoch_steps
Section titled “Loader.epoch_steps”def epoch_steps(epochs: int = 1, window: int = 1) -> intReturn the number of steps that epochs passes over the records take, in updates of
window steps (Ramp.stages), refusing passes too short to fill a batch.
A batch ramp grows the batch over the run’s first records once, so its smaller early steps count once across all the passes, not once a pass. A stream without a record count has no epoch, so a run over it must give its length in steps instead.
Loader.example_steps
Section titled “Loader.example_steps”def example_steps(examples: int, window: int = 1) -> int | NoneThe steps that read the examples source examples this loader’s training read is bounded to
(examples), with every row the objective makes of each, its last batch short where they end
inside one, in whole updates of window steps, where a global order’s plan counts them ahead
(order_records); None for one read in its own order, as a pack that runs as rows arrive makes,
or one an objective expands after the bound, which no pass reads ahead to count, such as a pool’s
read whose shares each read records of their own and so end steps apart: its run keeps its
progress in the examples its batches consume (dew.batch.EXAMPLES_CONSUMED), a pool’s every share
reading on until each has run out (dew.training._fit.pooled_read). A pool’s read in one global
order plans as one process’s does, every share ending at its steps, a share the order’s last batch
ends before stepping batches of no real row (dew.data.lowering._evened). Refused where the read is
bounded to another count, and where the last steps would make no whole update, so some examples
would train nothing.
Loading
Section titled “Loading”class Loading( workers: int | Literal['auto'] = 0, threads: int = 16, read_buffer: int = 16, worker_buffer: int = WORKER_BUFFER, ram_budget_mb: int | None = None, max_workers: int | None = None, cache: bool = True,)Read throughput settings, as Grain’s four knobs.
None of the four changes which records a run sees or what they contain, so a host can tune them for its disk and still produce identical batches. The shuffle seed is not one of them: it sets the order records arrive in and keys the per-record rng that augments and captions them.
Each knob counts something different:
workers: worker processes;threads: reads one worker keeps in flight;read_buffer: reads one worker holds ahead;worker_buffer: whole batches one worker holds ready for the training process.
threads and read_buffer count the elements a read produces: a whole
batch where the order is indexed, which is cut into batches by index
before any reader starts, as Grain’s own indexed pipelines are, and a
record where the order streams and is batched as it arrives.
The defaults, 16 reads in flight and 16 held ahead, keep enough reads in flight for a remote store’s latency while holding few batches. On an 8-core armada host with two workers, Dew read 128-pixel images at 0.91 to 1.04 times plain Grain under the same settings, within the run-to-run noise of parity, and held about 32 batches, 50 MB for 32 such images, per worker. 64 and 128 buffered 300 MB per worker and matched Grain only because Grain itself slowed down there.
Zero workers is Grain’s own default: threads in the training process read the records. Each worker process imports the program again, so it costs seconds and a process’s memory before the first batch, and pays off only once decoding or augmentation is slower than the threads. On two cores of a shared workstation, the first batch of a 100-record pipeline took 17.8 s and 7.07 GiB resident across 33 processes with 32 workers, against under 0.1 s and 0.23 GiB with none.
cache: bool-
Whether a run reads its chains’ leading deterministic steps from the records an earlier run made of the same records (
dew.data.cache.automatic_root), which changes none of them. stop_seconds: float-
How long stopping this many workers may take before it counts as a hang.
This is Grain’s own bound. A stop first waits for the batch being read. Grain then stops the worker processes one after another, each finishing its current batch before it exits, and kills a worker that has not exited within Grain’s kill timeout; the read gets the same time. Slow stops are normal: four workers took 5.1 to 7.4 s to stop on 12 vCPUs.
Loading.resolved
Section titled “Loading.resolved”def resolved( records: pygrain.IterDataset, *, reads: int, host: Host | None = None,) -> LoadingThese settings with automatic workers and read buffer sized for host (this process’s
by default) from its CPUs and its memory together, as tf.data’s autotuning budgets both.
Grain’s own pick (pick_performance_config) reads memory alone, takes the machine’s CPU
count where a container may run on a few of them, and counts one batch for each worker.
A worker for each CPU, as tf.data’s parallelism takes each schedulable one, where there is
more than one: on one CPU a worker adds a process and no parallelism, so the reading process
reads. On four CPUs of an eight-core host an image read in four workers read as many images
a second as in eight, within 2%, and in three 11% fewer (armada, 2026-10-26). At most
max_workers of them, and no more than half the memory the process may still take holds
(ram_budget_mb where given), each a process of its own (WORKER_PROCESS_BYTES) holding
its worker_buffer batches and read_buffer reads, the read buffer itself no more reads
than that half holds. records are the batches the read makes without workers,
SAMPLED_BATCHES of which are measured; reads is how many reads make one: one where the
order is read by batch, its rows where it streams records. A read that makes no batch reads
without workers.
Loading.announced_stop
Section titled “Loading.announced_stop”def announced_stop() -> Iterator[None]Stop the workers inside this context, explaining on stderr if the stop takes long.
If the stop runs past 5 seconds (_QUIET_STOP_SECONDS), it prints once
what it is waiting for, so a long stop does not look like a hang.
PhasedStream
Section titled “PhasedStream”class PhasedStream( phases: Sequence[tuple[Callable[[tuple[int, ...]], GlobalReader], int | None]], stop_seconds: float, *, tokens: Sequence[int | None] | None = None,)Reads one global order per phase, switching at step boundaries.
Each phase opens a GlobalReader, such as a compiled global order, and
has the global record count it ends at, or None for the last phase, which
runs on. Phase k starts its own order at its own record zero when the
run’s count reaches the end of phase k - 1, so the records a step reads
depend only on the step and the phase list, at any process count.
The saved position is the run’s record count, the current phase’s order,
and each completed phase’s order with its end (position.Global). A
restore checks the completed phases and the current phase’s order against
this list, and nothing after them. So a run may append phases, or move a
boundary it has not reached yet, and still resume where it stopped. A
one-order run’s position is phase 0 with no completed phases, so a run that
trained on one mixture resumes into a phase list that starts with it.
Changing a phase the run has already read, or ending the current phase
before the records the run already read in it, is refused.
A run on the committed-token clock (tokens, the committed tokens each
phase ends at, the last’s where the data ends or None, dew.data.pipeline.schedule) knows no phase’s
end in records ahead: the trainer ends the phase it stands in at the step
whose commit reaches the phase’s tokens (ending), and the saved position
carries that end, as the end of a phase the run completed. Each phase opens
with the record ends of the phases before it (a corpus they read continues
in it from where they left it), so one after an end the run has yet to
reach opens once the run reaches it. A phase the run’s commits passed
within one step ends where it starts, holding no record.
PhasedStream.get_state
Section titled “PhasedStream.get_state”def get_state() -> bytesPhasedStream.set_state
Section titled “PhasedStream.set_state”def set_state(state: bytes) -> NonePhasedStream.ending
Section titled “PhasedStream.ending”def ending(state: bytes) -> bytesstate, a position of this run on the committed-token clock, with the phase it stands in
ended where it stands and the next opened there: where the run’s commits reached the phase’s
tokens, which the trainer reads (dew.training.Trainer.fit).
PhasedStream.identity
Section titled “PhasedStream.identity”def identity() -> strPhasedStream.close
Section titled “PhasedStream.close”def close() -> Noneclass Ramp(start: int, increment: int, samples: int)Grows the global batch over the run’s first records.
A run starts at start records a step and adds increment each time the
records read since the last increment reach samples divided by the
number of increments. It stops at the batch the dataset was loaded with,
and the difference must be a whole number of increments, as in MaxText’s
batch rampup.
MaxText counts the batch per device, while Dew counts the global batch
everywhere, so start and increment are records a step: MaxText’s
per_device_batch_size_start times the device count is start here.
The stage a run is in depends only on the records it has read, which a
checkpoint already holds as the data position, so a resumed run continues
the ramp with nothing else saved. A stage lasts
ceil(samples / increments / batch) steps, computed as one integer ratio
instead of MaxText’s two floating-point divisions, so the boundaries are
exact. A run whose updates pool several steps (DataPartition.window)
begins each stage where an update does, at the first update boundary at or
after the step it would begin at, as MaxText and torchtitan apply a batch
schedule per optimizer step: no update pools two stages’ batches, and a
boundary moves by less than one update. The records stay the position, so a
run resumed under another window cuts its stages again. Where the ramp
ends, MaxText’s loader drops what is left of its buffered batch while Dew
keeps reading, so from then on MaxText reads later records than Dew in each
step, although the batch sizes match.
Ramp.stages
Section titled “Ramp.stages”def stages(final: int, window: int = 1) -> tuple[Stage, ...]Return every stage of the ramp up to final, the run’s own global batch, each beginning
where an update of window steps does.
Each stage’s batch has to split evenly into the mesh’s rows, and so into the shares its readers read; the trainer checks every stage against the mesh before the run starts.
Ramp.steps_for
Section titled “Ramp.steps_for”def steps_for(records: int, final: int, window: int = 1, tail: Tail = 'dropped') -> intReturn the number of steps a run of updates of window steps takes to read records
records under this ramp, a last step that holds fewer than a batch counted where it is read
(tail).
Early steps read fewer records, so a pass over a corpus takes more steps than it would with the final batch from the start.
RampedStream
Section titled “RampedStream”class RampedStream( source: Resumable, stages: Sequence[Stage], partition: DataPartition, *, tail: Tail = 'dropped', order_records: int | None = None,)Grows a training stream’s step over the run’s first records.
It wraps a stream whose position is a global record count (a
GlobalStream or a host transform over one) and decides only how many of its
records each step reads. Reads run one whole final-size batch ahead, and
each step is cut from a rolling buffer, as in MaxText’s loader. A step
therefore reads the records the run would have read without the ramp, in
the same order, and a stage change neither skips a record nor reads one
twice.
Records left in the buffer have not been trained on and are not counted in the position, which is the number of records handed out so far. A restore gives the source that count, so it reopens its read there.
stages: tuple[Stage, ...]-
The batch this stream reads at each stage, and where each stage begins.
The trainer reads them from the stream it opened. One compiled step per stage covers every shape a ramped run uses, and the mesh has to divide each of them.
RampedStream.get_state
Section titled “RampedStream.get_state”def get_state() -> bytesRampedStream.set_state
Section titled “RampedStream.set_state”def set_state(state: bytes) -> NoneRampedStream.close
Section titled “RampedStream.close”def close() -> NoneReader
Section titled “Reader”Opens a fresh iterator over one share of every global batch, the share the partition names. The iterator is its caller’s to close.
Records
Section titled “Records”class Records(Protocol)Returns records by index, which is how the loaders read a source.
A record is one example’s fields, in the shape Batch names, or the
packed bytes an ArrayRecord file holds, which the spec’s own transform
unpacks into fields. A record may be missing (None). A memory read and a
cache leave a missing record out and count it, records held in memory
leave it out of an order batched by index, and a stream skips it; any
other source read in an order batched by index must hold every record.
Grain’s own RandomAccessDataSource declares the same two methods with the
record as a type parameter. That parameter is invariant, so a source
declared through it cannot be passed to a loader that names a different
record type. This protocol names what the loaders actually read.
Resumable
Section titled “Resumable”class Resumable(Checkpointable, Protocol)A batch stream that can resume where it stopped; this is what a batch ramp wraps.
SourceSlice
Section titled “SourceSlice”class SourceSlice(source: Indexed, start: int, stop: int)Reads source[start:stop] by index.
It gives the train and validation loaders disjoint index ranges while Grain still handles the shuffle, the epochs and the sharding. The attributes stay plain because Grain pickles the source to its workers.
class Stage(batch: int, records: int)One stage of a batch ramp: the global batch a step reads in it, and the record it starts at.
Stoppable
Section titled “Stoppable”class Stoppable(Protocol)A stream that can be asked to stop before it is drained.
Stoppable.request_stop
Section titled “Stoppable.request_stop”def request_stop() -> Nonemapped
Section titled “mapped”def mapped(stream: Reader, stage: Callable[[Batch], Batch]) -> ReaderReturn stream with stage applied to each batch, forwarding stop, close and position.