dew.interop.openenv
OpenEnv environments as episode environments: one OpenEnv session per episode, Dew drawing each turn.
OpenEnv (huggingface/OpenEnv, the openenv distribution, 0.9) serves an
environment over a session a client opens, EnvClient or its synchronous
.sync(): reset(**options) and step(action) each answer a StepResult
holding the observation, the reward of the step and whether the episode is
done. OpenEnv draws no model turn; a trainer does. OpenEnv adapts one
session to the Environment an EnvironmentSource runs: the rollout server
draws each turn from the exact context ids, and the session steps on the
action the drawn ids stand for.
Three adapters are the caller’s, since only it knows the model’s chat format and the environment’s action type:
prompt(observation)gives the first context’s ids, and the chat messages they hold (Said).reply(observation)gives the ids the environment’s answer appends after the model’s turn, and the messages they hold. The turn’s drawn ids, EOS included, stay as drawn: the next context is the turn’s context, its drawn ids, then these.action(ids)gives the environment’s action for a turn’s drawn ids. Decoding them is the caller’s.
Each step’s reward becomes that turn’s step reward (Observation.reward), so
a verifier that returns 0 trains on the environment’s rewards alone, and a
done result completes the episode, a reset’s too, before any turn. The
messages each observation holds become the episode’s transcript
(Observation.messages).
Dew imports nothing of OpenEnv. session(task, identity) opens the caller’s
client, a context manager whose exit closes the session, and the protocols
below are what Dew reads of it. OpenEnv’s client speaks to its server over a
WebSocket, and a connection that drops while a reset or a step waits raises
websockets.exceptions.ConnectionClosed: that call raises it as an
InfrastructureError, an infrastructure failure the source retries.
| Name | Summary |
|---|---|
OpenEnv | Runs each episode in its own OpenEnv session (an environment factory for EnvironmentSource). |
Said | What an observation adds to the model’s context: its ids, and the chat messages they hold. |
Session | One open OpenEnv session, as EnvClient.sync() serves it. |
StepResult | What an OpenEnv session answers a reset or a step with (openenv.core.StepResult). |
OpenEnv
Section titled “OpenEnv”class OpenEnv( session: Callable[[Task, EpisodeId], contextlib.AbstractContextManager[Session[ObservationT, ActionT]]], prompt: Callable[[ObservationT], Said], reply: Callable[[ObservationT], Said], action: Callable[[tuple[int, ...]], ActionT], options: Callable[[Task, EpisodeId], Mapping[str, JSON]] = lambda task, identity: {},)Runs each episode in its own OpenEnv session (an environment factory for EnvironmentSource).
session(task, identity) opens it, options(task, identity) are its
reset’s keyword arguments, and prompt, reply and action are the
adapters the module docstring describes.
class Said(ids: tuple[int, ...], messages: tuple[Mapping[str, object], ...] = ())What an observation adds to the model’s context: its ids, and the chat messages they hold.
Session
Section titled “Session”class Session(Protocol)One open OpenEnv session, as EnvClient.sync() serves it.
Session.reset
Section titled “Session.reset”def reset(**options: JSON = {}) -> StepResult[ObservationT]Session.step
Section titled “Session.step”def step(action: ActionT) -> StepResult[ObservationT]StepResult
Section titled “StepResult”class StepResult(Protocol)What an OpenEnv session answers a reset or a step with (openenv.core.StepResult).