Skip to content

dew.rl.sandbox

Run untrusted programs and tool sessions in bounded Linux subprocesses.

SubprocessEnvironment is an episode environment whose worker runs under the caller’s OS permissions with bounded CPU, memory, time and IO. It has no filesystem or network isolation, so hostile code needs an outer boundary. Dew never selects or executes it by default.

A Program holds the files to write into a fresh temporary directory, an argv to run there without a shell, and the text to feed to its stdin. A runner executes one program under SandboxLimits and reports an Outcome: how it ended, its exit code, its capped output and the seconds it took. SandboxFleet runs programs on workers threads at once, each program in its own process, and a verifiable reward uses it to score completions.

Dew has two runners. ProcessRunner starts the program through the same launcher as SubprocessEnvironment. The program gets RLIMIT_CPU and RLIMIT_AS, no core dumps, its own session, SIGKILL on parent death and a minimal environment, and its process group is killed at the wall deadline. It keeps the caller’s user, filesystem and network, so a program can read and reach whatever you can. ContainerRunner runs the program in a fresh Docker or Podman container with no network, a read-only root, the job directory mounted read-only, no capabilities, an unprivileged user, and limits on memory, CPU, process count and CPU time. Use it for hostile code.

A program that fails returns an Outcome, and the reward decides what a timeout or a crash is worth. A runner raises only when it cannot start a program at all: an OSError when the process cannot start, and an InfrastructureError when a container runtime starts no container or stalls removing one. A session source retries either as an infrastructure failure.

MathReward is a verifiable reward that runs no program. It reads the completion’s final answer and compares it with the record’s reference as an exact rational number.

NameSummary
ContainerRunnerRuns each program in a fresh container of image with no network.
MathRewardScores 1.0 when the final answer equals the reference as a rational number, and 0.0 otherwise.
Outcome
ProcessRunnerRuns each program as a resource-limited Linux process in a temporary directory.
ProgramThe files to write, the argv to run beside them, and the text for its stdin.
RunnerRuns one program to its end under limits, and raises only when it cannot start it.
SandboxFleetRuns programs on workers sandboxed workers at once.
SandboxLimitsLimits for one worker: RLIMIT_CPU and RLIMIT_AS, plus session and IO limits the parent enforces.
SubprocessEnvironmentRuns an episode environment as a JSON-lines worker process that you choose.
VerdictHow a program ended.
outputs_matchReturn whether the program completed and its stdout equals expected.

dataclass source

class ContainerRunner(
image: str,
runtime: str = 'docker',
cpus: float = 1.0,
pids: int = 64,
user: str = '65534:65534',
python: tuple[str, ...] = ('python', '-I', '-S'),
)

Runs each program in a fresh container of image with no network.

runtime is the Docker-compatible CLI (docker or podman). The job directory is mounted read-only at /work, which is the working directory, and /tmp is a small writable tmpfs. The container caps memory at memory_bytes with no swap, CPU time at cpu_seconds (SIGXCPU, then SIGKILL a second later), the processor share at cpus and the process count at pids, and it runs as user. At the wall deadline the runner kills the client process, then force-removes the container by name. A runtime that exits without creating the container (no daemon, no permission, no image), or that stalls removing it, raises InfrastructureError.

python: tuple[str, ...]

The image’s own interpreter, since the host’s path does not exist in the container.

def command(
program: Program,
limits: SandboxLimits,
directory: str,
name: str,
cidfile: str,
) -> list[str]

Return the runtime argv that runs program from directory in a container called name.

The runtime writes the container’s id to cidfile once it creates one.

dataclass source

class MathReward(require_boxed: bool = True)

Scores 1.0 when the final answer equals the reference as a rational number, and 0.0 otherwise.

The answer is the last \boxed{} in the completion. With require_boxed=False, a completion without one uses its last number instead. Integers, decimals, a/b and \frac{a}{b} compare exactly, so 0.5, 1/2 and \frac{1}{2} agree. The reference is the record’s ground_truth; one that is not a number is compared as trimmed text.

dataclass source

class Outcome(
verdict: Verdict,
exit_code: int | None,
stdout: str,
stderr: str,
seconds: float,
)

dataclass source

class ProcessRunner(python: tuple[str, ...] = (sys.executable, '-I', '-S'))

Runs each program as a resource-limited Linux process in a temporary directory.

The runner kills the whole process group at the wall deadline, on excess output, and after a normal exit, so no child the program forked outlives it unless the child left the group on purpose. python is this interpreter, isolated from the environment, site-packages and user site.

dataclass source

class Program(files: Mapping[str, str], command: tuple[str, ...], stdin: str = '')

The files to write, the argv to run beside them, and the text for its stdin.

File names must be relative paths inside the job directory, and command must be a nonempty argv tuple; anything else raises ValueError.

class source

class Runner(Protocol)

Runs one program to its end under limits, and raises only when it cannot start it.

python is the argv that runs a Python file in the place this runner runs programs.

class source

class SandboxFleet(
runner: Runner = _DEFAULT_RUNNER,
*,
limits: SandboxLimits = _DEFAULT_LIMITS,
workers: int = os.cpu_count() or 1,
)

Runs programs on workers sandboxed workers at once.

runner defaults to a ProcessRunner and workers to the CPU count. Use it as a context manager, or call close, which waits for the running programs and cancels the queued ones. submit returns a future at once, and programs wait in a queue while every worker is busy.

def submit(program: Program) -> Future[Outcome]
def run(programs: Iterable[Program]) -> list[Outcome]

Run every program concurrently and return their outcomes in input order.

def close() -> None

dataclass source

class SandboxLimits(
wall_seconds: float = 30.0,
cpu_seconds: int = 10,
memory_bytes: int = 256 * 1024 ** 2,
message_bytes: int = 1024 ** 2,
)

Limits for one worker: RLIMIT_CPU and RLIMIT_AS, plus session and IO limits the parent enforces.

cpu_seconds and memory_bytes set RLIMIT_CPU and RLIMIT_AS. wall_seconds is the wall-clock deadline. message_bytes caps one JSON message to or from a SubprocessEnvironment worker, and a program’s stdout and stderr together.

Forked children inherit the resource limits, which apply to each process separately; they are not a cgroup limit on the group’s total. Killing the process group cleans up the descendants that stay in that group.

dataclass source

class SubprocessEnvironment(
command: tuple[str, ...],
limits: SandboxLimits = SandboxLimits(),
)

Runs an episode environment as a JSON-lines worker process that you choose.

The worker implements reset and step. command is its argv tuple, run without a shell in a temporary working directory. Each request has an operation and either an episode identity or an action record. Each reply holds context (integer ids), status (running, completed, truncated, cancelled or error), an optional string detail, an optional number reward, the step’s reward (Observation.reward), and an optional list messages of the chat messages it added (Observation.messages). It runs on Linux only.

Leaving the session kills the process group, whether the episode succeeded, failed or was cancelled. The worker itself also gets SIGKILL if its parent dies. Neither replaces filesystem and network isolation, and neither stops descendants that deliberately leave the group.

class source

class Verdict(Enum)

How a program ended.

COMPLETED

The program exited with status zero.

FAILED

The program exited with a nonzero status, including after an uncaught exception or a MemoryError.

TIMEOUT

The program ran past the wall deadline or its CPU-time limit and was killed.

CRASHED

The program was killed by a signal it did not ask for.

OUTPUT_LIMIT

The program wrote more than message_bytes to stdout and stderr together and was killed.

function source

def outputs_match(outcome: Outcome, expected: str) -> bool

Return whether the program completed and its stdout equals expected.

Whitespace around the whole output and at the end of each line is ignored.