dew.rl.flight
First-person drone gate racing in JAX: one drone’s race as a dew.rl.env.Env (FlightEnv), on device and
jittable whole, flown by dew.rl.quadrotor’s physics; and a batch of them seen through a renderer’s camera
as a dew.rl.simulator.Simulator (GateRace).
The reward is the one drone-racing RL has converged on: each step, the progress made toward the next gate’s
centre (Kaufmann et al., Nature 2023, Swift, Eq 8; Song et al., Sci. Robot. 2023, Eq 5; SkyDreamer, arXiv
2510.14783, Sec II-C), less a cost on the body rates (Song 2023, b = 0.01 a step at 100 Hz, here 1 a second)
and on the action’s rate of change (Swift’s lambda5, here 0.005 a step at 60 Hz as a rate a second); a bonus
on passing a gate, the more for passing it centred (SkyDreamer’s r_gate); a reward for the camera’s optical
axis pointing at the next gate’s centre (Swift’s perception term, Eq 8: lambda2 exp(lambda3 delta_cam^4), the
angle in radians), without which a drone flies its yaw where it likes: a teacher of the privileged views flew
tail first, the next gate out of its camera’s view on 95% of its steps (on a G4, 10-31), a flight no policy of
the camera alone can learn. Its weights are those of Geles et al.’s policy from pixels (RSS 2024, arXiv
2406.12505, Sec III-A4: 0.025 exp(-delta^4) a step, here 1.5 exp(-delta^4) a second, 0.025 a step at 60 Hz),
whose term pays four fifths of its most at a gate 40 degrees off the axis, where Swift’s sharper one (Extended
Data Table 1a: 0.02 exp(-10 delta^4)) pays a tenth: a teacher on Swift’s turned to the gate on 59% of its
steps at the curriculum’s level 0.8; and a crash’s penalty, which ends the episode (Song 2023’s -10). A gate
counts only when its plane is crossed through its opening in its direction; crossing the next gate’s plane
beside it misses it, a crash (SkyDreamer; Ferede et al., arXiv 2504.21586). The ground, the gates’ frames and
the obstacles are solid, the drone’s path through each physics step tested against them (struck), so a fast
drone cannot pass through one between two steps. An open track ends, a success, on passing its last gate.
Each episode is flown on a course of its own (drawn_course), of a family:
layout: a published racing layout (uzh-rpg/agile_flight, MIT: Split-S and its real-world variant, AlphaPilot; and FlyDreamer’s loop), each gate moved by up tojitter(Song et al., IROS 2021, Sec IV-C);loop: a closed loop of gates about an ellipse;chain: Song et al. 2021’s gate primitive chained into an open track, G_{j+1} = f(G_j, p, R) (Eq 6): each gate a relative turn, climb and distance from the last, with split-S (a gate below the last, facing back) and dive primitives; the paper gives no ranges, so these match its tracks’ 110 to 150 m and climbs up to 17 m (Fig 7), and its curriculum from near-straight (Sec III-C.3);figure8: a closed figure-eight whose lobes cross at different heights. Thedifficulty, from 0 to 1, scales the jitter, turns, climbs and primitives, and the obstacles’ number, a curriculum (Song 2021 raised it on the crash rate). Obstacles are vertical pillars beside or on the racing line. An episode starts anywhere on its course: hovering before a random gate (Swift; SkyDreamer’s Table III), or already flying toward it from along the segment before it.
The policy sees the camera’s image alone, the next gate lit (highlight: its colours and pattern drawn per
episode, as AI Grand Prix 2026 lights its gates; FlyDreamer highlights its target): which gate is next is in
the image. The state, the next gates’ corners, the obstacles and the drone’s drawn parameters are privileged,
for a world model’s auxiliary heads (SkyDreamer’s privileged decoding) and a teacher (Swift’s and Song 2023’s
observation with the parameters). An action keeps FlyDreamer’s semantics (env.py _action_to_command): its
first entry is the throttle, zero the drone’s hover thrust, -1 none and 1 the nominal drone’s full thrust;
the rest the body rates over the drone’s rate limit. A command reaches the flight controller after the drone’s
latency (Swift measured 40 ms; SkyDreamer’s 11 ms action delay).
The world frame is x forward, y left, z up (FLU), the body frame forward, left, up.
| Name | Summary |
|---|---|
ASPECTS | A pillar’s height over its radius, one of these: a renderer draws each as its shape scaled. |
FAMILIES | The course families (drawn_course). |
GATES | The most gates a course holds; a shorter one’s last are inactive. |
INTERFACE | The width of the teacher’s observation in its first race: its first entries keep their meaning. |
LAYOUTS | Racing layouts’ gate centres in metres, FLU: uzh-rpg/agile_flight at 03dc1179c011f414aba1264bb6db92679be66b4f (envsim/parameters/tracks/SplitS.yaml, real_track.yaml, AlphaPilot.yaml, MIT) and FlyDreamer’s loop (AshishKumar4/FlyDreamer dreamerv3/configs.yaml static_gates, turned from NED). |
NEAREST | The obstacles a teacher is shown, the nearest first. |
OBSTACLES | The most obstacles a course holds; an inactive one has no radius. |
TEACHER | The width of the teacher’s observation (privileged). |
VIEWS | What a gate race can show: the camera’s image, and the privileged views (privileged). |
Camera | A drone’s camera for an episode: its uptilt and horizontal field of view (radians), its mount’s roll and yaw, and its image’s exposure, white balance [3], gamma, read noise and shutter (seconds); and the looks its renderer draws (look [4]: the gates’, the lit gate’s, the ground’s and the sky’s, each an index the renderer takes modulo its variants). |
Commanded | A drone a flight controller commands (commanded): its drawn body, its rate_limit, its latency in whole control steps (delay) and the commands pending (delayed), a dataclass whose pending a command replaces. |
Course | A course: its gates’ centres [G, 3] and facings [G] (the yaw of their passing direction), how many it holds, whether it is closed (a loop) or open (a track ending at its last gate), its family’s index in FAMILIES, and its obstacles [O, 4], each a vertical pillar’s centre’s x and y, radius and height (an inactive one’s radius 0). |
FlightEnv | One drone racing racing’s courses as a dew.rl.env.Env: actions [4] in [-1, 1] (commanded); observations a dict of the privileged views asked for (privileged). |
GateRace | A batch of renderer.count drones racing racing’s courses (FlightEnv, stepped on device) and seen through renderer’s cameras (Seen): actions [B, 4] in [-1, 1]; observations a dict of the views asked for (VIEWS), each [B, ...]. |
Race | One drone’s race: its course, the gate it heads for and its distance from it after the last step, the steps taken and gates passed; the last action and the commands awaiting their latency [L, 4] (collective thrust in N and body rates in rad/s), its latency in control steps and its rate limit; its drawn body and its flight; its camera; the key its physics draws from; and the curriculum it was drawn at (difficulty, ramp). |
Racing | The race. |
Renderer | A renderer of a batch of count drones’ cameras: render(scene, cameras, opened) answers each drone’s image [B, H, W, 3] uint8 of its Scene from its camera at the shutter’s close [B, 12] (cameras) and at its open (for motion blur). |
Scene | What a renderer draws for each drone: its course’s gates [B, G, 3], facings [B, G] and count [B], the gate lit (lit [B], -1 for none), its obstacles [B, O, 4], and its camera’s looks [B, 4] (Camera.look). |
Seen | A batch of drones of a JAX environment env (stepped on device), seen through renderer’s cameras, as a dew.rl.simulator.Simulator: actions [B, A]; observations a dict of the views asked for, some of those offered by what, each [B, ...], image the renderer’s and the rest env’s, which shows the privileged views asked for (privileged). |
Viewed | A drone’s state a camera sees from (cameras): its camera and its flight. |
airframe | The drone’s parameters a world model learns to read off its latent and a teacher is shown [6]: its thrust-to-weight at its start’s charge over the nominal drone’s, its rate limit over nominal, its motors’ response a control step (one less the decay of their lag over it), its latency in control steps, and its camera’s uptilt and field of view. |
bars | A gate’s four frame bars [4, 3, 2], each a box’s extent along the gate’s normal, its left and up from its centre, grown by the drone’s radius on every side: where the drone’s centre meets the frame. |
cameras | A drone’s camera [12] as HyperLux takes it (set_cameras: the position, the ray to the image’s first pixel’s corner and the rays across a pixel row and down a column), its mount tilted up by the camera’s uptilt and turned by its mount’s roll and yaw, its field of view the camera’s: HyperDrone’s camera_base_one (hyperdrone/_native/dynamics/impl.cpp) of the mounted camera; before seconds earlier, the drone moved back along its velocity and turned back by its body rates (a shutter’s open). |
commanded | A drone’s action [4] in [-1, 1] as the flight controller takes it after the drone’s latency: the collective thrust (N), the throttle’s share of the hover below zero and of the span from the hover to the nominal quadrotor’s full thrust above; the body rates (rad/s), its share of the drone’s rate limit. |
corners | The corners of the openings of the gate headed for and of the ahead - 1 after it (on an open track, its last gate in place of any beyond it), in the body frame [12 * ahead] (Song 2023’s track observation). |
delayed | A command sent after delay control steps: the commands pending [D + 1, ...], newest first, with command pushed, and the one delay steps old, which the flight controller takes now. |
developed | A rendered image [H, W, 3] uint8 as a drone’s camera develops it, in the camera’s dtype: its exposure, white balance and gamma, and read noise of the camera’s std (full scale). |
drawn_camera | A drone’s camera for an episode (Camera), each quantity uniform in its range, the image’s spreads about neutral scaled by ramp; each of the scene’s looks drawn for a ramp share of episodes, else the first, so a curriculum’s first raise varies a tenth of its scenes rather than every one. |
drawn_course | An episode’s course (Course): its family drawn at racing.families’ weights, then its gates (_layout, _loop, _chain, _figure8), facings and obstacles. |
facings | Each gate’s facing [G]: the yaw of the level line from the gate before it to the gate after it, on a closed course of count gates wrapping round, on an open one the first gate’s from itself and the last’s to itself. |
gate_axes | A gate’s normal (its passing direction) and its left [..., 3], level, from its facing. |
looking | The angle in radians between the drone’s camera’s optical axis (its mount’s forward, mounted) and the direction from the drone to point (Swift’s delta_cam). |
mounted | A camera’s mount [3, 3]: its columns the camera’s forward, left and up in the body frame, the body’s rolled by the mount’s roll, tilted up by the camera’s uptilt and turned by the mount’s yaw. |
nearest | The NEAREST active pillars nearest the drone, each its axis’s closest point’s offset in the body frame (at the drone’s height, held within the pillar) and its radius [4 * NEAREST], an absent one zeros. |
physics | The drone’s physics a teacher is shown and a world model learns to read off [20], each over its nominal: its mass, inertia [3], thrust and drag coefficients and top speed; its rotor, quadratic [2] and vertical drag; its ground effect’s strength; its battery’s voltage now over full and its charge drawn over the pack’s; and its constant force over its mass times the disturbance’s limit and its moment over the nominal inertia times its limit [3] each, within [-1, 1]. |
privileged | The privileged views: the state [13] (position, velocity, orientation, body rates), the next two gates’ corners in the body frame [24], the drone’s airframe [6] and physics [20], the obstacles nearest [12], the gates passed so far, whether the gate headed for is an open track’s last (finishing), and the teacher’s observation [80]: the velocity and the direction of gravity in the body frame, the body rates, the corners, the last action and the airframe (the first 43, its interface since FlyDreamer’s first race), then the physics, the obstacles and the finishing flag, each about unit size. |
quaternion | Unit quaternions (w, x, y, z) of a yaw, then a pitch, then a roll. |
raced | A step’s race, reward, and whether its episode terminated or was truncated, from the action taken and the drone’s path through the step to its flight after it. |
shown | Refuses views but where they are some of the views what offered. |
started | A drone’s episode (Race) at a curriculum’s difficulty and ramp: its course, its start on it, its drawn body (dew.rl.quadrotor.drawn_body) and camera, its latency and rate limit; its rotors at the speed that holds its weight, its commands awaiting their latency its hover; no pillar within 1.5 m of its start. |
struck | Whether a drone’s path through a step [S + 1, 3] (its positions before it and after each physics step, each pair joined by a straight segment) crashed, passed the gate it heads for, and how far across from its centre it passed. |
ASPECTS
Section titled “ASPECTS”ASPECTS = (6.0, 12.0, 24.0)A pillar’s height over its radius, one of these: a renderer draws each as its shape scaled.
FAMILIES
Section titled “FAMILIES”FAMILIES = ('layout', 'loop', 'chain', 'figure8')The course families (drawn_course).
GATES = 16The most gates a course holds; a shorter one’s last are inactive.
INTERFACE
Section titled “INTERFACE”INTERFACE = 43The width of the teacher’s observation in its first race: its first entries keep their meaning.
LAYOUTS
Section titled “LAYOUTS”LAYOUTSRacing layouts’ gate centres in metres, FLU: uzh-rpg/agile_flight at
03dc1179c011f414aba1264bb6db92679be66b4f (envsim/parameters/tracks/SplitS.yaml, real_track.yaml,
AlphaPilot.yaml, MIT) and FlyDreamer’s loop (AshishKumar4/FlyDreamer dreamerv3/configs.yaml static_gates,
turned from NED). Each gate faces along the course (facings): the layouts’ own gate rotations follow their
simulator’s model’s axes, which the files do not state.
NEAREST
Section titled “NEAREST”NEAREST = 3The obstacles a teacher is shown, the nearest first.
OBSTACLES
Section titled “OBSTACLES”OBSTACLES = 8The most obstacles a course holds; an inactive one has no radius.
TEACHER
Section titled “TEACHER”TEACHER = 43 + 20 + 4 * NEAREST + 1The width of the teacher’s observation (privileged).
VIEWS = ('image', 'state', 'gates', 'airframe', 'physics', 'obstacles', 'passed', 'finishing', 'teacher')What a gate race can show: the camera’s image, and the privileged views (privileged).
Camera
Section titled “Camera”class Camera()A drone’s camera for an episode: its uptilt and horizontal field of view (radians), its mount’s roll
and yaw, and its image’s exposure, white balance [3], gamma, read noise and shutter (seconds); and the
looks its renderer draws (look [4]: the gates’, the lit gate’s, the ground’s and the sky’s, each an
index the renderer takes modulo its variants).
Commanded
Section titled “Commanded”class Commanded(Protocol)A drone a flight controller commands (commanded): its drawn body, its rate_limit, its latency in
whole control steps (delay) and the commands pending (delayed), a dataclass whose pending a
command replaces.
Course
Section titled “Course”class Course()A course: its gates’ centres [G, 3] and facings [G] (the yaw of their passing direction), how many
it holds, whether it is closed (a loop) or open (a track ending at its last gate), its family’s index in
FAMILIES, and its obstacles [O, 4], each a vertical pillar’s centre’s x and y, radius and height (an
inactive one’s radius 0).
FlightEnv
Section titled “FlightEnv”class FlightEnv( racing: Racing = struct.field(pytree_node=False), views: tuple[str, ...] = struct.field(pytree_node=False, default=('teacher',)), difficulty: float = struct.field(pytree_node=False, default=1.0), ramp: float = struct.field(pytree_node=False, default=1.0),)One drone racing racing’s courses as a dew.rl.env.Env: actions [4] in [-1, 1] (commanded);
observations a dict of the privileged views asked for (privileged). An episode is drawn at its
curriculum, difficulty and ramp (started); a curriculum changes the env, its programs compiled again
(a few times a run).
FlightEnv.observed
Section titled “FlightEnv.observed”def observed(race: Race) -> dict[str, jax.Array]The privileged views of race asked for (privileged).
FlightEnv.scene
Section titled “FlightEnv.scene”def scene(race: Race) -> SceneWhat a renderer draws of race (_scene).
FlightEnv.reset
Section titled “FlightEnv.reset”def reset(key: jax.Array) -> tuple[Race, TimeStep]FlightEnv.flown
Section titled “FlightEnv.flown”def flown(race: Race, action: jax.Array) -> tuple[Race, jax.Array]The race after the drone flies a control step under action, and its path through it.
FlightEnv.step
Section titled “FlightEnv.step”def step(race: Race, action: jax.Array) -> tuple[Race, TimeStep]GateRace
Section titled “GateRace”class GateRace( renderer: Renderer | None, racing: Racing | None = None, views: tuple[str, ...] = VIEWS, count: int | None = None, difficulty: float = 1.0, ramp: float = 1.0,)A batch of renderer.count drones racing racing’s courses (FlightEnv, stepped on device) and seen
through renderer’s cameras (Seen): actions [B, 4] in [-1, 1]; observations a dict of the views
asked for (VIEWS), each [B, ...]. An episode restarts on a course drawn afresh at the race’s
curriculum, difficulty and ramp, which a race starts at and which may change between steps.
class Race()One drone’s race: its course, the gate it heads for and its distance from it after the last step, the
steps taken and gates passed; the last action and the commands awaiting their latency [L, 4]
(collective thrust in N and body rates in rad/s), its latency in control steps and its rate limit; its
drawn body and its flight; its camera; the key its physics draws from; and the curriculum it was
drawn at (difficulty, ramp).
Racing
Section titled “Racing”class Racing( families: tuple[tuple[str, float], ...] = (('loop', 0.4), ('chain', 0.6)), layouts: tuple[str, ...] = tuple(LAYOUTS), jitter: tuple[float, float] = (1.5, 0.75), obstacles: int = OBSTACLES, on_line: float = 0.3, hz: float = 60.0, episode_seconds: float = 25.0, opening: float = 1.5, frame: float = 0.25, depth: float = 0.1, radius: float = 0.2, progress_scale: float = 1.0, rate_cost: float = 1.0, smoothness_cost: float = 0.005 / 60, gate_bonus: float = 6.0, perception_scale: float = 1.5, perception_sharpness: float = 1.0, crash_penalty: float = 10.0, miss_radius: float = 3.0, ceiling: float = 25.0, rate_limit: float = 10.0, rate_spread: float = 0.2, latency: tuple[float, float] = (0.01, 0.06), uptilt: tuple[float, float] = (30.0, 30.0), fov: float = 100.0, fov_spread: float = 0.0, mount_spread: float = 0.0, exposure: tuple[float, float] = (0.5, 1.5), white_balance: float = 0.1, gamma: tuple[float, float] = (0.8, 1.25), noise: tuple[float, float] = (0.0, 0.03), shutter: tuple[float, float] = (0.002, 0.008), highlight: bool = True, flying_share: float = 0.5, start_speed: float = 8.0, quadrotor: Quadrotor = Quadrotor(),)The race. families weighs the course families drawn (FAMILIES), layouts names the layouts a
layout course is drawn from, jitter the most a layout’s gate moves across and up at full difficulty;
obstacles the most obstacles a course holds at full difficulty and on_line the share placed on the
racing line. The reward’s weights are raced’s; hz the control rate, 60 Hz (a teacher at 60 Hz passed
27% more gates than at 30 in the same time; Swift’s and Song’s run at 90 to 100 Hz); an episode is cut
after episode_seconds (Swift’s 1500 steps). Gates’ openings are opening square (1.5 m, SkyDreamer’s
and Ferede’s) in a frame frame wide and depth deep; the drone collides as a cube of half-width
radius aligned with the gate it meets and as a sphere of radius with the ground and obstacles
(struck). rate_limit is the nominal body-rate command’s limit and rate_spread its randomization,
latency the command latency’s range in seconds; the camera’s uptilt above the body’s forward axis in
degrees, its horizontal field of view fov and fov_spread, its mount’s mount_spread, by default one
fixed geometry, 30 degrees up and 100 across, as a real FPV quadrotor’s camera is mounted; the image’s
exposure, white_balance, gamma, noise (read noise’s std, of full scale) and shutter (seconds)
ranges; highlight whether the next gate is lit; flying_share the share of starts already flying and
start_speed their fastest; and the quadrotor flown.
max_steps: int-
The steps an episode is cut after.
delays: tuple[int, int]-
The latency’s range in whole control steps, each rounded; none where
latencyis (0, 0).
Renderer
Section titled “Renderer”class Renderer(Protocol)A renderer of a batch of count drones’ cameras: render(scene, cameras, opened) answers each drone’s
image [B, H, W, 3] uint8 of its Scene from its camera at the shutter’s close [B, 12] (cameras)
and at its open (for motion blur). Its aspect is its images’ width over height, and gate the opening,
frame and depth of the gates it draws, which a race’s must be. Arrays cross by DLPack
(dew.rl.simulator.Handed).
Renderer.render
Section titled “Renderer.render”def render(scene: Scene, cameras: jax.Array, opened: jax.Array) -> Handedclass Scene()What a renderer draws for each drone: its course’s gates [B, G, 3], facings [B, G] and count [B],
the gate lit (lit [B], -1 for none), its obstacles [B, O, 4], and its camera’s looks [B, 4]
(Camera.look).
class Seen( renderer: Renderer | None, env, views: tuple[str, ...], offered: tuple[str, ...], what: str, count: int | None = None,)A batch of drones of a JAX environment env (stepped on device), seen through renderer’s cameras,
as a dew.rl.simulator.Simulator: actions [B, A]; observations a dict of the views asked for, some
of those offered by what, each [B, ...], image the renderer’s and the rest env’s, which shows
the privileged views asked for (privileged). env steps one drone (reset, step) and shows its
views (observed, its states Viewed) and its scene (scene). The batch is the renderer’s, or count
drones where there is none, and then no image.
Seen.privileged
Section titled “Seen.privileged”def privileged(views: tuple[str, ...]) -> tuple[str, ...]The views of views an environment shows, all but the renderer’s image (the gates passed where
there are none).
Seen.reset
Section titled “Seen.reset”def reset(seed: int) -> dict[str, jax.Array]Seen.restart
Section titled “Seen.restart”def restart(mask: jax.Array) -> dict[str, jax.Array]Seen.step
Section titled “Seen.step”def step(action: jax.Array) -> TimeStepSeen.branch
Section titled “Seen.branch”def branch() -> Iterator[None]A cooldown branch’s steps (dew.rl.simulator.branched), after which every drone’s state and the
key its images’ noise is drawn from stand as the run left them: no step donates them, so keeping
them keeps them.
Viewed
Section titled “Viewed”class Viewed(Protocol)A drone’s state a camera sees from (cameras): its camera and its flight.
airframe
Section titled “airframe”def airframe(racing: Racing, race: Race) -> jax.ArrayThe drone’s parameters a world model learns to read off its latent and a teacher is shown [6]: its
thrust-to-weight at its start’s charge over the nominal drone’s, its rate limit over nominal, its motors’
response a control step (one less the decay of their lag over it), its latency in control steps, and its
camera’s uptilt and field of view.
def bars(racing: Racing) -> np.ndarrayA gate’s four frame bars [4, 3, 2], each a box’s extent along the gate’s normal, its left and up from
its centre, grown by the drone’s radius on every side: where the drone’s centre meets the frame.
cameras
Section titled “cameras”def cameras(race: Viewed, aspect: float, before: float = 0.0) -> jax.ArrayA drone’s camera [12] as HyperLux takes it (set_cameras: the position, the ray to the image’s first
pixel’s corner and the rays across a pixel row and down a column), its mount tilted up by the camera’s
uptilt and turned by its mount’s roll and yaw, its field of view the camera’s: HyperDrone’s
camera_base_one (hyperdrone/_native/dynamics/impl.cpp) of the mounted camera; before seconds earlier,
the drone moved back along its velocity and turned back by its body rates (a shutter’s open).
commanded
Section titled “commanded”def commanded( quadrotor: Quadrotor, drone: D, action: jax.Array,) -> tuple[D, jax.Array, jax.Array]A drone’s action [4] in [-1, 1] as the flight controller takes it after the drone’s latency: the
collective thrust (N), the throttle’s share of the hover below zero and of the span from the hover to the
nominal quadrotor’s full thrust above; the body rates (rad/s), its share of the drone’s rate limit. The
drone, a gate race’s (Race) or any other with its fields, comes back with the command pushed.
corners
Section titled “corners”def corners(racing: Racing, race: Race, ahead: int = 2) -> jax.ArrayThe corners of the openings of the gate headed for and of the ahead - 1 after it (on an open track,
its last gate in place of any beyond it), in the body frame [12 * ahead] (Song 2023’s track
observation).
delayed
Section titled “delayed”def delayed( pending: jax.Array, delay: jax.Array, command: jax.Array,) -> tuple[jax.Array, jax.Array]A command sent after delay control steps: the commands pending [D + 1, ...], newest first, with
command pushed, and the one delay steps old, which the flight controller takes now.
developed
Section titled “developed”def developed(image: jax.Array, camera: Camera, key: jax.Array) -> jax.ArrayA rendered image [H, W, 3] uint8 as a drone’s camera develops it, in the camera’s dtype: its
exposure, white balance and gamma, and read noise of the camera’s std (full scale).
drawn_camera
Section titled “drawn_camera”def drawn_camera(racing: Racing, key: jax.Array, ramp: jax.Array) -> CameraA drone’s camera for an episode (Camera), each quantity uniform in its range, the image’s spreads
about neutral scaled by ramp; each of the scene’s looks drawn for a ramp share of episodes, else the
first, so a curriculum’s first raise varies a tenth of its scenes rather than every one.
drawn_course
Section titled “drawn_course”def drawn_course(racing: Racing, key: jax.Array, difficulty: jax.Array) -> CourseAn episode’s course (Course): its family drawn at racing.families’ weights, then its gates
(_layout, _loop, _chain, _figure8), facings and obstacles.
facings
Section titled “facings”def facings(center: jax.Array, count: jax.Array, closed: jax.Array) -> jax.ArrayEach gate’s facing [G]: the yaw of the level line from the gate before it to the gate after it, on a
closed course of count gates wrapping round, on an open one the first gate’s from itself and the last’s
to itself.
gate_axes
Section titled “gate_axes”def gate_axes(facing: jax.Array) -> tuple[jax.Array, jax.Array]A gate’s normal (its passing direction) and its left [..., 3], level, from its facing.
looking
Section titled “looking”def looking(race: Race, point: jax.Array) -> jax.ArrayThe angle in radians between the drone’s camera’s optical axis (its mount’s forward, mounted) and
the direction from the drone to point (Swift’s delta_cam).
mounted
Section titled “mounted”def mounted(camera: Camera) -> jax.ArrayA camera’s mount [3, 3]: its columns the camera’s forward, left and up in the body frame, the body’s
rolled by the mount’s roll, tilted up by the camera’s uptilt and turned by the mount’s yaw.
nearest
Section titled “nearest”def nearest(race: Race) -> jax.ArrayThe NEAREST active pillars nearest the drone, each its axis’s closest point’s offset in the body
frame (at the drone’s height, held within the pillar) and its radius [4 * NEAREST], an absent one
zeros.
physics
Section titled “physics”def physics(racing: Racing, race: Race) -> jax.ArrayThe drone’s physics a teacher is shown and a world model learns to read off [20], each over its
nominal: its mass, inertia [3], thrust and drag coefficients and top speed; its rotor, quadratic [2]
and vertical drag; its ground effect’s strength; its battery’s voltage now over full and its charge drawn
over the pack’s; and its constant force over its mass times the disturbance’s limit and its moment over
the nominal inertia times its limit [3] each, within [-1, 1]. A term a quadrotor turns off (its nominal
or limit zero, so every drone’s zero too) reads as the nominal’s, 1, and an absent disturbance as 0.
privileged
Section titled “privileged”def privileged(racing: Racing, race: Race) -> dict[str, jax.Array]The privileged views: the state [13] (position, velocity, orientation, body rates), the next two
gates’ corners in the body frame [24], the drone’s airframe [6] and physics [20], the
obstacles nearest [12], the gates passed so far, whether the gate headed for is an open track’s
last (finishing), and the teacher’s observation [80]: the velocity and the direction of gravity in
the body frame, the body rates, the corners, the last action and the airframe (the first 43, its
interface since FlyDreamer’s first race), then the physics, the obstacles and the finishing flag, each
about unit size.
quaternion
Section titled “quaternion”def quaternion(roll: jax.Array, pitch: jax.Array, yaw: jax.Array) -> jax.ArrayUnit quaternions (w, x, y, z) of a yaw, then a pitch, then a roll.
def raced( racing: Racing, race: Race, action: jax.Array, path: jax.Array,) -> tuple[Race, jax.Array, jax.Array, jax.Array]A step’s race, reward, and whether its episode terminated or was truncated, from the action taken
and the drone’s path through the step to its flight after it. The reward is the progress toward the
gate headed for, its distance before the step less its distance after (times progress_scale), less
rate_cost times the body rates’ norm over the step, less smoothness_cost times the action’s
squared rate of change over the step (its squared change over the step’s seconds, a rate a second);
plus perception_scale times exp(-perception_sharpness delta^4) over the step (a rate a second,
times its seconds), delta the angle in radians between the camera’s optical axis and the centre of the
gate headed for after the step (looking); on passing that gate, gate_bonus times one less its
offset’s share of half the opening, the drone heading for the next; on a crash (struck), less
crash_penalty, which ends the episode. Passing an open track’s last gate ends it, a success. An
episode is cut after max_steps steps.
def shown(views: tuple[str, ...], offered: tuple[str, ...], what: str) -> NoneRefuses views but where they are some of the views what offered.
started
Section titled “started”def started( racing: Racing, key: jax.Array, difficulty: jax.Array, ramp: jax.Array,) -> RaceA drone’s episode (Race) at a curriculum’s difficulty and ramp: its course, its start on it, its
drawn body (dew.rl.quadrotor.drawn_body) and camera, its latency and rate limit; its rotors at the
speed that holds its weight, its commands awaiting their latency its hover; no pillar within 1.5 m of
its start.
struck
Section titled “struck”def struck( racing: Racing, race: Race, path: jax.Array,) -> tuple[jax.Array, jax.Array, jax.Array]Whether a drone’s path through a step [S + 1, 3] (its positions before it and after each physics
step, each pair joined by a straight segment) crashed, passed the gate it heads for, and how far across
from its centre it passed. It crashes on coming within radius of the ground or climbing past ceiling,
on a segment entering a gate’s frame bar grown by radius (bars), on coming within radius of a
pillar, or on crossing the next gate’s plane in its direction outside the frame within miss_radius (a
missed gate). It passes that gate on crossing its plane in its direction with its centre within radius
of the opening’s inside, the larger of its offsets across and up from the centre no more than half the
opening less radius.