A racing drone that flies from its camera
A first-person racing drone sees the world through one wide camera and flies by commanding a collective thrust and three body rates. This tutorial trains such a pilot from scratch, reading its camera image, the body rates and rotor speeds the drone senses, and the flight plan’s next three gates, with no demonstrations and no privileged state at flight time: an agent that learns a world model of what its camera sees and how the drone moves, and learns to fly inside that model’s imagination.
It trains on SkyDreamer’s circle of gates from a 64-pixel camera image for a million environment steps, about two hours on one Colab G4. The trained pilot passes 44.4 gates in a 2000-step episode with no crash from Table III’s evaluation starts and 36.2 from held-out training starts, where the untrained policy passes none; the film at the end shows it flying.
The recipe is SkyDreamer’s (Verraest et al., arXiv 2510.14783),
DreamerV3 with an informed world model: its latent learns to predict the drone’s privileged state, which
the policy never reads. Everything here runs through Dew’s public API: a race simulated in JAX and drawn
by a ray tracer, a run described by a record that dew train reads, and an evaluation that flies the
trained policy against the untrained one from the same starts.
An NVIDIA GPU with OptiX draws the camera’s images (HyperLux, the hyperlux extra). The race’s gates, ground and sky are CC0 textures that tools/fpv_assets.py fetches from Poly Haven and builds into the renderer’s assets.
%pip install -q "dewml[cuda13,hyperlux] @ git+https://github.com/AshishKumar4/dew@19860eb66b76325cf15a06fefcd6f51c1f55b018" --extra-index-url https://pypi.dewml.dev/simple!curl -sLO https://raw.githubusercontent.com/AshishKumar4/dew/19860eb66b76325cf15a06fefcd6f51c1f55b018/tools/fpv_assets.py!python fpv_assets.py assets > /dev/nullinstall log
Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done
import timefrom pathlib import Path
import jaximport jax.numpy as jnpimport numpy as np
from dew import figuresfrom dew.rl.hyperlux import HyperLuxRendererfrom dew.rl.skydreamer import TRACKS, Sky, SkyDreamerRace
ASSETS = Path("assets")print(jax.devices()[0])cuda:0
The race
Section titled “The race”SkyDreamerRace is a batch of drones flying SkyDreamer’s task, stepped on the GPU in JAX and seen through
a renderer. Each episode starts at rest a few metres before a gate and asks the drone to pass the track’s
gates in order. An action is four numbers in [-1, 1]: a throttle (0 holds the hover’s thrust) and three
body rates, a share of the drone’s rate limit. The reward is SkyDreamer’s: progress toward the next gate,
a small cost on the body rates, and a bonus for each pass through a gate, larger nearer its centre. An
episode ends where the drone crosses a gate’s plane outside it or touches the ground.
The observation holds the camera’s image and the vectors a real drone senses: its body rates, its
rotors’ speeds (motors) and the flight plan, the next three gates’ positions and headings. It also
holds the privileged state (position, velocity and orientation in the world and in the gate’s frame),
which the world model learns to predict and the policy never sees.
race = SkyDreamerRace(HyperLuxRenderer(16, ASSETS, size=64))seen = race.reset(0)print({name: tuple(value.shape) for name, value in seen.items()})print("tracks:", list(TRACKS), "| action size", race.action_size)
# Eight drones' cameras at their starts, 64 x 64.figures.images([np.asarray(seen["image"][:8])], columns=8, size=96){'image': (16, 64, 64, 3), 'rates': (16, 3), 'motors': (16, 4), 'plan': (16, 24), 'state': (16, 16), 'passed': (16,)}
tracks: ['circle', 'inverted_loop', 'ladder'] | action size 4The recipe as a run
Section titled “The recipe as a run”A run of the kind env (EnvRunConfig) names everything that trains: the environment, the agent’s
networks and objective, the rollout that collects its experience, the optimizer, and the splits each
validation flies. These are SkyDreamer’s settings (Sec III-A):
- a world model of DreamerV3’s
size12m(an RSSM of 2048 units, 32 latents of 16 classes), reading the image through convolutions and the sensed vectors (proprio) through an MLP; - the informed decoder: no reconstruction of the image (a
decoderscale of 0), and heads that predict the privilegedstate, the rates and the rotor speeds, each its width as the race declares it; - the actor and critic of DreamerV3, the imagined returns bootstrapped from the slow critic
(
slow_targets), and a cost on the change of the policy’s mean between imagined steps (smoothness, Eq 11); train_ratio128: each environment step is replayed 128 times;- LaProp at 4e-5 after a 1000-update warmup.
Each validation flies the greedy policy through whole episodes on two splits of its own: evaluation,
every start before the first gate with the gates’ reward area 1 m either side of their centre, where
training’s is 0.8 m (Table III’s evaluation), and
held_out, starts drawn as in training. It logs each split’s mean return, steps and gates passed.
from dew.checkpoints import Keepfrom dew.config import ObjectiveConfig, TrainerConfigfrom dew.objectives.rl.config import EnvConfig, EnvRunConfig, EpisodesConfig, RolloutConfigfrom dew.optim import Linear, OptimConfigfrom dew.registry import ModelConfig
DRONES, EPISODE = 16, 2000renderer = {"class": "dew.rl.hyperlux:HyperLuxRenderer", "fields": {"count": DRONES, "assets": str(ASSETS), "size": 64}}evaluation_sky = {"class": "dew.rl.skydreamer:Sky", "fields": {"anywhere": 0.0, "size": 1.0}}
run = EnvRunConfig( env=EnvConfig("skydreamer", {"renderer": renderer}), model=ModelConfig("dew.objectives.rl.dreamer:WorldModel", { "reads": "image", "proprio": ["rates", "motors", "plan"], "auxiliary": [["state", 256], ["rates", 256], ["motors", 256]], "deter": 2048, "hidden": 256, "stoch": 32, "classes": 16, "blocks": 8, "units": 256, "conv_depth": 16, "dtype": "bfloat16"}), objective=ObjectiveConfig("dew.objectives.rl.dreamer:DreamerV3Objective", {"slow_targets": True, "smoothness": 0.002, "scales": {"decoder": 0.0}}), rollout=RolloutConfig("dreamer", {"batch": 16, "length": 64, "train_ratio": 128.0, "capacity": 2_000_000 // DRONES, "prefill": 64}), optim=OptimConfig(optimizer="laprop", learning_rate=Linear(peak_value=4e-5, warmup_steps=1000, end_value=4e-5)), episodes={ "evaluation": EpisodesConfig(steps=EPISODE, reached=("passed",), env=EnvConfig( "skydreamer", {"renderer": renderer, "sky": evaluation_sky})), "held_out": EpisodesConfig(steps=EPISODE, reached=("passed",), env=EnvConfig( "skydreamer", {"renderer": renderer}))}, trainer=TrainerConfig(name="skydreamer", checkpoint_dir="runs", steps=125_000, log_every=500, eval_every=12_500, checkpoint_every=12_500, keep=Keep(latest=1)))print(run.save("runs/skydreamer"))runs/skydreamer/run.json
The record, runs/skydreamer/run.json, is the whole run: dew train runs/skydreamer/run.json trains the
same run from the command line, and a run stopped part way goes on from its last checkpoint the same way.
Training
Section titled “Training”125,000 updates replay 128 steps for each of the million environment steps the agent takes, about two hours on one NVIDIA RTX PRO 6000 (a Colab G4). The run logs its losses every 500 updates and flies both validation splits every 12,500.
began = time.perf_counter()state = run.prepare().train("skydreamer")print(f"{int(state.step)} updates in {(time.perf_counter() - began) / 60:.0f} min")Experiment_Name: skydreamer Local tracking: /content/t13/runs/skydreamer/tracking
╭─ dew · WorldModel ────────────────────────────────────────────────────────────────────────────────────────────╮ │ 12.9M parameters · on 1 × NVIDIA RTX PRO 6000 Blackwell Server Edition · batch 1 · float32 parameters, │ │ bfloat16 compute │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125000/125000 100% 19.9 step/s 1:49:42 │ │ │ │ train │ │ loss 9.143 ▂▃▂▃▂▁▂▂▂▂▂▂▂▁▂▂▂▃█▂▄█▂▂▁▂▄▂▃▂▃▂▂▃▂▂▂▂▃▂▂▂▃▂▂▁▁▂ ↑ 2.5% │ │ imagined_return 479.5 ▄▁▃▃▃▆▃▅▃▄▂▇▃▇▆▆▆▂▄▅▂▅▇▅▇▄▃█▁▆▅▃█▂▇▆▆▄▅▆▇▇▄▄▇█▇▆ ↑ 10% │ │ loss_continue 0.02594 ▄▅▃▄▄▁▅▄▇▆▅▄▅▃▂▆▅▇▁▆▆▅▂▅▄▄█▂▇▆▆█▆▆▄▃▃▆▄▅▅▆█▆▃▄▄▂ ↓ 16% │ │ loss_dynamics 4.235 ▅█▅▇▅▄▆▅▆▅▅▄▆▃▅▄▄▇█▆▇▇▅▅▃▅▅▄▆▄█▇▃▇▄▅▄▇▇▃▄▃▆▄▄▁▂▅ ↑ 2.0% │ │ loss_motors 0.003900 ▅▄▂▆▃▅▄▆▄▆▄▅▄▆▄▆▂▃▆▄█▇▆▅▄▄▆▄▃▅▆▂▃▆▃▁▃▆█▂▅▆▃▅▂▃▄▂ ↓ 27% │ │ loss_policy -0.008525 ▄▃▃▁▄▆▁▇▃▆▄▆▆▅▅▅▆▇▄▃▂▆▇▁▄▄▃▇▁▄▅▃▃▇▃▄▄▂▅▃▅▆▄▂█▆▆▃ ↓ 12% │ │ loss_rates 0.3102 ▄▄▃▄▅▃▆▄▇▄▃▄▄▃▅▄▄▅█▃▆▆▅▅▅▂▇▄▅▄▃▄▃▆▅▄▃▄▆▄▅▃▃▂▂▂▁▄ ↓ 2.1% │ │ loss_replay_value 1.876 ▅▆▅▆▇▁▅▅▆▄▆▄▇▃▃▃▄▇▅▆▇▇▅▆▂▆▇▃█▆▇▆▄▇▅▆▄▆▅▄▆▄▆▇▄▂▄▃ ↓ 21% │ │ loss_representation 4.235 ▅█▅▇▅▄▆▅▆▅▅▄▆▃▅▄▄▇█▆▇▇▅▅▃▅▅▄▆▄█▇▃▇▄▅▄▇▇▃▄▃▆▄▄▁▂▅ ↑ 2.0% │ │ loss_reward 0.7170 ▅▇▅▆▆▅▂▄▄▄▃▄▄▃▃▂▅▄▅▃▇█▅▂▂▁▄▅▂▄▆▂▁▃▃▂▃▅▅▂▃▂▅▂▂▂▃▆ ↑ 3.7% │ │ loss_state 3.085 ▂▂▁▃▂▂▂▂▂▂▁▁▂▁▂▃▂▂█▂▄█▂▂▁▁▅▂▂▂▃▂▂▃▂▂▂▂▃▁▁▂▂▂▂▁▂▂ ↑ 21% │ │ loss_value 1.488 ▅▇▆▆▆▁▆▄▅▄▇▂▆▂▄▂▄█▆▆█▅▅▇▄█▆▅█▅▆▆▄▆▄▅▅▆▅▅▃▄▆▅▄▃▄▅ ↓ 0.3% │ │ return_scale 555.5 ▁▁▁▂▂▂▄▃▄▄▅▄▄▅▅▅▅▆▇▇▇▇▇█▇██▇▇▇▆▆▇▇▆▆▆▅▆▆▄▄▅▅▄▄▄▂ ↑ 1.4% │ │ data_wait_ms 0.01785 ▂▅▇▄▃▃▅▅▃▄▄▂▃▂▄▆▄▆▅▃▄▂▃▅▄▆██▅▄▃▅▂▂▁▆▅▅▆▅▇▃▄▆▂▄▃▄ ↑ 4.2% │ │ grad_norm 170.1 ▂▃▆▆▄▃▄▅▅▄▃▂▃▃▂▄▃▃▃▂▄▂▄▄▄▃▆▃▆▄█▄▂▆▂▅▂▂▁▂▄▄▃▁▂▂▅▆ ↑ 102% │ │ flops 9.318e+11 compiles 1.000 cache_hits 174.0 cache_misses 2.000 peak_memory_gib 25.41 │ │ throughput │ │ step_time_ms 51.10 ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▂▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ↑ 1.6% │ │ samples_per_sec 313.1 ███████████████████████▁█▇███▇██████████████████ ↓ 1.6% │ │ mfu 3.6% ███████████████████████▁█▇███▇██████████████████ ↓ 0.1 pt │ │ hlo_flops_util 3.5% ███████████████████████▁█▇███▇██████████████████ ↓ 0.1 pt │ │ hfu 3.5% ███████████████████████▁█▇███▇██████████████████ ↓ 0.1 pt │ │ rollout_seconds 8.215 ▂▃▄▄▄▅▄▅▃▅▄▁▄▃▆▅▃▄▅▄▄▁▅█▆▇▅▅▃▄▅▅▅▅▄▆▅▅▄▄▅▄▄▄▅▄▆▅ ↑ 13% │ │ optimizer │ │ accepted 100.0% learning_rate 4.000e-05 │ │ rollout │ │ episode_return nan ▁ │ │ │ │ evaluation (ema) · step 125000 held_out (ema) · step 125000 │ │ return 2,731 ▁▁▂▁▂███▆█ ↑ 203% return 2,094 ▁▁▁▂▂██▇▂▇ ↑ 22× │ │ steps 2,000 ▁▁▁▁▃███▆█ ↑ 116% steps 1,851 ▁▁▂▁▂██▇▁█ ↑ 16× │ │ passed 44.31 ▁▁▁▁▂███▇█ ↑ 104% passed 40.88 ▁▁▁▁▂██▇▂█ ↑ 26× │ ╰────────────────────────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯
╭─ ✓ WorldModel ────────────────────────────────────────────────────────────────────────────────────────────────╮ │ trained 125000 steps in 1:49:48 │ │ first step after 51.72 s │ │ then 19.6 step/s │ │ goodput 96.7% of the wall time in steps │ │ final loss 9.143 │ │ evaluation (ema) at 125000 return 2,731 ↑ 203% ▁▁▂▁▂███▆█ steps 2,000 ↑ 116% ▁▁▁▁▃███▆█ passed 44.31 ↑ │ │ 104% ▁▁▁▁▂███▇█ │ │ held_out (ema) at 125000 return 2,094 ↑ 22× ▁▁▁▂▂██▇▂▇ steps 1,851 ↑ 16× ▁▁▂▁▂██▇▁█ passed 40.88 ↑ 26× │ │ ▁▁▁▁▂██▇▂█ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
125000 updates in 110 min
The matched evaluation
Section titled “The matched evaluation”A score means something beside what the untrained policy earns on the same flights. episodes flies a
policy’s greedy actions (greedy, the policy’s mean) through fresh episodes from a key and scores each
drone’s first episode; flown from one key, the trained and the untrained policies start from the same
places and see the same race.
from dew.rl.evaluation import episodesfrom dew.rl.simulator import DrivenSimulator
agent = run.agent(race)judged = {name: SkyDreamerRace(HyperLuxRenderer(DRONES, ASSETS, size=64), sky) for name, sky in (("evaluation", Sky(anywhere=0.0, size=1.0)), ("held_out", Sky()))}for policy, variables in (("untrained", agent.init(jax.random.key(0))), ("trained", state.variables)): for name, judged_race in judged.items(): score = episodes(agent.greedy, DrivenSimulator(judged_race), EPISODE, memory=agent.memory, reached=("passed",))(variables, jax.random.key(7)) print(f"{policy:>9} {name:>10}: {score['passed']:5.1f} gates, return {score['return']:7.1f}, " f"{score['steps']:6.0f} steps of {EPISODE}")untrained evaluation: 0.0 gates, return -193.7, 515 steps of 2000 untrained held_out: 0.0 gates, return -168.2, 511 steps of 2000 trained evaluation: 44.4 gates, return 2723.7, 2000 steps of 2000 trained held_out: 36.2 gates, return 1859.8, 1646 steps of 2000
Curves
Section titled “Curves”The run’s tracker holds every validation it flew. Each point is the mean over 16 drones of their first episode: the gates they passed and how long they flew before an episode’s end (a crash, as the race ends no episode otherwise before its 2000 steps).
from dew.training.tracker import LocalTracker
logged = LocalTracker("runs/skydreamer/tracking").scalars()splits = ("evaluation", "held_out")# Each update replays 16 windows of 64 steps, 128 for each environment step taken: 8 steps an update.points = {name: [(step * 8, scalars) for step, scalars in logged if f"{name}/passed" in scalars] for name in splits}steps = {name: [step for step, _ in points[name]] for name in splits}display(figures.curve({name: [scalars[f"{name}/passed"] for _, scalars in points[name]] for name in splits}, x=steps, x_label="environment steps", y_label="gates passed in an episode"))display(figures.curve({name: [scalars[f"{name}/steps"] for _, scalars in points[name]] for name in splits}, x=steps, x_label="environment steps", y_label="steps flown of 2000"))The pilot’s view
Section titled “The pilot’s view”The policy flies from 64-pixel images; to watch it, a second race draws the same flights at 256 pixels. Both races start from the same seed, so each step’s action, chosen from the small image, flies the drone in both.
import imageio.v2 as imageiofrom IPython.display import Video
from dew.rl.env import viewfrom dew.rl.simulator import simulator_start
FILMED = 4flying = DrivenSimulator(SkyDreamerRace(HyperLuxRenderer(FILMED, ASSETS, size=64), Sky(anywhere=0.0)))filmed = SkyDreamerRace(HyperLuxRenderer(FILMED, ASSETS, size=256), Sky(anywhere=0.0))key = jax.random.key(11)carry = simulator_start(flying, key, agent.memory(FILMED))seed_key, _ = jax.random.split(key)filmed.reset(int(jax.random.randint(seed_key, (), 0, jnp.iinfo(jnp.int32).max)))step, memory, frames = carry.observation, carry.memory, []for t in range(1800): action, _, memory = agent.greedy(state.variables, step, jax.random.fold_in(key, t), memory) step = flying.step(action).observation frames.append(np.concatenate(list(np.asarray(view(filmed.step(action).observation, "image"))), 1))imageio.mimwrite("pilot.mp4", frames, fps=90, macro_block_size=1)Video("pilot.mp4", embed=True, width=1024)stderr
/usr/lib/python3.13/subprocess.py:1922: RuntimeWarning: os.fork() was called. os.fork() is incompatible with multithreaded code, and JAX is multithreaded, so this will likely lead to a deadlock. self.pid = _fork_exec(