Skip to content

A racing drone that flies from its camera

A first-person racing drone sees the world through one wide camera and flies by commanding a collective thrust and three body rates. This tutorial trains such a pilot from scratch, reading its camera image, the body rates and rotor speeds the drone senses, and the flight plan’s next three gates, with no demonstrations and no privileged state at flight time: an agent that learns a world model of what its camera sees and how the drone moves, and learns to fly inside that model’s imagination.

It trains on SkyDreamer’s circle of gates from a 64-pixel camera image for a million environment steps, about two hours on one Colab G4. The trained pilot passes 44.4 gates in a 2000-step episode with no crash from Table III’s evaluation starts and 36.2 from held-out training starts, where the untrained policy passes none; the film at the end shows it flying.

The recipe is SkyDreamer’s (Verraest et al., arXiv 2510.14783), DreamerV3 with an informed world model: its latent learns to predict the drone’s privileged state, which the policy never reads. Everything here runs through Dew’s public API: a race simulated in JAX and drawn by a ray tracer, a run described by a record that dew train reads, and an evaluation that flies the trained policy against the untrained one from the same starts.

An NVIDIA GPU with OptiX draws the camera’s images (HyperLux, the hyperlux extra). The race’s gates, ground and sky are CC0 textures that tools/fpv_assets.py fetches from Poly Haven and builds into the renderer’s assets.

%pip install -q "dewml[cuda13,hyperlux] @ git+https://github.com/AshishKumar4/dew@19860eb66b76325cf15a06fefcd6f51c1f55b018" --extra-index-url https://pypi.dewml.dev/simple
!curl -sLO https://raw.githubusercontent.com/AshishKumar4/dew/19860eb66b76325cf15a06fefcd6f51c1f55b018/tools/fpv_assets.py
!python fpv_assets.py assets > /dev/null
install log
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
import time
from pathlib import Path
import jax
import jax.numpy as jnp
import numpy as np
from dew import figures
from dew.rl.hyperlux import HyperLuxRenderer
from dew.rl.skydreamer import TRACKS, Sky, SkyDreamerRace
ASSETS = Path("assets")
print(jax.devices()[0])
cuda:0

SkyDreamerRace is a batch of drones flying SkyDreamer’s task, stepped on the GPU in JAX and seen through a renderer. Each episode starts at rest a few metres before a gate and asks the drone to pass the track’s gates in order. An action is four numbers in [-1, 1]: a throttle (0 holds the hover’s thrust) and three body rates, a share of the drone’s rate limit. The reward is SkyDreamer’s: progress toward the next gate, a small cost on the body rates, and a bonus for each pass through a gate, larger nearer its centre. An episode ends where the drone crosses a gate’s plane outside it or touches the ground.

The observation holds the camera’s image and the vectors a real drone senses: its body rates, its rotors’ speeds (motors) and the flight plan, the next three gates’ positions and headings. It also holds the privileged state (position, velocity and orientation in the world and in the gate’s frame), which the world model learns to predict and the policy never sees.

race = SkyDreamerRace(HyperLuxRenderer(16, ASSETS, size=64))
seen = race.reset(0)
print({name: tuple(value.shape) for name, value in seen.items()})
print("tracks:", list(TRACKS), "| action size", race.action_size)
# Eight drones' cameras at their starts, 64 x 64.
figures.images([np.asarray(seen["image"][:8])], columns=8, size=96)
{'image': (16, 64, 64, 3), 'rates': (16, 3), 'motors': (16, 4), 'plan': (16, 24), 'state': (16, 16), 'passed': (16,)}
tracks: ['circle', 'inverted_loop', 'ladder'] | action size 4
8 images

A run of the kind env (EnvRunConfig) names everything that trains: the environment, the agent’s networks and objective, the rollout that collects its experience, the optimizer, and the splits each validation flies. These are SkyDreamer’s settings (Sec III-A):

  • a world model of DreamerV3’s size12m (an RSSM of 2048 units, 32 latents of 16 classes), reading the image through convolutions and the sensed vectors (proprio) through an MLP;
  • the informed decoder: no reconstruction of the image (a decoder scale of 0), and heads that predict the privileged state, the rates and the rotor speeds, each its width as the race declares it;
  • the actor and critic of DreamerV3, the imagined returns bootstrapped from the slow critic (slow_targets), and a cost on the change of the policy’s mean between imagined steps (smoothness, Eq 11);
  • train_ratio 128: each environment step is replayed 128 times;
  • LaProp at 4e-5 after a 1000-update warmup.

Each validation flies the greedy policy through whole episodes on two splits of its own: evaluation, every start before the first gate with the gates’ reward area 1 m either side of their centre, where training’s is 0.8 m (Table III’s evaluation), and held_out, starts drawn as in training. It logs each split’s mean return, steps and gates passed.

from dew.checkpoints import Keep
from dew.config import ObjectiveConfig, TrainerConfig
from dew.objectives.rl.config import EnvConfig, EnvRunConfig, EpisodesConfig, RolloutConfig
from dew.optim import Linear, OptimConfig
from dew.registry import ModelConfig
DRONES, EPISODE = 16, 2000
renderer = {"class": "dew.rl.hyperlux:HyperLuxRenderer",
"fields": {"count": DRONES, "assets": str(ASSETS), "size": 64}}
evaluation_sky = {"class": "dew.rl.skydreamer:Sky", "fields": {"anywhere": 0.0, "size": 1.0}}
run = EnvRunConfig(
env=EnvConfig("skydreamer", {"renderer": renderer}),
model=ModelConfig("dew.objectives.rl.dreamer:WorldModel", {
"reads": "image", "proprio": ["rates", "motors", "plan"],
"auxiliary": [["state", 256], ["rates", 256], ["motors", 256]],
"deter": 2048, "hidden": 256, "stoch": 32, "classes": 16, "blocks": 8, "units": 256,
"conv_depth": 16, "dtype": "bfloat16"}),
objective=ObjectiveConfig("dew.objectives.rl.dreamer:DreamerV3Objective",
{"slow_targets": True, "smoothness": 0.002, "scales": {"decoder": 0.0}}),
rollout=RolloutConfig("dreamer", {"batch": 16, "length": 64, "train_ratio": 128.0,
"capacity": 2_000_000 // DRONES, "prefill": 64}),
optim=OptimConfig(optimizer="laprop",
learning_rate=Linear(peak_value=4e-5, warmup_steps=1000, end_value=4e-5)),
episodes={
"evaluation": EpisodesConfig(steps=EPISODE, reached=("passed",), env=EnvConfig(
"skydreamer", {"renderer": renderer, "sky": evaluation_sky})),
"held_out": EpisodesConfig(steps=EPISODE, reached=("passed",), env=EnvConfig(
"skydreamer", {"renderer": renderer}))},
trainer=TrainerConfig(name="skydreamer", checkpoint_dir="runs", steps=125_000, log_every=500,
eval_every=12_500, checkpoint_every=12_500, keep=Keep(latest=1)))
print(run.save("runs/skydreamer"))
runs/skydreamer/run.json

The record, runs/skydreamer/run.json, is the whole run: dew train runs/skydreamer/run.json trains the same run from the command line, and a run stopped part way goes on from its last checkpoint the same way.

125,000 updates replay 128 steps for each of the million environment steps the agent takes, about two hours on one NVIDIA RTX PRO 6000 (a Colab G4). The run logs its losses every 500 updates and flies both validation splits every 12,500.

began = time.perf_counter()
state = run.prepare().train("skydreamer")
print(f"{int(state.step)} updates in {(time.perf_counter() - began) / 60:.0f} min")
Experiment_Name: skydreamer
Local tracking: /content/t13/runs/skydreamer/tracking
╭─  dew · WorldModel  ────────────────────────────────────────────────────────────────────────────────────────────╮
│ 12.9M parameters  ·  on 1 × NVIDIA RTX PRO 6000 Blackwell Server Edition  ·  batch 1  ·  float32 parameters,    │
│ bfloat16 compute                                                                                                │
│                                                                                                                 │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  125000/125000  100%  19.9 step/s  1:49:42 │
│                                                                                                                 │
│ train                                                                                                           │
│   loss                        9.143  ▂▃▂▃▂▁▂▂▂▂▂▂▂▁▂▂▂▃█▂▄█▂▂▁▂▄▂▃▂▃▂▂▃▂▂▂▂▃▂▂▂▃▂▂▁▁▂       ↑ 2.5%              │
│   imagined_return             479.5  ▄▁▃▃▃▆▃▅▃▄▂▇▃▇▆▆▆▂▄▅▂▅▇▅▇▄▃█▁▆▅▃█▂▇▆▆▄▅▆▇▇▄▄▇█▇▆        ↑ 10%              │
│   loss_continue             0.02594  ▄▅▃▄▄▁▅▄▇▆▅▄▅▃▂▆▅▇▁▆▆▅▂▅▄▄█▂▇▆▆█▆▆▄▃▃▆▄▅▅▆█▆▃▄▄▂        ↓ 16%              │
│   loss_dynamics               4.235  ▅█▅▇▅▄▆▅▆▅▅▄▆▃▅▄▄▇█▆▇▇▅▅▃▅▅▄▆▄█▇▃▇▄▅▄▇▇▃▄▃▆▄▄▁▂▅       ↑ 2.0%              │
│   loss_motors              0.003900  ▅▄▂▆▃▅▄▆▄▆▄▅▄▆▄▆▂▃▆▄█▇▆▅▄▄▆▄▃▅▆▂▃▆▃▁▃▆█▂▅▆▃▅▂▃▄▂        ↓ 27%              │
│   loss_policy             -0.008525  ▄▃▃▁▄▆▁▇▃▆▄▆▆▅▅▅▆▇▄▃▂▆▇▁▄▄▃▇▁▄▅▃▃▇▃▄▄▂▅▃▅▆▄▂█▆▆▃        ↓ 12%              │
│   loss_rates                 0.3102  ▄▄▃▄▅▃▆▄▇▄▃▄▄▃▅▄▄▅█▃▆▆▅▅▅▂▇▄▅▄▃▄▃▆▅▄▃▄▆▄▅▃▃▂▂▂▁▄       ↓ 2.1%              │
│   loss_replay_value           1.876  ▅▆▅▆▇▁▅▅▆▄▆▄▇▃▃▃▄▇▅▆▇▇▅▆▂▆▇▃█▆▇▆▄▇▅▆▄▆▅▄▆▄▆▇▄▂▄▃        ↓ 21%              │
│   loss_representation         4.235  ▅█▅▇▅▄▆▅▆▅▅▄▆▃▅▄▄▇█▆▇▇▅▅▃▅▅▄▆▄█▇▃▇▄▅▄▇▇▃▄▃▆▄▄▁▂▅       ↑ 2.0%              │
│   loss_reward                0.7170  ▅▇▅▆▆▅▂▄▄▄▃▄▄▃▃▂▅▄▅▃▇█▅▂▂▁▄▅▂▄▆▂▁▃▃▂▃▅▅▂▃▂▅▂▂▂▃▆       ↑ 3.7%              │
│   loss_state                  3.085  ▂▂▁▃▂▂▂▂▂▂▁▁▂▁▂▃▂▂█▂▄█▂▂▁▁▅▂▂▂▃▂▂▃▂▂▂▂▃▁▁▂▂▂▂▁▂▂        ↑ 21%              │
│   loss_value                  1.488  ▅▇▆▆▆▁▆▄▅▄▇▂▆▂▄▂▄█▆▆█▅▅▇▄█▆▅█▅▆▆▄▆▄▅▅▆▅▅▃▄▆▅▄▃▄▅       ↓ 0.3%              │
│   return_scale                555.5  ▁▁▁▂▂▂▄▃▄▄▅▄▄▅▅▅▅▆▇▇▇▇▇█▇██▇▇▇▆▆▇▇▆▆▆▅▆▆▄▄▅▅▄▄▄▂       ↑ 1.4%              │
│   data_wait_ms              0.01785  ▂▅▇▄▃▃▅▅▃▄▄▂▃▂▄▆▄▆▅▃▄▂▃▅▄▆██▅▄▃▅▂▂▁▆▅▅▆▅▇▃▄▆▂▄▃▄       ↑ 4.2%              │
│   grad_norm                   170.1  ▂▃▆▆▄▃▄▅▅▄▃▂▃▃▂▄▃▃▃▂▄▂▄▄▄▃▆▃▆▄█▄▂▆▂▅▂▂▁▂▄▄▃▁▂▂▅▆       ↑ 102%              │
│   flops 9.318e+11   compiles 1.000   cache_hits 174.0   cache_misses 2.000   peak_memory_gib 25.41              │
│ throughput                                                                                                      │
│   step_time_ms                51.10  ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▂▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁       ↑ 1.6%              │
│   samples_per_sec             313.1  ███████████████████████▁█▇███▇██████████████████       ↓ 1.6%              │
│   mfu                          3.6%  ███████████████████████▁█▇███▇██████████████████     ↓ 0.1 pt              │
│   hlo_flops_util               3.5%  ███████████████████████▁█▇███▇██████████████████     ↓ 0.1 pt              │
│   hfu                          3.5%  ███████████████████████▁█▇███▇██████████████████     ↓ 0.1 pt              │
│   rollout_seconds             8.215  ▂▃▄▄▄▅▄▅▃▅▄▁▄▃▆▅▃▄▅▄▄▁▅█▆▇▅▅▃▄▅▅▅▅▄▆▅▅▄▄▅▄▄▄▅▄▆▅        ↑ 13%              │
│ optimizer                                                                                                       │
│   accepted 100.0%   learning_rate 4.000e-05                                                                     │
│ rollout                                                                                                         │
│   episode_return                nan  ▁                                                                          │
│                                                                                                                 │
│ evaluation (ema) · step 125000                           held_out (ema) · step 125000                           │
│   return         2,731  ▁▁▂▁▂███▆█       ↑ 203%            return         2,094  ▁▁▁▂▂██▇▂▇        ↑ 22×        │
│   steps          2,000  ▁▁▁▁▃███▆█       ↑ 116%            steps          1,851  ▁▁▂▁▂██▇▁█        ↑ 16×        │
│   passed         44.31  ▁▁▁▁▂███▇█       ↑ 104%            passed         40.88  ▁▁▁▁▂██▇▂█        ↑ 26×        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────  ✓ finished  ─╯
╭─  ✓ WorldModel  ────────────────────────────────────────────────────────────────────────────────────────────────╮
│                    trained  125000 steps in 1:49:48                                                             │
│                 first step  after 51.72 s                                                                       │
│                       then  19.6 step/s                                                                         │
│                    goodput  96.7% of the wall time in steps                                                     │
│                 final loss  9.143                                                                               │
│ evaluation (ema) at 125000  return 2,731 ↑ 203% ▁▁▂▁▂███▆█   steps 2,000 ↑ 116% ▁▁▁▁▃███▆█   passed 44.31 ↑     │
│                             104% ▁▁▁▁▂███▇█                                                                     │
│   held_out (ema) at 125000  return 2,094 ↑ 22× ▁▁▁▂▂██▇▂▇   steps 1,851 ↑ 16× ▁▁▂▁▂██▇▁█   passed 40.88 ↑ 26×   │
│                             ▁▁▁▁▂██▇▂█                                                                          │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
125000 updates in 110 min

A score means something beside what the untrained policy earns on the same flights. episodes flies a policy’s greedy actions (greedy, the policy’s mean) through fresh episodes from a key and scores each drone’s first episode; flown from one key, the trained and the untrained policies start from the same places and see the same race.

from dew.rl.evaluation import episodes
from dew.rl.simulator import DrivenSimulator
agent = run.agent(race)
judged = {name: SkyDreamerRace(HyperLuxRenderer(DRONES, ASSETS, size=64), sky)
for name, sky in (("evaluation", Sky(anywhere=0.0, size=1.0)), ("held_out", Sky()))}
for policy, variables in (("untrained", agent.init(jax.random.key(0))), ("trained", state.variables)):
for name, judged_race in judged.items():
score = episodes(agent.greedy, DrivenSimulator(judged_race), EPISODE, memory=agent.memory,
reached=("passed",))(variables, jax.random.key(7))
print(f"{policy:>9} {name:>10}: {score['passed']:5.1f} gates, return {score['return']:7.1f}, "
f"{score['steps']:6.0f} steps of {EPISODE}")
untrained evaluation:   0.0 gates, return  -193.7,    515 steps of 2000
untrained   held_out:   0.0 gates, return  -168.2,    511 steps of 2000
  trained evaluation:  44.4 gates, return  2723.7,   2000 steps of 2000
  trained   held_out:  36.2 gates, return  1859.8,   1646 steps of 2000

The run’s tracker holds every validation it flew. Each point is the mean over 16 drones of their first episode: the gates they passed and how long they flew before an episode’s end (a crash, as the race ends no episode otherwise before its 2000 steps).

from dew.training.tracker import LocalTracker
logged = LocalTracker("runs/skydreamer/tracking").scalars()
splits = ("evaluation", "held_out")
# Each update replays 16 windows of 64 steps, 128 for each environment step taken: 8 steps an update.
points = {name: [(step * 8, scalars) for step, scalars in logged if f"{name}/passed" in scalars]
for name in splits}
steps = {name: [step for step, _ in points[name]] for name in splits}
display(figures.curve({name: [scalars[f"{name}/passed"] for _, scalars in points[name]] for name in splits},
x=steps, x_label="environment steps", y_label="gates passed in an episode"))
display(figures.curve({name: [scalars[f"{name}/steps"] for _, scalars in points[name]] for name in splits},
x=steps, x_label="environment steps", y_label="steps flown of 2000"))
gates passed in an episode against environment steps20402500005000007500001e+06environment stepsgates passed in an episodeevaluation 44.31held_out 40.88
steps flown of 2000 against environment steps5001000150020002500005000007500001e+06environment stepssteps flown of 2000evaluation 2000.00held_out 1851.25

The policy flies from 64-pixel images; to watch it, a second race draws the same flights at 256 pixels. Both races start from the same seed, so each step’s action, chosen from the small image, flies the drone in both.

import imageio.v2 as imageio
from IPython.display import Video
from dew.rl.env import view
from dew.rl.simulator import simulator_start
FILMED = 4
flying = DrivenSimulator(SkyDreamerRace(HyperLuxRenderer(FILMED, ASSETS, size=64), Sky(anywhere=0.0)))
filmed = SkyDreamerRace(HyperLuxRenderer(FILMED, ASSETS, size=256), Sky(anywhere=0.0))
key = jax.random.key(11)
carry = simulator_start(flying, key, agent.memory(FILMED))
seed_key, _ = jax.random.split(key)
filmed.reset(int(jax.random.randint(seed_key, (), 0, jnp.iinfo(jnp.int32).max)))
step, memory, frames = carry.observation, carry.memory, []
for t in range(1800):
action, _, memory = agent.greedy(state.variables, step, jax.random.fold_in(key, t), memory)
step = flying.step(action).observation
frames.append(np.concatenate(list(np.asarray(view(filmed.step(action).observation, "image"))), 1))
imageio.mimwrite("pilot.mp4", frames, fps=90, macro_block_size=1)
Video("pilot.mp4", embed=True, width=1024)
stderr
/usr/lib/python3.13/subprocess.py:1922: RuntimeWarning: os.fork() was called. os.fork() is incompatible with multithreaded code, and JAX is multithreaded, so this will likely lead to a deadlock.
  self.pid = _fork_exec(