dew.rl.hyperlux
HyperLux’s batch ray tracer (rl-tools’ hyperlux 0.1.10, the hyperlux extra, on an NVIDIA GPU with OptiX)
drawing each drone’s front camera of the gate race (dew.rl.flight.Renderer), its scene built from the GLB
assets tools/fpv_assets.py writes.
HyperLux draws one scene for every camera, its environment and lights shared; a camera’s own content is the
overlay attached to it, whose placements move each frame (README; HyperLux.attach, spawn, set_pose).
Each drone’s overlay holds every look of its scene: the gates in each of their looks and the lit gate in each
of its own, the grounds, the skies, the pillars of each aspect in each material and the props; a frame shows
the looks its drone’s episode drew (dew.rl.flight.Camera.look) and sinks the rest out of sight, shrunk.
Its gates stand where its course’s do, facing their way, the lit one in its lit look; its ground and sky are
centred on its course, the sky turned; its pillars stand where its course’s do, each its aspect’s shape at its
radius; its props stand outside the course, placed by its look. A shutter’s open and close cameras blur the
image by the drone’s motion (motion_blur samples, at least two; HyperDrone’s paper, Sec 4.1, finds two
enough). At “medium” fidelity HyperLux draws base-color textures without lights; exposure, white balance,
gamma and noise are the race’s (dew.rl.flight.developed).
A fisheye lens is an equidistant fisheye of that horizontal field of view (an FPV quadrotor’s wide lens):
each output pixel at angle theta from the axis lies at a radius proportional to theta, theta reaching half the
field of view at the image’s side edges. HyperLux casts pinholes (set_cameras), so the renderer draws a
pinhole of the same horizontal field of view source pixels square and samples it, bilinearly, at each
output pixel’s direction (fisheye_map): a pinhole of that field of view reaches 67.8 degrees from its axis
at its corners where the fisheye’s corners lie further out, so a fisheye image’s outer corners, which no
pinhole of it holds, are black.
| Name | Summary |
|---|---|
PROPS | Each prop’s placements a drone’s scene holds. |
SUNK | Where a placement not shown stands, shrunk to a thousandth, out of every camera’s sight. |
HyperLuxRenderer | count drones’ front cameras size pixels square (dew.rl.flight.Renderer) over the assets in assets (tools/fpv_assets.py’s manifest), at HyperLux’s fidelity, blurred by motion_blur samples (1 for none). |
fisheye_map | Where each pixel of a size-square equidistant fisheye image of horizontal field of view fov (degrees) samples a source-square pinhole image of the same field of view: for each output pixel the four source pixels about its direction [size, size, 4] (each its row * source + column) and their bilinear weights [size, size, 4], 0 where the pinhole does not hold the direction; and where it does [size, size]. |
fisheyed | Pinhole images [B, S, S, C] resampled to a fisheye [B, N, N, C] by fisheye_map’s corners and weights, in the weights’ dtype. |
PROPS = 6Each prop’s placements a drone’s scene holds.
SUNK = (0.0, 0.0, -1000.0)Where a placement not shown stands, shrunk to a thousandth, out of every camera’s sight.
HyperLuxRenderer
Section titled “HyperLuxRenderer”class HyperLuxRenderer( count: int, assets: str | Path, *, size: int = 96, fidelity: str = 'medium', motion_blur: int = 2, fisheye: float | None = None, source: int | None = None,)count drones’ front cameras size pixels square (dew.rl.flight.Renderer) over the assets in
assets (tools/fpv_assets.py’s manifest), at HyperLux’s fidelity, blurred by motion_blur samples
(1 for none). Its images are [count, size, size, 3] uint8: pinholes of the cameras’ fields of view, or
with a fisheye lens equidistant fisheyes of that field of view, drawn from pinholes source pixels
square (the module docstring), whose cameras’ field of view must be the lens’s.
HyperLuxRenderer.render
Section titled “HyperLuxRenderer.render”def render(scene: Scene, cameras: jax.Array, opened: jax.Array) -> jax.Arrayfisheye_map
Section titled “fisheye_map”def fisheye_map( size: int, source: int, fov: float,) -> tuple[np.ndarray, np.ndarray, np.ndarray]Where each pixel of a size-square equidistant fisheye image of horizontal field of view fov
(degrees) samples a source-square pinhole image of the same field of view: for each output pixel the
four source pixels about its direction [size, size, 4] (each its row * source + column) and their
bilinear weights [size, size, 4], 0 where the pinhole does not hold the direction; and where it does
[size, size].
fisheyed
Section titled “fisheyed”def fisheyed(images: jax.Array, corners: jax.Array, weights: jax.Array) -> jax.ArrayPinhole images [B, S, S, C] resampled to a fisheye [B, N, N, C] by fisheye_map’s corners and
weights, in the weights’ dtype.