Skip to content

dew.rl.hyperlux

HyperLux’s batch ray tracer (rl-tools’ hyperlux 0.1.10, the hyperlux extra, on an NVIDIA GPU with OptiX) drawing each drone’s front camera of the gate race (dew.rl.flight.Renderer), its scene built from the GLB assets tools/fpv_assets.py writes.

HyperLux draws one scene for every camera, its environment and lights shared; a camera’s own content is the overlay attached to it, whose placements move each frame (README; HyperLux.attach, spawn, set_pose). Each drone’s overlay holds every look of its scene: the gates in each of their looks and the lit gate in each of its own, the grounds, the skies, the pillars of each aspect in each material and the props; a frame shows the looks its drone’s episode drew (dew.rl.flight.Camera.look) and sinks the rest out of sight, shrunk. Its gates stand where its course’s do, facing their way, the lit one in its lit look; its ground and sky are centred on its course, the sky turned; its pillars stand where its course’s do, each its aspect’s shape at its radius; its props stand outside the course, placed by its look. A shutter’s open and close cameras blur the image by the drone’s motion (motion_blur samples, at least two; HyperDrone’s paper, Sec 4.1, finds two enough). At “medium” fidelity HyperLux draws base-color textures without lights; exposure, white balance, gamma and noise are the race’s (dew.rl.flight.developed).

A fisheye lens is an equidistant fisheye of that horizontal field of view (an FPV quadrotor’s wide lens): each output pixel at angle theta from the axis lies at a radius proportional to theta, theta reaching half the field of view at the image’s side edges. HyperLux casts pinholes (set_cameras), so the renderer draws a pinhole of the same horizontal field of view source pixels square and samples it, bilinearly, at each output pixel’s direction (fisheye_map): a pinhole of that field of view reaches 67.8 degrees from its axis at its corners where the fisheye’s corners lie further out, so a fisheye image’s outer corners, which no pinhole of it holds, are black.

NameSummary
PROPSEach prop’s placements a drone’s scene holds.
SUNKWhere a placement not shown stands, shrunk to a thousandth, out of every camera’s sight.
HyperLuxRenderercount drones’ front cameras size pixels square (dew.rl.flight.Renderer) over the assets in assets (tools/fpv_assets.py’s manifest), at HyperLux’s fidelity, blurred by motion_blur samples (1 for none).
fisheye_mapWhere each pixel of a size-square equidistant fisheye image of horizontal field of view fov (degrees) samples a source-square pinhole image of the same field of view: for each output pixel the four source pixels about its direction [size, size, 4] (each its row * source + column) and their bilinear weights [size, size, 4], 0 where the pinhole does not hold the direction; and where it does [size, size].
fisheyedPinhole images [B, S, S, C] resampled to a fisheye [B, N, N, C] by fisheye_map’s corners and weights, in the weights’ dtype.

attribute source

PROPS = 6

Each prop’s placements a drone’s scene holds.

attribute source

SUNK = (0.0, 0.0, -1000.0)

Where a placement not shown stands, shrunk to a thousandth, out of every camera’s sight.

class source

class HyperLuxRenderer(
count: int,
assets: str | Path,
*,
size: int = 96,
fidelity: str = 'medium',
motion_blur: int = 2,
fisheye: float | None = None,
source: int | None = None,
)

count drones’ front cameras size pixels square (dew.rl.flight.Renderer) over the assets in assets (tools/fpv_assets.py’s manifest), at HyperLux’s fidelity, blurred by motion_blur samples (1 for none). Its images are [count, size, size, 3] uint8: pinholes of the cameras’ fields of view, or with a fisheye lens equidistant fisheyes of that field of view, drawn from pinholes source pixels square (the module docstring), whose cameras’ field of view must be the lens’s.

def render(scene: Scene, cameras: jax.Array, opened: jax.Array) -> jax.Array

function source

def fisheye_map(
size: int,
source: int,
fov: float,
) -> tuple[np.ndarray, np.ndarray, np.ndarray]

Where each pixel of a size-square equidistant fisheye image of horizontal field of view fov (degrees) samples a source-square pinhole image of the same field of view: for each output pixel the four source pixels about its direction [size, size, 4] (each its row * source + column) and their bilinear weights [size, size, 4], 0 where the pinhole does not hold the direction; and where it does [size, size].

function source

def fisheyed(images: jax.Array, corners: jax.Array, weights: jax.Array) -> jax.Array

Pinhole images [B, S, S, C] resampled to a fisheye [B, N, N, C] by fisheye_map’s corners and weights, in the weights’ dtype.