Skip to content

dew.nn.fourier

Fourier feature pyramids: learnable periodic feature grids at coarse-to-fine resolutions, read at any coordinate by Fourier interpolation.

The pyramid is beignet’s (Zhao et al., “Fourier Feature Pyramids for Physics-Informed Neural Networks”, arXiv 2605.24278): level l holds a grid of features channels at its own resolution, read at a coordinate x in [0, 1)^d by its trigonometric interpolant,

g_l(x) = Re( sum_k theta_hat_l[k] exp(2 pi i k . x) ),

theta_hat_l the grid’s DFT over its size, k over each axis’s signed FFT modes {-N/2, …, N/2 - 1}. The levels’ features, each scaled by level_scale to its index and all by global_scale (the paper’s per-level and global preconditioners), are concatenated for a decoder to read. A level’s resolution bounds the frequencies it represents, so the finest grid is the representation’s bandlimit. The interpolant is smooth in x, so a residual’s derivatives in the coordinates are the spectral derivatives of each level (the paper’s Eq. 8), which automatic differentiation of this sum computes exactly.

NameSummary
interpolatedThe trigonometric interpolant of grid ([*sizes, channels], periodic on [0, 1)^d) at coordinates ([…, d]): Re of the sum over each axis’s signed FFT modes of the grid’s DFT over its size times exp(2 pi i k .
FourierPyramidLearnable periodic feature grids at the resolutions grids (one size per axis a level, or one int for a 1-D level), each of features channels, read at coordinates in [0, 1)^d by Fourier interpolation (interpolated) and concatenated, […, levels * features].

function source

def interpolated(grid: jax.Array, coordinates: jax.Array) -> jax.Array

The trigonometric interpolant of grid ([*sizes, channels], periodic on [0, 1)^d) at coordinates ([…, d]): Re of the sum over each axis’s signed FFT modes of the grid’s DFT over its size times exp(2 pi i k . x), […, channels]. At a grid point i / N it is the grid’s own value.

The modes are summed one axis at a time, each a complex product written as real products through float32_einsum, so a float32 grid multiplies at float32’s own precision on every backend (a GPU’s complex product would take TF32) and a float64 one in float64.

Flax module source

class FourierPyramid(
global_scale: float = 1.0,
level_scale: float = 1.0,
init_noise: float = 0.0001,
param_dtype: DTypeLike = jnp.float32,
)

Learnable periodic feature grids at the resolutions grids (one size per axis a level, or one int for a 1-D level), each of features channels, read at coordinates in [0, 1)^d by Fourier interpolation (interpolated) and concatenated, […, levels * features].

Level l’s features are scaled by level_scale ** l and all by global_scale: a decoder reads features of order one while the grids train at the scale of init_noise, the half-width of the uniform draw every grid value starts from. Coordinates are periodic with period one; a problem on another domain maps onto [0, 1) first, onto half of it where its solution is not periodic.