Multi Layer Perceptrons#

MLP backbones (ReLU or sine) for parameterizing implicit neural representations.

class spherical_inr.mlp.ReLUMLP(input_features: int, output_features: int, hidden_sizes: List[int], bias: bool = True)[source]#

Bases: Module

ReLU-activated multi-layer perceptron.

Hidden layers apply:

\[h_k = \mathrm{ReLU}(W_k h_{k-1} + b_k), \quad k=1,\dots,L-1,\]

and the output layer is linear:

\[f_\theta(x) = W_L h_{L-1} + b_L.\]
Parameters:

  • input_features – Input dimension.

  • output_features – Output dimension.

  • hidden_sizes – List of hidden layer widths.

  • bias – Whether to include biases in each linear layer.

forward(x: Tensor) → Tensor[source]#

Forward pass of the ReLU-activated MLP.

Applies a sequence of linear layers with ReLU activation on all hidden layers, followed by a final linear output layer without activation.

Parameters:

x (torch.Tensor) – Input tensor of shape (..., input_features).

Returns:

Output tensor of shape (..., output_features).

Return type:

torch.Tensor

class spherical_inr.mlp.SineMLP(input_features: int, output_features: int, hidden_sizes: List[int], bias: bool = True, omega0: float = 1.0)[source]#

Bases: Module

Sine-activated multi-layer perceptron.

This module is identical to ReluMLP except the hidden activation is a sine nonlinearity with frequency factor \(\omega_0\). Given an input \(x\), the hidden activations are

\[h_0 = x, \qquad h_k = \sin\!\bigl(\omega_0 (W_k h_{k-1} + b_k)\bigr), \quad k=1,\dots,L-1,\]

and the output layer is linear:

\[f_\theta(x) = W_L h_{L-1} + b_L.\]

The weights are initialized uniformly (per layer) as

\[W_k \sim \mathcal{U}\!\left[-\frac{\sqrt{6/n_k}}{\omega_0}, \frac{\sqrt{6/n_k}}{\omega_0}\right],\]

where \(n_k\) is the fan-in (number of input features) of layer \(k\). Biases are initialized to zero when present.

Parameters:

  • input_features – Input dimension.

  • output_features – Output dimension.

  • hidden_sizes – List of hidden layer widths.

  • bias – Whether to include biases in each linear layer.

  • omega0 – Frequency factor \(\omega_0\) used in the sine activation and in the weight initialization bound.

forward(x: Tensor) → Tensor[source]#

Forward pass of the sine-activated MLP.

Applies sine nonlinearities with frequency scaling omega0 after each hidden linear layer, followed by a final linear output layer.

Parameters:

x (torch.Tensor) – Input tensor of shape (..., input_features).

Returns:

Output tensor of shape (..., output_features).

Return type:

torch.Tensor