Construct a graph and its spectral positional features using affinities between inputs after optimally aligning one input over the known symmetry group. Feed these quotient-space eigenvectors to a transformer or graph neural network, so symmetry-equivalent examples receive the same structural coordinates without storing augmented copies.
Replace ordinary pairwise attention similarity by an affinity averaged over transformed keys or values. The resulting attention is invariant to the group action on either input and avoids requiring the network to learn identical attention patterns for every rotated or transformed copy.
Represent each token value in an attention head as a bounded three-dimensional Lie-algebra vector and impose a weighted polygon-closure condition on the aggregate value vectors. The module is invariant to a common \(SU(2)\simeq SO(3)\) rotation of all token vectors, preventing the head from spending capacity on an arbitrary global orientation. A soft closure penalty gives a drop-in experiment, while projection onto the zero-sum manifold provides a harder constrained variant.
Add a profile-preserving exchange layer that mixes representations through pairwise state swaps, with weights shared over all positions. Unlike ordinary token-wise MLP mixing, the layer explicitly models interactions between two categorical states while respecting the symmetry of position permutations.
Replace an unconstrained feature vector of size n+1 by the coefficients of a homogeneous degree-n binary polynomial and make the layer transform through the irreducible symmetric-power representation of GL_2(R). For n=4 this creates a five-channel equivariant feature block whose transformation law is exact rather than learned through augmentation.
Build an input-conditioned unitary transformation as an ordered product of exponentials of anti-Hermitian matrices, with each factor controlled by a univariate function of one input coordinate or one learned scalar projection. This replaces a dense multivariate matrix-valued controller with separable scalar nonlinearities while preserving exact unitarity at every forward pass.
Replace a learned linear operator by Monte Carlo averages of unitary-conjugated copies, where the conjugators are discrete time-frequency shifts sampled from a narrow phase-space distribution. The conjugation preserves the singular spectrum of each sampled operator, while the expectation penalizes sensitivity to small translations and modulations without directly shrinking the operator norm.
Replace or augment standard sinusoidal or RoPE position features with bracket-quadratic phases $e(-\theta n\lfloor\beta n\rfloor)$ generated by a Heisenberg nilmanifold orbit. Multiple irrational coefficients and output frequencies produce a cheap deterministic encoding whose empirical cross-position correlations should exhibit cancellation instead of the periodic aliasing of rational or finite-frequency encodings.
Regularize learned skew generators so that their iterated Lie brackets span many independent feature-mixing directions rather than collapsing to commuting or redundant matrices. This turns the paper's controllability family into a differentiable diversity objective for structured neural layers.
Paper: Nonlinear Controllability and the Propagation of Local Information: From the Kalman Family to Lie Brackets, Rotation Groups, and Reachable SubgroupsarXiv:2608.20094
Represent each token or graph node by an anti-Hermitian matrix latent state and replace a standard residual transformation with a discretized Lie-algebra vortex flow. The commutator nonlinearities are equivariant under global unitary conjugation, so the block can learn interactions without selecting a basis and preserves the anti-Hermitian state space when initialized there.
Construct a neural mixing layer on Fourier or positional modes using a small set of exponentiated Virasoro generators instead of a dense mode-to-mode matrix. The generator coefficients are shared across all inputs, while the Lie bracket fixes how different mode shifts interact; an optional central channel captures the special coupling between modes whose indices sum to zero.