# Frieze-consistent multiplicative feature block

- ID: 495
- Canonical URL: https://synthcore.org/idea/495/frieze-consistent-multiplicative-feature-block
- API JSON: https://synthcore.org/api/idea/495.json
- API Markdown: https://synthcore.org/api/idea/495.md
- Verification status: unverified
- Source: [arXiv:2607.06767](https://arxiv.org/abs/2607.06767)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: mlp, cnn, training-dynamics, regularization
- Math tags: algebra, combinatorics, dynamical-systems, linear-algebra
- Ratings: usefulness 5/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a standard two-layer multiplicative interaction block with auxiliary positive features X whose neighboring products generate two coupled feature grids x and y. Add the Y-diamond recurrence as either a hard recurrent update or a differentiable consistency loss, forcing local interactions to obey the same compatibility structure as an SL2/Y-frieze.

## Mathematical statement

The construction uses auxiliary variables X_{i,k}, where i is a depth or vertical-layer index and k is a cyclic feature-position index. Lemma 7 states that Y-frieze entries satisfy y_{i,k}+1=X_{i,k}X_{i,k+1}; neighboring SL2-style entries satisfy x_{i,k}=X_{i,k}X_{i-1,k}. The Y-diamond rule propagates the next x-row through (x_{i+1,k}+1)(x_{i,k}+1)=y_{i,k}y_{i,k-1}, hence x_{i+1,k}=y_{i,k}y_{i,k-1}/(x_{i,k}+1)-1. Positivity of the integer factors X_{i,k} is proved inductively for valid Y-frieze boundary data. In a neural adaptation, use X_{i,k}=softplus(U_{i,k})+epsilon, define y_{i,k}=X_{i,k}X_{i,k+1}-1 and x_{i,k}=X_{i,k}X_{i-1,k}, and penalize or enforce the local residual R_{i,k}=(x_{i+1,k}+1)(x_{i,k}+1)-y_{i,k}y_{i,k-1}. Here epsilon>0 prevents zero factors, U is an unconstrained network output, and all k indices are taken modulo K for periodic features.

## Key formulas

- $$y_{i,k}+1=X_{i,k}X_{i,k+1}$$
- $$x_{i,k}=X_{i,k}X_{i-1,k}$$
- $$(x_{i+1,k}+1)(x_{i,k}+1)=y_{i,k}y_{i,k-1},\qquad x_{i+1,k}=\frac{y_{i,k}y_{i,k-1}}{x_{i,k}+1}-1$$
- $$\mathcal{L}_{\mathrm{frieze}}=\frac{1}{IK}\sum_{i,k}\left[(x_{i+1,k}+1)(x_{i,k}+1)-y_{i,k}y_{i,k-1}\right]^2$$

## Implementation notes

(1) Integration point: insert the block after a Transformer or MLP hidden projection, or between two 1D convolution layers. Reshape a hidden vector h in R^d into U in R^{I x K}, with d=I K; k is a circular feature axis and i indexes internal interaction layers. Compute positive factors X=softplus(U)+epsilon, with epsilon=1e-4. Produce two feature grids y and x using y[i,k]=X[i,k]*X[i,(k+1) mod K]-1 and x[i,k]=X[i,k]*X[i-1,k], using a learned or fixed positive boundary X[-1,k]. Flatten concat(x,y), project back to the original hidden dimension, and add a residual connection to h. (2) Pseudocode: U=reshape(linear(h)); X=softplus(U)+eps; y=X*roll(X,-1,axis=k)-1; x=X*shift(X,+1,axis=i); residual=(x[1:]+1)*(x[:-1]+1)-y[:-1]*roll(y[:-1],+1,axis=k); output=h+linear(flatten(concat(x,y))); loss=task_loss+lambda_f*mean(residual**2). A stricter variant computes x[i+1,k]=y[i,k]*y[i,k-1]/(x[i,k]+1)-1 and uses the resulting x as the next layer state, with clipping if numerical instability occurs. (3) The exact mathematical quantities are the adjacent products and Y-diamond residual; no theorem constants are estimated. Empirically sweep lambda_f, K, and the number of layers, and monitor the fraction of negative y/x values, residual magnitude, activation variance, and gradient norm. (4) First experiment: train a 4-layer MLP and a small CIFAR-10 ConvNet with identical parameter counts, comparing a standard gated multiplicative block against this block at K=16 or 32. Use AdamW, identical initialization and learning-rate schedule, and three seeds. Success means lower gradient-norm variance, fewer NaNs or activation explosions, and either better validation accuracy at equal parameters or equal accuracy with at least 20% fewer hidden parameters.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.