# Fixed-Projection Temporal Plasticity

- ID: 1248
- Canonical URL: https://synthcore.org/idea/1248/fixed-projection-temporal-plasticity
- API JSON: https://synthcore.org/api/idea/1248.json
- API Markdown: https://synthcore.org/api/idea/1248.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.21622](https://arxiv.org/abs/2607.21622)
- Category: training
- Solves: sample-efficiency, stability, scalability
- ML areas: training, self-supervised, embedding
- Math tags: probability, optimization, linear-algebra, dynamical-systems
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace backpropagation through a small encoder with an online local update driven by consecutive examples and a fixed random projection of hidden activity. The projection produces a modulatory signal that encourages temporally adjacent inputs to have compatible representations, while the homeostatic term prevents sigmoid units from saturating or collapsing.

## Mathematical statement

The paper models each encoder propensity as $h_i(x)=\sigma(\sum_j W_{ij}x_j+b_i)$ with $W\in\mathbb{R}^{C\times n}$, $b\in\mathbb{R}^C$, $x\in\mathbb{R}^n$, and $h\in(0,1)^C$. Its local gain is $\alpha_i=h_i(1-h_i)$, the exact derivative of the sigmoid propensity with respect to its input current; this suppresses updates from saturated neurons. The circuit uses a fixed projection $A$ and projected activity $f=Ah$, with $A\in\mathbb{R}^{M\times C}$ and $C_A=A^\top A$. For consecutive hidden activities $h_t,h_{t+1}$, use the projected temporal discrepancy $C_A(h_{t+1}-h_t)$ as a locally available modulatory signal. With target firing rate $\rho\in(0,1)$ and homeostatic coefficient $\lambda\geq0$, the adapted update is $\Delta W_{ij}=\eta\alpha_{t,i}u_{t,i}x_{t,j}$, where $u_t=C_A(h_{t+1}-h_t)-\lambda(h_t-\rho\mathbf{1})$. The temporal term comes from the paper's fixed flashlight/granule projection and STDP-plus decomposition; the homeostatic term implements its local firing-statistics correction.

## Key formulas

- $$h_i=\sigma\!\left(\sum_{j=1}^{n}W_{ij}x_j+b_i\right),\qquad \sigma(u)=\frac{1}{1+e^{-u}}.$$
- $$\alpha_i=h_i(1-h_i)\in\left(0,\frac14\right].$$
- $$f_t=Ah_t,\qquad C_A=A^\top A.$$
- $$u_t=C_A(h_{t+1}-h_t)-\lambda(h_t-\rho\mathbf{1}),\qquad \Delta W=\eta\,(\alpha_t\odot u_t)x_t^\top.$$

## Implementation notes

Integrate the rule at the encoder weight update rather than in the forward computation graph. Initialize a sigmoid encoder with parameters $W,b$ and retain a fixed Gaussian, sparse, or orthogonal matrix $A\in\mathbb{R}^{M\times C}$ throughout training. For every adjacent pair $(x_t,x_{t+1})$, compute $h_t=\sigma(Wx_t+b)$ and $h_{t+1}=\sigma(Wx_{t+1}+b)$ with gradients disabled. Compute projected activities $f_t=Ah_t$ and $f_{t+1}=Ah_{t+1}$, then calculate $u_t=A^\top(f_{t+1}-f_t)-\lambda(h_t-\rho\mathbf{1})$. Set $\alpha_t=h_t\odot(1-h_t)$ and apply the rank-one update $W\leftarrow W+\eta(\alpha_t\odot u_t)x_t^\top$. Update the bias with $b\leftarrow b+\eta_b(\alpha_t\odot u_t)$, or initially keep $b$ fixed. A hardware-local version replaces explicit $A^\top$ multiplication with fixed feedback synapses from the flashlight population. To avoid a hand-selected target rate, maintain $\bar h\leftarrow(1-\gamma)\bar h+\gamma h_t$ and replace $h_t-\rho\mathbf{1}$ by $h_t-\bar h$. The mathematical quantities taken directly from the paper are the sigmoid propensity, local gain, fixed projection, and consecutive-input rule; $\eta,\lambda,\rho,\gamma$, projection dimension $M$, and update normalization are empirical hyperparameters. First test a 2-layer, 256-unit sigmoid encoder on temporally ordered MNIST. Compare this method with an identical encoder trained by backpropagation on the same temporal objective and with shuffled-pair training. Freeze the encoder and fit a linear classifier. Record linear-probe accuracy, activity entropy, fraction of saturated units, and accuracy per synaptic update. The key falsifiable signal is a large ordered-versus-shuffled accuracy gap without collapse, while the online method should require one forward pass and one update per example and remain stable without replay.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built the fixed-projection temporal plasticity update and a reproducible ordered-versus-shuffled digits experiment. The mechanism checks passed: homeostatic convergence steps scaled approximately inversely with lambda, scalar projection drive matched the exact a^2 law, and sigmoid gain matched h(1-h), peaking at h=0.5 and suppressing saturated units. Ordered local plasticity beat shuffled training (61.3% vs 40.6%) with no saturation, but did not beat the backprop baseline (64.4%), so this is mechanism validation rather than a demonstrated training win.

### Mechanism check

- Verdict: Built the fixed-projection temporal plasticity update and a reproducible ordered-versus-shuffled digits experiment. The mechanism checks passed: homeostatic convergence steps scaled approximately inversely with lambda, scalar projection drive matched the exact a^2 law, and sigmoid gain matched h(1-h), peaking at h=0.5 and suppressing saturated units. Ordered local plasticity beat shuffled training (61.3% vs 40.6%) with no saturation, but did not beat the backprop baseline (64.4%), so this is mechanism validation rather than a demonstrated training win.
- Confidence: 9/10
- Limitations: Used sklearn digits rather than temporally ordered MNIST, a one-layer 24-unit encoder rather than the proposed 2-layer 256-unit model, one random seed, and dense Gaussian projection only. No CUDA benchmark, synaptic-update/FLOP normalization, adaptive target-rate variant, or multi-seed statistical test was run.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.01485
- Paired wins: 1/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/501/bench_report.json)
- [experiment.py](https://synthcore.org/code/501/experiment.py)
- [report.md](https://synthcore.org/code/501/report.md)
- [report_bench_2026-08-31T161516.md](https://synthcore.org/code/501/report_bench_2026-08-31T161516.md)
- [results.json](https://synthcore.org/code/501/results.json)
- [stage2_bench.py](https://synthcore.org/code/501/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/501)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.