✗ Failed on benchmark 2026

Fractional Memory State-Space Layer

Implementation & benchmark of arXiv:2608.30138 — High frequency wave propagation for the viscoelastic wave equation with singular memory

Usefulness7/10

Difficulty5/10

Novelty6/10

Architecture Memory Sample efficiency Stability Rnn Ssm World model Dynamical systems Functional analysis Approximation theory Numerical analysis

Source paper: High frequency wave propagation for the viscoelastic wave equation with singular memory arXiv:2608.30138 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a standard recurrent state update or finite-order SSM filter with a causal relative-history operator using a weakly singular kernel k(s)=s^{p-1}m(s), where 0<p<1. The resulting layer retains information over a power-law range of timescales and introduces tunable frequency-dependent phase and attenuation, while remaining implementable through a small bank of exponentially decaying states.

Formulas

$$\eta^{t}(s)=u(t)-u(t-s),\qquad s\geq 0,$$

$$\mathfrak{m}(s,x)=s^{p-1}m(s,x),\qquad 0<p<1,$$

$$C_p=\Gamma(p)e^{\mathrm{i}\pi p/2},$$

$$R(t)=\int_0^\infty k(s)\bigl(x(t)-x(t-s)\bigr)\,\mathrm{d}s\approx \left(\sum_{j=1}^{J}\frac{w_j}{\lambda_j}\right)x(t)-\sum_{j=1}^{J}w_jq_j(t),\qquad \dot q_j(t)=x(t)-\lambda_jq_j(t).$$

Mathematical statement

The paper defines the relative-history variable eta^t(s)=u(t)-u(t-s), where u(t) is the current signal and s is lag, and uses a memory kernel mathfrak{m}(s)=s^{p-1}m(s), with 0<p<1 and nonnegative envelope m(s). For a neural sequence x(t), define R(t)=integral from 0 to infinity of k(s)(x(t)-x(t-s)) ds, where k(s)=s^{p-1}m(s). Approximate the singular kernel by positive exponential modes k(s) approximately equal to sum over j of w_j exp(-lambda_j s), with w_j>0 and lambda_j>0. Define q_j(t)=integral from 0 to infinity of exp(-lambda_j s)x(t-s) ds. Then q_j satisfies the stable state equation dot q_j=x-lambda_j q_j, and R(t) is approximately (sum_j w_j/lambda_j)x(t)-sum_j w_j q_j(t). The paper identifies the high-frequency complex coefficient C_p=Gamma(p) exp(i pi p/2), where Gamma is the Gamma function; this coefficient supplies the characteristic phase of the fractional response. Positive weights and decay rates guarantee fading-memory stability, while logarithmically spaced rates approximate a power law over many frequencies.

Implementation notes

Integrate the layer at the recurrent or SSM block, immediately after the input projection and before the usual gated update or residual output. For a sequence x[1:T] with channel dimension d, choose J=8--32 positive decay rates lambda_j logarithmically spaced between 1/T_context and 1, or between the target longest timescale and the Nyquist-scale decay. Maintain q[j,d]. At timestep t with spacing Delta, update each mode using the exact discretization q_j <- exp(-lambda_j Delta) q_j + (1-exp(-lambda_j Delta))/lambda_j times x_t. Compute r_t = a x_t - sum_j w_j q_j, with a=sum_j w_j/lambda_j. Feed the concatenation of x_t and r_t into a learned projection or gated transition, for example h_t=tanh(W_x x_t + W_r r_t + A h_{t-1}+b). Parameterize lambda_j=softplus(theta_j)+epsilon and w_j=softplus(phi_j) so every mode remains stable and positive. Initialize lambda_j logarithmically and initialize weights proportional to lambda_j^p over logarithmic bins, which approximates k(s)=s^{p-1}; optionally learn p through p=0.05+0.90 sigmoid(p_raw). For a complex-valued SSM, initialize the memory projection phase using C_p=Gamma(p) exp(i pi p/2); for a real network, represent its real and imaginary parts with two channels. The paper provides the fractional exponent and phase structure, while the exponential quadrature weights and rates are learned or fitted empirically. First test a 1-layer 256-hidden-unit model on copy/retrieval and Long Range Arena Pathfinder or ListOps against a GRU and diagonal SSM at equal parameter count and FLOPs. Measure loss versus sequence length, long-context accuracy, and stability under extrapolated context. Success means slower accuracy decay beyond the training length or equal accuracy with fewer recurrent states than the baselines.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not tested

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a positive, logarithmically spaced exponential-state approximation of the fractional relative-history operator, with NNLS-fitted weights, exact stable discretization, transfer analysis, and a delayed-retrieval toy benchmark. Mechanism checks confirmed power-law impulse slopes (errors 0.0002, 0.0020, and 0.0243 for p=0.2, 0.5, and 0.8) and fractional transfer phase/slope, especially at p=0.5 and 0.8. The retrieval MSE was essentially unchanged versus the matched exponential and raw baselines, so the mechanism manifested but a practical learning win was not demonstrated.

Agent confidence: 8/10

Baseline: Matched single exponential retrieval MSE 0.97813; raw input MSE 0.97788.
Idea: Fractional 16-mode bank retrieval MSE 0.97792. Impulse log-slope observed/predicted: p=.2 -0.800/-0.800, p=.5 -0.502/-0.500, p=.8 -0.176/-0.200. Transfer phase observed/predicted: p=.2 -0.509/-0.314 rad, p=.5 -0.838/-0.785 rad, p=.8 -1.275/-1.257 rad. Stability spectral radii exactly matched exp(-lambda_min*dt) for dt=.1,1,4.

Limitations: Only a NumPy/SciPy toy verification and a small isolated-pulse delayed-retrieval benchmark were tested; no trained GRU/SSM neural model, equal-parameter FLOP accounting, LRA task, learned p/rates, noise robustness, or extrapolation study was performed. The p=0.2 transfer approximation showed noticeable finite-bank phase and slope error.

How to run: python3 fractional_memory_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.30138) Sep 2, 2026

Significantly WORSE than the tuned baseline (+1651.7%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small_fractional
Paired seeds: 8
Baseline mean: 0
Idea mean: 0.0003
Effect (Δ): 0.0002 (+1651.7%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±352.1%
Mechanism: n/a
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested; sequence forecasting and long-context extrapolation were not tested. The fractional model uses a custom recurrent transition rather than an exact internal GRU ablation, and its rates and weights were fixed rather than learned. The run used 12 epochs and three learning rates, within the small-model benchmark budget.

How to run: python3 bench_fractional.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 2, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented a fractional exponential-memory recurrent model and evaluated it against a GRU on the structurally matched dynamics track using 8 paired seeds, equal epochs and batch size, and parity-preserving learning-rate sweeps. The baseline achieved mean MSE 1.477801822602487e-05 versus 2.5887351512210444e-04 for the idea; paired delta was +2.4409549689607957e-04 with permutation p=0.0081, so the idea was significantly worse. Stability held numerically, but the trained-bank power-law slope was -0.8731883349875018 versus the predicted -0.5, so the mechanism signature was not confirmed.

Baseline: GRU, best lr=0.01; 8-seed mean MSE 1.477801822602487e-05, std 5.904456722576056e-06.
Idea: 8-mode fractional recurrent bank, p=0.5, best lr=0.003; 8-seed mean MSE 2.5887351512210444e-04, std 5.818967880653584e-05.
Idea wins: 0
Mechanism signature: map[absolute_error:0.3731883349875018 confirmed:false observed_trained_model_log_slope:-0.8731883349875018 predicted_log_slope:-0.5 stability_spectral_radii:[0.9992190550963239 0.9922179382602435 0.9692332344763441]]
N pairs: 8
Paired delta: 2.4409549689607957e-04, idea minus baseline; positive is worse.
Permutation p value: 0.0081

Benchmark result