# Error-budgeted local log-signature tokens

- ID: 1437
- Canonical URL: https://synthcore.org/idea/1437/error-budgeted-local-log-signature-tokens
- API JSON: https://synthcore.org/api/idea/1437.json
- API Markdown: https://synthcore.org/api/idea/1437.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.26281](https://arxiv.org/abs/2607.26281)
- Category: memory
- Solves: memory, scalability, accuracy
- ML areas: transformer, ssm, embedding, memory
- Math tags: tensor-decomposition, approximation-theory, dynamical-systems, combinatorics
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Encode a continuous or sampled multivariate time series as degree-N log-signatures on m local intervals, choosing (m,N) to minimize the number of stored scalars under an explicit linear-CDE approximation budget. Unlike fixed-size temporal patches, the representation increases temporal resolution when the path is oscillatory and increases algebraic degree when the path is smooth but strongly nonlinear.

## Mathematical statement

The paper represents a path X:[0,T]->R^d using iterated integrals S^n(X)=integral over 0<u_1<...<u_n<T of dX_{u_1} tensor-product ... tensor-product dX_{u_n}, and approximates the linear controlled differential equation dY=A Y dX, where Y is the hidden state and A is a linear control operator with operator norm |A|<=r. For a segment j with 1-variation length ell_j and degree N, the omitted Taylor-signature tail is bounded by R_N(r ell_j)=exp(r ell_j)-sum_{k=0}^N (r ell_j)^k/k!. A practical global error proxy is E(m,N)=sum_{j=1}^m R_N(r ell_j), which conservatively sums local propagation errors. If all m segments have equal variation ell=L/m, then E(m,N)<=m[exp(rL/m)-sum_{k=0}^N (rL/m)^k/k!], where L is total path variation. The storage cost of local truncated log-signatures is C(m,N)=m sum_{k=1}^N l_k(d), where l_k(d) is the dimension of the degree-k free-Lie component for d input channels; the Witt formula is l_k(d)=(1/k) sum_{q|k} mu(q)d^(k/q), with mu the Moebius function. Select the smallest C(m,N) subject to E(m,N)<=epsilon, where epsilon is the desired approximation budget.

## Key formulas

- $$S^n(X)=\int_{0<u_1<\cdots<u_n<T} dX_{u_1}\otimes\cdots\otimes dX_{u_n}$$
- $$dY=A Y\,dX,\qquad \lVert A\rVert\le r$$
- $$R_N(z)=e^z-\sum_{k=0}^{N}\frac{z^k}{k!},\qquad E(m,N)=\sum_{j=1}^{m}R_N(r\ell_j)$$
- $$C(m,N)=m\sum_{k=1}^{N}\ell_k(d),\qquad \ell_k(d)=\frac{1}{k}\sum_{q\mid k}\mu(q)d^{k/q},\qquad (m^*,N^*)=\arg\min_{m,N}C(m,N)\ \text{s.t.}\ E(m,N)\le\varepsilon$$

## Implementation notes

Integrate this at the sequence-to-token interface before a transformer, SSM, or neural CDE. Given a sampled sequence x[0:T] with d channels, first compute cumulative path variation v_t=sum_{s<t} ||x[s+1]-x[s]||_2. For each candidate interval count m in a small grid such as {1,2,4,8,16,32} and degree N in {1,...,5}, partition the sequence at equal-variation quantiles rather than equal time. On every segment j, compute the truncated log-signature z_j=(log S_j)_{1:N}; concatenate z_j with segment duration and endpoint increment, then project it to the model width with one shared linear layer. Estimate ell_j from observed variation in segment j and set r to a tunable sensitivity scale, initially r=1/(median segment norm) or a validation parameter. Compute R_N(r ell_j) using a recurrence for Taylor terms rather than direct factorials. Enumerate candidate (m,N), retain the lowest C(m,N) satisfying E(m,N)<=epsilon, and cache the selected representation during preprocessing. Pseudocode: `v=variation(x); cuts=variation_quantiles(v,m); for j: ell[j]=segment_variation; z[j]=logsignature(x[cuts[j]:cuts[j+1]],N); E=sum(exp(r*ell[j])-taylor_exp(r*ell[j],N)); choose lowest_cost(E<=eps); tokens=Linear(concat(z,duration,delta_x))`. The mathematical quantities are ell_j, R_N, C, and the feasibility test; log-signatures and r are computed numerically or estimated from data. First test on Speech Commands or a UEA multivariate time-series dataset with a 2-layer transformer, comparing fixed raw patches, fixed degree-2 signatures, and adaptive tokens at equal input-memory budgets. Measure validation accuracy, token count, training throughput, and accuracy at fixed GPU memory. Success means at least 20% fewer tokens or 1.5x longer sequences at no accuracy loss, with tighter epsilon producing a monotone accuracy improvement.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a readable MVP for recurrence-based Taylor tails, Witt/free-Lie storage cost, variation-quantile segmentation, local degree-1/2 log-signature-like features, budget selection, and a toy classifier. The mechanism manifested: tails decreased strictly with degree, equal-variation subdivision decreased E monotonically, and tightening epsilon monotonically increased selected cost. The recurrence matched factorial-reference tails to 4.5e-13 absolute error. The classification signal was inconclusive because fixed degree-2 signatures and adaptive tokens both reached 100% while raw patches reached 50.5%, so this is evidence for the mathematical mechanism, not a demonstrated ML win.

### Mechanism check

- Verdict: Built a readable MVP for recurrence-based Taylor tails, Witt/free-Lie storage cost, variation-quantile segmentation, local degree-1/2 log-signature-like features, budget selection, and a toy classifier. The mechanism manifested: tails decreased strictly with degree, equal-variation subdivision decreased E monotonically, and tightening epsilon monotonically increased selected cost. The recurrence matched factorial-reference tails to 4.5e-13 absolute error. The classification signal was inconclusive because fixed degree-2 signatures and adaptive tokens both reached 100% while raw patches reached 50.5%, so this is evidence for the mathematical mechanism, not a demonstrated ML win.
- Confidence: 8/10
- Limitations: The toy data is small and easily separable; no transformer, Speech Commands, UEA dataset, throughput-at-equal-memory, or GPU benchmark was tested. The feature implementation includes exact degree-1 and antisymmetric degree-2 log-signature coordinates only; degrees 3-5 are used in the mathematical selector but not fully encoded. The unequal-segment proxy can be substantially larger than the equal-variation expression, as expected from the conservative local sum.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/561/bench_report.json)
- [experiment.py](https://synthcore.org/code/561/experiment.py)
- [report.md](https://synthcore.org/code/561/report.md)
- [report_bench_2026-08-31T175136.md](https://synthcore.org/code/561/report_bench_2026-08-31T175136.md)
- [stage2_bench.py](https://synthcore.org/code/561/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/561)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.