# Inverse-Gain Structured Privileged Distillation

- ID: 2086
- Canonical URL: https://synthcore.org/idea/2086/inverse-gain-structured-privileged-distillation
- API JSON: https://synthcore.org/api/idea/2086.json
- API Markdown: https://synthcore.org/api/idea/2086.md
- Verification status: beats_baseline
- Source: [arXiv:2608.10453](https://arxiv.org/abs/2608.10453)
- Category: architecture
- Solves: stability, accuracy, sample-efficiency
- ML areas: rnn, world-model, rl, training-dynamics
- Math tags: control-theory, dynamical-systems, optimization, stochastic-processes
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Replace direct action imitation with a causal recurrent estimator of the inverse input gain. The neural network predicts the latent quantity needed by the expert controller, and a fixed algebraic wrapper converts that prediction into an action using the measured state difference and tracking error, thereby removing the additive disturbance exactly under the sampled timing model.

## Mathematical statement

The plant is sampled as $x_{1,k}=x_{1,k-1}+T_c(a_kx_{2,k-1}+d_k)$ and $x_{2,k}=x_{2,k-1}+T_cu_{k-1}$, where $x_1$ is the regulated state, $x_2$ is the measured velocity-like state, $T_c$ is the sampling period, $a_k>0$ is an unknown multiplicative input gain, and $d_k$ is an unknown additive disturbance. Define the causal difference $\delta x_{1,k}=(x_{1,k}-x_{1,k-1})/T_c=a_kx_{2,k-1}+d_k$, the reference $r$, and tracking error $e_k=r-x_{1,k}$. The privileged expert uses $x_{2r,k}=(k_1e_k-d_k)/a_k$, with control gain $k_1>0$. Eliminating $d_k$ gives $x_{2r,k}=x_{2,k-1}+q_k(k_1e_k-\delta x_{1,k})$, where $q_k=1/a_k$. Thus the student only needs to estimate $q_k$ from causal history. Let $\hat q_k=f_\theta(h_k)$, where $h_k$ contains recent measured states, references, actions, and finite differences. The deployed action is $\hat x_{2r,k}=x_{2,k-1}+\hat q_k(k_1e_k-\delta x_{1,k})$. The crucial property is that disturbance dependence disappears from the wrapper; action error is $\hat x_{2r,k}-x_{2r,k}=(\hat q_k-q_k)(k_1e_k-\delta x_{1,k})$.

## Key formulas

- $$\delta x_{1,k}:=\frac{x_{1,k}-x_{1,k-1}}{T_c}=a_kx_{2,k-1}+d_k$$
- $$x_{2r,k}=\frac{k_1e_k-d_k}{a_k}=x_{2,k-1}+\frac{1}{a_k}\left(k_1e_k-\delta x_{1,k}\right),\qquad e_k=r-x_{1,k}$$
- $$\hat q_k=f_\theta(h_k),\qquad \hat x_{2r,k}=x_{2,k-1}+\hat q_k\left(k_1e_k-\delta x_{1,k}\right)$$
- $$\hat x_{2r,k}-x_{2r,k}=(\hat q_k-q_k)z_k,\qquad q_k=\frac{1}{a_k},\quad z_k=k_1e_k-\delta x_{1,k}$$

## Implementation notes

1. Integration point: use this as a deployable policy head for a partially observed control network, or as a structured recurrent policy inside an RL or distillation system. The GRU or LSTM receives only deployment-visible quantities: a window of $(x_1,x_2,r)$, previous actions, and causally computed differences. It outputs a scalar inverse-gain estimate $\hat q_k$ or a bounded residual around a nominal inverse gain. The fixed wrapper computes $e_k$, $\delta x_{1,k}$, and $\hat x_{2r,k}$; an existing low-level actuator converts this reference into $u_k$. 2. Pseudocode: `delta=(x1[k]-x1[k-1])/Tc; e=r-x1[k]; h=GRU(concat(x1,x2,r,u_prev)); qhat=q_min+(q_max-q_min)*sigmoid(head(h)); z=k1*e-delta; x2_ref=x2[k-1]+qhat*z; u=inner_controller(x2_ref,x2[k])`. During privileged training, compute the expert action `x2r=(k1*e-d)/a`, but do not provide $a$ or $d$ to the student. Minimize Huber loss between `x2_ref` and `x2r`, plus a temporal smoothness penalty on $\hat q_k$, and optionally train on closed-loop rollouts rather than shuffled samples. 3. Computed exactly from the paper: the sampled difference, algebraic cancellation, and structured action wrapper. Estimated empirically: the recurrent map $f_\theta$, the admissible gain interval $[q_{min},q_{max}]$, sensor-noise filtering, and inner-loop dynamics. Use a causal moving average or one-pole filter for $\delta x_1$ only if the delay is acceptable. 4. First cheap experiment: simulate the two-state plant with $a_k$ randomly switching among $\{0.25,0.5,1,2\}$ and piecewise-constant disturbances, then hold out gain sequences and 60-second reference trajectories. Compare a direct MLP action imitator, a GRU action imitator, and the structured GRU against the privileged controller and a tuned nominal observer. Measure closed-loop tracking RMSE, action error, gain-estimation error, and failure rate. The quantitative prediction is that $|\hat x_{2r}-x_{2r}|$ scales linearly with $|z_k|$ and inverse-gain error according to $|\hat x_{2r}-x_{2r}|=|\hat q-q||z|$; binning by $|z|$ should produce slope $|\hat q-q|$. The structured policy should retain performance on unseen $(a,d)$ combinations, while direct action imitation should show larger closed-loop degradation despite similar offline action MSE.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built the inverse-gain structured policy wrapper and a fixed-seed toy verification. Algebraic disturbance cancellation and the factorized action-error identity held to about 5e-14 across disturbance amplitudes, and the inverse-gain noise sweep matched the predicted linear scaling closely. The offline learning comparison did not show a practical win: structured GRU RMSE was 0.513 versus 0.501 for the MLP and 0.555 for the GRU, so the mechanism is validated but superiority is not established.

### Mechanism check

- Verdict: Built the inverse-gain structured policy wrapper and a fixed-seed toy verification. Algebraic disturbance cancellation and the factorized action-error identity held to about 5e-14 across disturbance amplitudes, and the inverse-gain noise sweep matched the predicted linear scaling closely. The offline learning comparison did not show a practical win: structured GRU RMSE was 0.513 versus 0.501 for the MLP and 0.555 for the GRU, so the mechanism is validated but superiority is not established.
- Confidence: 9/10
- Limitations: The learning comparison is offline, teacher-forced action prediction rather than closed-loop control. Sensor noise, online adaptation, inner-loop actuator dynamics, stability/failure-rate analysis, FLOP accounting, and repeated random seeds were not tested. results.json contains the original learning run; verification_report.json contains the additional mechanism sweeps.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/779/bench_report.json)
- [bench_run.py](https://synthcore.org/code/779/bench_run.py)
- [custom_inverse_gain_track.py](https://synthcore.org/code/779/custom_inverse_gain_track.py)
- [experiment.py](https://synthcore.org/code/779/experiment.py)
- [report.md](https://synthcore.org/code/779/report.md)
- [report_bench_2026-09-01T102825.md](https://synthcore.org/code/779/report_bench_2026-09-01T102825.md)
- [results.json](https://synthcore.org/code/779/results.json)
- [verification_report.json](https://synthcore.org/code/779/verification_report.json)
- [Download all files as ZIP](https://synthcore.org/download/779)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.