Companion Observer Memory for Neural Policies

Integrate the module between the environment observation and the policy/value network. Let the observation dimension be p, action dimension be m, and choose memory order n, initially 4, 8, or 16. Build a block-diagonal fixed matrix F_A=I_{m+p}\otimes Acal, where Acal is the companion matrix with last row -alpha_0,...,-alpha_{n-1}. Select the coefficients so every eigenvalue has modulus rho<1; a simple choice is a stable polynomial corresponding to repeated or real poles near rho=0.8. Construct G_u=[I_m\otimes b;0] and G_y=[0;I_p\otimes b]. Maintain zeta separately for every parallel environment. At each step, receive y_t, compute h_t=M zeta_t, concatenate h_t with y_t if desired, and produce u_t=pi_theta(h_t,y_t). After executing u_t and receiving y_{t+1}, update zeta_{t+1}=F_A zeta_t+G_u u_t+G_y y_{t+1}. Learn M jointly with policy parameters theta using PPO, SAC, or imitation learning. If privileged simulator states x_t are available, add L_obs=(1/T)sum_t ||M zeta_t-x_t||_2^2; otherwise train only from task return or prediction loss. Apply gradient clipping and optionally penalize ||M||_F^2. The fixed matrices are computed exactly from the construction; M, theta, and any observation encoder are estimated empirically. First test on partially observed CartPole, DeepMind Control with hidden velocity, and a small visual POMDP. Compare against a same-size GRU and a transformer over the last n observations/actions. Record return, samples to target return, wall-clock training cost, hidden-state norm, and robustness to longer episodes. Success is comparable or better return with lower recurrent compute, faster early learning, and bounded hidden-state norms as rho remains below one.