Spectrally safeguarded DFP

Integrate this into a limited-memory or blockwise DFP optimizer, because a dense full-network inverse Hessian is infeasible. Partition parameters into blocks of 256 to 2048 parameters, or use a low-rank representation H_k = gamma_k I + U U^T, and apply the safeguard independently to each block after every accepted step. First compute g_k, choose d_k = -H_k g_k, and run a line search satisfying the Armijo and strong-Wolfe tests. Store s_k = x_{k+1} - x_k and y_k = g_{k+1} - g_k. Reject the quasi-Newton update when s_k^T y_k is less than or equal to delta times the product of the norms of s_k and y_k. Otherwise form the DFP update using a dense block or limited-memory representation. Estimate the smallest and largest eigenvalues with 3 to 5 Lanczos iterations. Track P_k, the projector onto the r smallest estimated eigenvectors, and compute q_k = ||P_k - P_{k-1}||_2. If lambda_min is below epsilon, q_k exceeds tau, or the condition estimate exceeds kappa_max, use H_{k+1} = (1-rho)H_k + rho H_{k+1}^{DFP}, backtracking on rho, and clip eigenvalues to [epsilon, kappa_max epsilon]. If clipping is expensive, reset the block to gamma I with gamma = (s_k^T y_k)/(y_k^T y_k). The paper determines the failure mechanism; epsilon, tau, kappa_max, rho, and delta are empirical hyperparameters. First test on a two-layer MLP with 1,000 to 10,000 parameters on MNIST or Fashion-MNIST, comparing SGD, BFGS, ordinary DFP, and safeguarded DFP at equal gradient evaluations. Also test a two-dimensional strongly convex synthetic objective whose Hessian lies between (1/2)I and (3/2)I. Success means a lower gradient-norm plateau, bounded block condition numbers, fewer line-search failures, and lower loss at equal function and gradient evaluations.