# Gradient-Projected Rectified Flow

- ID: 1709
- Canonical URL: https://synthcore.org/idea/1709/gradient-projected-rectified-flow
- API JSON: https://synthcore.org/api/idea/1709.json
- API Markdown: https://synthcore.org/api/idea/1709.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.02487](https://arxiv.org/abs/2608.02487)
- Category: sampling
- Solves: accuracy, stability, sample-efficiency
- ML areas: diffusion, diffusion-sampling, optimizer
- Math tags: optimal-transport, convex-analysis, dynamical-systems, functional-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace the unconstrained rectified-flow velocity predictor with the gradient of a learned scalar potential. At every rectification round, fit the potential by weighted least squares to the current displacement field, then integrate the resulting conservative velocity from the source distribution to the target distribution. The gradient restriction is intended to eliminate non-transport rotational motion and improve convergence toward the quadratic optimal-transport coupling.

## Mathematical statement

The transferable construction is the projection of a velocity field onto a gradient class. Let $\pi_k$ be the current coupling of source $X\sim P$ and target $Y\sim Q$, let $Z_t=(1-t)X+tY$ be its displacement interpolation, and let $D=Y-X$ be the displacement. The unconstrained rectified-flow field is the conditional velocity $v_k(z,t)=\mathbb{E}_{\pi_k}[D\mid Z_t=z]$. The c-rectified replacement is a gradient field $u_k(z,t)=\nabla_z\phi_k(z,t)$ obtained by minimizing the weighted Hilbert-space projection objective $\mathbb{E}_{(X,Y)\sim\pi_k,\,t\sim U[0,1]}[\|\nabla_z\phi_k(Z_t,t)-D\|_2^2]$. Here $\phi_k$ is a scalar potential, $z\in\mathbb{R}^d$ is the flow state, and $t\in[0,1]$ is time. The gradient class consists of conservative vector fields, so its restriction removes rotational components. The induced flow solves $\dot Z_t=u_k(Z_t,t)$ with $Z_0\sim P$. For quadratic displacement cost, the target map is $T^*=\arg\min_{T:T_{\#}P=Q}\mathbb{E}_{X\sim P}\|T(X)-X\|_2^2$. In implementation, endpoint preservation is checked explicitly by comparing the generated endpoint distribution with $Q$ and can be enforced through a marginal penalty or a target re-coupling step.

## Key formulas

- $$X_{1},\ldots,X_{n}\overset{\mathrm{i.i.d.}}{\sim}P,\qquad Y_{1},\ldots,Y_{n}\overset{\mathrm{i.i.d.}}{\sim}Q.$$
- $$Z_t=(1-t)X+tY,\qquad D=Y-X,\qquad v_k(z,t)=\mathbb{E}_{\pi_k}[D\mid Z_t=z].$$
- $$\phi_k=\arg\min_{\phi}\;\mathbb{E}_{(X,Y)\sim\pi_k,\,t\sim\mathrm{Unif}[0,1]}\left[\left\|\nabla_z\phi(Z_t,t)-(Y-X)\right\|_2^2\right],\qquad u_k(z,t)=\nabla_z\phi_k(z,t).$$
- $$\frac{dZ_t}{dt}=u_k(Z_t,t),\qquad Z_0\sim P,\qquad T_k(x)=Z_1\ \text{when }Z_0=x.$$

## Implementation notes

Integrate this as a replacement for the velocity network in a flow-matching or rectified-flow generator. Use an MLP potential $\phi_\theta(z,t)$ with one scalar output, and obtain the vector field by automatic differentiation with respect to $z$; do not directly output a vector velocity. For each rectification round $k$, construct a minibatch of pairs $(x_i,y_i)$ from the current coupling $\pi_k$, sample $t_i\sim\mathrm{Uniform}[0,1]$, and compute $z_i=(1-t_i)x_i+t_i y_i$ and $d_i=y_i-x_i$. Minimize $L_k=B^{-1}\sum_{i=1}^B\|\nabla_z\phi_\theta(z_i,t_i)-d_i\|_2^2$. Pseudocode: `for k: sample (x,y) from pi_k; sample t; z=(1-t)*x+t*y; d=y-x; v=grad_z(phi_theta(z,t)); loss=mean((v-d)^2); update theta; integrate dz/dt=grad_z(phi_theta(z,t)) from x to obtain yhat; update pi_{k+1}`. Update $\pi_{k+1}$ by nearest-neighbor matching between generated endpoints $\hat y$ and fresh target samples, or by solving a minibatch entropic Sinkhorn coupling. Add an optional endpoint penalty $\lambda\,\mathrm{MMD}(\{\hat y_i\},\{y_i\})$. The paper-specific computation is the gradient-class projection; the empirical contraction factor, endpoint discrepancy, and transport cost are measured rather than assumed. Begin with two-dimensional rotated Gaussian distributions and Gaussian mixtures, using a four-layer width-128 potential. Compare against ordinary rectified flow and a direct vector-field flow-matching network at equal training steps and ODE function evaluations. Record quadratic coupling cost, endpoint MMD or Wasserstein distance, curl residual $\|\nabla u-(\nabla u)^T\|_F$, and generation quality. Success is lower transport cost and endpoint error after the same number of rectification rounds, particularly for noncommuting rotated covariance matrices, without a substantial increase in inference-time ODE evaluations.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a readable MVP comparing a direct vector-field flow-matching network with a scalar-potential gradient field. The mechanism checks passed quantitatively: rotational energy removal was 1.000 of the injected rotational energy for alpha=0.25, 0.5, 1, and 2; removed energy scaled exactly as alpha^2; input curl scaled as alpha and projected curl stayed near 1e-15. In the small Gaussian experiment, the potential model also showed lower endpoint MMD, covariance error, and transport cost, though this comparison used one seed and is only a promising toy signal.

### Mechanism check

- Verdict: Built a readable MVP comparing a direct vector-field flow-matching network with a scalar-potential gradient field. The mechanism checks passed quantitatively: rotational energy removal was 1.000 of the injected rotational energy for alpha=0.25, 0.5, 1, and 2; removed energy scaled exactly as alpha^2; input curl scaled as alpha and projected curl stayed near 1e-15. In the small Gaussian experiment, the potential model also showed lower endpoint MMD, covariance error, and transport cost, though this comparison used one seed and is only a promising toy signal.
- Confidence: 8/10
- Limitations: The learned comparison is a single fixed-seed, small 2D Gaussian test with 700 updates and no multiple-seed statistics, rectification-round sweep, Sinkhorn re-coupling, explicit learned-field curl measurement, FLOP-normalized timing, or Gaussian-mixture evaluation. The exact math verification uses a periodic spectral projection rather than the finite MLP training procedure.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/632/bench_report.json)
- [experiment.py](https://synthcore.org/code/632/experiment.py)
- [report.md](https://synthcore.org/code/632/report.md)
- [report_bench_2026-08-31T201623.md](https://synthcore.org/code/632/report_bench_2026-08-31T201623.md)
- [results.json](https://synthcore.org/code/632/results.json)
- [run_bench.py](https://synthcore.org/code/632/run_bench.py)
- [run_registered_bench.py](https://synthcore.org/code/632/run_registered_bench.py)
- [transport_track.py](https://synthcore.org/code/632/transport_track.py)
- [Download all files as ZIP](https://synthcore.org/download/632)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.