Unverified 2026

Inverse-square count-aware Gaussian release

Implementation & benchmark of arXiv:2607.14406 — Better Privacy Guarantees for Larger Groups

Usefulness5/10
Difficulty5/10
Novelty6/10

Source paper: Better Privacy Guarantees for Larger Groups arXiv:2607.14406 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace the constant-noise release used for private group aggregates with noise whose standard deviation grows linearly with the group count. The resulting relative error remains approximately constant, while the zCDP privacy loss decreases as the inverse square of group size; this is especially relevant to federated gradient aggregation or private expert-load statistics.

Formulas

$$v(n)=\Theta_r(n^{-2}),\qquad \liminf_{n\to\infty}n^{2}v(n)\geq\frac{(1-r)^{6}}{128r^{2}(1+r)^{2}}>0.$$
$$\rho(n)=\frac{\Delta^{2}}{2\sigma(n)^{2}},\qquad v(n)=\frac{C}{(n+s)^2}\ \Longrightarrow\ \sigma(n)=\frac{\Delta(n+s)}{\sqrt{2C}}.$$
$$\frac{\mathbb E|Y-x|}{n}=\sqrt{\frac{2}{\pi}}\frac{\sigma(n)}{n}\xrightarrow[n\to\infty]{}\frac{\Delta}{\sqrt{\pi C}}.$$

Mathematical statement

Theorem 1.1 states that for every fixed relative-error tolerance 0<r<1, the optimal nonincreasing count-dependent zCDP budget satisfies v(n)=Theta_r(n^{-2}), while every valid mechanism obeys liminf_{n o\infty} n^2v(n)\ge (1-r)^6/[128r^2(1+r)^2]. Here n is the affected group count, r is the allowed relative mean absolute error, and v(n) bounds both Rényi-divergence directions at every order. A Gaussian release Y=x+Z with sensitivity \Delta and Z\sim\mathcal N(0,\sigma(n)^2) has zCDP parameter \rho(n)=\Delta^2/[2\sigma(n)^2]. Setting v(n)=C/(n+s)^2 therefore gives \sigma(n)=\Delta(n+s)/\sqrt{2C}; for unit sensitivity and large n, E|Z|/n=\sqrt{2/\pi}\,\sigma(n)/n\approx 1/\sqrt{\pi C}. The shift s>0 prevents an undefined or excessively small scale at n=0. The paper's certified upper bound uses a shifted-log Gaussian mechanism; the Gaussian rule here is an implementable approximation and does not inherit the theorem's exact privacy guarantee unless that shift construction is reproduced.

Implementation notes

Integrate this at the release boundary of a private federated learner, or at the all-reduce that publishes per-expert token counts in a private MoE router. For a federated experiment, let each round aggregate a clipped group sum G=\sum_{j=1}^{n}g_j, with per-example clipping ||g_j||_2\leq L. Use sensitivity \Delta=L for add/remove adjacency on the sum, or \Delta=L/n for releasing an average with a fixed public n. First choose a target asymptotic zCDP constant C and a numerical shift s, then compute sigma=Delta*(n+s)/sqrt(2*C). Pseudocode: (1) clip each gradient g_j to norm L; (2) form G=sum(g_j); (3) obtain n from a public group-size field or from a separately privatized lagged count; (4) set sigma as above; (5) release G+Normal(0,sigma^2 I); (6) update the server model with the released aggregate. For MoE, apply the same operation to the vector of expert counts before capacity-factor or auxiliary-load-balancing decisions. The mathematical quantities are the inverse-square budget law, the Gaussian zCDP identity, and the relative-error calculation. Estimate empirically the achieved relative error E||noise||/||G||, privacy accountant composition across rounds, and sensitivity after clipping. Begin with CIFAR-10 or FEMNIST using a 2-layer CNN and 20-client FedAvg, comparing standard constant Gaussian noise against count-aware noise at matched average relative error and matched total zCDP. Success is a lower validation-loss variance for heterogeneous client sizes, or equal accuracy at a smaller privacy budget; also test whether expert-load estimates have fewer false overload decisions at equal privacy.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.