Consider the problem of minimizing a \(C^4\) function \(f \colon \Rn \to \reals\) over the probability simplex: \[ \min_{x \in \Delta^{n-1}} f(x), \qquad \Delta^{n-1} = \{ x \in \Rn : x \geq 0, \ \mathbf{1}^\top x = 1 \}. \tag{1}\] As discussed in a prior post about the Fisher-Rao metric, the simplex can be smoothly parameterized via entrywise squaring (Hadamard product) from a sphere \[ \Sn = \{ y \in \Rn : \|y\|^2 = y_1^2 + \cdots + y_n^2 = 1 \}. \] Indeed, the image of \[ \varphi \colon \Sn \to \Rn, \qquad \varphi(y) = y \odot y \] is exactly the simplex: \(\varphi(\Sn) = \Delta^{n-1}\).
Thus, the problem in Eq. 1 is equivalent to the following smooth problem, \[ \min_{y \in \Sn} g(y), \qquad g(y) := f(y \odot y), \tag{2}\] in the sense that \(y\) solves Eq. 2 if and only if \(\varphi(y)\) solves Eq. 1.
This raises a natural question:
If \(y\) only satisfies necessary optimality conditions for the lifted problem Eq. 2, does its image \(x = \varphi(y)\) also satisfy certain optimality conditions for Eq. 1?
This blog post proves the following result. The precise notions of necessary optimality conditions are important: we discuss them later.
Theorem 1 (\(4 \Rightarrow 2\) for the sphere-to-simplex lift) Let \(f \colon \Rn \to \reals\) be \(C^4\).
If \(y \in \Sn\) satisfies necessary optimality conditions up to order 4 for Eq. 2, then \(x = y \odot y\) satisfies necessary optimality conditions up to order 2 for Eq. 1.
This is sharp, in the sense that the conclusion may fail if \(y\) satisfies necessary optimality conditions only up to order 3.
Why four? This seems to come from the fact that the lift \(\varphi\) is quadratic, with the effect that \(g = f \circ \varphi\) must be inspected at double the order at which one would have to inspect \(f\).
That is compatible with the fact that the lift also satisfies \(2 \Rightarrow 1\) (Levin, Kileel, and Boumal 2025; Li, McKenzie, and Yin 2023), and it transpires in the proofs (see Lemma 1 and Corollary 1 below).
However, this is only a heuristic take: we should not expect every quadratic lift to fare the same way. (For example, even a linear lift can lead to situations where a local minimum \(y\) maps to a point that is not even stationary (Levin, Kileel, and Boumal 2025, Ex. 3.7).) The difficulties stem from the fact that the geometry of the (nonsmooth) image of the lift (here, the simplex) matters a great deal, if only to define necessary optimality conditions in the first place.
This post results from Pablo’s BSc semester project during Spring 2026.
Context
Optimization problems on the simplex are common, as the variable \(x\) may represent probabilities, weights, or mixture coefficients. The lift is appealing for algorithmic reasons, because the simplex is nonsmooth (it has a boundary, faces, and vertices), while the sphere is a smooth manifold: we can apply advanced tools from Riemannian optimization to Eq. 2 but not to Eq. 1.
The lift \(\varphi\) is known as the Hadamard lift, and the transfer of guarantees through such lifts is studied in general in (Levin, Kileel, and Boumal 2025). Global minima of Eq. 2 map to global minima of Eq. 1 simply because \(\varphi\) is onto (take entrywise square roots). Local minima transfer too, because \(\varphi\) is an open map onto \(\Delta^{n-1}\) (see that reference for a proof).
First-order criticality is already subtle: for every tangent direction \(u\), the differential \(\D\varphi(y)[u] = 2\, y \odot u\) vanishes in each coordinate \(i\) with \(y_i = 0\), so that, at a boundary point of the simplex, the first derivative of the lift is blind to the directions that leave the active face, that is, those along which a zero coordinate of \(x\) becomes positive. A net effect is that first-order critical points of Eq. 2 may not map to stationary points of Eq. 1.
Fortunately, a second-order critical point \(y\) of \(g\) on the sphere does map to a stationary point \(x = \varphi(y)\) of \(f\) on the simplex (\(2 \Rightarrow 1\)). Moreover, it is known that \(x\) then also satisfies certain second-order necessary optimality conditions (Li, McKenzie, and Yin 2023), namely, those restricted to the active face of the simplex (see Theorem 2 below).
The full second-order conditions ask for more: nonnegative curvature along every critical direction, including those that leave the active face. This is what Theorem 1 provides, at the expense of requiring \(y\) to be fourth-order critical (\(4 \Rightarrow 2\)).
Optimality conditions on the simplex
A point \(x \in \Delta^{n-1}\) is stationary for \(f\) if there is no feasible first-order descent direction: \(\langle \nabla f(x), v \rangle = v^\top \nabla f(x) \geq 0\) for every \(v\) in the tangent cone \[ \T_x \Delta^{n-1} = \{ v \in \Rn : \mathbf{1}^\top v = 0, \ v_i \geq 0 \textrm{ if } x_i = 0 \}. \] At a stationary point \(x\), second-order conditions test the curvature of \(f\) along the critical directions: feasible directions orthogonal to the gradient. These form the critical cone \[ \mathcal{C}(x) = \bigl\{ q \in \T_x \Delta^{n-1} : \nabla f(x)^\top q = 0 \bigr\}. \tag{3}\] We say \(x\) is second-order stationary for \(f\) on \(\Delta^{n-1}\) if it is stationary and \(q^\top \nabla^2 f(x)\, q \geq 0\) for all \(q \in \mathcal{C}(x)\).
This is the standard second-order necessary optimality condition of nonlinear programming (see, e.g., Ruszczyński 2006, Thm. 3.46). (More generally, the test involves the Hessian of the Lagrangian; since the simplex constraints are affine, that Hessian reduces to \(\nabla^2 f(x)\).)
The subspace of directions tangent to the active face of the simplex is \[ \mathcal{C}_{\Delta}(x) = \bigl\{ q \in \Rn : \mathbf{1}^\top q = 0, \ \ q_i = 0 \textrm{ for all } i \textrm{ with } x_i = 0 \bigr\}. \tag{4}\] At a stationary point \(x\), it sits inside \(\mathcal{C}(x)\): for such \(q\), both \(q\) and \(-q\) lie in the tangent cone, so stationarity forces \(\nabla f(x)^\top q = 0\).
At a boundary point, the inclusion may be strict: some critical directions leave the active face, with \(q_j > 0\) at a boundary coordinate \(j\) (a coordinate with \(x_j = 0\)). We call these the gap directions.
Optimality conditions on the sphere
On the sphere, we need criticality conditions up to order \(4\). The first- and second-order conditions are as usual: Riemannian gradient equal to zero, and Riemannian Hessian positive semidefinite. To go to higher order, the following curve-based notion is convenient.
Definition 1 (Higher-order criticality on a manifold) Fix an integer \(p \geq 1\), and let \(g \colon \calM \to \reals\) be \(C^p\) on a smooth manifold \(\calM\).
A point \(y \in \calM\) is \(p\)-critical for \(g\) if, for every \(\varepsilon > 0\) and every smooth curve \(\gamma \colon (-\varepsilon, \varepsilon) \to \calM\) with \(\gamma(0) = y\) and \(\gamma'(0) \neq 0\), the function \(\psi(t) := g(\gamma(t))\) satisfies the following:
- Either \(\psi^{(k)}(0) = 0\) for all \(k \in \{1, \ldots, p\}\),
- or the smallest \(k \in \{1, \ldots, p\}\) with \(\psi^{(k)}(0) \neq 0\) is even and \(\psi^{(k)}(0) > 0\).
The even-and-positive clause prevents \(\psi\) from decreasing on either side of \(0\) at any order up to \(p\). This is the natural manifold version of the curve-based notion discussed in the blog post of Criscitiello (2024). For \(p = 1\) and \(p = 2\), it recovers the usual Riemannian conditions.
Of course, local minimizers are \(p\)-critical, and \(p\)-criticality implies \(k\)-criticality for all \(1 \leq k \leq p\).
The known second-order correspondence
As \(u\) ranges over the tangent space of the sphere at \(y\), the downstairs direction \(\D\varphi(y)[u] = 2 \, y \odot u\) ranges exactly over \(\mathcal{C}_{\Delta}(x)\) (Eq. 4) with \(x = \varphi(y)\).
The gap directions are precisely the critical directions that the differential of the lift cannot reach. Second-order criticality upstairs therefore probes the curvature of \(f\) only along the active face. This leads to a known correspondence, first noted by Li, McKenzie, and Yin (2023), as follows:
Theorem 2 (Restricted cone guarantee) A point \(y \in \Sn\) is \(2\)-critical for \(g = f \circ \varphi\) on \(\Sn\) (Definition 1) if and only if \(x = \varphi(y)\) is stationary for \(f\) and \[ q^\top \nabla^2 f(x)\, q \geq 0 \qquad \textrm{ for all } q \in \mathcal{C}_{\Delta}(x). \]
Plain second-order criticality upstairs thus provides no information along the gap directions, and the counterexample below shows that the curvature there can indeed be negative. To control the curvature on the full cone \(\mathcal{C}(x)\) (Eq. 3), we look at higher orders upstairs.
Fourth order suffices
We can now state Theorem 1 precisely: if \(y\) is \(4\)-critical for \(g\) (Definition 1), then \(x = \varphi(y)\) is second-order stationary for \(f\). The engine of the proof is a curve construction: every gap direction, invisible to the differential of the lift, is nevertheless realized at second order by some curve on the sphere.
Lemma 1 (Realization of gap directions) Let \(y \in \Sn\) be such that \(x = \varphi(y)\) is stationary for \(f\). Let \(q \in \mathcal{C}(x)\) have a nontrivial boundary part, meaning \(q_j > 0\) for at least one \(j\) with \(x_j = 0\) (in other words, \(q\) is a gap direction). Then there exist \(\varepsilon > 0\) and a \(C^\infty\) curve \(\gamma \colon (-\varepsilon, \varepsilon) \to \Sn\) with \(\gamma(0) = y\), \(\gamma'(0) \neq 0\), and \[ \varphi(\gamma(t)) = x + t^2 q + O(t^4). \]
Proof. We construct the curve by moving at order \(t\) in the boundary coordinates, where squaring creates the boundary part of \(t^2 q\), and at order \(t^2\) in the support coordinates, with coefficients tuned to create the remaining part of \(t^2 q\) while keeping the unnormalized curve on the sphere up to \(O(t^4)\).
Write \(I = \{i : x_i > 0\}\) and \(I^c = \{i : x_i = 0\}\); since \(x = y \odot y\), these are exactly \(\{i : y_i \neq 0\}\) and \(\{i : y_i = 0\}\). Since \(q\) lies in the tangent cone, \(q_j \geq 0\) on \(I^c\); define \(d, e \in \Rn\) by \[ d_i = 0 \textrm{ for } i \in I, \quad d_j = \sqrt{q_j} \textrm{ for } j \in I^c, \qquad e_i = \frac{q_i}{2 y_i} \textrm{ for } i \in I, \quad e_j = 0 \textrm{ for } j \in I^c, \] and consider the curve \(\gamma(t) = \tilde\gamma(t) / \|\tilde\gamma(t)\|\), where \(\tilde\gamma(t) = y + t\, d + t^2 e\). The vector \(d\) is supported where \(y\) vanishes, and \(d\) and \(e\) have disjoint supports; this gives the first two relations below, while the third holds coordinate-wise (on \(I\) it reads \(2 y_i \frac{q_i}{2 y_i} = q_i\), and on \(I^c\) it reads \(d_j^2 = q_j\)): \[ y \odot d = 0 \ (\textrm{hence } y^\top d = 0), \qquad d \odot e = 0 \ (\textrm{hence } d^\top e = 0), \qquad 2 (y \odot e) + d \odot d = q. \] Moreover, using \(\mathbf{1}^\top q = 0\), \[ y^\top e = \sum_{i \in I} y_i \frac{q_i}{2 y_i} = \frac{1}{2} \sum_{i \in I} q_i = -\frac{1}{2} \sum_{j \in I^c} q_j = -\frac{1}{2} \|d\|^2. \] The relations above kill every term of order \(t\), \(t^2\), and \(t^3\) in the squared norm: \[ \|\tilde\gamma(t)\|^2 = 1 + 2t\, y^\top d + t^2 \bigl( 2 y^\top e + \|d\|^2 \bigr) + 2 t^3\, d^\top e + t^4 \|e\|^2 = 1 + t^4 \|e\|^2, \] so \(\|\tilde\gamma(t)\|^2 \geq 1\) for all \(t\) (hence \(\gamma\) is well defined and \(C^\infty\)), the normalization perturbs \(\tilde\gamma\) only at order \(t^4\), and \(\gamma(t) = y + t\, d + t^2 e + O(t^4)\). In particular, \(\gamma(0) = y\) and \(\gamma'(0) = d \neq 0\) (the nontrivial boundary part guarantees \(d \neq 0\)). Finally, using the first two relations again and then the third, \[ \tilde\gamma(t) \odot \tilde\gamma(t) = y \odot y + t^2 \bigl( 2(y \odot e) + d \odot d \bigr) + t^4 (e \odot e) = x + t^2 q + t^4 (e \odot e), \] and dividing by \(\|\tilde\gamma(t)\|^2 = 1 + t^4\|e\|^2\) gives \(\varphi(\gamma(t)) = x + t^2 q + O(t^4)\).
Along this curve, the first three derivatives of the function \[ \psi(t) = g(\gamma(t)) = f\big( x + t^2 q + O(t^4) \big) \] vanish at \(t = 0\), and the fourth derivative at \(t = 0\) is a positive multiple of the downstairs quadratic form we want to control.
Corollary 1 (Fourth-order expansion along the lifted curve) Under the assumptions of Lemma 1, and with \(\gamma\) the curve constructed in its proof, the function \(\psi(t) := g(\gamma(t))\) satisfies \[ \psi'(0) = \psi''(0) = \psi'''(0) = 0 \qquad \textrm{and} \qquad \psi^{(4)}(0) = 12\, q^\top \nabla^2 f(x)\, q. \]
Proof. We expand \(f\) at \(x\) along the downstairs displacement of the curve. Let \(\delta(t) := \varphi(\gamma(t)) - x = t^2 q + O(t^4)\). The Taylor expansion of \(f\) at \(x\) gives \[ \psi(t) = f(x) + \langle \nabla f(x), \delta(t) \rangle + \frac{1}{2}\, t^4\, q^\top \nabla^2 f(x)\, q + o(t^4). \] The gradient term vanishes exactly. Since the simplex satisfies LICQ at every point, stationarity is equivalent to the first-order KKT conditions: there exist \(\mu \in \reals\) and \(\lambda \in \Rn\) with \(\nabla f(x) = -\mu \mathbf{1} + \lambda\), where \(\lambda\) vanishes on \(I = \{i : x_i > 0\}\) and is nonnegative on \(I^c = \{i : x_i = 0\}\). Now \(q\) is critical, so \(0 = \nabla f(x)^\top q = -\mu\, \mathbf{1}^\top q + \lambda^\top q = \sum_{j \in I^c} \lambda_j q_j\), where the last equality uses \(\mathbf{1}^\top q = 0\) and the support of \(\lambda\); the terms are nonnegative, so \(q_j = 0\) whenever \(\lambda_j > 0\). Let \(J = \{j \in I^c : \lambda_j = 0\}\). The constructed curve has \(\gamma_j(t) = 0\) for every \(j \in I^c \setminus J\) (there, \(y_j = 0\); \(q_j = 0\) as just shown, hence \(d_j = \sqrt{q_j} = 0\); and \(e_j = 0\) on all of \(I^c\) by construction), so \(\delta(t)\) is supported on \(I \cup J\). On \(I \cup J\) the multipliers \(\lambda_k\) vanish, so \(\nabla f(x)_k = -\mu\) there, and \[ \langle \nabla f(x), \delta(t) \rangle = -\mu\, \mathbf{1}^\top \delta(t) = 0, \] where the last equality holds because \(\varphi(\gamma(t))\) stays in the simplex: \(\mathbf{1}^\top \varphi(\gamma(t)) = 1 = \mathbf{1}^\top x\). Hence \(\psi(t) = f(x) + \frac{1}{2} t^4\, q^\top \nabla^2 f(x)\, q + o(t^4)\); since \(\psi\) is \(C^4\) (\(f\) is \(C^4\) and \(\gamma\) is \(C^\infty\)), the claims follow by identifying Taylor coefficients: \(\psi^{(4)}(0) = 4! \cdot \frac{1}{2}\, q^\top \nabla^2 f(x)\, q = 12\, q^\top \nabla^2 f(x)\, q\).
We now have the appropriate tools to prove the main theorem.
Proof (Theorem 1). We treat the two kinds of critical directions separately: those tangent to the active face are covered by Theorem 2 (due to Li, McKenzie, and Yin (2023)), while Lemma 1 and Corollary 1 handle the gap directions.
Since \(y\) is \(4\)-critical for \(g\) on \(\Sn\), it is in particular \(2\)-critical. By Theorem 2, \(x = \varphi(y)\) is stationary for \(f\) and \(q^\top \nabla^2 f(x)\, q \geq 0\) holds for every \(q \in \mathcal{C}_{\Delta}(x)\).
Now let \(q \in \mathcal{C}(x)\) be arbitrary. Since \(q\) lies in the tangent cone, \(q_j \geq 0\) at the boundary coordinates. Either \(q_j = 0\) for all of them, in which case \(q \in \mathcal{C}_{\Delta}(x)\) and we are done, or \(q\) has a nontrivial boundary part. In the latter case, the curve \(\gamma\) constructed in the proof of Lemma 1 is an admissible test curve, and Corollary 1 gives \(\psi'(0) = \psi''(0) = \psi'''(0) = 0\) with \(\psi^{(4)}(0) = 12\, q^\top \nabla^2 f(x)\, q\). Since \(y\) is \(4\)-critical, the first nonzero derivative of \(\psi\) at \(0\) up to order \(4\) (if any) occurs at an even order and is positive. The first three vanish, so \(\psi^{(4)}(0)\) cannot be negative: \(q^\top \nabla^2 f(x)\, q \geq 0\).
The Hessian condition therefore holds on all of \(\mathcal{C}(x)\), so \(x\) is second-order stationary.
The sharpness claim follows from the counterexample below.
We call this mechanism order doubling: the lift \(\varphi\) has degree \(2\), and it so happens that, to certify second-order stationarity downstairs, it suffices to certify twice that order upstairs.
Second order does not suffice
We now show that \(2\)-criticality upstairs does not guarantee second-order stationarity downstairs. Take \(n = 2\) and \[ f(x) = -x_2^2, \qquad \textrm{so that} \qquad g(y) = -y_2^4, \] and consider the point \(y = (1, 0)\), which maps to the vertex \(x = \varphi(y) = (1, 0)\). Upstairs, one checks that \(y\) is \(2\)-critical for \(g\) on \(\mathbb{S}^1\), as the Riemannian gradient and Hessian of \(g\) vanish at \(y\). Downstairs, \(\nabla f(x) = 0\), so \(x\) is stationary; one checks that \(q = (-1, 1)\) lies in the critical cone \(\mathcal{C}(x)\) and that \(q^\top \nabla^2 f(x)\, q = -2 < 0\), so \(x\) is not second-order stationary. Hence \(2 \not\Rightarrow 2\), and the failure occurs exactly where expected: \(q\) is a gap direction, invisible to the differential of the lift.
Neither does third order
The same example shows that \(3\)-criticality upstairs does not guarantee second-order stationarity downstairs either. For every \(\varepsilon > 0\), every smooth curve \(\gamma \colon (-\varepsilon, \varepsilon) \to \mathbb{S}^1\) with \(\gamma(0) = y\) and \(\gamma'(0) \neq 0\) can be written as \(\gamma(t) = (\cos \theta(t), \sin \theta(t))\) for some smooth function \(\theta\) with \(\theta(0) = 0\), so that \(\psi(t) = g(\gamma(t)) = -\sin(\theta(t))^4\). Since the first three derivatives of \(s \mapsto -\sin(s)^4\) all vanish at \(0\), the chain rule gives \(\psi'(0) = \psi''(0) = \psi'''(0) = 0\) for every such curve: \(y\) is \(3\)-critical, yet \(x\) is not second-order stationary. Hence \(3 \not\Rightarrow 2\): the threshold \(4\) in Theorem 1 cannot be lowered. Note that taking \(\theta(t) = t\) gives \(\psi^{(4)}(0) = -24 < 0\), an even-order negative derivative: \(y\) is indeed not \(4\)-critical, consistent with Theorem 1.
Optimization over stochastic matrices
The results above extend blockwise to products of simplices \(\Delta^{n_1 - 1} \times \cdots \times \Delta^{n_m - 1}\), lifted to products of spheres, where the variable consists of \(m\) probability vectors. When all the \(n_i\) are equal to a common \(n\), the product may be identified with the set of column-stochastic \(n \times m\) matrices.
Open questions
Two questions stand out.
Computational tractability. Finding (or simply recognizing) a fourth-order local minimum is NP-hard even for smooth unconstrained problems on \(\Rn\) (Anandkumar and Ge 2016). That hardness does not directly apply here, for three reasons: the notion of criticality that leads to hardness (neighborhood-based) is stronger than ours; we may not need full \(4\)-criticality (even curves-based); and the lifted functions \(g = f \circ \varphi\) have a special structure, since each \(y_i\) enters \(g\) only through \(y_i^2\). It is natural to ask whether \(4\)-criticality is tractable for this class. A positive answer would give a practical route from smooth optimization on the sphere to certified second-order stationary points on the simplex.
Order doubling for other lifts. Since the sphere-to-simplex lift is quadratic, it is natural to expect “order doubling” as a requirement. The more surprising part is that doubling is enough (we do indeed have \(2 \Rightarrow 1\) and \(4 \Rightarrow 2\)). Does the same order-doubling phenomenon hold for other quadratic lifts of interest, such as the Burer–Monteiro parametrization \(Y \mapsto YY^\top\) of bounded-rank positive semidefinite matrices? The latter does enjoy \(2 \Rightarrow 1\) (Levin, Kileel, and Boumal 2025). Investigating \(4 \Rightarrow 2\) would require taking a close look at the tangent cones and second-order tangent sets of the image of the lift.
References
Citation
@online{habib2026,
author = {Habib, Pablo and Musat, Andreea and Boumal, Nicolas},
title = {The Sphere-to-Simplex Lift Enjoys \$4 {\textbackslash
Rightarrow} 2\$ (and \$3\$ Is Not Enough)},
date = {2026-07-14},
url = {www.racetothebottom.xyz/posts/sphere-to-simplex/},
langid = {en},
abstract = {Entrywise squaring (\$y \textbackslash mapsto y
\textbackslash odot y\$) maps the unit sphere onto the probability
simplex. As a result, to minimize \$f\$ over the simplex, one can
minimize \$g(y) = f(y \textbackslash odot y)\$ on the sphere
instead. If \$y\$ is second-order critical for \$g\$, then its image
\$x = y \textbackslash odot y\$ satisfies first-order optimality
conditions for \$f\$, but \$x\$ may only satisfy part of the
second-order conditions. We show that, for \$x\$ to satisfy the full
second-order conditions, it is enough to require \$y\$ to be
fourth-order critical (and this is sharp).}
}