Hyperspherical Simplex Representations for Backward-Compatible Retrieval

Ricci, Simone; Biondi, Niccolò; Pernici, Federico; Del Bimbo, Alberto

Hyperspherical Simplex Representations for Backward-Compatible Retrieval

Softmax outputs and logits of independently trained models share a simplex geometry fixed by the label space. We turn it into backward-compatible features, with no compatibility training.

Simone Ricci^* Niccolò Biondi^* Federico Pernici Alberto Del Bimbo

Media Integration and Communication Center (MICC), University of Florence ^*equal contribution

Paper Code soon Video

0:00 / 0:00 Full screen ↗

TL;DR Softmax cross-entropy with one-hot labels drives every model toward the same simplex, fixed by the label space. We center and normalize the softmax outputs (PSP) or normalize the logits (LSP), and correct for new classes with a fixed linear map. We prove that the backward-compatibility inequalities then hold in expectation on the hypersphere. Independently trained models become compatible without compatibility losses, learned mappings or architecture changes.

Abstract

Training modern AI models has become increasingly expensive, and updating deployed models can alter their internal feature representations, affecting applications built upon them. In retrieval systems, such updates often require a complete gallery re-indexing, which is computationally expensive at scale. Existing backward-compatibility methods enable direct comparison between representations produced by updated and earlier models, avoiding re-indexing. However, they typically require compatibility-specific training, learned mapping functions, or modifications to the model architecture, and often depend explicitly on the previous model. In this paper, we propose hyperspherical simplex representations derived from softmax outputs and logits for backward-compatible retrieval across independently trained models. These representations leverage the simplex structure induced by softmax cross-entropy training with one-hot labels, which provides a common geometric reference across model updates. Since this reference depends on the number of classes, introducing new ones during an update expands the simplex, thereby misaligning the prototypes of existing classes with their earlier counterparts. We therefore derive a deterministic linear transformation to restore their alignment. We further provide a theoretical analysis of the formal backward-compatibility definition, identifying two geometric conditions under which its inequalities hold in expectation, assuming von Mises–Fisher class distributions whose concentration increases across updates. Aligned regular-simplex representations satisfy these conditions, providing a theoretical basis for the proposed representations under the stated assumptions. Without using additional compatibility-specific losses, learned mappings, or modifications to the model architecture, the proposed representations achieve competitive results on standard compatibility benchmarks, including scenarios with expanded class sets, more expressive architectures, and independently trained models from public repositories.

The compatibility problem

A retrieval gallery is encoded once with the deployed model $\phi^k$. When the model is updated to $\phi^t$, new query features must be compared with the stored gallery features. Independently trained models do not share a feature space, so the comparison fails unless the whole gallery is re-encoded.

Definition 3.1 · Backward compatibility (Shen et al., 2020)

$$d\big(\mathbf h^t_i,\mathbf h^k_j\big)\le d\big(\mathbf h^k_i,\mathbf h^k_j\big)\;\;\forall\,(i,j):y_i=y_j\qquad(1a)$$

$$d\big(\mathbf h^t_i,\mathbf h^k_j\big)\ge d\big(\mathbf h^k_i,\mathbf h^k_j\big)\;\;\forall\,(i,j):y_i\ne y_j\qquad(1b)$$

New-to-old comparisons must group same-class items, and separate different classes, at least as well as old-to-old comparisons. Empirically, this is checked with the criterion $M(\phi^t,\phi^k)>M(\phi^k,\phi^k)$ on a retrieval metric $M$.

Most methods obtain compatibility by training the new model against the old one (e.g., BCT), by learning mappings between feature spaces, or by sharing a pre-allocated fixed classifier across updates. We need none of these: the reference is already provided by the label space.

Method

1Center

Map softmax outputs from the probability simplex onto a regular simplex centered at the origin with $\mathbf P_{k,k}=\mathbf I-\tfrac1{C^k}\mathbf J$. Logits are already centered and aligned with the same prototypes.

2Normalize

Project onto the unit hypersphere. Class prototypes $\mathbf v_c=\mathbf e_c-\mathbf o$ are fixed by the one-hot coordinates, so they are the same for every model trained on the same classes.

3Correct

If the update adds classes, apply $\mathbf P_{t,k}=[\,\mathbf P_{k,k}\mid\mathbf 0\,]$ to new queries. It is deterministic and data-free, and restores the alignment of the old-class prototypes.

$$\textbf{PSP:}\;\;\mathbf h^k=\frac{\mathbf P_{k,k}\,\sigma(\mathbf z^k)}{\big\|\mathbf P_{k,k}\,\sigma(\mathbf z^k)\big\|_2}\qquad\qquad\textbf{LSP:}\;\;\mathbf h^k=\frac{\mathbf z^k}{\|\mathbf z^k\|_2}$$

Why the softmax gives a shared reference

Cross-entropy with one-hot labels drives softmax outputs to the vertices $\mathbf e_c$ of the probability simplex $\Delta^{C-1}$. These vertices depend only on the label space, not on the model. The simplex, however, lies in the non-negative orthant, so normalized softmax outputs of different classes are at most $\pi/2$ apart.

Centering moves the simplex to the origin. The prototypes $\mathbf v^k_c=\mathbf e_c-\mathbf o^k$ form a regular simplex with pairwise angle $\arccos\!\big(-\tfrac1{C-1}\big)>\pi/2$. Under a logit unconstrained-features model, logits align with the same directions (Prop. 3.3), which gives LSP.

Probability simplex in three dimensions with softmax outputs near its vertices, the centered prototypes v1, v2, v3, and von Mises-Fisher distributions on the sphere. — Fig. 1. Softmax outputs lie on $\Delta^{C-1}$ (colored), are driven to its vertices, and are mapped by $\mathbf P_{k,k}$ onto a regular simplex whose vertices define the class prototypes. After normalization, each class is modeled by a vMF distribution around its prototype.

New classes rotate the simplex

Adding classes moves the simplex center, so every existing prototype rotates by the same angle (Eq. 6), and the prototypes of the new classes are orthogonal to the old ones (Prop. 3.4). The correction $\mathbf P_{t,k}$ centers the old-class coordinates and drops the new ones, which maps each updated prototype back onto its step-$k$ counterpart (Prop. 3.5).

$$\theta_c^{k\to t}=\arccos\!\left(\sqrt{\frac{C^k-1}{C^k}\cdot\frac{C^t}{C^t-1}}\right)\qquad 2\!\to\!3:\ 30.0^\circ\quad 5\!\to\!10:\ 19.5^\circ$$

Three update steps: a 1-simplex with two classes, a 2-simplex after adding a third class, and a 3-simplex after adding a fourth, with the prototypes of existing classes rotating and the correction mapping them back. — Fig. 4. Prototypes across updates that add classes. Dashed arrows: the rotation caused by the new simplex. Solid black arrows: $\mathbf P_{t,k}$, which restores alignment with the earlier step.

When does backward compatibility hold?

We model features on the unit hypersphere and derive closed-form expected cosine distances. Two geometric conditions then make both inequalities of Definition 3.1 hold in expectation.

Assumptions on the hypersphere

the features of each class $c$ are drawn independently from a von Mises–Fisher distribution $\mathrm{vMF}(\boldsymbol\mu_c,\kappa_c)$ on $\mathbb S^{d-1}$, whose mean direction $\boldsymbol\mu_c$ is the class prototype;
the concentration of each class increases with the update, $\kappa^t_c>\kappa^k_c$.

Theorem 3.6 · Compatibility theorem Under (i) and (ii), the compatibility inequalities (1a) and (1b) are satisfied in expectation if

corresponding class prototypes are aligned across updates, and
the angle between prototypes of different classes is greater than $\pi/2$.

The proof rests on Lemma G.1. For independent $\mathbf X_1\sim\mathrm{vMF}(\boldsymbol\mu_1,\kappa_1)$ and $\mathbf X_2\sim\mathrm{vMF}(\boldsymbol\mu_2,\kappa_2)$,

$$\mathbb E\big[1-\mathbf X_1^\top\mathbf X_2\big]=1-m_d(\kappa_1)\,m_d(\kappa_2)\,\boldsymbol\mu_1^\top\boldsymbol\mu_2,\qquad m_d(\kappa)=\frac{I_{d/2}(\kappa)}{I_{d/2-1}(\kappa)},$$

where $m_d(\kappa)$ is the mean resultant length, $\mathbb E[\mathbf X]=m_d(\kappa)\,\boldsymbol\mu$, which increases with $\kappa$. A more concentrated new model multiplies $\boldsymbol\mu_1^\top\boldsymbol\mu_2$ by a larger factor. This brings aligned same-class pairs closer, but pushes different-class pairs apart only when $\boldsymbol\mu_1^\top\boldsymbol\mu_2<0$.

Corollary 3.7 · Aligned regular simplex Prototypes forming a regular simplex centered at the origin have $\boldsymbol\mu_i^\top\boldsymbol\mu_j=-\tfrac1{C-1}<0$. If they are also aligned across updates, both conditions hold. PSP satisfies this geometry by construction, and LSP does so under the logit-UFM: the centering $\mathbf P_{k,k}$ provides (b), and the correction $\mathbf P_{t,k}$ provides (a).

Explore the theorem

Two classes on the hypersphere. Old features ($\phi^k$) follow $\mathrm{vMF}(\boldsymbol\mu_c,\kappa^k)$ and new ones ($\phi^t$) follow $\mathrm{vMF}(\boldsymbol\mu_c,\kappa^t)$. The bars are the exact expected cosine distances from Lemma G.1. Break one condition and watch the corresponding inequality fail.

Preset

Dashed arrows: $\mathbb E[\mathbf X]$ for the old model. Solid arrows: the new model. Red: class 1, blue: class 2.

angle $\angle(\boldsymbol\mu_1,\boldsymbol\mu_2)$ misalignment $\delta$ old concentration $\kappa^k$ new concentration $\kappa^t$ dimension $d$

$\delta$ rotates the new prototype of class 1 away from class 2, so only (1a) is affected. Presets use the regular-simplex angle $\arccos(-1/(C-1))$ and the misalignment of Eq. 6. Margins are defined as in Appendix I: old→old minus new→old distance for (1a), and the reverse for (1b).

Each operation provides one condition

On CIFAR-10 (ResNet-18, 5 → 10 classes, 9 independently trained model pairs), removing either operation of PSP breaks the inequality that the theorem predicts.

Representation	$\mathbf P_{k,k}$	$\mathbf P_{t,k}$	margin (1a)	margin (1b)	both hold
Softmax	–	–	−11.1 ± 3.0	−5.5 ± 0.9	0 / 9
PSP without $\mathbf P_{t,k}$	✓	–	−56.8 ± 3.3	−14.2 ± 0.8	0 / 9
PSP without $\mathbf P_{k,k}$	–	✓^*	+12.0 ± 1.9	−13.3 ± 1.0	0 / 9
PSP	✓	✓	+15.5 ± 2.5	+3.9 ± 0.6	9 / 9

Average margins of (1a) and (1b), ×10⁻³, mean ± std over 9 pairs. Positive values mean the inequality holds in expectation. ^*$\mathbf P_{t,k}$ with $\mathbf P_{k,k}$ replaced by the identity. The measured misalignment without correction is 18.8°–20.0°, against the 19.5° predicted by Eq. 6.

Results

We evaluate on the standard compatibility benchmarks (extended classes, extended backbone, and both), and between independently pretrained public models. Compatibility across $T$ updates is summarized by AC, the fraction of update pairs that satisfy the empirical criterion, and ACA, which averages the cross-model accuracy over compatible pairs only.

~~0.11~~0.39

AC on CIFAR-100 after 50 updates: best prior method (CoReS) vs PSP (LSP: 0.36)

~~0.20~~1.00

AC on ImageNet-1K with 5 updates: best prior method (BCT) vs PSP

010 / 10

compatible pairs among 5 public torchvision models, encoder features vs PSP

159×

less gallery storage on Google Landmarks v2 with top-256 sparsification, with unchanged results

Independently trained models from a public repository

Five torchvision models pretrained on ImageNet-1K, of increasing capacity. They share nothing but the label space. Each entry is CMC@1 on ImageNet-1K, with queries from the row model and the gallery from the column model. Red entries fail the empirical criterion.

Features

Extended classes, many updates (CIFAR-100, ResNet-18)

Method	2 steps			5 steps			20 steps			50 steps
Method	AC	AA	ACA	AC	AA	ACA	AC	AA	ACA	AC	AA	ACA
Baseline	0	29.63	0	0	13.60	0	0	4.22	0	0	2.29	0
BCT	1	40.64	35.51	0	31.03	0	0	22.66	0	0.01	15.78	0.84
RACT	1	44.19	45.09	0.60	30.43	16.53	0.05	21.61	2.42	0	16.71	0
AdvBCT	0	35.32	0	0.40	26.13	11.67	0.02	19.79	0.19	0	14.70	0.03
LCE	1	43.48	40.71	0.10	32.63	4.63	0	20.95	0.25	0	13.81	0.03
CoReS	1	40.94	35.21	0	29.63	0	0.12	24.88	4.61	0.11	23.50	3.91
PSP	1	36.31	29.05	0.90	26.04	21.76	0.52	20.56	12.99	0.39	19.42	10.76
LSP	1	41.14	36.38	0.70	30.36	21.91	0.44	24.11	13.26	0.36	22.68	11.25

From Table 1. Dark and light blue mark the highest and second-highest value of each metric. ImageNet-1K and Google Landmarks v2 (Table 2), extended-backbone and combined benchmarks (Table 4), and Places365 and ViT sequences (Table 5) are in the paper.

Takeaways

A common reference instead of pairwise constraints. Methods that enforce compatibility with the previous model accumulate misalignment over many updates. PSP and LSP refer every model to the same simplex, so their advantage grows with the number of updates.
PSP vs LSP. PSP collapses more tightly onto the simplex and is compatible more often. LSP keeps more within-class spread, and reaches higher accuracy (AA) on the standard benchmarks and under domain shift (Places365).
Public models, closed and open set. On torchvision models, PSP reaches AC = 1 on ImageNet-1K for both the CNN and the ViT sequences, and AC = 0.90 / 1.00 on Places365. Encoder features reach AC = 0.
Scales with the number of classes. Softmax is rank-preserving, so keeping the top-k entries keeps the ranking. Top-256 leaves PSP unchanged on ImageNet-1K and GLDv2, and raises LSP's ImageNet-1K AC from 0.50 to 0.70.

Code

The code will be released in the repository github.com/miccunifi/PSP.

Limitations

PSP and LSP need models trained with softmax cross-entropy on one-hot labels. Self-supervised encoders can be used by training a linear classifier on a frozen backbone (Appendix L.1).
The mapping between class labels and output indices must be preserved across updates. A known reordering can be undone with a fixed permutation.
The guarantees hold in expectation under the vMF and increasing-concentration assumptions, which approximate learned representations rather than characterize them exactly.

Citation

@article{ricci2026hyperspherical,
  title   = {Hyperspherical Simplex Representations for Backward-Compatible Retrieval},
  author  = {Ricci, Simone and Biondi, Niccol{\`o} and Pernici, Federico and Del Bimbo, Alberto},
  journal = {arXiv preprint arXiv:2610.29550},
  year    = {2026}
}