Hyperbolic Embedding as a Retriever

[HamedBabaei]

• reference: https://github.com/Melika-Ayoughi/Optimal-Hierarchy
• paper: https://link.springer.com/chapter/10.1007/978-3-031-94575-5_20

Overall, we need a Hyperbolic Embedding-based Retriever, so the above references would give you an idea of what hyperbolic embeddings are. In the end, a hyperbolic embedding should give any given text an embedding (this is the final goal of this task).

[HamedBabaei]

Q1: Do we have any pre-trained model that works for hyperbolic embeddings?

Note: There is currently no widely available pretrained text embedding model that directly outputs hyperbolic embeddings.

Study Material:
Hyperbolic Embedding Space:
https://github.com/marlin-codes/Awesome-Hyperbolic-Representation-and-Deep-Learning

Hyperbolic Deep Learning for Foundation Models [Hiren Madhu, Neil He, Menglin Yang]
https://arxiv.org/html/2507.17787v1

PoinCare Glove (Hyperbolic Word Embeddings) [Alexandru Tifrea and Gary Becigneul and Octavian-Eugen Gane]

https://arxiv.org/abs/1810.06546

Multi-relational Poincaré Graph Embeddings [Ivana Balažević, Carl Allen, Timothy Hospedales]
https://arxiv.org/abs/1905.09791
Github: https://github.com/ibalazevic/multirelational-poincare (Need to train the model) (Example trained with WN18RR Dataset)

Experimental Models:
Hierarchical Mamba (HiM):
Hierarchical Mamba Meets Hyperbolic Geometry: A New Paradigm for Structured Language Embeddings
Hyperbolic Sentence Embeddings (HSE) (Nickel & Kiela, 2018):
Paper: https://arxiv.org/abs/1806.04313
Github: https://github.com/facebookresearch/poincare-embeddings

Multi-relational Poincaré Graph Embeddings [Ivana Balažević, Carl Allen, Timothy Hospedales]
Github: https://github.com/ibalazevic/multirelational-poincare
Paper: https://arxiv.org/abs/1905.09791

Q2: Can we pass Ontology to these models, if yes then how?
Yes — but only to models that are built for graph or hierarchical data, such as:
• MuRP

Q3: What if we don’t have an Ontology? Then what can we do?
If you don’t have an ontology, you have two main options:
Option 1: Use pretrained text embeddings → project to hyperbolic space

1. Get embeddings from a pretrained Euclidean model (e.g., sentence-transformers, Qwen2.5-Instruct).
2. Apply a hyperbolic projection (e.g., map vectors to a Poincaré ball)