You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- Remove redundant "more" from "The more broader term"
- Fix spelling of "architecture" (was "arhitecture") in multiple code examples
- Add missing article "a" in "is a large set corpus"
- Fix spelling of "Overview" (was "Overiew") in documentation reference
Copy file name to clipboardExpand all lines: train-sparse-encoder.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -57,7 +57,7 @@ We list the most prominent ones along with their benchmark results in [Pretraine
57
57
58
58
## What are Sparse Embedding models?
59
59
60
-
The more broader term "embedding models" refer to models that convert some input, usually text, into a vector representation (embedding) that captures the semantic meaning of the input. Unlike with the raw inputs, you can perform mathematical operations on these embeddings, resulting in similarity scores that can be used for various tasks, such as search, clustering, or classification.
60
+
The broader term "embedding models" refer to models that convert some input, usually text, into a vector representation (embedding) that captures the semantic meaning of the input. Unlike with the raw inputs, you can perform mathematical operations on these embeddings, resulting in similarity scores that can be used for various tasks, such as search, clustering, or classification.
61
61
62
62
With dense embedding models, i.e. the common variety, the embeddings are typically low-dimensional vectors (e.g., 384, 768, or 1024 dimensions) where most values are non-zero. Sparse embedding models, on the other hand, produce high-dimensional vectors (e.g., 30,000+ dimensions) where most values are zero. Usually, each active dimension (i.e. the dimension with a non-zero value) in a sparse embedding corresponds to a specific token in the model's vocabulary, allowing for interpretability.
63
63
@@ -210,7 +210,7 @@ from sentence_transformers import SparseEncoder
210
210
211
211
model = SparseEncoder("google-bert/bert-base-uncased")
@@ -831,7 +831,7 @@ The following pages contain training examples with explanations as well as links
831
831
- [MSMARCO](https://sbert.net/examples/sparse_encoder/training/ms_marco/README.html) - Example training scripts for training on the MSMARCO information retrieval dataset.
832
832
- [Retrievers](https://sbert.net/examples/sparse_encoder/training/retrievers/README.html) - Example training scripts for training on generic information retrieval datasets.
833
833
- [Natural Language Inference](https://sbert.net/examples/sparse_encoder/training/nli/README.html) - Natural Language Inference (NLI) data can be quite helpful to pre-train and fine-tune models to create meaningful sparse embeddings.
834
-
- [Quora Duplicate Questions](https://sbert.net/examples/sparse_encoder/training/quora_duplicate_questions/README.html) - Quora Duplicate Questions is large set corpus with duplicate questions from the Quora community. The folder contains examples how to train models for duplicate questions mining andfor semantic search.
834
+
- [Quora Duplicate Questions](https://sbert.net/examples/sparse_encoder/training/quora_duplicate_questions/README.html) - Quora Duplicate Questions isa large set corpus with duplicate questions from the Quora community. The folder contains examples how to train models for duplicate questions mining andfor semantic search.
835
835
- [STS](https://sbert.net/examples/sparse_encoder/training/sts/README.html) - The most basic method to train models is using Semantic Textual Similarity (STS) data. Here, we use sentence pairs and a score indicating the semantic similarity.
836
836
837
837
### Documentation
@@ -842,12 +842,12 @@ Additionally, the following pages may be useful to learn more about Sentence Tra
* [Training Overview](https://sbert.net/docs/sparse_encoder/training_overview.html) (This blogpost is a distillation of the Training Overiew documentation)
845
+
* [Training Overview](https://sbert.net/docs/sparse_encoder/training_overview.html) (This blogpost is a distillation of the Training Overview documentation)
0 commit comments