Skip to content

Commit fd9d0ce

Browse files
fix: correct typos (#2925)
- Remove redundant "more" from "The more broader term" - Fix spelling of "architecture" (was "arhitecture") in multiple code examples - Add missing article "a" in "is a large set corpus" - Fix spelling of "Overview" (was "Overiew") in documentation reference
1 parent ea837a5 commit fd9d0ce

File tree

1 file changed

+7
-7
lines changed

1 file changed

+7
-7
lines changed

train-sparse-encoder.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ We list the most prominent ones along with their benchmark results in [Pretraine
5757

5858
## What are Sparse Embedding models?
5959

60-
The more broader term "embedding models" refer to models that convert some input, usually text, into a vector representation (embedding) that captures the semantic meaning of the input. Unlike with the raw inputs, you can perform mathematical operations on these embeddings, resulting in similarity scores that can be used for various tasks, such as search, clustering, or classification.
60+
The broader term "embedding models" refer to models that convert some input, usually text, into a vector representation (embedding) that captures the semantic meaning of the input. Unlike with the raw inputs, you can perform mathematical operations on these embeddings, resulting in similarity scores that can be used for various tasks, such as search, clustering, or classification.
6161

6262
With dense embedding models, i.e. the common variety, the embeddings are typically low-dimensional vectors (e.g., 384, 768, or 1024 dimensions) where most values are non-zero. Sparse embedding models, on the other hand, produce high-dimensional vectors (e.g., 30,000+ dimensions) where most values are zero. Usually, each active dimension (i.e. the dimension with a non-zero value) in a sparse embedding corresponds to a specific token in the model's vocabulary, allowing for interpretability.
6363

@@ -210,7 +210,7 @@ from sentence_transformers import SparseEncoder
210210

211211
model = SparseEncoder("google-bert/bert-base-uncased")
212212
# SparseEncoder(
213-
# (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'arhitecture': 'BertForMaskedLM'})
213+
# (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
214214
# (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
215215
# )
216216
```
@@ -239,7 +239,7 @@ model = SparseEncoder(modules=[router], similarity_fn_name="dot")
239239
# SparseEncoder(
240240
# (0): Router(
241241
# (query_0_SparseStaticEmbedding): SparseStaticEmbedding ({'frozen': False}, dim:30522, tokenizer: BertTokenizerFast)
242-
# (document_0_MLMTransformer): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'arhitecture': 'BertForMaskedLM'})
242+
# (document_0_MLMTransformer): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertForMaskedLM'})
243243
# (document_1_SpladePooling): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
244244
# )
245245
# )
@@ -307,7 +307,7 @@ from sentence_transformers import SparseEncoder
307307
308308
model = SparseEncoder("mixedbread-ai/mxbai-embed-large-v1")
309309
# SparseEncoder(
310-
# (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'arhitecture': 'BertModel'})
310+
# (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
311311
# (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
312312
# (2): SparseAutoEncoder({'input_dim': 1024, 'hidden_dim': 4096, 'k': 256, 'k_aux': 512, 'normalize': False, 'dead_threshold': 30})
313313
# )
@@ -831,7 +831,7 @@ The following pages contain training examples with explanations as well as links
831831
- [MS MARCO](https://sbert.net/examples/sparse_encoder/training/ms_marco/README.html) - Example training scripts for training on the MS MARCO information retrieval dataset.
832832
- [Retrievers](https://sbert.net/examples/sparse_encoder/training/retrievers/README.html) - Example training scripts for training on generic information retrieval datasets.
833833
- [Natural Language Inference](https://sbert.net/examples/sparse_encoder/training/nli/README.html) - Natural Language Inference (NLI) data can be quite helpful to pre-train and fine-tune models to create meaningful sparse embeddings.
834-
- [Quora Duplicate Questions](https://sbert.net/examples/sparse_encoder/training/quora_duplicate_questions/README.html) - Quora Duplicate Questions is large set corpus with duplicate questions from the Quora community. The folder contains examples how to train models for duplicate questions mining and for semantic search.
834+
- [Quora Duplicate Questions](https://sbert.net/examples/sparse_encoder/training/quora_duplicate_questions/README.html) - Quora Duplicate Questions is a large set corpus with duplicate questions from the Quora community. The folder contains examples how to train models for duplicate questions mining and for semantic search.
835835
- [STS](https://sbert.net/examples/sparse_encoder/training/sts/README.html) - The most basic method to train models is using Semantic Textual Similarity (STS) data. Here, we use sentence pairs and a score indicating the semantic similarity.
836836
837837
### Documentation
@@ -842,12 +842,12 @@ Additionally, the following pages may be useful to learn more about Sentence Tra
842842
* [Quickstart](https://sbert.net/docs/quickstart.html)
843843
* [Usage](https://sbert.net/docs/sparse_encoder/usage/usage.html)
844844
* [Pretrained Models](https://sbert.net/docs/sparse_encoder/pretrained_models.html)
845-
* [Training Overview](https://sbert.net/docs/sparse_encoder/training_overview.html) (This blogpost is a distillation of the Training Overiew documentation)
845+
* [Training Overview](https://sbert.net/docs/sparse_encoder/training_overview.html) (This blogpost is a distillation of the Training Overview documentation)
846846
* [Dataset Overview](https://sbert.net/docs/sentence_transformer/dataset_overview.html)
847847
* [Loss Overview](https://sbert.net/docs/sparse_encoder/loss_overview.html)
848848
* [API Reference](https://sbert.net/docs/package_reference/sparse_encoder/index.html)
849849
850850
And lastly, here are some advanced pages that might interest you:
851851
852852
* [Hyperparameter Optimization](https://sbert.net/examples/sentence_transformer/training/hpo/README.html)
853-
* [Distributed Training](https://sbert.net/docs/sentence_transformer/training/distributed.html)
853+
* [Distributed Training](https://sbert.net/docs/sentence_transformer/training/distributed.html)

0 commit comments

Comments
 (0)