You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: api-reference/how-to/embedding.mdx
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -43,7 +43,7 @@ To use the Ingest CLI or Ingest Python library to generate embeddings, do the fo
43
43
44
44
1. Choose an embedding provider that you want to use from among the following allowed providers, and note the provider's ID:
45
45
46
-
- The provider ID `aws-bedrock` for [Amazon Bedrock](https://aws.amazon.com/bedrock/). [Learn more](https://python.langchain.com/v0.2/docs/integrations/text_embedding/bedrock/).
46
+
- The provider ID `bedrock` for [Amazon Bedrock](https://aws.amazon.com/bedrock/). [Learn more](https://python.langchain.com/v0.2/docs/integrations/text_embedding/bedrock/).
47
47
-`huggingface` for [Hugging Face](https://huggingface.co/). [Learn more](https://python.langchain.com/v0.2/docs/integrations/text_embedding/huggingfacehub/).
48
48
-`mixedbread-ai` for [Mixedbread](https://www.mixedbread.ai/). [Learn more](https://www.mixedbread.ai/docs/embeddings/overview).
49
49
-`octoai` for [Octo AI](https://octo.ai/). [Learn more](https://octo.ai/docs/text-gen-solution/using-unstructured-io-for-embedding-documents).
@@ -54,7 +54,7 @@ To use the Ingest CLI or Ingest Python library to generate embeddings, do the fo
54
54
55
55
2. Run the following command to install the required Python package for the embedding provider:
56
56
57
-
- For `aws-bedrock`, run `pip install "unstructured-ingest[bedrock]"`.
57
+
- For `bedrock`, run `pip install "unstructured-ingest[bedrock]"`.
58
58
- For `huggingface`, run `pip install "unstructured-ingest[embed-huggingface]"`.
59
59
- For `mixedbread-ai`, run `pip install "unstructured-ingest[embed-mixedbreadai]"`.
60
60
- For `octoai`, run `pip install "unstructured-ingest[embed-octoai]"`.
@@ -65,7 +65,7 @@ To use the Ingest CLI or Ingest Python library to generate embeddings, do the fo
65
65
66
66
3. For the following embedding providers, you can choose the model that you want to use. If you do choose a model, note the model's name:
67
67
68
-
-`aws-bedrock`. [Choose a model](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html). No default model is provided. [Learn more about the supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html).
68
+
-`bedrock`. [Choose a model](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html). No default model is provided. [Learn more about the supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html).
69
69
-`huggingface`. [Choose a model](https://huggingface.co/models?other=embeddings), or use the default model [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2).
70
70
-`mixedbread-ai`. [Choose a model](https://www.mixedbread.ai/docs/embeddings/models), or use the default model [mixedbread-ai/mxbai-embed-large-v1](https://www.mixedbread.ai/docs/embeddings/mxbai-embed-large-v1).
71
71
-`octoai`. [Choose a model](https://octo.ai/blog/supercharge-rag-performance-using-octoai-and-unstructured-embeddings/), or use the default model `thenlper/gte-large`.
@@ -76,7 +76,7 @@ To use the Ingest CLI or Ingest Python library to generate embeddings, do the fo
76
76
77
77
4. Note the special settings to connect to the provider:
78
78
79
-
- For `aws-bedrock`, you'll need an AWS access key value, the corresponding AWS secret access key value, and the corresponding AWS Region identifier. [Get an AWS access key and secret access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html).
79
+
- For `bedrock`, you'll need an AWS access key value, the corresponding AWS secret access key value, and the corresponding AWS Region identifier. [Get an AWS access key and secret access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html).
80
80
- For `huggingface`, if you use a gated model (a model with special conditions that you must accept before you can use it, or a privately published model), you'll need an HF inference API key value, beginning with `hf_`. [Get an HF inference API key](https://huggingface.co/docs/api-inference/en/quicktour#get-your-api-token). To learn whether your model requires an HF inference API key, see your model provider's documentation.
81
81
- For `mixedbread-ai`, you'll need a Mixedbread API key value. [Get a Mixedbread API key](https://www.mixedbread.ai/dashboard?next=api-keys).
82
82
- For `octoai`, you'll need an Octo AI API token value. [Get an Octo AI API token](https://octo.ai/docs/getting-started/how-to-create-octoai-access-token).
@@ -94,7 +94,7 @@ To use the Ingest CLI or Ingest Python library to generate embeddings, do the fo
94
94
- Set the command's `--embedding-provider` to the provider's ID, for example `huggingface`.
95
95
- Set `--embedding-model-name` to the model name, as applicable, for example `sentence-transformers/sentence-t5-xl`. Or omit this to use the default model, as applicable.
96
96
- Set `--embedding-api-key` to the provider's required API key value or credentials JSON file path, as appropriate.
97
-
- For `aws-bedrock`:
97
+
- For `bedrock`:
98
98
99
99
- Set `--embedding-aws-access-key-id` to the AWS access key value.
100
100
- Set `--embedding-aws-secret-access-key` to the corresponding AWS secret access key value.
@@ -106,7 +106,7 @@ To use the Ingest CLI or Ingest Python library to generate embeddings, do the fo
106
106
- Set the `embedding_provider` parameter to the provider's ID, for example `huggingface`.
107
107
- Set `embedding_model_name` to the model name, as applicable, for example `sentence-transformers/sentence-t5-xl`. Or omit this to use the default model, as applicable.
108
108
- Set `embedding_api_key` to the provider's required API key value or credentials JSON file path, as appropriate.
109
-
- For `aws-bedrock`:
109
+
- For `bedrock`:
110
110
111
111
- Set `embedding_aws_access_key_id` to the AWS access key value.
112
112
- Set `embedding_aws_secret_access_key` to the corresponding AWS secret access key value.
Copy file name to clipboardExpand all lines: snippets/ingest-configuration-shared/embedding-configuration.mdx
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ A common embedding configuration is a critical component that allows for dynamic
10
10
11
11
* <Iconicon="v"/><Iconicon="1"/> `aws_secret_access_key`: The AWS secret access key to be used for AWS-based embedders, such as Amazon Bedrock.
12
12
13
-
* <Iconicon="v"/><Iconicon="2"/> `embedding_provider`: The embedding provider to use while doing embedding. Available values include `aws-bedrock`, `azure-openai`, `huggingface`, `mixedbread-ai`, `octoai`, `openai`, `togetherai`, `vertexai`, and `voyageai`.
13
+
* <Iconicon="v"/><Iconicon="2"/> `embedding_provider`: The embedding provider to use while doing embedding. Available values include `bedrock`, `azure-openai`, `huggingface`, `mixedbread-ai`, `octoai`, `openai`, `togetherai`, `vertexai`, and `voyageai`.
14
14
15
15
* <Iconicon="v"/><Iconicon="2"/> `embedding_api_key`: The API key to use, if one is required to generate the embeddings through an API service, such as OpenAI.
16
16
@@ -24,12 +24,12 @@ A common embedding configuration is a critical component that allows for dynamic
24
24
25
25
* <Iconicon="v"/><Iconicon="1"/> `model_name`: The specific model to use for the embedding provider, if necessary.
26
26
27
-
* <Iconicon="v"/><Iconicon="1"/> `provider`: The embedding provider to use while doing embedding. Available values include `aws-bedrock`, `azure-openai`, `huggingface`, `mixedbread-ai`, `octoai`, `openai`, `togetherai`, `vertexai`, and `voyageai`.
27
+
* <Iconicon="v"/><Iconicon="1"/> `provider`: The embedding provider to use while doing embedding. Available values include `bedrock`, `azure-openai`, `huggingface`, `mixedbread-ai`, `octoai`, `openai`, `togetherai`, `vertexai`, and `voyageai`.
0 commit comments