You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/config/models.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,9 +6,9 @@ This page contains information on selecting a model to use and options to supply
6
6
7
7
GraphRAG was built and tested using OpenAI models, so this is the default model set we support. This is not intended to be a limiter or statement of quality or fitness for your use case, only that it's the set we are most familiar with for prompting, tuning, and debugging.
8
8
9
-
Starting with version 2.6.0, GraphRAG supports using[LiteLLM](https://docs.litellm.ai/) for calling language models. LiteLLM provides support for 100+ models though it is important to note that when choosing a model it must support returning [structured outputs](https://openai.com/index/introducing-structured-outputs-in-the-api/) adhering to a [JSON schema](https://docs.litellm.ai/docs/completion/json_mode).
9
+
GraphRAG uses[LiteLLM](https://docs.litellm.ai/) for calling language models. LiteLLM provides support for 100+ models though it is important to note that when choosing a model it must support returning [structured outputs](https://openai.com/index/introducing-structured-outputs-in-the-api/) adhering to a [JSON schema](https://docs.litellm.ai/docs/completion/json_mode).
10
10
11
-
Example using LiteLLm as the language model tool for GraphRAG:
11
+
Example using LiteLLM as the language model manager for GraphRAG:
12
12
13
13
```yaml
14
14
models:
@@ -37,7 +37,7 @@ See [Detailed Configuration](yaml.md) for more details on configuration. [View L
37
37
38
38
## Model Selection Considerations
39
39
40
-
GraphRAG has been most thoroughly tested with the gpt-4 series of models from OpenAI, including gpt-4 gpt-4-turbo, gpt-4o, and gpt-4o-mini. Our [arXiv paper](https://arxiv.org/abs/2404.16130), for example, performed quality evaluation using gpt-4-turbo. As stated above, non-OpenAI models are now supported with GraphRAG 2.6.0 and onwards through the use of LiteLLM but the suite of gpt-4 series of models from OpenAI remain the most tested and supported suite of models for GraphRAG.
40
+
GraphRAG has been most thoroughly tested with the gpt-4 series of models from OpenAI, including gpt-4 gpt-4-turbo, gpt-4o, and gpt-4o-mini. Our [arXiv paper](https://arxiv.org/abs/2404.16130), for example, performed quality evaluation using gpt-4-turbo. As stated above, non-OpenAI models are supported through the use of LiteLLM but the suite of gpt-4 series of models from OpenAI remain the most tested and supported suite of models for GraphRAG – in other words, these are the models we know best and can help resolve issues with.
41
41
42
42
Versions of GraphRAG before 2.2.0 made extensive use of `max_tokens` and `logit_bias` to control generated response length or content. The introduction of the o-series of models added new, non-compatible parameters because these models include a reasoning component that has different consumption patterns and response generation attributes than non-reasoning models. GraphRAG 2.2.0 now supports these models, but there are important differences that need to be understood before you switch.
43
43
@@ -85,30 +85,30 @@ global_search:
85
85
86
86
Another option would be to avoid using a language model at all for the graph extraction, instead using the `fast` [indexing method](../index/methods.md) that uses NLP for portions of the indexing phase in lieu of LLM APIs.
87
87
88
-
## Using Non-OpenAI Models
88
+
## Using Custom Models
89
89
90
-
As shown above, non-OpenAI models may be used via LiteLLM starting with GraphRAG version 2.6.0 but cases may still exist in which some users wish to use models not supported by LiteLLM. There are two approaches one can use to connect to unsupported models:
90
+
LiteLLM supports hundreds of models, but cases may still exist in which some users wish to use models not supported by LiteLLM. There are two approaches one can use to connect to unsupported models:
91
91
92
92
### Proxy APIs
93
93
94
94
Many users have used platforms such as [ollama](https://ollama.com/) and [LiteLLM Proxy Server](https://docs.litellm.ai/docs/simple_proxy) to proxy the underlying model HTTP calls to a different model provider. This seems to work reasonably well, but we frequently see issues with malformed responses (especially JSON), so if you do this please understand that your model needs to reliably return the specific response formats that GraphRAG expects. If you're having trouble with a model, you may need to try prompting to coax the format, or intercepting the response within your proxy to try and handle malformed responses.
95
95
96
96
### Model Protocol
97
97
98
-
As of GraphRAG 2.0.0, we support model injection through the use of a standard chat and embedding Protocol and an accompanying factories that you can use to register your model implementation. This is not supported with the CLI, so you'll need to use GraphRAG as a library.
98
+
We support model injection through the use of a standard chat and embedding Protocol and accompanying factories that you can use to register your model implementation. This is not supported with the CLI, so you'll need to use GraphRAG as a library.
99
99
100
100
- Our Protocol is [defined here](https://github.com/microsoft/graphrag/blob/main/graphrag/language_model/protocol/base.py)
101
101
- We have a simple mock implementation in our tests that you can [reference here](https://github.com/microsoft/graphrag/blob/main/tests/mock_provider.py)
102
102
103
103
Once you have a model implementation, you need to register it with our ChatModelFactory or EmbeddingModelFactory:
Copy file name to clipboardExpand all lines: docs/config/yaml.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,7 +21,7 @@ default_chat_model:
21
21
22
22
### models
23
23
24
-
This is a dict of model configurations. The dict key is used to reference this configuration elsewhere when a model instance is desired. In this way, you can specify as many different models as you need, and reference them differentially in the workflow steps.
24
+
This is a dict of model configurations. The dict key is used to reference this configuration elsewhere when a model instance is desired. In this way, you can specify as many different models as you need, and reference them independently in the workflow steps.
25
25
26
26
For example:
27
27
```yml
@@ -173,7 +173,7 @@ Where to put all vectors for the system. Configured for lancedb by default. This
173
173
- `audience`**str** (only for AI Search) - Audience for managed identity token if managed identity authentication is used.
174
174
- `index_prefix`**str** - (optional) A prefix for the indexes you will create for embeddings. This stores all indexes (tables) for a given dataset ingest.
175
175
- `database_name`**str** - (cosmosdb only) Name of the database.
176
-
- `embeddings_schema`**list[dict[str, str]]** (optional) - Enables customization for each of your embeddings.
176
+
- `embeddings_schema`**dict[str, dict[str, str]]** (optional) - Enables customization for each of your embeddings.
177
177
- `<supported_embedding>`:
178
178
- `index_name` **str**: (optional) - Name for the specific embedding index table.
179
179
- `id_field` **str**: (optional) - Field name to be used as id. Default=`id`
@@ -332,7 +332,7 @@ These are the settings used for Leiden hierarchical clustering of the graph to c
332
332
#### Fields
333
333
334
334
- `embeddings`**bool** - Export embeddings snapshots to parquet.
335
-
- `graphml`**bool** - Export graph snapshots to GraphML.
335
+
- `graphml`**bool** - Export graph snapshot to GraphML.
0 commit comments