-
Notifications
You must be signed in to change notification settings - Fork 948
document model profiles feature #1568
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ccurme
wants to merge
6
commits into
main
Choose a base branch
from
cc/model_profiles
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+106
−2
Open
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1190,6 +1190,89 @@ LangChain supports all major model providers, including OpenAI, Anthropic, Googl | |
| ## Advanced topics | ||
| ### Model profiles | ||
| <Warning> This is a beta feature. The format of model profiles is subject to change. </Warning> | ||
| <Info> Model profiles require `langchain>=1.1`. </Info> | ||
| :::python | ||
| LangChain chat models expose supported features and capabilities through a `.profile` attribute: | ||
| ```python | ||
| model.profile | ||
| # { | ||
| # "max_input_tokens": 400000, | ||
| # "image_inputs": True, | ||
| # "reasoning_output": True, | ||
| # "tool_calling": True, | ||
| # ... | ||
| # } | ||
| ``` | ||
| Refer to the full set of fields in the [API reference](https://reference.langchain.com/python/langchain_core/language_models/). | ||
| Much of the model profile data is powered by the [models.dev](https://github.com/sst/models.dev) project, an open source initiative that provides model capability data. These data are augmented with additional fields for purposes of use with LangChain. These augmentations are kept aligned with the upstream project as it evolves. | ||
| Model profile data allow applications to work around model capabilities dynamically. For example: | ||
| 1. [Summarization middleware](/oss/langchain/middleware/built-in#summarization) can trigger summarization based on a model's context window size. | ||
| 2. [Structured output](/oss/langchain/structured-output) strategies in `create_agent` can be inferred automatically (e.g., by checking support for native structured output features). | ||
| 3. Model inputs can be gated based on supported [modalities](#multimodal) and maximum input tokens. | ||
| <Accordion title="Updating or overwriting profile data" icon="link"> | ||
|
||
| Model profile data can be changed if it is missing, stale, or incorrect. | ||
| **Option 1 (quick fix)** | ||
| You can instantiate a chat model with any valid profile: | ||
| ```python | ||
| custom_profile = { | ||
| "max_input_tokens": 100_000, | ||
| "tool_calling": True, | ||
| "structured_output": True, | ||
| # ... | ||
| } | ||
| model = init_chat_model("...", profile=custom_profile) | ||
| ``` | ||
| The `profile` is also a regular `dict` and can be updated in place. If the model instance is shared, consider using | ||
| ```python | ||
| new_profile = model.profile | {"key": "value"} | ||
| model.model_copy(update={"profile": new_profile}) | ||
| ``` | ||
| to avoid mutating shared state. | ||
| **Option 2 (fix data upstream)** | ||
| The primary source for the data is the [models.dev](https://models.dev/) project. These data are merged with additional fields and overrides in LangChain [integration packages](/oss/python/integrations/providers/overview) and are shipped with those packages. | ||
| Model profile data can be updated through the following process: | ||
| 1. (If needed) update the source data at [models.dev](https://models.dev/) through a pull request to its [repository on Github](https://github.com/sst/models.dev). | ||
| 2. (If needed) update additional fields and overrides in `langchain_<package>/data/profile_augmentations.toml` through a pull request to the LangChain [integration package](/oss/python/integrations/providers/overview)`. | ||
| 3. Use the [langchain-model-profiles](https://pypi.org/project/langchain-model-profiles/) CLI tool to pull the latest data from [models.dev](https://models.dev/), merge in the augmentations and update the profile data: | ||
|
|
||
| ```bash | ||
| pip install langchain-model-profiles | ||
| ``` | ||
| ```bash | ||
| langchain-profiles refresh --provider <provider> --data-dir <data_dir> | ||
| ``` | ||
| That command will: | ||
| - Download the latest data for `<provider>` from models.dev | ||
| - Merge in augmentations from `profile_augmentations.toml` in `<data_dir>` | ||
| - Write the merged profiles to `profiles.py` in `<data_dir>` | ||
|
|
||
| Example, from [libs/partners/anthropic](https://github.com/langchain-ai/langchain/tree/master/libs/partners/anthropic) in the LangChain monorepo: | ||
| ```bash | ||
| uv run --with langchain-model-profiles --provider anthropic --data-dir langchain_anthropic/data | ||
| ``` | ||
| </Accordion> | ||
|
|
||
| ::: | ||
|
|
||
| :::js | ||
| LangChain chat models expose supported features and capabilities through a `.profile` accessor. | ||
|
|
||
| ::: | ||
|
|
||
| ### Multimodal | ||
|
|
||
| Certain models can process and return non-textual data such as images, audio, and video. You can pass non-textual data to a model by providing [content blocks](/oss/langchain/messages#message-content). | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
below where?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clarified this!