Commit 6ec8e89
MLFlow Release Finishing Touches (#2359)
* add missed mkdocs link
* Fix MLflow integration issues with TruLens scorers
1. Fix AttributeError when instrumenting LiteLLM endpoint
The _instrument_class method was failing with:
AttributeError: 'CallTypes' object has no attribute '__name__'
This occurred because the method tried to wrap non-callable attributes
(like enum values) that happened to have the same name as the target
method. Added a callable() check to skip non-callable attributes.
2. Suppress noisy third-party library warnings
Added warning filters to suppress:
- pkg_resources is deprecated warning from munch library
- python-dotenv could not parse statement warnings/logs
These warnings were cluttering output when using TruLens scorers
via MLflow GenAI evaluation framework.
.... Generated with [Cortex Code](https://docs.snowflake.com/user-guide/snowflake-cortex/cortex-agents)
Co-Authored-By: Cortex Code <noreply@snowflake.com>
* Add MLflow + TruLens scorers example notebook
New example notebook demonstrating how to use TruLens feedback functions
as first-class scorers in MLflow GenAI evaluation framework (MLflow 3.10+).
Covers:
- RAG evaluation scorers (Groundedness, ContextRelevance, AnswerRelevance)
- Output scorers (Coherence)
- Agent trace scorers (ToolSelection, ToolCalling)
- Batch evaluation with mlflow.genai.evaluate
- Threshold configuration and multi-provider support
.... Generated with [Cortex Code](https://docs.snowflake.com/user-guide/snowflake-cortex/cortex-agents)
Co-Authored-By: Cortex Code <noreply@snowflake.com>
* Update MLflow + TruLens scorers example notebook
Removed redundant Best Practices section to keep notebook focused on
demonstrating scorer usage.
.... Generated with [Cortex Code](https://docs.snowflake.com/user-guide/snowflake-cortex/cortex-agents)
Co-Authored-By: Cortex Code <noreply@snowflake.com>
* Add agent evaluation examples to MLflow integration docs
Added new "Agent Evaluation" section covering:
- Batch agent evaluation with predict_fn
- Evaluating individual agent traces
- Note clarifying when to use agent vs RAG scorers
.... Generated with [Cortex Code](https://docs.snowflake.com/user-guide/snowflake-cortex/cortex-agents)
Co-Authored-By: Cortex Code <noreply@snowflake.com>
* Rename 'Running Feedback Functions' to 'Running Metrics'
---------
Co-authored-by: sfc-gh-jreini <sfc-gh-jreini@users.noreply.github.com>
Co-authored-by: Cortex Code <noreply@snowflake.com>1 parent fd6268b commit 6ec8e89
File tree
7 files changed
+1016
-31
lines changed- docs/component_guides/evaluation
- examples/expositional/frameworks/mlflow
- src
- core/trulens/core
- feedback
- providers/litellm/trulens/providers/litellm
7 files changed
+1016
-31
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
176 | 176 | | |
177 | 177 | | |
178 | 178 | | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
179 | 259 | | |
180 | 260 | | |
181 | 261 | | |
| |||
0 commit comments